In the previous work, it was demonstrated that one can effectively employ CTRNN-EH (a neuromorphic variant of EH method) methodology to evolve neuromorphic flight controllers for a flapping wing robot. This paper describes a novel frequency grouping-based analysis technique, developed to qualitatively decompose the evolved controllers into explainable functional control blocks. A summary of the previous work related to evolving flight controllers for two categories of the controller types, called autonomous and nonautonomous controllers, is provided, and the applicability of the newly developed decomposition analysis for both controller categories is demonstrated. Further, the paper concludes with appropriate discussion of ongoing work and implications for possible future work related to employing the CTRNN-EH methodology and the decomposition analysis techniques presented in this paper.
Mosaforementionedt, if not all, existing bird-sized and insect-sized flapping-wing vehicles possess only a small number of actively controlled degrees of freedom. In these vehicles, the bulk of the wing motions are generated via a combination of actively driven linkages (motors and armatures, piezoelectric beams, etc.) and passively driven elements (wing flex or rotation via dynamic pressure loading, etc.) [
The CTRNN-EH framework introduced in the previous works [
Schematic of CTRNN-EH Framework.
The above-mentioned CTRNN-EH framework has been successfully employed to control legged locomotion in both real and simulated hexapod walkers [
However, conceptually the flapping-wing flight problem shares the requirement of generating optimal oscillatory dynamics for desired flight behavior; with the hexapod walker problem, the former has inherent instability in its body dynamics, introduced by virtue of the medium of its flight (i.e., in three, dimensional space with constantly varying center of mass). This possible inherent instable body dynamics present in flapping-wing vehicles might make the CTRNN-EH based learning more challenging to be effective than when applied to the hexapod walkers, to generate optimal actuator dynamics. Further, the dynamical module analysis [
These are neural network configurations that produce oscillatory signals without any external sensory inputs [
Illustrative representation of the assembly of neurons forming the autonomous neuromorphic (shown in (a)) and nonautonomous Neuromorphic (shown in (b)) controllers. “
These are neural network configurations that can produce appropriate oscillatory patterns only when coupled to some other oscillatory system. They are more completely discussed in [
This section describes authors’s successful efforts aimed at evolving autonomous and non-autonomous CTRNN-EH controllers for a number of flapping-wing vehicle flight modes [
Micromechanical flying insect model was developed by MFI team at UC Berkeley [
In brief, the implemented MFWR model takes wing (left and right) actuation parametric inputs like stroke and rotation trajectories and produces the position and attitude information of the MFWR in the world coordinate system as shown in Figure
Schematic of the simulated Microlevel Flapping Winged Robot (MFWR).
After some preliminary experimentation conducted with the MFWR model, the custom CTRNN-EH control architecture shown in Figure
Interfacing CTRNN Architecture to MFWR Model to evolve and analyze flight controllers. In the figure,
As mentioned earlier, each individual CTRNN neuron is specified by one bias, one time constant, and eight weighted connections from all neurons in the network (seven connections to other neurons, one self-connection, and one sensor). Thus, the central core network, with eight-neurons, is fully specified by the numeric value settings of eighty-eight parameters, with eleven parameters for each neuron in a fully connected eight-neuron network. The aforementioned minipop EA is implemented with a population size of 4 and a mutation rate of 0.005. The genome length chosen was equal to total number of bits employed to encode a given CTRNN configuration. Each neuron parameter in the configuration is encoded in eight bits, which aggregates to a genome length of 704 bits to represent the central core network. The delay input interval duration for gate networks is encoded in an eight-bit string. Employing the aforementioned architecture and algorithm specifications the next sections provide the details of the evolutionary runs and evaluation criterion applied to evolving autonomous and nonautonomous controllers.
Three kinds of autonomous flight controllers, namely, cruising, altitude gain, and steering, were successfully evolved using the aforementioned architecture and algorithm, but with type-specific fitness evaluation criterion. For example, an acceptable behavior of MFWR under an evolved cruise mode controller is to produce motion in a forward direction that is greater than the motion in altitude or sideward directions. Moreover, it should also maintain zero angular velocity along the three vehicle frame axes (zero pitch, roll, and yaw). The later criteria of the expected controller can be met by employing preevolved CTRNN-EH gate networks with symmetric delays. But the first and primary criteria of the controller should be evolved in central core, since this is the only module capable of generating any dynamics to drive the wings. Thus, an evaluation function to capture this established cruise criteria should observe the motion of the MFWR under the control of the potential controller and reward the controller on generation of the forward motion and penalize it on generation of altitude variations. A pictorial representation of the expected autonomous cruise behavior and the established relation to its potential evaluation function is shown in Figure
The figure shows the relation between the expected cruise behavior of MFWR and the fitness evaluation function employed to evolve the CTRNN-EH controllers to achieve the same behavior under control. An acceptable cruise controller has to propel the MFWR in forward direction and minimize the variation in the altitude. Thus, the fitness score employed to evolve the cruise controllers should reward any forward motion (in
It can be observed that the above evaluation function captures the expected forward motion by placing constraints on the controller to maximize
The motion of the MFWR in three dimensions (shown in (a)) controlled by a Cruise Mode Controller actuated wing kinematics (shown in (b)). Here the stroke kinematics has relatively higher beats rate than that of the rotation for the above controller shown in the figure, thus the oscillation cycles are cluttered, making it hard to visualize them with respect to the rotation kinematics.
Two kinds of non-autonomous flight controllers were successfully evolved, namely, adaptive cruise mode controllers and polymorphic controllers [
(a) shows the Wing Kinematics generated by the Central Core network of a Non-Autonomous Cruising Controller and the corresponding insect motion and sensory update (the stroke kinematics relatively has higher beats rate than that in rotation for the above controller). (b) shows the Wing Kinematics generated by the Central Core network of a Polymorphic flight Controller and the corresponding insect motion, when acted upon by external sensory inputs. One can see the initial cruising behavior is been switched to Alt. Gain behavior (with brief switching delay in wing kinematics).
The evolved CTRNN-EH flight controllers would be better accepted for practical deployment, at least for engineers, if their functionality can be explained using known general principles of engineering. As with all evolvable hardware-based methods, there exists a possibility that the acceptance of the evolved flight controllers, merely in terms of fitness score value (which is based on closely approximating the acceptable overall body trajectory behavior), could have been exploited the possible underlying noise in the MFWR model to gain optimal controller status. Thus, the first possible analysis to accept the evolved flight controller is to diligently observe and validate the insect’s temporal behavior when coupled with the evolved controller’s dynamics and determine if they satisfy the known principal physical characteristics of the MFWR model flight behavior. Further, it would be of interest to explain the evolved controllers by possible decomposition of the CTRNN-EH layer in terms of logical control blocks. The next subsections deal with analyzing the evolved controllers with two deduced approaches mentioned below.
During the course of this work, it was deduced that the acceptability of the physical behavior of the MFWR flight, produced by the evolved flight controllers could be readily understood by qualitatively contrasting them, with the information discerned from the empirical study conducted on the MFI insect model [
It was demonstrated that the autonomous and nonautonomous controllers, evolved merely based on simple fitness evaluation functions, produced an acceptable physical behavior in the MFWR model in terms of overall body trajectory [
It would be of interest to interpret the evolved controllers by possible decomposition into easily explainable logical control blocks, and there exists a previous work [
The evolved autonomous altitude gain, cruising, and steering and controllers are suspected to fall under the CPG template and could be decomposed into a collection of explainable oscillatory and nonoscillatory neuron groups that produced desired control of the evolved flight behaviors.
On other hand, the nonautonomous cruising mode controllers and polymorphic mode controller (as a whole) are likely to fall under the RPG template and could be decomposed into a collection of sensor-dependent or -independent oscillatory neuron groups. Thus, it would be necessary to find and separate the possible independent and dependent oscillatory control modules in an evolved controller that could aid in characterizing a given controller using known CPG or RPG templates. Further this decomposition process could provide a qualitative view and human understandable structure of the lower-level coordination among these separated modules, which primarily govern the behavior of a given evolved controller. In this vein, a three-step frequency-based analysis procedure is proposed to qualitatively decompose the evolved controllers.
To simplify the process of decomposing, the evolved controllers into a group of functional units, a step-by-step neuron elimination technique, shown in Figure
A pictorial representation of the “dynamics-deprived neuron elimination” process. The primary neurons are labeled as “
Based on the previously mentioned general principle of acceptable controller dynamics, it was deduced that the steady oscillatory dynamics in the wing (stroke or rotation) dictate the flight behavior. Thus, based on this controller acceptability knowledge, it would be appropriate to group the neurons in the reduced network, based on their individual time constants, into no more than two groups (one each for rotation and stroke). As shown in the Figure
A pictorial representation of the “frequency-based grouping” process. The first step of the process, as shown in (a), is to determine a relative threshold time constant (Tau) for the reduced network, reduced by “dynamics-deprived neuron elimination” process, followed by grouping the neurons in the architecture based on the frequency of the output produced by individual neuron (i.e., the neurons with time constant less than the relative threshold are clustered into high frequency group, and neurons with time constants more than the relative threshold are clustered into low frequency group) as shown in (b).
Once the frequency clustered neuron control modules are obtained for a given controller, it is necessary to understand the interactions of the individual neurons within those control modules and with the other existing control modules to qualitatively deduce the underlying governing principle of the controller functionality. Thus, in this lesion study, a general method of diligently observing the variations in the dynamics of an individual or group of neurons, while some of its connections are amputated from rest of the network, has been adopted. Though the number of lesion operations cannot be quantified and will vary depending on the complexity of the evolved controller, but as shown in Figure
A pictorial representation of the “lesion study” process. The lesion study is based on the idea that it is possible to determine the underlying governing functional principle of the network with rigorously observing the behavior changes in the network for appropriate combinations of the amputations. Based on the complexity of the controller, the lesion study can be performed between neurons in distinct frequency groups, which is performing intergroup amputations, shown in (a), or between the neurons in the same frequency group, that is, intragroup amputations shown in (b).
This section provides the detailed qualitative decomposition process for one of the best evolved autonomous cruise mode controllers, using the above-mentioned three general steps. For qualitative comparisons and to better understand the controller decomposition an unaltered original eight-neuron architecture of the controller to be analyzed is shown in Figure
A pictorial representation of the fully connected eight neuron architecture of the autonomous cruise mode controller chosen for qualitative decomposition analysis. As mentioned earlier, the stroke and rotation neurons are marked “
The above figure shows the neuron output state dynamics of each neuron in the fully connected original eight neuron controllers produced during the flight control of the MFWR to provide optimal cruise behavior.
Moreover, the flight trajectory of the MFWR under the control of the original controller in the context is shown in Figure
The MFWR trajectory produced by the original fully connected eight-neuron controller. It can be observed that the evolved controller was successful in producing forward motion in the MFWR without any overall gain in the altitude.
A pictorial representation of the reduced five-neuron architecture of the cruise controller referred in Figure
The later condition eliminates the possibility that the reduced architecture could have changed dramatically and lost its internal dynamics, although it could have satisfied the primary condition to produce the desired cruise behavior in MFWR. Thus, the reduced five neuron controller is evaluated against the MFWR, and the individual neuron output state envelope of the five-neurons is captured and shown in Figure
The above figure shows the neuron output state dynamics of each neuron in the reduced five-neuron architecture of the cruise controller architecture shown in Figure
The MFWR trajectory produced by the reduced five neuron controller. It can be observed that the architectural reduced controller was successful in producing qualitatively same cruise behavior possible by the fully connected eight neuron network. The MFWR trajectory produced by the eight-neuron network is shown in Figure
A pictorial representation of the “frequency-based grouping” process for the cruise mode controller shown in Figure
Moreover, the most convincing evidence that the reduced controller qualitatively controls the MFWR trajectory to produce desired cruise behavior justifies that the dynamics-deprived neuron elimination process is applicable for this controller. Thus, moving forward with the reduced five-neuron architecture, applying frequency-based grouping would be uncomplicated, since it can be observed from the five neuron output envelopes that the primary stroke neuron and third secondary neuron seem to share a peculiar in sync frequency and amplitude variations, intuitively belonging to high frequency group. Moreover, the evolved time constant for both of these neurons is same and is 0.010000 units and on other hand, the neurons 1 and 4 along with the rotation primary neuron can be allocated to low frequency group with corresponding time constants 10.546157, 9.558393, and 20.176863, respectively. Thus, if a relative time constant threshold of 9 units is chosen, then there exist two distinct frequency-based groups as shown in Figure
The output state dynamics of each neuron in the interfrequency group amputated network, amputated as part of the lesion study on the reduced five-neuron network. It can be observed that the primary stroke neuron and the third secondary neuron produced perfect in sync oscillations forming a two-neuron independent oscillator. On the other hand, the primary rotation neuron with second neuron formed a feeble two neuron oscillator. It should be noticed that the fourth neuron dynamics are saturated, in the amputated network, compared to its original oscillatory behavior seen in Figure
The trajectory of MFWR produced under the control of the amputated cruise controller. It can be noticed that the controller, with two independent oscillators for stroke, and rotation produces an acceptable cruise behavior during initial phases of the flight, but immediately loose its ability to control and reduce the altitude variations, in absence of the monitor neuron.
Based on the above analysis, it can be deduced that the evolved autonomous cruise mode controllers can be qualitatively explained as a composition of two steady and independent frequency oscillators, one governing the stroke kinematics of the wing with higher beat rate and another it is rotation with lower beat rate, in presence of a monitoring neuron which periodically tunes the amplitude and frequency of the stroke oscillator, which periodicity synchronized with the rotation oscillator. A pictorial representation of the above deduced compositional template is shown Figure
The Qualitative functional decomposition template derived for the autonomous cruise mode. Most of the autonomous cruise mode controllers can be decomposed into the above-shown template with a high frequency stroke control oscillator module and a low frequency rotation control oscillator along with an intermediate neuron called monitor neuron, which is responsible to coordinate and fine-tune the amplitude and frequency of the stroke oscillator with a period derived from the rotation oscillator. This functional template explains the general evolved behavior of the amplitude and frequency modulation of the stroke kinematics with rotation period for optimal cruise control of MFWR.
The above decomposition analysis mentioned in the context of the cruise mode controllers is performed on the entire best five autonomous altitude gain mode and steer mode controllers. The individual controller architectures were reducible from an 8-neuron to 4-neuron architecture using “Dynamics-deprived Neuron Elimination” process in both categories. Only some of the best altitude gain controllers were complaint with clustering criteria and thus two functional templates were derived using the lesion study performed on the individual neurons in the reduced network. As shown in Figure
The qualitative functional decomposition derived for the evolved autonomous altitude gain controllers and steer controllers. Most of the steer controller’s wing kinematics can be decomposed with a typical CPG-like functional template shown in (a) as a closely coupled stroke and rotation oscillators with steady beat rate and steady amplitude. Most of the altitude gain controllers can be decomposed with the functional template shown in (b), with a dedicated stroke oscillator along with a saturated rotation control module.
This section provides the detailed qualitative decomposition process for one of the best evolved non-autonomous cruise mode controllers. The applicability of the established three-step decomposition using Dynamics-Deprived Neuron Elimination, Frequency-based grouping, and Lesion Study methods will be presented and possible oscillatory level decomposition will be deduced. For qualitative comparisons and to better understand the controller decomposition, an unaltered original eight-neuron architecture of the controller to be analyzed is shown in Figure
A pictorial representation of the fully connected eight-neuron architecture with a single altitude sensor, of the nonautonomous cruise mode controller chosen for qualitative decomposition analysis. Following the general neuron representation, the stroke and rotation neurons are marked with “
The above figure shows the neuron output state dynamics of each neuron in the fully connected original eight-neuron controller and the external altitude sensor, produced during the flight control of the MFWR to provide optimal cruise behavior. It can be observed that the output states of neurons 1 and 4 entrain with altitude sensor in phase and out of phase, respectively.
The MFWR trajectory produced by the original fully connected eight-neuron nonautonomous controller. It can be observed that the evolved controller was successful in producing forward motion in the MFWR without any overall gain in the altitude.
A pictorial representation of the reduced six-neuron architecture of the cruise controller referred in Figure
Further, the interesting entrainment behavior between the sensor output and the rotation neuron output (and if possible the third (old designated position-fourth) secondary neuron output) should be maintained, at least qualitatively. Thus, the reduced six-neuron controller is evaluated against the MFWR, and the individual neuron output state envelope of the six neurons and the sensor status are captured and shown in Figure
The above figure shows the neuron output state dynamics of each neuron in the reduced six neuron architecture of the cruise controller architecture shown in Figure
The MFWR trajectory produced by the reduced six-neuron controller. It can be observed that the architectural reduced controller was successful in producing qualitatively same cruise behavior possiblely by the fully connected eight-neuron network. The MFWR trajectory produced by the eight-neuron network is shown in Figure
Thus, moving forward with the reduced six-neuron architecture, applying frequency-based grouping would be complicated, since it can observed from the six-neuron output envelopes that the primary stroke neuron, along with second, fourth, and fifth secondary neurons, seems to share the same frequency bandwidth, intuitively belonging to high frequency group.
Moreover, the evolved time constants for these neurons are in the range of 0.010000 to 0.05000 units. But, on the other hand, the rotation primary neuron and the third secondary neuron can be allocated to low frequency group with corresponding time range of 10.034 to 16.532 units. Moreover, since the sensor module output can be treated as a pseudoneuron (with dynamics equivalent to the MFWR model and with interneuron connections to the primary neurons only), there exist two options to decompose the architecture further. The first approach is to group the sensor pseudoneuron into the low frequency group and perform the intergroup lesion study, which will provide the insight into the high frequency group oscillator’s (if at all the group exhibits independent oscillatory nature) dependency on the sensor state and further the same dependency can be derived by performing intragroup lesion study on the low frequency group by amputating the sensor pseudoneuron. The second approach is to group only the real neurons by completely ignoring the sensor signal (i.e., amputating the sensor signal) into a high and low frequency groups and study their behavior independently, checking for independent oscillatory behavior, in the absence of the external sensor signal, followed by introducing the sensor signal to detect any significant behavior changes for deducing any possible independent control modules. Though both approaches would yield the same conclusions, the second approach is chosen since the sensor dynamics of the MFWR can be treated separately from the actual neuron dynamics, in two easy steps of complete sensor-independent neuron dynamics decomposition (frequency grouping and intragroup lesion study) followed by the sensor status injection into the possible neuron-level decomposed modules. Thus, moving forward, the six-neuron architecture is disconnected from the external sensor and a frequency-based grouping, with groups mentioned earlier is performed as shown in the pictorial representation Figure
A pictorial representation of the “frequency-based grouping” process combined with intergroup lesion study in absence of the external sensor input for the nonautonomous cruise mode controller shown in Figure
The above figure shows the neuron output state dynamics of each neuron in the low frequency group (1 and 3) and high frequency group (0, 2, 4, and 5) after intergroup amputation is performed and evaluated in absence of the external altitude sensor signal (represented in Figure
The MFWR trajectory produced by the controller during the lesion study performed with the techniques shown in Figure
The Qualitative functional decomposition template derived for the non-autonomous cruise mode controllers. Most of the nonautonomous cruise mode controllers can be decomposed into the above-shown template, as a combination of two independent oscillator, of which the high-frequency oscillator, controlled the stroke kinematics of the wing with steady amplitude and frequency and the low-frequency oscillator which was evolved to monitor the altitude variations in the MFWR, through the available external sensor module, altered the rotation dynamics continuously to limit the variations in the altitude of the MFWR and simultaneously provided the forward motion in it, by generating required lift and anti-lift with the behavior verified by the general principles of the empirical study.
Since the polymorphic controllers embed in their architecture both the autonomous altitude gain and cruise mode controllers, which can be invoked as a separate controllers in isolation with a static external signal not a continuous dynamic signal, the qualitative functional decomposition templates presented for the autonomous cruise and altitude gain controllers in the previous section would be applicable for decomposing the polymorphic controllers into two isolated general templates pictorial represented in Figure
(a) shows the neuron output state dynamics of each neuron in the polymorphic controller when presented with a cruise command, whose the external sensor signal is “0.” It can be seen that the neurons 0, 1, 4, and 5 form a composite module with same frequency and would not comply with established criteria for frequency-based grouping. But, the cruise mode controller has been verified to form a single frequency composite stroke and rotation control module as shown in Figure
Moving forward, it can be observed from Figure
In this paper, we have summarized author’s prior efforts using the Neuromorphic Evolvable Hardware (CTRNN-EH) framework to successfully evolve locomotion and different flight mode controllers, with detailed emphasis on the flight mode controllers. Further, a new frequency-based analysis procedure has been introduced to analyze the different evolved flight mode controllers, besides providing a brief qualitative analysis suggesting the acceptability of the evolved controllers for the given flight mode in the context. Moreover, the proposed frequency-based analysis methodology has been successfully applied to the evolved autonomous and nonautonomous controllers, and it has been demonstrated that the methodology can be indeed used to decompose the evolved controllers into logically explainable control blocks for further control analysis. Finally, it can be perceived from the presented results and discussion that the proposed Neuromorphic Evolvable Hardware (CTRNN-EH) and frequency-based analysis methodologies can be employed to control problems that are similar to the flapping flight domain, using tabularasa approach. Though, it is not always an appropriate recommendation to employ a tabula-rasa approach to the control problems at hand; it can serve as an only approach where a suitably impressive closed-form traditional controller does not exist. Moreover, the above-proposed CTRNN-EH methodologies have also been successfully employed to design and evolve hybrid controllers, with evolvable module in the base traditional controller being evolved to supplement the control characteristics of the traditional controllers with rich dynamics of CTRNNs [