4.5.1 Appetitive Control
The appetitive controller is composed of four main subsystems. The first three will be described in this section and the fourth will be discussed in the next section. The three that will be described here are the chemotaxis, feeding arousal, and appetite behavior integration controls. Chemotaxis is the orientation or movement of an organism or cell in relation to chemical agents. This is a very common process that is used by all kinds of organisms, from insects to amoeba, to find food. Granted, such different entities use completely separate mechanisms to do this, but the underlying principle is the same. Insects like the cockroach use their antennas to detect chemical odors. And just like we use our ears to detect the direction a sound is coming from, they use their antennas to detect the direction where the food is. The left and right antenna are separated on the head and the insect can compare the strength of the odor scent at each one to determine which direction it should turn towards. Hunger is a very important behavior for any animal. It gives the organism some idea of what its current energy status is. If an insect is getting low on energy it will become hungry and actively try to find and eat food. However, if an insect is currently sated then it may simply ignore food that it stumbles across because it is not hungry at the moment. The final behavior of an insect is dependent on many internal and external variables, and hunger is one of the most important. Next, a number of the insects different behaviors are mutually exclusive. Attempting to perform random wandering at the same time you are trying to move towards food are not really compatible behaviors. There must be some controls that determine when one behavior is active and another is not. That is what the behavior integration neurons are for. Finally, The next section will discuss eating.
Figure 1 is the neural layout for the appetitive controller. The eating subsystem will be discussed in the next section. The chemotaxis neurons are the ones with the white background, the feeding arousal neurons have the light blue background, and the integration neurons have the light green background.
Before the insect can eat it must first be able to find the food. The purpose of the chemotaxis control neurons is to orient the insect so that it walks towards a food source that it can smell. It will be up to the feeding arousal controls to determine if chemotaxis should be something that is actually being used for a behavior or not at the given time. The antenna chemical sensor neurons are critical for chemotaxis. These neurons are the Antenna Chemical Sensor Low Left (ACSll), Antenna Chemical Sensor Low Right (ACSlr), Antenna Chemical Sensor High Left (ACShl), and Antenna Chemical Sensor High Right (ACShr). Each different type of food has some graph parameters that define how its odor diffuses. In all of the examples currently being used the inverse square law determines the quantity of diffused odor chemicals from the tip of an insect antenna to the center of a food pile. A summation is taken of this value for all food piles in the environment.
Figure 2 demonstrates how the chemical odor value is calculated. Z1, Z2, and Z3 are the linear distances between the tip of the left antenna and the center of each pile of food of a specific type. These values are then plugged into the equations that calculate the value of odor molecules that could diffuse that far for each pile: D1, D2, and D3. The value A is a diffusion constant. The larger A is the more odor is diffused for a given distance, and if A is zero then no odor is diffused at all. These diffusion values are then added up as the total quantity of odor chemicals that are detectable for that antenna. Obstacles do not effect the ability of odors to diffuse. The linear distance is still used to calculate the diffusion quantity even if it passes directly over an obstacle. Finally, each antenna then has a gain which controls the sensitivity of that antenna. The low and high in the chemical sensor names refers to this gain on the antenna. A chemical sensor with a very high gain will be able to detect smaller quantities of chemicals, and will thus be able to detect food at a much longer distance. However, since it has such a high gain it also means that as it gets closer to the food the high gain sensors will saturate and the animal will no longer be able to get any useful orientation information from them. That is where the low gain sensors come into play. It takes a good deal more of the chemicals to be present for them to respond. This means that they will not be able to detect food unless they are very near it. But it also means that they will be able to guide the insect even when they are standing almost directly on top of the food. The outputs from these chemical sensor neurons are fed into the Left Odor Strength (LOS) and Right Odor Strength (ROS) neurons. Chemical sensors from the left side of the body excite the LOS neuron and inhibit the ROS neuron, and chemical sensors from the right side excite the ROS neuron and inhibit the LOS neuron. This performs a comparison of the odor strength values that are coming from the left and right side that can be used to determine which way to turn in order to orient the insect towards the food it is smelling. For example, if the odor strength is such that the chemical sensor of the right antenna was firing at 0.40 Hz and the chemical sensor on the left was firing at 0.34 Hz, then LOS would be getting (1.7 na - 2.0 na) = -0.3 na inhibition, and ROS would get (2.0 na - 1.7 na) = 0.3 na excitation. This means that the insect needs to turn to the right to orient towards the food, so ROS stimulates the RT neuron. If the insect is pointed directly at the food then the distance from the left antenna to the food and the distance from the right antenna to the food will be the same, and the excitation and inhibition to neurons LOS and ROS will cancel out so that neither one will attempt to turn the insect.
3. Chemotaxis Analysis
In video 1 the insect is placed in an environment that is empty except for one pile of food. It has an energy level set that will make it hungry, and the eating subsystem is disabled so that it can not stop to eat food once it finds it. This example is just to demonstrate how it uses odor to orient its body so that it walks towards the food. Looking at the neural output in the graph for video 2 some important facts pop out. First notice how the ACS high gain neurons are firing at some non-saturated level at the start, and the left one is higher than the right. The low gain antennas are barely firing at all. Since there is such a large difference between the outputs of the left and right high gain antenna LOS is firing at maximum, and ROS is silent. As the insect moves closer to the food both high gain antenna saturate at maximum, and the low gain antenna take over. When the insect is almost directly over the top of the food then even slight turns move it off center and that is what causes the alternation between LOS and ROS firing, and causes the insect to swag left and right slightly. Normally, the insect would begin eating once it was over the top of the food when it is this hungry. But with the eating system disabled it simply keeps on walking. Once it has walked far enough away the high gain sensors begin to show enough differences that they start to turn the insect again and the whole thing is repeated.
The neurons with the light blue background in figure 1 are the ones involved in hunger. The two key neurons here are the Energy Sensor (ES) neuron and the Feeding Arousal (FA) neuron. The energy sensor neuron maps the amount of energy that the insect currently has to a given intrinsic current. It uses a quadratic equation for this mapping. So when the insects energy is high, ES is firing at full strength. The quadratic is setup so that ES continues firing somewhat strongly until a certain level is reached. FA is a tonic neuron that is always trying to fire. It is basically the hunger neuron. When it is firing then the insect is hungry, and when it is silent the insect has a full tummy. ES inhibits the feeding arousal neuron. So when the insect has a full stomach, ES is actively inhibiting FA and keeping it from firing. But when the energy level drops, ES no longer fires and FA begins firing, signaling that the insect is hungry. The Search Command (SC) neuron is in charge of determining whether the insect should use chemotaxis to try and orient towards food or not. If the insect is hungry then FA will be firing and will stimulate SC. SC stimulates the Following Odor Motor Command (FOMC) neuron. This neuron is basically a pass through control neuron that allows other systems to turn off the chemotaxis behavior. When FOMC fires it turns on the gated synapses allowing LOS and ROS to turn the insect. When FOMC is not firing then those two neurons can not turn the insect even if they are firing away like mad.
5. Hunger Analysis
In this example the insect again has its eating neurons disabled so that it can not stop and eat once it gets to food. It also has its wandering system disabled so that it will walk in a straight line. Finally, its energy usage has been increased dramatically from the normal 0.2 energy units per moving time step up to 1.0. This is so that the effects of feeding arousal can be demonstrated. When the simulation begins the insect is almost right on top of a food pile. However, at that point it has a full energy supply and it is not hungry. This can be seen in video 4. The energy level is at 1.0 and FA is completely silent. Because of this the insect ignores that food and walks right past it. But since the energy usage was increased so much, its energy reservoirs quickly begin dropping and it soon starts getting hungry. FA begins increasing at around the six second mark, and soon thereafter SC also begins firing to tell the insect to look for food. It then makes a beeline for the other food pile. In this instance it walks right over the food. But that is because its eating system is disabled. This is a good example of how the internal state of the insect has a profound impact on how it reacts to outside stimuli. Just because there is food nearby, or even directly under it, does not mean that it will stop and eat. The insect will only eat if it is hungry.
6. Appetite Behavior Integration
The other neurons that are involved in hunger are those that are at the top of this layout with the light green background. They are the neurons that integrate the appetitive behavior with the edge following and random wandering systems. When an insect is trying to find food so it can survive it should not be randomly wandering. It would make no sense to be homing in on some succulent scrap of food, and then just take a hard left turn for no reason. So when the insect is following food the wandering controller must be completely shut off. However, the edge following system is a good deal more complex. It can not just be shut off while following food. What happens if the food that the insect is trying to get to is on the other side of a wall that is in its way? The wall poses no boundary to the odor molecules. So the insect can still smell it even though there is a wall blocking its way. If the edge following system were simply shut down then the insect would slam head first into the wall and be stuck. What is needed is a more selective system. If the insect is following food that is on the right side of its body, and if it comes in contact with an obstacle with the right side, then the edge follower should take control. However, if the insect is following an odor on the left side of its body, and comes in contact with an obstacle on its right side, then the edge follower should be suppressed and the insect should follow the odor to the food. The neurons described here implement a neural network that performs this behavior. The Following Odor (FO) neuron is active when the insect can follow odors. Even if other systems are currently inhibiting that behavior. It receives stimulation from the SC neuron and from the two high gain antenna chemical sensors. It is configured so that both SC and one of the chemical sensors must be firing at some minimum level before it will fire. The SC neuron firing basically says that the appetitive controller wants to look for food, and the chemical sensors say that the insect can currently smell food. So when both are active then it wants to follow an odor to food. When FO is active it excites the Non-Wandering Control (NWC) neuron to inhibit the random wandering system. Also, FO connects to two companion neurons called Following Odor Left (FOl) and Following Odor Right (FOr). These two neurons are similar to FO, but they are for specific sides of the body. If the insect smells an odor from its left side, and it is following an odor, then FOl will fire. Otherwise it is silent. These two neurons are used to control the edge following system. If FOl is firing then it excites the Non-Following Control Right (NFCr) neuron that shuts down the edge following system for the right side of the insect. And FOr shuts down the edge following on the left side. So if the insect is trying to follow food that is to its left, and an obstacle is on its right, then it will ignore the obstacle and orient towards the food. However, if the food was on its right, then the edge following system for the right side would still be active and it would continue to follow that edge rather than orient to the food. There are some connections that still need to be made though in order for this to work correctly. As things stand now the insect would be trying to turn to orient towards the food and trying to follow the edge. This would cause a good deal of confusion and would likely get the insect stuck. What is needed is a way for the edge following systems to inhibit the chemotaxis system when it is actively trying to follow an edge. This is accomplished by having the Fr and Fl neurons inhibit the FOMC neuron. This prevents the chemotaxis system from actually turning the insect. By following the edge around towards the food one of two things will happen. First, the insect may go around the obstacle so that the food is now on the opposite side of the insect. For example, If the insect smells food on its left and is blocked by a square obstacle, then it will follow the obstacle. Eventually it will come around to the other side, and at this point the food will now be on its right side and the obstacle on its left side. The edge following system for the left side will be ignored and the insect can make its way to the food. Second, The obstacle may be so large that the insect loses the scent of the food and just continues following the edge of the obstacle in the hope that it will run into more food somewhere else. All of these neurons working together create an insect with some very impressive dynamic behavior.
7. Appetite Behavior Integration Analysis
Video 5 is very similar to the feeding arousal video from above. However, the big difference is that the wandering controls are no longer being manually inhibited. When the simulation starts the insect begins its random wandering to explore the environment. It goes up and right for a little while and then makes a hard left turn. At around eight seconds the SC neuron begins firing. When it reaches about 70% of its max value FO begins firing signaling that it is hot on the trail of some food, and this excites NWC and shuts down the random wandering system. In this example the insect is already pointed almost directly at the food by sheer coincidence. So it just keeps on trucking until it gets to the food. It passed right over the food because its eating system is still disabled.
Video 6 demonstrates what happens when the insect is hungry and their is an obstacle between it and its food. There must be a complex interaction between the appetitive system and the edge following system in order for the insect to successfully get to the food. When the simulation starts the insect is parallel to the wall, but is close enough to smell the food. It immediately begins to turn towards the food and then runs into the obstacle. Notice that as soon as the left antenna comes into contact with the wall the neuron FOMC ceases to fire. This stops the chemotaxis neurons from controlling the movements of the insect. Also, NFCr is firing at a high rate, but NFCl is not firing at all. This is because the insect is following an odor with its left antenna. Since NFCl is not firing, and it has hit an obstacle on its left side, it begins following the edge of the obstacle. It eventually rounds the corner. After it does so the antenna that is following the odor suddenly switches from the left antenna to the right. So now it is the left edge follower that is inhibited and not the right. Almost immediately Fl quits firing and FOMC resumes firing and once again the chemotaxis neurons are in control of maneuvering the animal to the food. At this point please just stop a moment and think about what took place in this example. It does not seem like that big of a deal at first. We see insects doing stuff like this all of the time. But this is not a real insect. It is a simple virtual insect with less than 200 simulated neurons. Yet it was able to detect a food source, move towards that food source, detect that an obstacle was in its way and, all on its own, maneuver around that obstacle until it could get to the food that would keep it alive. None of this behavior was ever explicitly programmed into this simulation. Nowhere in the code was there a snippet that said something like:
All of this behavior resulted solely from the dynamic interactions between the neural network and the environment. So while this may seem like a simple little behavior from a real insect, it is a very important behavior to behold in such a simple simulated insect.
The appetitive control system has added something that until now the insect did not have. This is internal motivation. Before it could explore its environment and wander aimlessly. But now the insect knows hunger, and can find food that can quench that hunger. This adds a very important aspect to the dynamics of the insects behavior. Until this point all behavior elicited from the insect was prompted either by outside stimuli or random neuron firing. Now these external stimuli are placed in a context that is related to a goal that the insect has, to eat and survive. However, the systems described so far only get the insect to the food. It still has no way to actually consume the food.