Information Flow in Animal-Robot Interactions

The nonverbal transmission of information between social animals is a primary driving force behind their actions and, therefore, an important quantity to measure in animal behavior studies. Despite its key role in social behavior, the flow of information has only been inferred by correlating the actions of individuals with a simplifying assumption of linearity. In this paper, we leverage information-theoretic tools to relax this assumption. To demonstrate the feasibility of our approach, we focus on a robotics-based experimental paradigm, which affords consistent and controllable delivery of visual stimuli to zebrafish. Specifically, we use a robotic arm to maneuver a life-sized replica of a zebrafish in a predetermined trajectory as it interacts with a focal subject in a test tank. We track the fish and the replica through time and use the resulting trajectory data to measure the transfer entropy between the replica and the focal subject, which, in turn, is used to quantify one-directional information flow from the robot to the fish. In agreement with our expectations, we find that the information flow from the replica to the zebrafish is significantly more than the other way around. Notably, such information is specifically related to the response of the fish to the replica, whereby we observe that the information flow is reduced significantly if the motion of the replica is randomly delayed in a surrogate dataset. In addition, comparison with a control experiment, where the replica is replaced by a conspecific, shows that the information flow toward the focal fish is significantly more for a robotic than a live stimulus. These findings support the reliability of using transfer entropy as a measure of information flow, while providing indirect evidence for the efficacy of a robotics-based platform in animal behavioral studies. Entropy 2014, 16 1316


Introduction
Animals interact with a large number of sources in their environment through multiple senses, while adapting their behavior in response to a few important cues [1].In social organisms that move in groups, it has been shown that the actions of a few nearest neighbors contribute largely to such cues [2].In this respect, determining the flow of information between interacting individuals can reveal how they prioritize sensory modalities [3], infer the directionality of information flow [4][5][6], quantify leadership roles in animal groups [7], and determine the effectiveness of engineered stimuli in the laboratory studies [8][9][10].These motivations to measure information flow are similar to those encountered in the study of networks of dynamical systems, where tools from information theory have been used to detect causal relationships [11][12][13], locate driving nodes [14], and quantify the strength of network connections [15].
Information flow between two systems is measured as the reduction in the uncertainty of one system given the knowledge of the other [16].Here, uncertainty is quantified by the entropy rate of a stochastic process (or the entropy in the case of a random variable) [16,17].Information flow measured in this sense is able to highlight nonlinear relationships between coupled systems over large interaction delays, a major advantage over correlation-based methods that assume linearity [18].This has resulted in a multitude of applications of information-theoretic tools ranging from neuroscience [19], to structure health monitoring [20], financial time series analysis [21] and weather forecasting [22].
The application of information-theoretic measures to study animal behavior is not new [23][24][25].In [23], the entropy of a dynamical system is used to study rat locomotor behavior; in [24], relative entropy is used to quantify behavior patterns in Pacific bluefin tuna after release; and in [25], entropy rates are used to quantify aphid behavior under different conditions.Across these studies, the advantage of information-theoretic quantities over traditional methods are cited in terms of independence from observational noise and sensitivity to higher order moments in parameter distributions.While these efforts focus on the movement of a single individual, similar opportunities exist in the study of social animals, whose behavior is driven by the complex exchange of information through multiple sensory cues [3,26].Interestingly, entropy-based measures have been used as an efficient sensing approach in swarm navigation algorithms [27] and multi-robot systems [28].
Here, we utilize information-theoretic tools to study the social behavior of zebrafish [29].Zebrafish is selected for its extensive use as an animal model [29][30][31], its prominent shoaling tendency [32] and its use of vision as a dominant sensing modality [30,33].To offer a consistent and controllable visual stimulus to the focal subjects, we consider a robotics-based experimental paradigm, in which fish interact with a live fish and a life-sized replica maneuvered along predetermined trajectories using a robotic arm.We compute transfer entropy between the trajectories of the live fish and the replica to demonstrate the reliability of this approach in elucidating the direction of information flow between the focal subject and the replica.
The paper is organized as follows.In Section 2, we introduce the entropy of a random process and define transfer entropy to measure information flow between two dynamical systems.Section 3 presents materials and methods used to perform and analyze the animal behavior experiments.Results are presented in Section 4, followed by their discussion in Section 5.

Preliminaries on Information Flow Measurements
In information theory, the uncertainty of a random variable is measured by its Shannon entropy, which quantifies the expected value of the amount of information contained in the population [17].Specifically, the Shannon entropy H(X) of a discrete random variable X that can take values in the set X with a probability distribution P X (x), x ∈ X , is defined as: The logarithmic function in Equation ( 1) can have any base; for the selected base 2, H(X) is measured in bits.In terms of expectation, entropy can also be defined as the expected value of − log 2 P X (x), where X is drawn according to P X (x), as . Notably, Rényi [34] generalized this definition of information entropy by proposing a parametric family that includes Shannon entropy as a limiting case.For two random variables X and Y , with Y taking values in Y, the joint entropy, H(X, Y ), is [16]: where P X,Y (x, y) is the joint distribution.The reduction in uncertainty of a random variable X given another random variable Y is quantified by the mutual information defined as [16]: where H(X|Y ) is the conditional entropy that follows the chain rule H(X, Y ) = H(X) + H(Y |X).
Mutual information is nonnegative and symmetric with respect to its arguments, that is, I(X; Y ) = I(Y ; X).If X and Y are independent, knowledge about one does not alter the uncertainty in the other, and therefore, mutual information between two independent random variables is zero.
To extend these ideas to dynamical systems, the above quantities are generalized to a stochastic process as follows.For a stochastic process X, considered as a sequence of random variables, X 1 , X 2 , . .., the entropy rate is defined as [16,35]: where the joint entropy, H(X 1 , . . ., X n ), for any finite n is computed on the basis of the joint probability distribution, P (x 1 , . . ., x n ), where x 1 ∈ X 1 , . . ., x n ∈ X n , similarly to Equation (2).For two stochastic processes X and Y the mutual information rate is defined by [17]: where I(n) = I(X 1 , . . ., X n ; Y 1 , . . ., Y n ) is the mutual information computed over sequences of length n.
Unlike the mutual information rate, which is symmetric with respect to the two stochastic processes it is computed for, Schreiber proposed the use of another quantity, called transfer entropy, to measure directional information flow between two dynamical systems [36].(A different, but related information-theoretic measure, called conditional mutual information, was proposed by Paluš et al. [37].)Assuming that X and Y are first-order Markov processes, so that for example P (x n+1 |x n , . . ., x 1 ) = P (x n+1 |x n ) for any n, the transfer entropy quantifies the reduction in uncertainty of the state, X n+1 , given X n and Y n .This quantity measures the flow of information from Y to X and is given by [21,36]: where x n and y n take values in the sets X n and Y n , which are generally different for each n.The definition implies TE X→Y = TE Y →X .In this sense, given the direction of dominant information flow, the transfer entropy is associated with the cause-effect relationship between two dynamical systems [11,38].
In practice, information-theoretic measures, such as mutual information rate and transfer entropy, are difficult to compute with experimental data, which is often limited and noisy [39].As a result, shortrange correlations between noise in two processes may be interpreted as information flow undermining important long-range correlations [40].One way to address this issue is to downsample the time series while ensuring that the Markovianity of the process is not lost [41].Probability distributions can then be computed from the down-sampled data using binning methods, such as histograms [12] or a correlation integral [42].To ensure that the experimental data is well approximated by a first-order Markov process, we calculate the sampling time interval t and the number of bins for computing histograms following the method described in [43].In [43], expansion rates, instead of probabilities, are used to calculate mutual information rates and its bounds.The evolution of the coupled system of two one-dimensional processes X and Y is represented as points on a two-dimensional grid that is quantized into N square boxes each with side 1/N .The expansion rates, e k , k = 1, 2, are [43]: where ÑC is the number of boxes on the two-dimensional grid occupied by points on the combined trajectory of X and Y , = 1/N , L i 1 (t) = ∆ i (t)/δ i measures the maximum growth rate of nearby points in -square box i that have spread out after time t and L i 2 (t) = A i (t)/ 2 quantifies the spread of a region enclosing such points.Accordingly, δ i is the maximum distance between pairs of points in a box i and ∆ i (t) is the largest distance between the same pair of points after time t.Similarly, A i (t) is the area occupied by points after time t that were initially in -square box i.Given a range of values of t and , the following constrained minimization problem is solved [43]: where T = N/e 1 (t) represents the amount of time that is needed for the correlation of the combined trajectory of X and Y to decay to zero.The optimization reduces the correlation by ensuring that the number of boxes occupied in a time window of duration T is the same as that over the full trajectory.In other words, points on the two-dimensional grid occupy approximately the same number of boxes in T as for the full distribution.

Animals
Experiments were conducted with adult zebrafish, Danio rerio, bought from an online store (LiveAquaria, Rhinelander, WI, USA).At the time of the experiments, the fish were approximately 3 cm in body length and were maintained in 37.8-liter (10-gallon) housing tanks at a housing density of no more than 4 fish per liter.Housing conditions were maintained in accordance with the approved animal care protocol, with lighting at a 12 h dark/12 h light photoperiod [44], pH between 7 and 7.2 and the temperature set at 27 ± 1 • C.

Experimental Setup
A classical two-choice experimental setup [45], comprising a 76 × 30 × 30 cm tank with two 10 × 30 × 30 cm stimulus compartments on either side of the tank and a 56 × 30 × 30 cm swimming compartment (Figure 1), was used to conduct the experiments.The tank was lit by two 25 W fluorescent lamps (All-Glass Aquarium, preheat aquarium lamp, U.K.) mounted above the longitudinal sides.An overhead view of the setup was recorded with a web camera (Logitech, Newark, CA, USA) mounted approximately 65 cm above the surface of the water.Experiments were filmed at 15 frames per second at a resolution of 800 × 600 pixels.A similar setup was used for other zebrafish-robot interactions experiments by our group [46][47][48][49][50][51].Fish replicas, 3 cm in length, were fabricated using Acrylonitrile Butadiene Styrene (ABS) thermoplastic in a 3D prototyping machine (Dimension Elite, Stratasys Ltd., Eden Prairie, MN, USA).The replicas were designed in a Computer Aided Design (CAD) software (SolidWorks, SolidWorks Corp., Waltham, MA, USA) and featured dorsal, ventral and caudal fins.The replicas were painted with non-toxic pigments to match the zebrafish skin coloration and were mounted onto a movable platform with a transparent acrylic rod (Figure 1).The rod was attached to a custom-made robotic arm installed at the edge of the tank.The robotic arm consisted of three rigid links each driven by a separate servo-motor (Futaba Corporation of America, Schaumburg, IL, USA; and Hitec RCD USA Inc., Poway, CA, USA) [52].The servo motors were controlled using a pre-programmed microcontroller (Arduino Uno, Arduino, Italy), so that the combined end-effector trajectory followed a periodic figure-eight pattern along the breadth of the tank at a speed of 5.7 ± 1.5 cm/s.The spatial trajectory was selected to achieve the large coverage of the stimulus region of the tank while being distinctly visible to the focal subject.A servo motor at the top end of the attachment rod was oscillated about the vertical axis at 3 Hz.This local oscillation gave an appearance of undulating movement to the replica in addition to spatial coverage and was selected to match previous values of tail-beat frequency in studies with bioinspired robots [46,49,51].Two such robotic arms were installed on either side of the tank to provide a uniform visual background to the live subject with respect to the two choices available on the sides of the tank.

Experimental Procedure
A hypothesis-driven biological experiment in animal behavior typically involves conducting experimental trials by changing independent design parameters while observing specific behaviors [53].The null hypothesis, namely that no change in behavior will be detected, is rejected using a statistical test when the alternate hypothesis is true with a high probability, which is typically 95% [54].The number of trials for each value of the independent design parameter must be selected to avoid Type II errors, that is, a failure to reject a false null hypothesis [54].Here, we test two hypotheses: first, that the directional information transfer will be dominant from the replica to the live fish; and second, that the information flow from the replica to the live fish is more than if the replica were replaced by a live stimulus.
To investigate the transfer of information between a live fish and the moving replica, an experimental condition was conducted where a single zebrafish was filmed as the replica was maneuvered on one of the tank sides (the experiments performed in this study were approved by the Animal Welfare Oversight Committee (AWOC) of the Polytechnic Institute of New York University, protocol AWOC-2013-103).To compare this with information flow between two live animals, a control condition was tested where a live conspecific was released in the stimulus area.To achieve statistical power, ten trials were conducted for each of the two conditions with a single experimentally naive fish for each trial; experimentally naive fish were observed for each trial to avoid pseudo replications [54].Thus, a total of thirty subjects (twenty focal fish and ten stimulus fish) were considered in this study.In accordance with an established protocol that ensures that zebrafish are acclimatized to the experimental tank [55], fish were transferred from the holding tank using a hand net and were habituated to the experimental tank for ten minutes during which the replica, if present, was moving.Post-habituation, the fish were filmed for five minutes.Observations made during these five minutes were used in the analysis.The side on which the stimuli were presented was balanced across trials to avoid a possible left-right bias in the results.Note that in all the tests with the replica, the replica was always started from the same relative location in the stimulus area at the initial time of the recording to ensure the repeatability of the stimulus across trials.

Data Assimilation and Statistical Analysis
Video data was post-processed using a target-tracking system that output the trajectories of the focal fish and the stimulus (conspecific or replica) in the tank frame.The tracking system, described in detail in [56], used a Kalman filter to track the fish centroid through the five-minute experimental time.Fish tracks were verified and repaired in a MATLAB R graphical user interface (GUI) for analysis.The trajectory data was used to compute average speed along the tank-wide direction and validate the time series sampling strategy.To reconstruct the replica trajectory, the same was tracked manually using the GUI for the first ten seconds of each trial.The tank-wide component of the ten-second trajectory was then fit using the MATLAB nlinfit routine onto a four-parameter sinusoidal function of the form f (k) = B sin(ωk + φ) + C, k being the time in seconds.Instead, the orthogonal tank-long component was fit using least-squares estimation onto a twentieth order Fourier series expansion [57].The full periodic trajectory was then extrapolated for the remaining of the trial and projected back onto the video for visual verification.Figure 2 shows sample trajectories from each experimental condition.Referring to Section 2, we use the tank-wide position of the fish (Figure 1) and the stimulus (replica or conspecific) as the dynamic variable for computing information flow.When working with experimentally generated time series, noise and limited data may undermine the application of theoretical approaches [58].Coarse-grained treatments of the experimental data address these issues; however, their incorrect implementation could lead to the possibility of loosing important information [41].Following the method detailed in Section 2, a sampling interval and a number of bins were computed for all trials to obtain common average values to be used for the entire dataset [43].Thus, we set t = 2 seconds for the sampling interval and N = 7 for the number of bins.Notably, the 2-second sampling interval represents the average time during which the correlation of the trajectory decays to zero within a square box of size equal to the bin width.The facts that fish speed is generally on the order of a few body lengths per second [59] and that the size of each bin is on the order of one fish body length guarantee that a time interval of 2 seconds suffices for the fish to reach a different bin in the given time step.To ensure that these values are representative of the directionality of the dominant information transfer [41], we computed transfer entropy over a range of values of t ∈ [0.06, 5]s and N ∈ [4,30] values for a random trial.Data from stimulus and focal fish trajectory was normalized so that the values lie between −1 and 1.The normalized data was, in turn, used to build the joint distributions using histograms.Transfer entropy was computed for each direction (fish → stimulus, stimulus → fish), where fish is the focal subject, and stimulus can be the replica or the conspecific.Figure 3 shows the tank-wide position for the sample trajectories for a single (fish ↔ fish) and (fish ↔ replica) trials, respectively.Figure 3. Transfer entropy computation.(a) Sample (normalized) tank-wide position trajectories for a fish (red) interacting with a conspecific (blue); for this sample, TE fish→stimulus = 0.615 bits and TE stimulus→fish = 0.524 bits, where "stimulus" is a conspecific; (b) Sample tank-wide position trajectories for a fish (red) interacting with a replica (blue); for this sample, TE fish→stimulus = 0.357 bits and TE stimulus→fish = 0.638 bits, where "stimulus" is the replica.As an additional test to verify the inference about the directionality of the information flow, we considered an artificial dataset, for which we generated surrogate replica trajectories using the time trace of the position of the replica in the tank-wide direction as the reference.In particular, we simulated artificial motions as sinusoidal functions with the same frequency of oscillation, and a random phase difference (drawn from a uniform distribution sampled from [0, π]).Thus, the surrogate data shares the same behavior of the replica motion, while being dissimilar from that for most times (Figure 4).A total of 10 surrogate trajectories with phase delays of 1.32 ± 0.93 rad were generated and used in the statistical analysis.To each of these trajectories, we randomly associated the response of a focal fish in the condition with the replica.3b) and surrogate (dashed) data for a section of the trial generated to evaluate transfer from surrogate data to the live fish.Information flow was compared within each condition using one-way analysis of variance (ANOVA) [54] with the direction of flow as the independent parameter and the transfer entropy as the dependent parameter.Trials with abnormal fish behavior, such as excessive freezing [60], were discarded.This resulted in 9 trials for the control condition and 10 for the one with replica.The ANOVA comparison [54] reports the probability of the null hypothesis, namely that the means of the values are equal.If this probability is less than 5% (p < 0.05), we assume that the null hypothesis is rejected and that the means are significantly different.The output of a one-way ANOVA comparison consists of the probability p of accepting the null hypothesis and the F-test statistics reported in the form, F df1,df2 , where df1, df2 are the degrees of freedom between groups and between single observations, respectively.The value of the F-test statistic is computed as F = between-group variability/within-group variability, where variability is computed as the variance between and within groups [54].A large value of the F-test statistic corresponds to a higher between-group variability than within-group variability, implying a difference in populations.Significance is evaluated by comparing with an F-distribution whose shape depends on the number of degrees of freedom [54].Information flow in a given direction is compared between conditions using one-way ANOVA, with the condition as the independent parameter and the transfer entropy for that direction as the dependent parameter.Similarly, one-way ANOVA was used to compare information flow from the surrogate trajectories to the fish with the information flow from the replica to the fish.

Results
Figure 5 shows the experimental results of this study.We found that the transfer entropy from the focal fish to replica TE fish→replica was significantly less than from the replica to the fish TE replica→fish (p = 0.007, F 1,9 = 9.127).Differently, transfer entropy from the focal fish to the conspecific TE fish→fish was not significantly different between directions (p = 0.193, F 1,8 = 1.845).Between conditions, the transfer entropy from the replica to the focal fish was significantly more than from the conspecific to the fish (p = 0.024, F 1,17 = 6.05; Figure 5).With respect to the surrogate dataset, we also found that the transfer entropy from the simulated replica to the fish was significantly less than the transfer entropy from the replica to the focal subject (TE surrogate→fish = 0.25 ± 0.08 bits; p < 0.001, F 1,17 = 40.99).Finally, transfer entropy from the focal fish to the conspecific and transfer entropy from the focal fish to the replica were not significantly different (p = 0.086, F 1,17 = 3.326).To evaluate the sensitivity of the dominant flow of information with respect to variations in the selection of the parameters t and N , we considered the effect of varying them in the analysis of the single trial displayed in Figure 3b.Transfer entropy over a range of t values shows that the dominant direction of information flow, measured as the difference between the transfer entropies in each direction (TE replica→fish − TE fish→replica ), is preserved for a majority of the range.In particular, the difference in transfer entropy was positive for most values of the sampling time interval more than 0.5 s (Figure 6).3b for a range of values of time and number of bins.The sampling time interval and number of bins selected for the analysis is marked with a red cross.

Discussion
Our results show that the transfer entropy can be used to measure information flow in zebrafish.In agreement with our expectations, we found that when a live fish interacts with a replica moving along predetermined trajectories, the dominant flow of information is from the replica to the fish.The comparison of information flow between the control and the experimental condition shows that the lifesized replica of a real zebrafish maneuvered using a robotic arm influences the motion of the focal fish more than a conspecific.
While determining the direction of information flow, we note that a lower value of the sampling interval is typically recommended in the literature on unidirectionally coupled oscillators modeled as a second-order autoregression process [41].In this case, where the full system may be represented as a high-dimensional state vector with the past two states in time, a foreseeable problem associated with low temporal resolution is the loss of observability [41].In contrast, the relatively large value of sampling interval used here can be justified in terms of the typical fish speed, which is on the order of a few body lengths [59], and thus allows for a change in position from one bin to another in a single time step, preserving the process' Markovianity [41].We find that although the dominant direction of information flow stays replica → fish for the most part, the direction is reversed when the sampling time interval is less than 0.5 s.A possible reason for this reversal may be the correlation between the noise in the fish and the replica trajectories [39].In particular, correlations between the movement of the replica, which maneuvers tank-wide at a higher oscillation than the focal fish, and the noise from the fish trajectory may be detected at this temporal resolution.
The validation of the transfer entropy as a measure of directional information is offered by the significant difference between the information flow the replica to the zebrafish and the information flow computed the other way around.Combined with the fact that no such significance was found when the stimulus is a conspecific, this suggests that transfer entropy is a valid measure of information flow.In particular, the result that the replica influenced the focal subject more than the other way around was expected, since the replica was controlled independently.The conspecific, on the other hand, was always visually aware of the focal subject and, therefore, likely to be influenced by it.In addition, transfer entropy from the replica to the focal subject was significantly reduced in a surrogate dataset in which the replica was artificially delayed, confirming the validity of the directional information flow between the replica and the live subject.At the same time, the non-zero information flow from the live subject to the replica is misleading, but not unexpected.This is because of the generality of the method that exhaustively looks at all possible combinations for the reduction in the uncertainty in the dynamics of one system given the knowledge of the other [36].It is possible that within the confined space in which we observed the replica and the focal fish, some nonlinear relationships exist that can relate movements of the replica to those of the focal subject.At the same time, the significantly higher transfer of information from the replica to the fish is strong evidence in support of the use of this method to unravel information flow in zebrafish social response.
Between conditions, the amount of information flow from the stimulus to the focal subject was also significantly more when the stimulus was a replica.This is direct evidence in support of the efficacy of a robotics-based solution in animal behavior studies.For example, often cited benefits of using robotics in experimental paradigms addressing functional and dysfunctional processes are the reliability and repeatability of the animal response to the robot [47,48].In this respect, the marked one-directional influence of the replica on the focal fish indicates that consistent responses may be elicited more easily with a robot than a live stimulus.Although vision was the primary mode of sensing in this experiment, it is conceivable that a similar finding could be garnered with animals that utilize other cues, such as chemical [61], acoustic [62] and hydrodynamic [63], to sense their environment.

Conclusions
An important question in the study of the behavior of social species is "who follows whom?" [64], which can be answered by quantifying the direction of information flow between individuals [11].In this paper, we validated for the first time the use of transfer entropy as an information-theoretic measure to compute directional information flow in social behavior.To test the method in a scenario in which information flow is known, we designed an experimental paradigm based on the integration of a robotics-based platform in a dichotomous preference test.Ongoing work includes the classification of engineered stimuli using this method and the assessment of information flow in larger animal groups.

Figure 1 .
Figure 1.Experimental setup.(a) Top view of the experimental tank with two robotic arms on each side (the fish and the replica are circled); the tank-wide axis is marked in white bidirectional arrow; (b) painted plastic replica of the zebrafish that is attached to the end-effector.

Figure 4 .
Figure 4. Surrogate data for verification of directionality.Original (solid) (see also Figure3b) and surrogate (dashed) data for a section of the trial generated to evaluate transfer from surrogate data to the live fish.

Figure 5 .
Figure 5. Transfer entropy was significantly different between directions when a replica was present.Transfer entropy from the stimulus to the fish was significantly different between conditions.Overhead bars (blue) p < 0.05.

Figure 6 .
Figure 6.Transfer entropy difference (TE replica→fish −TE fish→replica ) for a single trial shown in Figure 3b for a range of values of time and number of bins.The sampling time interval and number of bins selected for the analysis is marked with a red cross.