Many biological systems are regarded as non-linear dynamical systems [1
] operating in high-dimensional spaces. Proteins, genes, and macromolecules interact in a variety of ways to create the dynamics of cells [2
]. Collections of cells interact and coordinate activity, forming cohesive units such as bacterial colonies, simple multicellular organisms, or tissues in more complex multicellular organisms. At each level of organization, ‘input’ signals (e.g., odors or hormones) are introduced into the system, processed by means of the system’s dynamics, and responded to accordingly, sometimes generating new ‘output’ signals as a by-product [3
]. Fundamentally, biological systems must process external signals in real time to inform a wide variety of response decisions. For example, the fruit fly olfactory system projects an input to a high-dimensional space before classifying an odor [4
In this vein, reservoir computing (RC) is a form of signal processing that is used for classification and online learning and uses such high-dimensional systems to process signals. Initially called echo state networks [2
] or liquid state machines [5
], the computational structure resembles an unorganized recurrent neural network. These networks were inspired by the most classical example of signal processing in biological systems: the brain. While initial research in neural networks was focused on modeling and understanding neural computation [6
], applications and directions for research quickly expanded in scope. For reservoir computers, this includes systems for predicting time series from chaotic systems [7
] as well as multiple real-world applications such as gas detection [9
], robotics [10
], and economic trends [13
]. Applications in processing human data are also found with human emotion [14
] and gesture recognition [15
], speech and text processing [16
], and health care monitoring [18
In biology, reservoir computers have been used to analyze biological data for identifying anomalous states such as cardiac arrhythmia [19
], seizures [20
], and microsleeps [22
], as well as for high-dimensional classification such as cell type detection [23
]. In these applications, RCs are used to interpret or classify biological data. RCs have also been used as models of biological systems, serving to explore the mechanisms and dynamics of said systems. For example, the questions of how the brain functions [24
] and how gene regulatory networks respond to external stimuli [27
] have been studied in an RC context.
In reservoir computing, an incoming signal is fed into the reservoir, which is a randomly connected recurrent network where nodes are connected via a variety of coupling functions. Thus, the reservoir transforms the signal into a high-dimensional representation. Finally, an output layer is trained, often with straightforward techniques such as regularized regression [29
], in order to perform a classification or regression task (Figure 1
RCs can also be trained to act as signal processors, filtering the input to produce a new output through approximating a function that is normally applied to a window sliding over the signal. The type of function can be extended to recursive functions, where some elements of the input window are replaced by values from previously generated outputs. These types of filters have a long history of use, which includes filtering biological signals [30
While reservoirs are most typically built using unorganized recurrent neural networks, it has been shown that any non-linear dynamical system that exhibits a fading memory property can theoretically be used to compute a time-invariant function on a signal [33
]. In fact, DNA, carbon nanotubes, memristor networks, and even buckets of water have been shown to work as reservoirs [34
]. One type of dynamical system that has only recently been explored as a reservoir is Boolean networks [37
]. Boolean networks (BNs) are well-suited to reservoir computing, since they are one of the simplest modeling approaches that (1) can capture the heterogeneity, both in wiring and coupling functions, that characterizes self-assembled systems; and (2) can exhibit the non-trivial dynamical behavior that is required for computation [38
]. Additionally, the fading memory property can easily be achieved in Boolean networks by adjusting two key parameters: the average in-degree of the Boolean functions,
, and the bias, p
, which is the probability that a function produces an output of one for any given set of inputs [39
]. The implementation of Boolean networks, both in software and in hardware, is also considerably easier and faster than more traditional approaches [41
Similar to neural networks, Boolean networks were first used as models of biological systems, namely gene regulatory networks [37
]. While often researched to understand cellular behavior in terms of protein/gene interaction networks in isolation [44
], BNs have also been used to understand how external signals impact biological systems [47
]. These systems rely on their ability to process external signals (relevant to survival or population function) and make decisions based on them in real time, thereby performing a similar computation to a reservoir [4
]. Just as the Boolean network framework has provided insight into how biological systems function, studying BN reservoir computers (BN RCs) may further our understanding of biological signal-response.
Previous work on BNs as reservoirs is still limited. Snyder et al. [38
] evaluated BN RCs with different numbers of nodes (
) and average node input degrees (
). Using a measure of reservoir quality, it was found that compared to networks with homogeneous in-degrees, networks with heterogeneous in-degree better separated inputs by class: the so-called separability property [5
]. Also, it was found that computational ability—when the difference in separability and fading memory is the greatest—was maximized when the average in-degree was
= 2 (out of
= 1,2,3) [50
], and that higher
values led to worse performance. This value for
is notable, as Boolean networks with
= 2 are known to be dynamically critical (Lyapunov exponent = 0). Critical dynamics, which lie at the border between order and chaos, have been shown to be important in computation and information processing, biological and otherwise [51
While previous work [38
] has laid the foundation showing that it is possible to use Boolean networks as reservoirs, the main focus has been on testing a small number of well-known functions, such as the median and parity functions, as benchmark measures. We are extending the body of research in three ways: (1) by performing a wider examination of how well these systems perform across different functions, including recursively defined operators; (2) by analyzing how flexible a single reservoir is for repurposing (i.e., retraining only the output layer and keeping the same network reservoir); and (3) characterizing the functions that are easier or harder for the reservoir to implement in terms of the function’s average sensitivity, which is a measure of its smoothness. We also evaluate reservoirs under different dynamical regimes: ordered, critical, and chaotic.
2. Materials and Methods
2.1. Reservoir Computer
A reservoir computer consists of three components: the input layer, the reservoir network, and the output layer (Figure 1
). The input represents a temporally changing signal that perturbs the reservoir, which performs computations on the signal in real time. The output node is the readout of the reservoir’s computation, estimating a given function operating on the signal. The value of the output node is given by a linear combination of the states of the reservoir nodes that are continuously being driven, either directly or indirectly (via other nodes), by the input signal. The weights of the linear regression are trained to be specific to a given objective function, which captures the error between the reservoir output and the function to be approximated.
In our implementation, the reservoir is constructed as a random Boolean network (RBN) [37
]. RBNs are networks with
binary-valued nodes with states:
The state of the network at any time,
, is a function of the state at the previous time,
, given by:
is the set of Boolean updating functions corresponding to each node. The node in-degree does not need to be constant, such that each function
has an independent number of arguments,
, so that:
The identity of each of the
arguments is chosen randomly from the
nodes, without replacement. The in-degree of the whole network,
, is characterized by the average in-degree across its nodes. The other primary descriptor of the network is the bias,
, which is the probability that a function outputs a one. Together,
can be varied to adjust the dynamics of the network, ranging from ordered, in which perturbations die out, to chaotic, in which perturbations are amplified [39
]. For this paper, we will fix
= 0.5 and tune the dynamics of the networks by varying
. Generally, we tune the dynamics using
< 2 for ordered,
= 2 for near critical,
> 2 for chaotic. To construct a network with a specific
, we take
edges uniformly distributed amongst all of the pairs of nodes.
In order for the RBN to act as a reservoir computer, it must be driven by an external signal, which is represented by a temporal sequence of binary values,
. Some percentage (expressed as a fraction) of reservoir nodes,
, is directly connected to the input signal. If a node,
, is connected to the input signal, then the input becomes an additional argument to its function, i.e.:
The nodes that have the input as an additional argument are chosen uniformly from the nodes of the reservoir without replacement.
At each time step, the reservoir produces a binary output value,
, which is defined as:
is the function that maps values greater than 0.5 to 1, and everything else to 0;
is a weight for each node of the reservoir, and
is a constant. The parameters
are trained to approximate the output,
, of a given Boolean function operating on a moving window over
, under some error criterion. Here, we use every node in the reservoir in the linear regression so that an optimal combination may be found. Pre-specifying which nodes are used in calculating the output would limit the success of the reservoir, since the structure and rules of the reservoir are fixed. In practice, when N = 100 and L = 0.5, ~25% of the nodes have non-zero weights, which are distributed equally across the nodes that are directly and indirectly perturbed by the signal.
2.5. Objective Functions
In this work, the computational performance of the reservoir computer is assessed by approximating the Boolean functions of three and five arguments evaluated over a temporally changing input signal. Moreover, we considered two types of functions:
Non-recursive functions, defined as
Recursive functions, defined as
where is the length of the sliding window on the input signal for which a function is approximated, and is a delay between when the signal perturbs the reservoir and when the corresponding output is computed. In this work, we explore = 3, 5 for non-recursive functions, and = 3 for recursive functions. Here, is kept fixed to = 1. We have tested all 256 three-bit recursive and non-recursive functions, and 1000 randomly sampled five-bit functions.
2.6. Training and Testing Algorithm
To train a BN RC for approximating a single function, we use random binary sequences as input streams,
, and compare the reservoir’s output value,
, to the function output,
, evaluated over the input stream. Specifically, we use a set of 150 random binary sequences of length
(e.g., 12 for 3-bit functions) to generate a set of
= 1500 different
pairs. It should be noted that for each
binary sequence, the reservoir is randomly initialized, and no transient period occurs before the outputs are used for training. With the
pairs, a regression model is fit in order to generate the coefficient weights on the output layer. We used Scikit-learn to perform lasso regression [56
] with an
= 0.1 for fitting.
To test a BN RC for approximating a single function, we again use a set of 150 random binary sequences to generate
pairs. We then measure the accuracy of the BN RC for that function, which is defined as:
indexes the Boolean function,
is a particular instantiation of an RC, and
is the summed error
, which is scaled by P
. Thus, it can be seen that the accuracy
is between 0–1. This process of training and testing is repeated for a set of different functions for each BN RC, thereby creating multiple estimation accuracies
2.7. Overall Strategy
The goals of this work centers around exploring how the approximation accuracy of RBN RCs varies across many different functions, providing a sense of ‘flexibility’. To do that, one reservoir is constructed and applied to estimating the output of a set of Boolean functions, such as all of the three-bit functions. This is done for a total of 100 RC instances for each combination of different values of , , and to test the effect of reservoir size, degree of network perturbation by the signal, and dynamical regime, respectively. We use = 10, 20, …, 50, 100, 200, …, 500; = 0.1, 0.2, …, 1; and = 1, 2, 3. For example, with the three-bit functions, we trained three (values of ) × 10 (values of ) × 10 (values of ) × 100 (RCs instances) × 256 (3-bit functions) = 7,680,000 RBN RCs.
Each RC is trained and tested for approximating a set of functions, including median and parity, reporting an accuracy for each function. We chose three sets of functions for testing the RCs. First, there are the three-bit functions, for which we test all possible functions. Second, there are five-bit functions, for which there are functions, which is a much larger function space than that for three bits. Working with the complete function set is impractical, so we made a random sample over the function space, choosing to test 1000 functions. Some additional hand-selected key functions, such as median and parity, were also used.
The last set of functions was recursive three-bit Boolean functions. This set of functions is defined as taking a set of arguments that includes the last produced output, . This is a more difficult task, because the BN RC must approximate the function with a hidden variable, .
In this work, we found that the flexibility of a particular BN RC instantiation is a function of the topology and size of the reservoir, as encoded by the parameter set (). Generally, higher values of and result in more flexible reservoirs; however, there is heterogeneity when or are low, and flexible reservoirs can be found at these values. As noted in research concerning BN RCs, the optimal parameter set includes tuning the dynamics toward criticality, which leads to more flexible systems. For signal processing, a flexible reservoir will be more accurate and more efficient, requiring less searching and training when different filters are being applied to the same data. For biological systems, where survival is dependent on signal response, , and can be tuned to balance the restraints of the system with the demands of the environment.
In terms of the challenge we provided to the RCs, we found three-bit functions to be relatively easy to approximate, while five-bit functions were more difficult, essentially requiring more resources. However, three-bit recursive functions were the most difficult, where we noted some recursive functions that are essentially impossible to approximate with this system, which is likely due to a long memory requirement.
Approximating the five-bit functions depends on a large reservoir and input size, and more strongly depends on
. The five-bit function space is massive, so while we were only able to sample a tiny fraction of it, we clearly saw the effect of the dynamics that were close to criticality on approximation accuracy. As
increased toward chaotic dynamics, the approximation became very poor, leading to a large separation between dynamical regimes. Looking forward, it is likely that seven-bit or nine-bit function approximations would require increasingly more resources in terms of reservoir size, and have a greater dependence on dynamics. This seems to be the case for the median and parity functions, at least [50
]. We found that the relative accuracy of reservoirs with
= 1, 2, or 3 could change, depending on the value of
, which is the input connectivity into the reservoir. This is most evident with the five-bit functions in the
= 2 accuracy plot, which showed a tapering off curve (convex), where improvement with reservoir size is not linear (Figure 6
). In the curves, as
increases, we see the
= 1 and
= 2 accuracy curves crossing each other. One possible explanation for this is that the external perturbations to the reservoir’s nodes shift the dynamics of the reservoir to a less ordered regime, creating a higher fraction of perturbed nodes and resulting in a greater shift. So, the dynamics in
= 1 approach criticality with higher
values, while the dynamics in the
= 2 case moving past criticality, toward chaos, thus dropping in approximation accuracy. However, the effect of time-varying external perturbation has not been well studied for Boolean network dynamics. [50
] showed that the mutual information between the reservoir and the signal increase monotonically as
= 1, so there may be other effects of increasing
at play. Although our results indicate a strong effect of mean in-degree (
) in accuracy, it is possible that accuracy is also affected by other properties of reservoir topology; thus, additional in-degree distributions should be investigated.
We found that Boolean function sensitivity also plays a role in the accuracy, where more sensitive functions are more difficult to approximate. The sensitivity can be broken down further into the activities of the function variables. Functions appear to be more difficult to approximate if the activities of temporally distant variables are higher. For non-recursive functions, increasing and is sufficient to enable BN RCs to approximate sensitive functions; however, for non-recursive functions, large reservoirs with high values of are not as affected. This points to a large portion of the reservoir needing perturbation to avoid the problems that are related to function argument sensitivity.
However, for recursive functions, even large reservoirs are affected by Boolean function sensitivity. This is likely due to the non-smooth output, which is difficult to approximate. Further, the recursive sliding-window filtering formulation essentially creates infinite memory, albeit fading. Many questions remain, such as what is needed for good approximations of recursive functions, and whether there is a change to the model that might help. That said, for the most part, many recursive functions are approximated very well, and that may be enough.
Overall, having an idea of the sensitivity of the processing task will inform the minimum parameter values that are necessary for a successful reservoir. This is intriguing from a biological standpoint, and it is worth exploring how signal responses that are more sensitive, especially to more historic signal values, are handled. The inability of approximating certain functions may not impinge on the use for modeling biology. When viewed through the lens of ‘complex adaptive systems’, recursion is a key characteristic, and one that BN RCs would need to address in order to model biological systems.