1. Introduction
A common goal of neuroscientists and roboticists is to understand how animal nervous systems interact with biomechanics and their environment and generate adaptive behavior [
1]. By understanding and modeling aspects of the nervous system, it is hoped that robots will be able to exhibit embodied intelligence [
2] and exhibit animal-like robustness and adaptability [
3]. One approach is to design Synthetic Nervous Systems (SNS), networks of conductance-based neurons and synapses which can be used to model animal nervous systems [
4,
5] and control robots [
6,
7,
8]. Some strengths of SNS networks include that they can be tuned using analytic design rules [
9,
10] and that results obtained controlling robotic hardware can propose neurobiological hypotheses [
11,
12].
In order to design SNS networks for robotic control, software tools are needed. Software for simulating SNS networks should support conductance-based modeling of neurons and synapses, as there are elements of neural behavior in conductance-based models which are incompatible with current-based models [
9,
13]. Bidirectional synaptic links, such as electrical synapses, should also be supported [
14]. Simulators should also support networks with heterogeneous neural models, potentially containing both spiking and non-spiking neurons [
7]. While individual spiking neurons can be more computationally powerful than non-spiking neurons [
15], non-spiking neurons are capable of capturing much of the dynamics of populations of spiking neurons while being more amenable to real-time simulation [
16]. Networks should be able to be constructed in a programmatic way, in order to aid the design of large but formulaic networks [
17]. SNS networks should be able to be simulated with faster than real-time performance using CPUs and GPUs, and the same networks should be easily interfaced with physics simulation engines and robotic hardware. Additionally, for accessibility and ease of use in laboratory and educational settings, a simulator software should be cross-platform compatible with the Windows (trademark Microsoft Corporation, Redmond, WA, USA), MacOS (trademark Apple Corporation, Cupertino, CA, USA), and Linux operating systems. A selected survey of available simulation software is presented in
Table 1.
Software for simulating conductance-based neural dynamics have long been available, with the most popular options being NEURON [
18], NEST [
19], GENESIS [
20], and Brian [
21]. These simulators are capable of simulating highly detailed and biologically accurate neural models, however they were originally designed for the purpose of performing digital experiments and collecting data over a long simulation run. As such, interfacing with external software and systems can be challenging [
22] and often requires dedicated software for memory management [
23]. Additionally, these simulators are limited to being run on conventional CPUs, although some have begun to be adapted for use with GPUs [
24].
Other simulators are capable of designing large networks of neurons using principles from machine learning, such as snnTorch [
25], SpykeTorch [
26], and BindsNET [
27]. These simulators are capable of executing at high speed on both CPUs and GPUs, but they do so at the cost of limiting simulations to reduced models of spiking neurons and current-based synapses. Simulators have also been designed to simulate the Izhikevich neuron [
28] and other spiking neurons at scale, including CARLSim [
29], NEMO [
30], GeNN [
31], CNS [
32], and NCS6 [
33]; however they typically do not support hybrid networks of spiking and non-spiking neurons.
Multiple solutions have been developed which combine a neural dynamics simulator with a physics engine. One approach is to combine an existing neural simulator with an existing physics engine using a middleware memory management software. This has been used to combine Brian [
21] and SOFA [
34] using CLONES [
35], as well as NEST [
19] with Gazebo [
36] using MUSIC [
23]. This approach allows users who are comfortable with a specific neural simulator to interface their networks with physics objects. However, it leads to complicated software dependencies which are difficult to translate to other systems. The first integrated system was AnimatLab [
37], which allows networks consisting of either a non-spiking or spiking neural model to control user-definable physics bodies. It also comes with an integrated plotting engine, allowing users to run experiments and analyze data within a single application. Numerous models have successfully been controlled with AnimatLab, both in simulation [
4,
38,
39] and with robotic hardware [
6,
7], however networks cannot be designed in a programmatic way and are difficult to scale to larger networks [
17]. The Neurorobotics Platform (NRP) is a large software suite which integrates multiple neural simulators, including Nengo [
40] and NEST [
19], with Gazebo [
36] in a cloud-based simulation environment. The NRP is a comprehensive toolbox which comes with a variety of advanced visualization tools, and has been used successfully for multiple neurorobotic controllers in simulation [
41,
42]. However, the NRP comes with much overhead and, as such, is unsuited for real-time control of robotic hardware.
In recent years, high-performance robots have been developed which are controlled using networks of spiking neurons [
43,
44]. These networks achieve state-of-the-art performance, but rely on specialized neuromorphic hardware, such as Intel’s Loihi processors [
45], which are not yet widely available. Lava [
46] is a relatively recent and promising software solution for designing spiking networks, but it is primarily designed for use with CPUs and Loihi. Currently, the most widely used software for implementing spiking networks and controlling real hardware is Nengo [
40], which has achieved impressive results [
44]. However, Nengo is optimized for networks designed using the Neural Engineering Framework [
47], and can have reduced performance without the use of neuromorphic hardware. One simulator which can simulate networks with a mixture of spiking and non-spiking neurons in real-time or faster is ANNarchy [
16], which does so using a C++ code generation system. However, this code generation system which enables high performance comes at the cost of incompatibility with the Microsoft Windows operating system, which reduces its level of accessibility.
Here, we present SNS-Toolbox, an open-source Python package for the design and simulation of synthetic nervous systems. SNS-Toolbox allows users to design SNS networks with a simple interface and simulate them using established numerical processing libraries on consumer-grade hardware. We focus on simulating a specified set of neural and synaptic dynamics, without dedicated ties to a GUI or a physics simulator. This focus allows the SNS-Toolbox to be easily interfaced with other systems, and for a given network design to be able to be reused without modification in multiple contexts. In previous work [
48], we presented an initial version of SNS-Toolbox with reduced functionality. Here we explain in detail the expanded neural and synaptic dynamics supported in the toolbox, and describe the workflow for designing and simulating networks. We provide results which demonstrate comparative performance with other neural simulators, and we showcase the use of SNS-Toolbox in two different applications, motor control of a muscle-actuated biomimetic system in Mujoco [
49], and navigation control of a robotic system in simulation using the Robotic Operating System (ROS) [
50].
2. Materials and Methods
Herein we describe the internal functionality of the SNS-Toolbox, how different neurons and synapses are simulated, designed, and compiled by the user.
Section 2.1 defines the neural models which are supported in the toolbox, and
Section 2.2 does the same for connection types.
Section 3.2.3 describes the design process using SNS-Toolbox, and how a network is compiled and simulated.
All software described is written in Python [
51], which was chosen due to its ease of development and wide compatibility. Unless otherwise specified, the units for all quantities are as follows, current (nA), voltage (mV), conductance (μS), capacitance (nF), and time (ms).
2.1. Neural Models
SNS-Toolbox is designed to simulate a small selection of neural models, which are variations of a standard leaky integrator. In this section, we present the parameters and dynamics of each neural model which can be simulated using SNS-Toolbox.
2.1.1. Non-Spiking Neuron
The base model for all neurons in SNS-Toolbox is the non-spiking leaky integrator, as has been used in continuous-time recurrent neural networks [
52]. This neural model can be used to model non-spiking interneurons, as well as approximate the rate-coding behavior of a population of spiking neurons [
9]. The membrane potential
V behaves according to the differential equation
where
is the membrane capacitance,
the membrane conductance, and
is the resting potential of the neuron.
is an injected current of constant magnitude, and
is any external applied current.
is the current induced via synapses from presynaptic neurons, the forms of which are defined in
Section 2.2.
During simulation, the vector of membrane potentials
is updated at each step
t by representing Equation (
1) in a forward-Euler step:
where ⊙ denotes the element-wise Hadamard product, and
represents the simulation timestep.
is the membrane time factor, which is set as
.
2.1.2. Spiking Neuron
Spiking neurons in SNS-Toolbox are represented as expanded leaky integrate-and-fire neurons [
53], with the membrane depolarization dynamics described in Equation (
1) and an additional dynamical variable for a firing threshold
[
10],
where
is a threshold time constant, and
is the initial threshold voltage.
m is a proportionality constant which describes how changes in
V affect the behavior of
, with
causing
to always equal
. When the neuron is subjected to a constant stimulus,
results in a firing rate which decreases over time, and
causes a firing rate which increases over time. Spikes are represented using a spiking variable
,
which also triggers the membrane potential to return to rest:
The vector of firing thresholds
is updated as
where
is the threshold time factor. Based on the threshold states, the spiking states are updated as
Note that for simplified implementation, all spikes with SNS-Toolbox are internally represented as impulses of magnitude −1. Using these spike states, the membrane potential of each neuron which spiked is reset to
2.1.3. Neuron with Voltage-Gated Ion Channels
The other neural model available within SNS-Toolbox is a non-spiking neuron with additional Hodgkin–Huxley [
54] style voltage-gated ion channels. The membrane dynamics are similar to Equation (
1), with the addition of an ionic current
[
55]:
This ionic current is the sum of multiple voltage-gated ion channels, all obeying the following structure:
Any neuron within a network can have any number of ion channels.
is the maximum ionic conductance of the
jth ion channel, and
is the ionic reversal potential.
b and
c are dynamical gating variables, and have the following dynamics
where functions of the form
are a voltage-dependent steady-state
and
is a voltage-dependent time constant
p denotes an exponent, and
is the gate reversal potential.
and
are parameters which shape the
and
functions.
is the maximum value of
. Note that depending on the desired ion channel, the exponent for various sections can be set to 0 in order to effectively remove it from Equation (
10). One particular example of this is a neuron with a persistent sodium current, which is also available as a preset in SNS-Toolbox,
which is the same as Equation (
10) with one dynamic variable disabled and some variable renaming.
2.2. Connection Models
Within SNS-Toolbox, neurons are connected using connection objects. These can either define links between individual neurons, or structures of connectivity between neural populations (see
Section 2.2.4).
2.2.1. Non-Spiking Chemical Synapse
When connecting non-spiking neurons, non-spiking chemical synapses are typically used. The amount of synaptic current
to post-synaptic neuron
i from presynaptic neuron
j is
where
is the synaptic reversal potential.
is the instantaneous synaptic conductance, which is a function of the presynaptic voltage
:
is the maximum synaptic conductance, and voltages
and
define the range of presynaptic voltages where the synaptic conductance depends linearly on the presynaptic neuron’s voltage.
When simulating, Equation (
15) is expanded to use matrices of synaptic parameters (denoted in bold),
and each term is summed column-wise to generate the presynaptic current for each neuron. Synaptic parameter matrices have an NxN structure, with the columns corresponding to the presynaptic neurons and the rows corresponding to the postsynaptic neurons. Equation (
16) is also expanded to use parameter matrices,
2.2.2. Spiking Chemical Synapse
Spiking chemical synapses produce a similar synaptic current as non-spiking chemical synapses (Equation (
15)), but a key difference is that
is a dynamical variable defined as
The conductance is reset to
, the maximum value, whenever the presynaptic neuron spikes. Otherwise, it decays to zero with a time constant of
. When simulated, these dynamics are represented as
where
is the matrix of spiking synaptic conductances, and
is the synaptic time factor matrix.
An additional feature available with spiking synapses is a synaptic propagation delay. This sets the number of simulation steps it takes for a spike to travel from one neuron to another using a specific synapse, a feature which is useful for performing some aspects of temporal computation [
17]. If the synapse between neurons
j and
i has a delay of
d timesteps, the delayed spike is represented as
For simulation, this propagation delay is implemented using a buffer matrix
with
N columns and
D rows, where
D is the longest delay within the network. The rows of
are shifted down at each timestep, and the first row is replaced with current spike state vector
.
is then transformed into a matrix of delayed spikes
by rearranging based on the delay of each synapse in the network.
is then used to simulate the synaptic reset dynamics from Equation (
20),
2.2.3. Electrical Synapses
Electrical synapses, or gap junctions, are resistive connections between neurons that do not use synaptic neurotransmitters. As a result, the neurons exchange current proportional to the difference between their voltage values. Their current is defined as
where
is the synaptic conductance. Electrical synapses are simulated in SNS-Toolbox using a similar formulation as Equation (
17),
SNS-Toolbox simulates electrical synapses as bidirectional by default, where current can flow in either direction between the connected neurons. Rectified connections are also supported, where current only flows from the presynaptic to the postsynaptic neuron and only if
. When simulating with rectified electrical synapses, a binary mask
is generated,
where ⊖ denotes outer subtraction. The voltage of each neuron is subtracted in a pairwise fashion, with the result processed by the heaviside step function
H. This generates a matrix where each element is 1 if current is allowed to flow in that direction, and 0 otherwise. This binary mask is then applied to a synaptic conductivity matrix
to obtain the masked conductance
,
To generate the opposite current flow in rectified synapses, the masked conductance is then added to its transpose with the diagonal entries removed,
This final masked, transformed conductance matrix
is substituted for
in Equation (
25)
2.2.4. Matrix and Pattern Connections
In their base form, each of the preceding connection models defines the connection between two individual neurons. However, the connections’ behavior can be extended to defining connections between populations of neurons. Following the model presented in [
10], in the simplest form of population-to-population connection all neurons become fully connected and the synaptic conductance is automatically scaled such that the total conductance into each postsynaptic neuron is the same as the original synapse.
For more complex desired behavior, more types of population-to-population connections are available. Matrix connections allow the user to specify the exact matrices for each synaptic parameter, and one-to-one connections result in each presynaptic neuron to be connected to exactly one postsynaptic neuron, with all synapses sharing the same properties. Pattern connections are also available, modeled after convolutional connections in ANNs [
56]. In pattern connections, a kernel matrix
can be given,
where the indices are values for a single synaptic parameter (
). If
describes the connection pattern between two
neural populations, then the resulting synaptic parameter matrix
will present the following structure:
2.3. Inputs and Outputs
In order for an SNS to interact with external systems, it must be capable of receiving inputs and sending outputs. For applying external stimulus to a network, input sources can be added. These sources can be either individual elements or a one-dimensional vector, and are applied to the network via
,
where
is an LxN binary masking matrix which routes each input to the correct target neuron. L is the number of input elements, and N is the number of neurons in the network. This external input vector is varied from step to step and could come from any source (e.g., static data, real-time sensors).
Output monitors can also be added, both for sending signals to other systems and for observing neural states during simulation. These outputs are assigned one-to-one to each desired neuron, meaning one output applied to a population of five neurons results in five individual outputs. Output monitors can be voltage-based or spike-based, where the output is the direct voltage or spiking state of the source neuron. During simulation, the output vector is computed as
where
and
are connectivity matrices for the voltage and spike-based monitors, respectively.
2.4. Software Design and Workflow
Using SNS-Toolbox, the design and implementation of an SNS is split across three phases, a design phase, a compilation phase, and a simulation phase.
2.4.1. Design
To design a network, users first define neuron and connection types. These describe the parameter values of the various neural and synaptic models in the network, which can be subsequently reused. Once the neuron and connection presets are defined, they can be incorporated into a Network object (for a complete inventory of the different elements which can be added to a Network, refer to
Section 2.1,
Section 2.2 and
Section 2.3). First, the user can add populations of neurons by giving the neuron type, the size or shape of the population, and a name to refer to the population. When simulated, all populations will be flattened into a one-dimensional vector, but during the design process they can be represented as a two-dimensional matrix for ease of interpretation (e.g., working with two-dimensional image data). After populations are defined and labeled, the user can add synapses or patterns of synapses between neurons/populations, giving an index or character-string corresponding to the source and destination neurons or populations and the connection preset.
Once a network is designed, it can also be used as an element within another network. In this way, a large network can be designed using large collections of predefined subnetworks, in a methodology referred to as the Functional Subnetwork Approach (FSA). Available within the SNS-Toolbox is a collection of subnetworks which perform simple arithmetic and dynamic functions. For a complete explanation of these networks, as well as the FSA, please refer to [
9].
2.4.2. Compilation
While it describes the full structure of an SNS, a Network object is merely a dictionary which contains all of the network parameters. In order to be simulated, it must be compiled into an executable state. Given a Network, the SNS-Toolbox can compile a model capable of being simulated using one of the four software backends, NumPy [
57], PyTorch [
58], a PyTorch-based sparse matrix library (torch.sparse), and an iterative evaluator which evaluates each synapse individually. These backends are all built on well-established numerical processing libraries, with PyTorch bringing native and simple GPU support. Each backend has different strengths and weaknesses, which are illustrated in
Section 3.2.2. Although each is different, all backends are compiled following the general procedure described in Algorithm 1. Once a network is compiled, it can either be immediately used for simulation or saved to disk for later use.
Algorithm 1 General compilation procedure. |
function Compile(net, ) Get network parameters Initialize state and parameter vectors and matrices Set parameter values of each neuron in each population Set input mapping and connectivity parameter values Set connection synaptic parameter values Calculate time factors Initialize propagation delay buffer Set output mapping and connectivity parameter values return model end function
|
2.4.3. Simulation
Since the SNS-Toolbox focuses on smaller networks which are connected with varying levels of feedback loops [
6,
7,
52,
59,
60] instead of multiple massively connected layers [
61], we optimize our computations by representing all networks as single-layer, fully-connected recurrent networks. During simulation, the neural dynamics are evaluated by unfolding the network through time. This is similar to the method developed by Werbos et al., for training recurrent ANNs [
62]. See
Figure 1 for a visual representation of this strategy. At each timestep, every neuron can receive input from any neuron at the previous step (including itself via an autapse [
63]). Although the SNS-Toolbox only acts as a neural dynamics simulator, it is extensible to interact with other systems for controlling robot (
Section 3.3) or musculoskeletal (
Section 3.4) dynamics.
4. Discussion
In this work, we present SNS-Toolbox, an open-source Python package for simulating synthetic nervous systems. We focus on simulating a specific subset of neural and synaptic models, which allows for improved performance over many existing neural simulators. To the best of our knowledge, the SNS-Toolbox is the only neural simulator available which meets all of the desired functionality for designing synthetic nervous systems. The SNS-Toolbox is not tied to a dedicated graphical user interface, allowing networks to be designed and simulated on embedded systems. Heterogeneous networks of both spiking and non-spiking neurons, as well as chemical and electrical synapses, can also be simulated in real-time on both CPU and GPU hardware. All of these capabilities are also fully available across all major operating systems, including Windows (trademark Microsoft Corporation), MacOS (trademark Apple Corporation), and Linux-based systems.
We find that SNS-Toolbox can simulate networks with hundreds to thousands of neurons in real-time on desktop hardware, and low hundreds of neurons on embedded hardware. The performance is also competitive with other popular neural simulators. Through a simple programming interface, it is relatively simple to combine networks made in SNS-Toolbox with other software. Using ROS [
50], we implemented a Braitenberg-inspired [
64] neural steering algorithm and controlled navigation of a simulated mobile robot through an environment in Gazebo [
36]. We also take an existing SNS network which controlled a musculoskeletal model [
68] implemented in AnimatLab [
37], and achieved cyclical limb motion after reimplementing the network in SNS-Toolbox and interfacing with Mujoco [
49].
One decision we made early in the design process was to provide a simplified design and compilation interface, as well as to build SNS-Toolbox on top of widely used Python numerical processing libraries in order to facilitate use across all computing platforms. This has allowed multiple researchers with varying degrees of programming experience within our laboratories to begin using SNS-Toolbox successfully, as well as an instructional tool in pilot classes on neurorobotics. While other tools, such as ANNarchy [
16], achieve higher performance by direct code generation in C++, they do so at the expense of easy cross-platform support. Future work may explore adding additional build systems for different operating systems in order to achieve comparable performance.
In order to allow network simulation on GPUs, multiple backends in SNS-Toolbox are built on top of PyTorch [
58]. However, PyTorch has a large infrastructure of features which are currently not supported by the structure of the SNS-Toolbox backend, such as layer-based organization of networks and gradient-based optimization using automatic differentiation. Additionally, models built using the formal PyTorch style are able to be compiled into the C++-adjacent Torchscript, allowing improved simulation performance. Work is currently underway to restructure the PyTorch backend within SNS-Toolbox to allow these benefits.
Previous SNS models have often been made using the software AnimatLab [
4,
6,
7,
37], which uses a different workflow than SNS-Toolbox. Within AnimatLab, users have an integrated GUI that contains a rigid-body modeler, canvas for dragging and dropping neurons and synapses into a network, and a plotting window for viewing simulation results. The SNS-Toolbox is designed to focus on the design and simulation of the neural and synaptic dynamics, with the physics simulation and plotting being relegated to external libraries. While this may be less convenient for a user who is either a beginner or is migrating from AnimatLab, we feel that this separation is beneficial as it allows networks made using the SNS-Toolbox to be more extensible to interfacing with other systems. When transitioning from AnimatLab, the primary difference in workflow is that networks in the SNS-Toolbox are described via code instead of drawn. If the transitioning user is familiar with writing code in Python or another similar language, this change is easily managed. The other difficulty when converting from AnimatLab to SNS-Toolbox is when using MuJoCo [
49] for physics simulation. As the native muscle model within MuJoCo is different than AnimatLab, we show in
Section 3.4 and
Figure A2 that the same network with the same parameter values will exhibit different behavior if not tuned to use the new muscle model.
Throughout the design process of SNS-Toolbox, we chose to focus on implementing specific sets of neural and synaptic dynamics. This brings enhanced performance, however it does mean that there is no method for a user to add a new neural or synaptic model to a network which has not been previously defined, without editing the source code for the toolbox itself. One workaround to this issue is to create more complicated models by treating individual non-spiking or spiking neurons as compartments which are connected together as a multi-compartment model, however, in general, we find that this is a limitation for the SNS-Toolbox at this time.
The SNS-Toolbox is currently available as open-source software on GitHub (trademark GitHub incorporated), and has an extensive suite of documentation freely available online. In addition to installing from source, the SNS-Toolbox is also available to install from the Python Package Index (PyPi). All of the features within the toolbox are built on top of standard, widely used Python libraries. As long as these libraries maintain backwards compatibility as they update, or the current versions remain available, the functionality of SNS-Toolbox should remain into the future.
Examples were shown using the SNS-Toolbox to interface with external software systems, particularly ROS [
50] and Mujoco [
49]. While the primary goal of SNS-Toolbox is a simplified interface which focuses on neural dynamics, other users may find our interface mappings between SNS-Toolbox and other software useful. As such, we intend to release supplementary Python packages which contain helper logic to interface the SNS-Toolbox with other software as we develop them.
Many robots have been built which use SNS networks for control, although these are usually tethered to an off-board computer [
6,
7] or require non-traditional computer hardware [
8] to operate. With the release of SNS-Toolbox, we have two forward-looking hopes. Firstly, that more researchers design and implement synthetic nervous systems for robotic control, and that members of the robotics community will find value in neural simulators which are capable of simulating heterogeneous networks of dynamic neurons.