# Optimization of Transcription Factor Genetic Circuits

## Abstract

**:**

## Simple Summary

## Abstract

## 1. Introduction

## 2. Materials and Methods

#### 2.1. Number of Parameters

#### 2.2. Biological Bounds on Parameters

#### 2.3. Optimization of Bounded Parameters

#### 2.4. Initial Parameters and Hyperparameters

#### 2.5. Stochastic Fluctuations Vary with Abundance

## 3. Results

#### 3.1. Dynamics of TF Networks

#### 3.2. TF Network as Input-Output Function

#### 3.3. Maintaining Circadian Rhythm as a Design Challenge

#### 3.4. Stochastic Molecular Dynamics

#### 3.5. Random External Light Signal for Entrainment

#### 3.6. Dynamics of an Optimized System

#### 3.7. TF Logic of an Optimized System

## 4. Discussion

#### 4.1. Optimize a Neural Network and Fit a TF Network

#### 4.2. Large Networks, Flat Fitness Surfaces, and Genetic Variation

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## Abbreviations

AI | artificial intelligence |

SciML | scientific machine learning (Julia language packages) |

TF | transcription factor |

## References

- Baydin, A.G.; Pearlmutter, B.A.; Radul, A.A.; Siskind, J.M. Automatic differentiation in machine learning: A survey. J. Mach. Learn. Res.
**2018**, 18, 1–43. [Google Scholar] - Margossian, C.C. A review of automatic differentiation and its efficient implementation. WIREs Data Min. Knowl. Discov.
**2019**, 9, e1305. [Google Scholar] [CrossRef] - Frank, S.A. Automatic differentiation and the optimization of differential equation models in biology. arXiv
**2022**, arXiv:2207.04487. [Google Scholar] [CrossRef] - Goodfellow, I.; Bengio, Y.; Courville, A. Deep Learning; MIT Press: Cambridge, MA, USA, 2016. [Google Scholar]
- Fröhlich, F.; Kessler, T.; Weindl, D.; Shadrin, A.; Schmiester, L.; Hache, H.; Muradyan, A.; Schütte, M.; Lim, J.H.; Heinig, M.; et al. Efficient parameter estimation enables the prediction of drug response using a mechanistic pan-cancer pathway model. Cell Syst.
**2018**, 7, 567–579. [Google Scholar] [CrossRef] [PubMed] - Hiscock, T.W. Adapting machine-learning algorithms to design gene circuits. BMC Bioinform.
**2019**, 20, 214. [Google Scholar] [CrossRef] - Liu, Y.; Barr, K.; Reinitz, J. Fully interpretable deep learning model of transcriptional control. Bioinformatics
**2020**, 36, i499–i507. [Google Scholar] [CrossRef] - Nilsson, A.; Peters, J.M.; Meimetis, N.; Bryson, B.; Lauffenburger, D.A. Artificial neural networks enable genome-scale simulations of intracellular signaling. Nat. Commun.
**2022**, 13, 3069. [Google Scholar] [CrossRef] - Lopatkin, A.J.; Collins, J.J. Predictive biology: Modelling, understanding and harnessing microbial complexity. Nat. Rev. Microbiol.
**2020**, 18, 507–520. [Google Scholar] [CrossRef] - Rackauckas, C.; Ma, Y.; Martensen, J.; Warner, C.; Zubov, K.; Supekar, R.; Skinner, D.; Ramadhan, A. Universal differential equations for scientific machine learning. arXiv
**2020**, arXiv:2001.04385. [Google Scholar] - Bezanson, J.; Edelman, A.; Karpinski, S.; Shah, V.B. Julia: A fresh approach to numerical computing. SIAM Rev.
**2017**, 59, 65–98. [Google Scholar] [CrossRef] - Frank, S.A. Optimization of Transcription Factor Genetic Circuits: Julia Software Code. Zenodo
**2022**. Available online: https://doi.org/10.5281/zenodo.6798420 (accessed on 7 August 2022). - Milo, R.; Phillips, R. Cell Biology by the Numbers; Garland Science: New York, NY, USA, 2015. [Google Scholar]
- Kingma, D.P.; Ba, J. Adam: A method for stochastic optimization. arXiv
**2014**, arXiv:1412.6980. [Google Scholar] [CrossRef] - Frank, S.A. Optimizing differential equations to fit data and predict outcomes. arXiv
**2022**, arXiv:2204.07833. [Google Scholar] [CrossRef] - Marbach, D.; Prill, R.J.; Schaffter, T.; Mattiussi, C.; Floreano, D.; Stolovitzky, G. Revealing strengths and weaknesses of methods for gene network inference. Proc. Natl. Acad. Sci. USA
**2010**, 107, 6286–6291. [Google Scholar] [CrossRef] [PubMed] - Bintu, L.; Buchler, N.E.; Garcia, H.G.; Gerland, U.; Hwa, T.; Kondev, J.; Phillips, R. Transcriptional regulation by the numbers: Models. Curr. Opin. Genet. Dev.
**2005**, 15, 116–124. [Google Scholar] [CrossRef] - Bintu, L.; Buchler, N.E.; Garcia, H.G.; Gerland, U.; Hwa, T.; Kondev, J.; Kuhlman, T.; Phillips, R. Transcriptional regulation by the numbers: Applications. Curr. Opin. Genet. Dev.
**2005**, 15, 125–135. [Google Scholar] [CrossRef] - Stadler, P.F. Fitness landscapes. In Biological Evolution and Statistical Physics; Lässig, M., Valleriani, A., Eds.; Springer: Berlin/Heidelberg, Germany, 2002; pp. 183–204. [Google Scholar] [CrossRef]
- Malan, K.M. A survey of advances in landscape analysis for optimisation. Algorithms
**2021**, 14, 40. [Google Scholar] [CrossRef] - De Jong, H.; Gouzé, J.L.; Hernandez, C.; Page, M.; Sari, T.; Geiselmann, J. Qualitative simulation of genetic regulatory networks using piecewise-linear models. Bull. Math. Biol.
**2004**, 66, 301–340. [Google Scholar] [CrossRef] - Radhakrishnan, A.; Belkin, M.; Uhler, C. Overparameterized neural networks implement associative memory. Proc. Natl. Acad. Sci. USA
**2020**, 117, 27162–27170. [Google Scholar] [CrossRef] - Elad, M.; Simon, D.; Aberdam, A. Another step toward demystifying deep neural networks. Proc. Natl. Acad. Sci. USA
**2020**, 117, 27070–27072. [Google Scholar] [CrossRef] - Baraniuk, R.; Donoho, D.; Gavish, M. The science of deep learning. Proc. Natl. Acad. Sci. USA
**2020**, 117, 30029–230032. [Google Scholar] [CrossRef] - Poggio, T.; Banburski, A.; Liao, Q. Theoretical issues in deep networks. Proc. Natl. Acad. Sci. USA
**2020**, 117, 30039–30045. [Google Scholar] [CrossRef] - Ciliberti, S.; Martin, O.C.; Wagner, A. Innovation and robustness in complex regulatory gene networks. Proc. Natl. Acad. Sci. USA
**2007**, 104, 13591–13596. [Google Scholar] [CrossRef] - Frank, S.A. Genetic variation in cancer predisposition: Mutational decay of a robust genetic control network. Proc. Natl. Acad. Sci. USA
**2004**, 101, 8061–8065. [Google Scholar] [CrossRef] - Frank, S.A. Evolutionary design of regulatory control. II. Robust error-correcting feedback increases genetic and phenotypic variability. J. Theor. Biol.
**2019**, 468, 72–81. [Google Scholar] [CrossRef] - Frank, S.A. Robustness increases heritability: Implications for familial disease. bioRxiv
**2022**. [Google Scholar] [CrossRef]

**Figure 1.**Circadian dynamics with stochastic fluctuations and random daylight signal. Stochastic dynamics of TF proteins (

**a**–

**d**) and the mRNAs that produce them (

**e**–

**h**) over six days. The parameters were obtained from the best result of all optimization runs, in which best means the closest match of cellular dynamics to a circadian pattern, as defined in the following paragraphs. The vertical lines in (

**a**,

**b**) show entry into daylight (dotted) and nighttime (solid). The y-axis is $log10(1+y)$ for number of molecules per cell, y. The optimization design goal is for the blue curve in (

**a**), the number of TF 1 molecules, to match a circadian rhythm. To define the optimization loss value to be minimized, the number of TF 1 molecules, y, is transformed by a Hill function, $\tilde{y}={y}^{2}/({1000}^{2}+{y}^{2})$, to yield the green curve, which traces values $1+4\tilde{y}$. The gold curve traces the target circadian pattern. The optimization loss value to be minimized is the sum of the squared deviations between the gold and green curves at 50 equally spaced time points per day. The number of TF 2 proteins in (

**b**) is influenced by the internal cellular dynamics and is also increased in response to an external daylight signal (see text). The availability of the light signal switches on and off randomly. It is initially off. The average waiting time for a random switch in the presence or absence of the signal is w, measured in days. In this example, $w=2$. The signal turns on around sunrise of day 3 and stays on for the remaining days shown. Because the switching is random, daylight can be present or absent for several days in a row, or it can switch on and off several times in one day. In this particular example, looking at the match between cellular state shown by green curve in (

**a**) compared with the target gold signal, stochastic molecular perturbations push the cellular rhythm behind the actual circadian pattern during the first few days. When the daylight signal appears in the middle of day 3, the system entrains to the external signal and closely matches the target circadian pattern for the remaining days shown in the plot. Panels (

**i**,

**j**) illustrate the match of internal cellular state (green) and target circadian pattern (gold) over 20 days. Each plot shows a sample of 20 stochastic trajectories for cellular state, showing the magnitude and the randomness in the degree of mismatch between actual and target trajectories. In (

**i**), the average waiting time between random switching of for the presence or absence of the external light signal is $w=2$ days. In (

**j**), the waiting time is $w=1000$ days. Because the signal starts in the off state, in (

**j**) the external signal essentially never comes on. Thus, the green stochastic cellular dynamics in (

**j**) illustrate the ability of the cell to hold a circadian rhythm over many days in the absence of an external light signal for entrainment.

**Figure 2.**Stochastic perturbations to entrainment (

**a**) and TF logic for mRNA expression (

**b**). Panel (

**a**) presents the deviation of cellular dynamics from the circadian target pattern. Each set of two vertical lines and a circle show the distribution of the deviation between the entry into daytime cellular state and the actual onset of daytime. The circle denotes the median of 1000 stochastic cellular trajectories. The upper line shows the 75th percentile at the bottom and the 95th percentile at the top. The lower line shows the 5th (bottom) and 25th (top) percentiles. Each set of three distributions shows, from left to right, the distribution of deviations measured at 10, 20, and 30 days. The w labels below denote the average waiting time in days for random switches between the presence or absence of the external daylight entrainment signal (see caption for Figure 1). From left to right, the waiting times vary over $2,4,8,16,1000$. Shorter waiting times provide more frequent entrainment signals, reducing the consequences of the intrinsic molecular stochasticity of cellular dynamics. For w1000, the external signal essentially never occurs, thus the set of distributions shows the intrinsic cellular stochasticity and the ability of the cell to maintain a circadian pattern in the absence of an external signal. TF protein levels control mRNA expression. The four TF proteins form the inputs, and the four expression levels for the associated mRNAs form the outputs. This panel shows all four TF inputs and the associated mRNA expression level for protein 1 as the surface levels of each plot. In each plot, the basal axes quantify the levels of TF 1 and TF 2, labeled as p1 and p2 in the bottom row of plots. The scale is $log10(1+y)$ for TF protein level y. The height of each plot shows the relative expression level triggered by the TF inputs, scaled from 0 for complete repression to 1 for maximum expression. The rows from top to bottom show increasing levels of TF 3, labeled as p3. The columns from left to right show increasing levels of TF 4, labeled as p4. See the text for interpretation of the plots.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Frank, S.A.
Optimization of Transcription Factor Genetic Circuits. *Biology* **2022**, *11*, 1294.
https://doi.org/10.3390/biology11091294

**AMA Style**

Frank SA.
Optimization of Transcription Factor Genetic Circuits. *Biology*. 2022; 11(9):1294.
https://doi.org/10.3390/biology11091294

**Chicago/Turabian Style**

Frank, Steven A.
2022. "Optimization of Transcription Factor Genetic Circuits" *Biology* 11, no. 9: 1294.
https://doi.org/10.3390/biology11091294