Single-Cell Reprogramming in Mouse Embryo Development through a Critical Transition State

Our previous work on the temporal development of the genome-expression profile in single-cell early mouse embryo indicated that reprogramming occurs via a critical transition state, where the critical-regulation pattern of the zygote state disappears. In this report, we unveil the detailed mechanism of how the dynamic interaction of thermodynamic states (critical states) enables the genome system to pass through the critical transition state to achieve genome reprogramming right after the late 2-cell state. Self-organized criticality (SOC) control of overall expression provides a snapshot of self-organization and explains the coexistence of critical states at a certain experimental time point. The time-development of self-organization is dynamically modulated by changes in expression flux between critical states through the cell nucleus milieu, where sequential global perturbations involving activation-inhibition of multiple critical states occur from the middle 2-cell to the 4-cell state. Two cyclic fluxes act as feedback flow and generate critical-state coherent oscillatory dynamics. Dynamic perturbation of these cyclic flows due to vivid activation of the ensemble of low-variance expression (sub-critical state) genes allows the genome system to overcome a transition state during reprogramming. Our findings imply that a universal mechanism of long-term global RNA oscillation underlies autonomous SOC control, and the critical gene ensemble at a critical point (CP) drives genome reprogramming. Identification of the corresponding molecular players will be essential for understanding single-cell reprogramming.


Introduction
In mammalian embryo development, many molecular-level epigenetic studies [1][2][3] have revealed the presence of stunning global epigenetic modifications on chromatins (DNA + histones) associated with reprogramming processes. However, the genome-wide principle that drives such extremely complex epigenetic modifications is still unknown.
In our previous studies, based on transcriptome experimental data for seven distinct cell fates [4], we recognized that a self-organized criticality (SOC) transition in whole-genome expression plays an essential role in the change in the genome expression state at both the population and single-cell levels (see Methods, for more details [4][5][6]). The essential points of SOC control of overall expression can be summarized as follows: SOC control of overall expression can be seen in an emergent level of collective behaviors. The presence of a "collective organization layer" in gene expression, while naturally encompassed by the existence of "tissue-specific gene-expression profiles" across the whole genome, seems at odds with most biological experimentation that strictly focuses on gene-specific rules. The contrast of "microscopic rules" versus "collective behavior" is a classical issue in science: it is worth stressing that the presence of microscopic laws does not falsify the statistical approach (which in this condition could be considered to be a surrogate of a more "fundamental" microscopic approach), while a statistical "emergent" approach is valid even in the absence of microscopic deterministic rules [11]. In this latter case, the "fundamental level" corresponds to the collective organization layer [11,12]. As for our specific case, a gene-by-gene set of deterministic microscopic rules that affects thousands of gene products (some of which have very low concentrations inside single cells and thus are undergoing high stochastic fluctuation) is expected to be both more unreliable and more energy-consuming than self-organized criticality that in turn very clearly emerges in different scaling regimens along the order parameter ( Figure 1). encompassed by the existence of 'tissue-specific gene-expression profiles' across the whole genome, seems at odds with most biological experimentation that strictly focuses on genespecific rules. The contrast of 'microscopic rules' versus 'collective behavior' is a classical issue in science: it is worth stressing that the presence of microscopic laws does not falsify the statistical approach (which in this condition could be considered to be a surrogate of a more 'fundamental' microscopic approach), while a statistical 'emergent' approach is valid even in the absence of microscopic deterministic rules [11]. In this latter case, the 'fundamental level' corresponds to the collective organization layer [11,12]. As for our specific case, a gene-bygene set of deterministic microscopic rules that affects thousands of gene products (some of which have very low concentrations inside single cells and thus are undergoing high stochastic fluctuation) is expected to be both more unreliable and more energy-consuming than selforganized criticality that in turn very clearly emerges in different scaling regimens along the order parameter ( Figure 1).   Recently, we demonstrated that essentially the same critical-state dynamics that we observed for cell differentiation processes [4,6] are also present in overall RNA expression in single-cell mouse embryo development, which presents further proof of SOC control as a universal characteristic [13]. Overall RNA expression and its dynamics exhibit typical features (genome avalanche and sandpile type criticality) of self-organized criticality (SOC) control in mouse embryo development; Figure 1 shows a genome avalanche, i.e., scaling-divergent behavior in a log-log-scale plot between expression and the temporal variance of expression (nrmsf ). Figure 2 depicts the phenomena in Figure 1 along a time arrow, and clarifies early embryo development with a shift toward erasure of the previous organization pattern relative to the zygote state: sandpile-type criticality (critical behavior) of the zygote state survives at the early 2-cell state and disappears after the late 2-cell state to reach a stochastic pattern in the 4-cell state (linear pattern revealed in randomly shuffled overall expression: Figure S2 in [4]).

VS.
Importantly, genome avalanches ( Figure 1) reveal that the regions of scaling and divergence in terms of nrmsf are opposite between embryo development and cell differentiation at a single-cell level. Since nrmsf is an order parameter [4][5][6], this reveals opposite orders of self-organization; similar scaling-divergent order for cell differentiation is also observed at a cell-population level in cancer cells [4,6]. Furthermore, since chromosomes exhibit fractal organization [14,15], this power law behavior may reveal a quantitative relation between the aggregation state of chromatin through nrmsf and the average expression of an ensemble of genes, which is opposite in embryo development and cell differentiation, and the existence of coherent waves of condensation/de-condensation in chromatin in cell development, which is transmitted from a region of low temporal expression (low nrmsf region) in reprogramming of the embryo genome, to high temporal expression (high nrmsf ) in terminal somatic cell differentiation.
On the other hand, the erasure of sandpile-type criticality ( Figure 2) indicates that (i) the memory of early embryogenesis in the zygote is lost after the late 2-cell state: a significant change in the genome-state, i.e., the occurrence of reprogramming; and (ii) the SOC control landscape, showing valley (SOC control) -ridge (non SOC control) -valley (SOC control), exhibits a near-critical transition state at the middle -late 2-cell states through a stochastic pattern. A complete erasure of the zygote state's criticality occurs after the late 2-cell state, such that a critical transition state should exist right after the late 2-cell state. Recently, we demonstrated that essentially the same critical-state dynamics that we observed for cell differentiation processes [4,6] are also present in overall RNA expression in   Figure 7). Sandpile-type critical behaviors, which exhibit diverging up-and down-regulation at a critical point (CP: ln<nrmsf>~− 5.5), emerge in overall expression sorted and grouped according to the fold-change in expression (see stochastic resonance in [6]). Each group (net 25 groups) contains n = 685 RNAs. The xand y-axes show the natural logarithm of the fold-change in each group average of RNA expression between the s i and s j states (ith and jth cell states; Methods) represented by ln(<ε(s i )/ε(s j )>) and of RNA expression (ln(<ε( s i )>)), respectively.
In this report, we investigate the self-organizing dynamics of whole RNA expression in mouse embryo development [16] to reveal how genome reprogramming occurs by passing through a critical transition state. This is elucidated by a dynamic expression flux analysis (quantitative evaluation of perturbation based on self-organization through SOC).
The results regarding the flux dynamics suggest that the critical gene ensemble of the critical point plays an essential role in reprogramming. Our results suggest that the SOC control mechanism of genome dynamics is rather universal among several distinct biological processes [4]. Our findings may provide a universal classification scheme for phenomena with far-from-equilibrium phase transitions, which has been missing in past studies [17].

Sloppiness of Mouse RNA Expression Dynamics: Coherent-Stochastic Behaviors in Critical States
As has been demonstrated in cell differentiation [4][5][6], mouse embryo overall RNA expression is self-organized into distinct response domains (critical states). This can be seen as a transitional change in the expression profiles of groups of genes according to nrmsf ( Figure 3A), which generates three different gene expression states: "super-critical", "near critical" and "sub-critical". Figure 3 demonstrates that distinct coherent-stochastic behaviors emerged in critical states based on (i) the law of large numbers in each critical state (convergence to its center of mass; Figure 3B); (ii) different degrees of stochasticity ( Figure 3B); (iii) distinct coherent dynamics ( Figure 3C) and (iv) non-convergence of mixed states to a dominant CM (i.e., sub-critical state; Figure 3D).
This emergent coherent behavior shows how a single cell can overcome the problem of local stochastic fluctuations in single genes (microscopic rules) by gene-expression regulation, which further affirms the statistical significance of self-organization through SOC control of overall expression: genome avalanche and sandpile-type criticality ( Figures 1A and 2). This collective control in early embryo development points, notably, to the fact that bewildering epigenomic reprogramming molecular processes are regulated by a few hidden control parameters through SOC in terms of collective behaviors even at a single-cell level (see [4] for examples of SOC control at the cell-population level).
Regarding a critical point (CP) in single-cell Th17 cell differentiation and mouse early embryo development, the results suggest that a single critical point (CP) in terms of nrmsf may exist in the range of ln<nrmsf> from −5.5 to −6.0 for both processes: the onset of the divergence from scaling occurs at around ln<nrmsf>~− 6.0 for Th17 cell differentiation ( Figures 1B and 4B), whereas linear regressions in scaling regions for mouse early embryo exhibit intersection at around ln<nrmsf> = −5.5~−6.0 ( Figure 4C; see also a critical point as in Figures 2 and 3A). 7 The results regarding the flux dynamics suggest that the critical gene ensemble of the critical point plays an essential role in reprogramming. Our results suggest that the SOC control mechanism of genome dynamics is rather universal among several distinct biological processes [4]. Our findings may provide a universal classification scheme for phenomena with far-fromequilibrium phase transitions, which has been missing in past studies [17].

Sloppiness of Mouse RNA Expression Dynamics: Coherent-Stochastic Behaviors in Critical States
As has been demonstrated in cell differentiation [4][5][6], mouse embryo overall RNA expression is self-organized into distinct response domains (critical states). This can be seen as a transitional change in the expression profiles of groups of genes according to nrmsf ( Figure   3A), which generates three different gene-expression states: 'super-critical', 'near critical' and 'sub-critical'.

Expression Flux Dynamics Representing a Change in Genetic Activity
Here, we elucidate a statistical mechanism to explain how a single mouse embryo cell achieves genome reprogramming by passing through a transition state ( Figure 2). Figure 3B reveals that the coherent-stochastic behavior in a critical state corresponds to the scalar dynamics of its CM, X(s j ), where X(s j ) represents the numerical value of a specific critical state (i.e., super-, near-or sub-critical state) at the jth cell state, s j . Thus, the change in the one-dimensional effective force acting on the CM determines the dynamics of X(s j ). Figure 5A confirms this point, in that the trend of the dynamics of the CM of a critical state follows its effective force (net self-flux dynamics: see the definition below). We now consider that the respective average values of the effective force can serve as baselines, and how perturbation from these baselines occurs dynamically.
The expression flux between critical states is interpreted as a non-equilibrium system and evaluated in terms of a dynamic network of effective forces, where interaction flux is driven by effective forces between different critical states and can be described by a second-order time difference. From a mathematical point of view, the oscillatory phenomenon interpreted using a second-order time difference equation with a single variable is equivalent to inhibitor-activator dynamics given by a couple of first-order time difference equations with two variables.  Figure 7); (C) The change in the net kinetic energy flux (see Equation (4)) shows that sequential global perturbations from OUT to IN flux occur at the middle 2-cell-4-cell states in the sub-critical state, as in the coherent dynamics of critical states ( Figure 3C); (D) The interaction dynamics reveal that the expression dynamics of the sub-critical state (the generator of perturbation) are determined by the net interaction flux between the near-and sub-critical states, and the dynamics of the super-critical state are determined by its interaction flux from the near-critical state (super-to near-critical interaction). This image shows how the critical dynamics (temporal change in criticality exists at the boundary of the near-and sub-critical states) affect the entire genome-expression system, since the sub-critical state generates autonomous SOC control of overall expression (see more in Section 2.1.4).
The genome is embedded in the intra-nuclear environment, where the expression flux represents the exchange of genetic energy or activity-the effective force produces work, and thus causes a change in the internal energy of critical states. This model shows a statistical thermodynamic picture of self-organized overall expression under environmental dynamic perturbations; the regulation of RNA expression is managed through the mutual interaction between critical states and the external connection with the cell nucleus milieu. The environment here is intended in the broad sense of the baseline gene expression activity adapted to the microenvironment; in thermodynamic terms, the environment is equivalent to a "thermal bath".
The effective force can be interpreted as a combination of incoming flux from the past to the present and outgoing flux from the present to the future cell state [4]: where ∆P is the change in momentum with a unit mass (i.e., impulse: The interaction flux of a critical state, X with respect to another critical state or the environment (milieu) Y ( Figure 5B) can be defined as: where, again, the first and second terms represent IN flux and OUT flux, respectively, and the net value, IN flux − OUT flux, represents incoming (IN) interaction flux from Y for a positive sign and outgoing (OUT) interaction flux to Y for a negative sign. Y ∈ {Super, Near, Sub, E} where Y = X: E represents the environment. Due to the law of force, the net self-flux of a critical state is the sum of the interaction fluxes with other critical states and the environment: where f X s j ; A i is an interaction flux of X with A i ∈ {Super, Near, Sub} with A i = X, and M is the number of internal interactions (M = 2), i.e., for a given critical state, there are two internal interactions with other critical states. Equation  (1) and (2)) for the mouse RNA-Seq data (Methods). Note: The balance does not hold at each time point/cell state. This model of expression flux dynamics shows environmental dynamic perturbations in the self-organization of overall expression; the regulation of RNA expression is managed through the mutual interaction between critical states through the cell nucleus.
Under non-equilibrium thermodynamically-open conditions, due to the break-down of the detailed balance, cyclic state-flux among different states emerges as a general property [18]. This is evident in Figures 6 and 7. Figure 6 that shows how two cyclic fluxes (between the Super-Sub critical states, and the Super-Near critical states respectively) emerge, while Figure 5B reveals the broken detailed balance of mutual fluxes between critical states at each time point (see the broken detailed balance more clearly in Figure 7), these cyclic fluxes are substantially perturbed right before passing the critical transition state (near-transition state between the middle-late 2 cell states; see more in Section 2.2). Therefore, it becomes possible to evaluate the temporal change in the thermodynamically-open genetic system, where the expression flux represents the exchange of genetic energy or activity. resulting in the anti-phase dynamics of two piston movements (refer to Discussion in [4]). This suggests that the genome engine, may be a universal mechanism in gene-expression regulation of mammalian cells. These cyclic fluxes are considered in light of self-regulatory gene expression by means of a complex epigenetic machinery (methylation processes, long non-coding RNAs, small non-coding RNAs such as miRNAs, etc.): the super-critical state (high variance RNAs) acts as a sink internally to receive genetic information and send it back to the other critical states through the cell environment. On the other hand, the sub-critical state (low-variance RNAs) acts as an internal source of information and sustains (like a generator in an electrical circuit) the interaction with cyclic fluxes.
This implies that the collective behavior of an ensemble of low-variance RNA expression (sub-critical state) plays an essential role in reprogramming in single cells (see a more dynamical discussion below). This dominant cyclic flow also shows that the dynamics of the sub-critical and super-critical states are anti-phase with respect to each other ( Figure 3C) to form strong coupling between them [6]. This suggests that the two cyclic fluxes act as feedback flow for the change in criticality to generate coherent oscillatory dynamics of critical states. This dynamic model also provides a mechanism for long-term global RNA oscillation underlying autonomous SOC control generated by the sub-critical state [4,6,19].
The formation of a dominant cyclic flux between a source and a sink provides a genome-engine metaphor for SOC control mechanisms to describe how expression flux is transmitted among critical states: the sub-critical state as a "large piston" for short moves and the super-critical state as a "small piston" for large moves with an "ignition switch" (near-critical state with a critical point) are connected through a dominant cyclic state flux as a "camshaft", resulting in the anti-phase dynamics of two piston movements (refer to Discussion in [4]). This suggests that the genome engine, may be a universal mechanism in gene-expression regulation of mammalian cells.

Change in Criticality: Global Impact on the Whole Genome-Expression System
Here, we further clarify the SOC control mechanism of the reprogramming of single-cell embryo development through the breakdown of an initial state's criticality.
Reprogramming occurs right after the late 2-cell states: the zygote-state SOC control of overall gene expression (i.e., initial-state global gene-expression regulation mechanism) is destroyed through erasure of the zygote-state criticality. Figure 5D shows that the mutual interactions of Sub-Near and Sub-Super determine the net self-flux of the sub-and super-critical states, respectively, representing the effective driving forces acting on their CMs and thus, determine their coherent oscillatory dynamics ( Figure 3C). This between-states interaction serves as the underlying mechanism of self-regulatory gene expression through the orchestrated cooperation of myriads of epigenetic modifications, transcriptional factors and non-coding RNA regulations to determine the critical-state coherent oscillatory behavior. The essential role played by interactions explains how the temporal change in criticality at the near-critical state, i.e., in expression of the critical gene ensemble of the CP, directly perturbs the sub-critical state (the generator of flux dynamics: Figure 6) through their mutual interaction, and perturbation of this generator can spread over the entire system (refer also to Figure 13A for cell differentiation in [4]).

SOC Control Mechanism of Genome Reprogramming through a Critical Transition State
The erasure of initial-state criticality (e.g., in the zygote state) suggests that genome reprogramming begins right after the late 2-cell state. As noted, the initial state can be the early 2-cell state instead of the zygote state; this independent choice of the initial state further supports the timing of the genome-state change [4].
Interaction flux dynamics ( Figures 5B and 7) represents erasure as a thermodynamical event that passes through a critical transition state, which shows how the genome system can pass through the critical transition state -dynamic perturbation in the average flux in terms of the enhancement-suppression of two cyclic flows around the reprogramming event ( The asymmetry becomes most significant for the IN-and-OUT fluxes between the superand sub-critical states at the late 2-cell state, i.e., the greatest time-reversal symmetry breaking is caused. In this respect, it is worth noting that, with no time-reversal symmetry breaking, interaction fluxes between the super-and sub-critical states should be balanced (equal), corresponding to the fundamental framework of a detailed balance in equilibrium statistical physics, while a detailed balance should be violated in a thermodynamically open system [18]. (iii) After reprogramming: In the 4-cell state, while the enhancement of cyclic flux (Super-Sub) becomes weak, the perturbation of average flux activity almost disappears due to passage through the transition state.
Therefore, two major global perturbations, which involve the activation-inhibition of multiple critical states, occur between the middle and late 2-cell states, and between the late 2-cell and 4-cell states during genome reprogramming ( Figure 3C). This global perturbation event is clearly seen in the net kinetic energy flux [4] in a critical state ( Figure 5C): where the kinetic energy of the CM for the critical state with unit mass at s = s j is defined as 1/2 v s j 2 with average velocity: v s j ≡ X(s j )−X(sj−1) The above processes (i)-(iii) regarding reprogramming can be described in terms of how the genome system can pass through the transition state on the SOC control landscape (Figure 2): the perturbation of self-organization shows a temporal change in the degree of SOC control: a high (low) degree of SOC control points to a weak-local (strong-global) perturbation of the average flux flow. The process that climbs the hill of the SOC control landscape encompasses the following steps ( Figure 7): (1) Release from SOC control in the middle 2-cell state due to activation of the near-critical state; (2) Passage through the transition state -the lowest degree of SOC control (i.e., non-SOC control) due to the strongest global perturbation at the late 2-cell state stemmed from activation of the sub-critical state, which changes from SOC control to non-control; (3) Return to SOC control in the 4-cell state.   Figure 3C). Interaction flux dynamics ( Figure 5B) reveal a SOC control mechanism of genome reprogramming: (i) At the middle 2-cell state (2-cell(M)), interaction flux between Super and Near suppresses cyclic flux. All critical states receive the net IN flux, which indicates that, inside the nucleus, the mouse genome system is activated through the cell milieu (environment), which makes the cell state move up from a valley on the SOC control landscape, as shown in the inset; (ii) At the late 2-cell state, a substantial change in the net interaction flux at critical states occurs from the middle 2-cell state to induce a major change in two cyclic fluxes: a change from suppression to the enhancement of cyclic flux (Super-Near), and the strongest enhancement of cyclic flux (Super-Sub). This leads to a reverse change in the net interaction fluxes at the near-critical state (IN to OUT and vice versa) to enhance the cyclic flux (Super-Near) from major suppression in the middle 2-cell state. This activation of the sub-critical state (the maximum net OUT flux sent to other critical states) enables the system to pass through the transition state (erasure of zygote criticality after the late 2-cell state: Figure 2) to reprogram the mouse embryo genomic state as described in the inset; and (iii) In the 4-cell state, while the enhancement of cyclic flux between super-and sub-critical states becomes weak, the perturbation of average flux activity almost disappears. While the near-critical state becomes balanced, the sub-critical state receives the net expression flux and the super-critical state sends the net flux. (i)-(iii) show the occurrence of sequential global perturbations to pass through the transition state to achieve single-cell reprogramming. Numerals represent net interaction flux values.

Discussion and Conclusions
Thermodynamics allows for incredibly precise predictions thanks to the "generality of its premises" (Einstein's words [20]). In the present study, this allowed us to grasp the essentials by skipping the (largely unknown) detailed biological mechanisms and focusing on the phenomenology of the global changes in genome expression. The erasure of epigenetic marks (refer to the Discussion in [4]) is consistent with a thermodynamics perspective in addition to its actual mechanism.
The time-development of sandpile criticality reveals the existence of a transition state which in turn suggests that programming of mouse embryo occurs through the transition state. As proof of this concept, it has been demonstrated that EGF-stimulated MCF-7 cells do not erase sandpile criticality (see Figure 5A in [4]), i.e., there is no genome-state change (consistent with the experiment in which