This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).

To gain biological insights, investigators sometimes compare sequences of gene expression measurements under two scenarios (such as two drugs or species). For this situation, we developed an algorithm to fit, identify, and compare biologically relevant response curves in terms of heteromorphy (different curves), heterochrony (different transition times), and heterometry (different magnitudes). The curves are flat, linear, sigmoid, hockey-stick (sigmoid missing a steady state), transient (sigmoid missing two steady states), impulse (with peak or trough), step (with intermediate-level plateau), impulse+ (impulse with an extra parameter), step+ (step with an extra parameter), further characterized by upward or downward trend. To reduce overfitting, we fit the curves to every other response, evaluated the fit in the remaining responses, and identified the most parsimonious curves that yielded a good fit. We measured goodness of fit using a statistic comparable over different genes, namely the square root of the mean squared prediction error as a percentage of the range of responses, which we call the relative prediction error (RPE). We illustrated the algorithm using data on gene expression at 14 times in the embryonic development in two species of frogs. Software written in Mathematica is freely available.

Many gene expression experiments involve serial measurements in response to a varying condition, such as temperature, oxygen availability, time, drug concentration, levels of pollutants, and exposures to ultraviolet light. Often investigators want to compare the time varying response between two scenarios, such as two species or two drugs. For this comparative analysis, we developed an algorithm to fit biologically relevant curves to serial response measurements from each gene, identify pairs of curves that fit well, and compare these curves under the two scenarios in terms of heteromorphy (different curves), heterochrony (different transition times) and heterometry (different magnitudes). In the context of ontogeny, Yanai

The fitting part of the algorithm involves the following models: flat, linear, sigmoid, double sigmoid [

The aforementioned response curves are biologically relevant, as opposed to polynomial curves of degree two or greater, which generally have little biological basis. Flat curves represent a steady state. Linear curves represent the constant addition or subtraction of reacting components. Sigmoid curves model the addition or subtraction of reacting components from one steady state to another steady state. Sigmoid curves also arise in transcription factor binding [

Although there is a large literature on the fitting of response curves to sequential gene expression measurements in dose-response and short time series studies [

For illustration, we applied our algorithm to mean gene expression levels (averaged over three technical replicates and three specimens) for 11,299 genes at 14 development times in two species of frogs,

As noted by Forster [

Consider a single gene. Let _{j}^{th} observed response and _{j}^{th} observed time. We fit the model to responses {_{1}_{3}_{5}_{7}_{9}_{11}_{13}_{1}_{3}_{5}_{7}_{9}_{11}_{13}_{1}_{1}_{3}_{3}_{5}_{5}_{7}_{7}_{9}_{9}_{11}_{11}_{2}_{2}_{4}_{4}_{6}_{6}_{8}_{8}_{10}_{10}_{12}_{12}_{2}_{4}_{6}_{8}_{10}_{12}_{14}

We needed a measure of how well the predicted responses fit the observed evaluation points. One measure considered was the mean squared error (MSE). The problem with using MSE is that it depends on the absolute sizes of responses, so two genes could have the same MSE’s for comparing predicted and observed responses, yet visually one may fit well and the other fit poorly. To circumvent this problem we introduced the Relative Prediction Error (RPE), which is the square root of the MSE of the predicted response divided by the difference between the largest and smallest predicted responses, expressed as a percentage. The reason for using the square root is to put the measure on the same scale as the responses, analogous to using a standard deviation instead of a variance. The reason for dividing by the range of responses is to make small deviations between predicted and observed response relative to the entire shape of the curve, which leads to a visually satisfying measure. Let _{j in J}_{j}_{j}^{2}/7]^{1/2}/[max_{j in J}_{j}_{j in J}_{j}

The formula for RPE can be readily modified for more than 14 points. Based on a visual inspection of curves with different values of RPE, we decided that a threshold of 10% was a reasonable indicator of a good fit. To put the idea of a threshold RPE into perspective, note that a likelihood ratio test comparing observed and fitted counts typically also involves a threshold, namely a 5% type I error.

When comparing predicted and observed results at

To introduce the curve selection algorithm, consider the following two hypothetical examples for a single gene. In the first example, suppose the RPE’s for flat, lineU, sigmoidU, impulseU, and impulse+U curves are 30%, 12%, 11%, 8%, and 9%, respectively (as explained in the next section, the “U” designates upward trend).

In the second hypothetical example, suppose the RPE’s for flat, lineU, sigmoidU, impulseU, and impulse+U curves are 30%, 22%, 14%, 8%, 9%, respectively.

We formalize the curve selection algorithm as follows.

_{A}. In the first example Curve A is impulseU. If RPE_{A} > 10%, report no curve; otherwise proceed t

_{B} denote the set of response curves with fewer parameters than Curve A. In the first example CurveSet_{B} = {flat, lineU, sigmoidU}. Let CurveSubset_{B} denote a subset of response curves in CurveSet_{B} such that RPE ≤ RPE_{A} + 5%. In the first example CurveSubset_{B} is {lineU, sigmoidU}. If CurveSubset_{B} is the empty set, we identify Curve B as the curve with the most parameters in CurveSet_{B} (sigmoidU in the second example) but select Curve A as the reported curve. If CurveSubset_{B} is not empty we identify Curve B as the curve in CurveSubset_{B} with the fewest parameters (lineU in the first example) and select Curve B as the reported curve.

When we report a pair of response curves for a gene, we require that each response curve in the pair yield a good fit to the data with RPE_{A} ≤ 10%. The curve reported for each gene in the pair is either Curve A or Curve B, whichever was selected via the curve selection algorithm.

In our application to frog data, the 5% RPE leeway agreed well with the sign of the change in AIC, where AIC = 7 log [Σ_{j in J}_{j}_{j}^{2}/7] + 2 × (number of parameters). Although this is a non-standard use of AIC because it applies to evaluation points instead of fitted points, it is still instructive. _{A} − RPE_{B} > 5%, correspond to AIC_{A} − AIC_{B} > 0 (the upper right quadrant). Most Curve B selected points, which require RPE_{A} − RPE_{B} ≤ 5%, correspond to AIC_{A} − AIC_{B} ≤ 0 (the lower left quadrant).

Comparison of a change in relative prediction error (RPE) with a change in Akaike Information Criterion (AIC) among response curve pairs. The red points corresponding to Curve A require RPE_{A}− RPE_{B} ≤ 5% (so are above the horizontal 5% line) The green points corresponding to Curve B require RPE_{A} − RPE_{B} ≤ 5% (so are below the horizontal 5% line). A value of AIC_{A} − AIC_{B} ≤ 0 (so on the left of vertical line) would indicate selection of Curve B.

We fit all models using iteratively reweighted least squares with modifications to incorporate starting values. Let

The flat curve has equation f_{FLA}(_{FLA}.

The linear curve has equation f_{LIN}(_{LIN} + _{LIN}·_{LIN} ≠ 0. Letting b_{LIN} denote the estimate of _{LIN}, we designated the linear model as lineD if b_{LIN} < 0 and lineU if b_{LIN} > 0, where D stands for downward and U stands for upward.

The sigmoid curve starts with a steady state and then monotonically increases or decreases and finishes with another steady state (_{SIG} and γ_{SIG} specify levels of the steady states. The parameter _{SIG} is the horizontal point corresponding to the maximum slope, _{SIG}, between the steady states. The sign of b_{LIN} is not always a reliable guide to the trend of the sigmoid curve, which we determine by simply comparing the first and last points on the sigmoid curve. We designated the downward and upward trending sigmoid curves as sigmoidD and sigmoidU, respectively.

The hockey stick curve is a sigmoid curve that is missing one steady state. We identified a steady state in a sigmoid curve as a slope at the beginning or the end of the curve that is less than or equal to 0.10, a value chosen based on visual inspection. We designated the downward and upward trending hockey-stick curves as hockeyD and hockeyU, respectively.

A transition curve is a sigmoid curve that is missing two steady states, leaving only the transition region between the missing steady states. We designated the downward and upward trending transition curves as transitionD and transitionU, respectively.

The impulse curve is one type of curve (along with the sigmoid and step curves) arising from the double sigmoid model. For flexibility, we fit one of two versions of the double sigmoid model, depending on the estimated slope of the linear model,

To avoid numerical problems, we only fit the double sigmoid model if the RPE of the sigmoid model was larger than the RPE of the linear model. Starting values come from the fit of the sigmoid model, namely _{dbs} = _{sig}, _{dbs} = (_{sig} + _{sig})/2, γ_{dbs} = _{sig}, _{dbs} = _{sig} , _{dbs} = 0, and _{dbs} = 0, where _{sig} and _{sig} are the estimates of α_{sig} and γ_{sig}, respectively.

The impulse curve has a peak or trough between steady states (_{dbs} and γ_{dbs} correspond to levels of the flat sections. For example with _{LIN}_{dbs} > 0, f_{DBS}(_{dbs}) × _{dbs} × _{dbs} = _{dbs} for small values of _{dbs}) × _{dbs} × γ_{dbs} = γ_{dbs} for large values of _{dbs} determines the level of the impulse. The parameter _{dbs}, which appears in each sigmoid factor, determines the slope of the peak or trough. Mathematically, we identified the impulse curve as a double sigmoid curve in which the minimum or maximum did not occur at the endpoints. We designated an impulse curve with a trough and peak as impulseD and impulseU, respectively.

The step curve is a double sigmoid curve with an intermediate plateau between steady states, although sometimes the steady states are missing. Mathematically, we identified the step curve as a double sigmoid curve in which both the minimum and maximum occur at the endpoints. Although this identification procedure would also detect a sigmoid curve, the sigmoid curve is preferentially selected via the sigmoid model. HoHWe designated the downward and upward trending step curves as stepD and stepU, respectively.

Sivriver, _{dbs} in Equation (3) by an additional parameter λ_{dbs}. We used the parameter estimates from the impulse curve as starting values with λ_{dbs} = 0. We identified the impulse+ curve as a generalized double sigmoid curve in which the minimum or maximum was not at the endpoints. We designated an impulse+ curve with a trough and peak as impulse+D and impulse+U, respectively.

We identified the step+ curve as a generalized double sigmoid curve in which both the minimum and maximum occur at the endpoints. We designated the downward and upward trending step+ curves as step+D and step+U, respectively.

We measured heterometry (HM) as the mean vertical difference between response curves expressed as a percentage of the vertical response range. We computed HM based on the following points on the response curves: (i) any point on the flat curve; (ii) the endpoints for linear, transient, hockey, sigmoid, or step and step+, curves; and (iii) the endpoints and the point at the peak or trough for impulse and impulse+ curves. For

Response curve pairs with at least five counts.

Total | Heterochrony only | Heterometry Only | Heterochrony and heterometry | ||
---|---|---|---|---|---|

sigmoidU | sigmoidU | 694 | 18 | 347 | 70 |

lineU | sigmoidU | 146 | 0 | 0 | 0 |

sigmoidU | hockeyU | 73 | 0 | 0 | 0 |

sigmoidD | sigmoidD | 48 | 3 | 22 | 0 |

lineU | lineU | 47 | 0 | 39 | 0 |

hockeyU | hockeyU | 30 | 0 | 18 | 1 |

sigmoidU | lineU | 20 | 0 | 0 | 0 |

lineU | hockeyU | 14 | 0 | 0 | 0 |

hockeyU | sigmoidU | 13 | 0 | 0 | 0 |

sigmoidU | impulseD | 9 | 0 | 0 | 0 |

impulseD | sigmoidU | 8 | 0 | 0 | 0 |

sigmoidD | lineD | 8 | 0 | 0 | 0 |

sigmoidD | hockeyD | 5 | 0 | 0 | 0 |

We measured heterochrony (HC) as the mean horizontal difference between response curves as a percentage of the horizontal response range. We computed HC based on the following points on the response curves: (i) the horizontal point at the maximum absolute value of slope for sigmoid, hockey, transition, step, and step+ curves; and (ii) the horizontal point corresponding to the peak or trough of the impulse and impulse+ curves. We did not compute HC for flat or linear curves. For

Of the 11,299 genes in the frog data, 10% of the response curves were good fits in both species of frogs (and used for the analysis) and 45% were good fits in only one species of frog.

Example of sigmoid curves for one gene pair.

Example of impulse curves for one gene pair.

Example of a hockey stick and line for one gene pair.

Example of a step curve and line for one gene pair.

Counts for response curve pairs with downward trends for both

flat | lineD | tranD | hocD | sigD | impD | stepD | imp+D | step+D | |

Flat | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

lineD | 0 | 3 | 0 | 0 | 1 | 0 | 0 | 0 | 0 |

tranD | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

hocD | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

sigD | 0 | 8 | 0 | 5 | 48 | 1 | 0 | 0 | 0 |

impD | 0 | 0 | 0 | 0 | 0 | 2 | 0 | 0 | 0 |

stepD | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

impD+ | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

stepD+ | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

Counts for response curve pairs with downward trends for

flat | lineU | tranU | hocU | sigU | impU | stepU | imp+U | step+U | |

flat | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

lineD | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

tranD | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

hocD | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

sigD | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 |

impD | 0 | 4 | 0 | 0 | 8 | 0 | 0 | 0 | 0 |

stepD | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

impD+ | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

stepD+ | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

Counts for response curve pairs with upward trends for

flat | lineD | tranD | hocD | sigD | impD | stepD | imp+D | step+D | |

flat | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

lineU | 0 | 0 | 0 | 0 | 1 | 2 | 0 | 0 | 0 |

tranU | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

hocU | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

sigU | 0 | 0 | 0 | 0 | 0 | 9 | 0 | 0 | 0 |

impU | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 0 | 0 |

stepU | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

imp+U | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

step+U | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

Counts for response curve pairs that are upward trends for both

flat | lineU | tranU | hocU | sigU | impU | stepU | imp+U | step+U | |

flat | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

lineU | 0 | 47 | 0 | 14 | 146 | 0 | 1 | 0 | 0 |

tranU | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

hocU | 2 | 4 | 0 | 30 | 13 | 1 | 0 | 0 | 0 |

sigU | 0 | 20 | 0 | 73 | 694 | 3 | 3 | 0 | 0 |

impU | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 |

stepU | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

imp+U | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

step+U | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

Our algorithm allows researchers to investigate heteromorphy, heterochrony, and heterometry of biologically relevant response curves in comparative gene expression studies. When the RPE is near a threshold, model selection can be ambiguous. For example some step and impulse curves are similar to sigmoid curves when the RPE for the sigmoid curve is close to the threshold for selecting the sigmoid curve. Also the distinction between hockey and sigmoid curves is not clear when the slope of the sigmoid curve at either the beginning or end is near the threshold for steady state determination. Therefore, when using this algorithm investigators should also examine the plots of the fitted curves.

To investigate how well our algorithm reduces overfitting (in the frog data), we also investigated polynomial models (with degrees three, five and seven) in addition to biologically relevant models. Because polynomial models have little biological rationale, there is no information in the responses at fitted times that is inherently relevant to the responses at evaluation times. For example a polynomial of degree seven would perfectly fit seven points, but that says little about how well the polynomial would interpolate or extrapolate to the evaluation points. Hence an algorithm that avoids overfitting would preferentially select biologically relevant response curves over polynomial response curves. This was, in fact, the case. We found that when we also fit polynomial curves, the algorithm yielded the same distribution of biologically relevant response curve pairs (

With modifications, it may be possible to reasonably apply this method to fewer than 14 time points. We used seven points so we could fit the seven parameters in the generalized double sigmoid model and used the remaining seven points spread evenly over the time range for evaluation. One approach for using fewer points is to simply not fit the generalized double sigmoid so that the similar double sigmoid is the most complex model investigated. Because the double sigmoid model involves six parameters, we would only need six time points for model fitting. A second approach, which can be used in conjunction with the first approach, is to use fewer evaluation points spread over the range of values at the “cost” of less information for discriminating between model fits.

To implement our algorithm we developed a set of Mathematica [

SGB was supported by the National Institutes of Health. The author thanks Leonid Peshkin for providing the data. This paper is published in accordance with the NIH Publication Policy.

The author declares no conflict of interest.