1. Introduction
Frequency response is a powerful technique to analyze the characteristics and properties of dynamical systems, describing the system response of a sinusoidal input with variable frequency. The system’s output magnitude and amplitude are both functions of the frequency; the output and input are then compared. The first frequency response technique (FRT) emerged in the 1930s during the development of the feedback amplifier stability [
1]. Later on, it involved transfer functions to produce Nyquist and Bode plots, which are mainly used to study feedback amplifiers. Meanwhile, its antagonist, the time response approach, involved ordinary differential equations with their associated algebraic equations to solve problems related to automatic control systems needed in mechanical, naval and chemical engineering during WW1 [
2]. However, as time went by, several FRTs were developed to solve problems in chemical engineering [
3] and physical chemistry [
4], among other areas.
In dynamical systems, when specific changes in the input (e.g., doubling the amplitude, two inputs summed, etc.) are reflected in the output spectrum (doubled amplitude, equivalent summed outputs, etc.), the systems are linear. For the specific case of time-invariant systems, the system response does not vary with time. Laplace transform is the method of solution of a linear system due to its simplicity by reducing ordinary differential equations to algebraic equations. For the frequency-response case, well-established stability criteria are important in the development of the frequency response methods in automatic control. To consult the classical FRT, the reader is referred to the collected works by MacFarlane [
2].
Opposite to the behavior of a stable linear system under a periodic input (where an output of the same frequency is produced), a nonlinear system output oscillates with different frequencies with non-periodic harmonics that are not multiples of each other. The behavior of a nonlinear system that is not stable can be more complicated, i.e., one that is not at equilibrium nor periodic with near-periodic oscillations that evolve to chaos. Even though the system is deterministic in nature, some of these chaotic outputs exhibit randomness. The nonlinear system’s output may exhibit several modes of behavior and may have more than one limit cycle if the system is unforced. Conversely, a forced system with periodic input may produce complicated steady-state outputs that exhibit harmonic or subharmonic behavior; the kind of output produced usually depends on the amplitude and frequency of the input. It is worthwhile to mention that in some cases, when the amplitude or frequency of the input is smoothly changed, the nonlinear system may even exhibit a discontinuous jump in the output mode [
2].
Among the methods proposed to solve the equations that arise in frequency response techniques, the Galerkin’s and Harmonic balance methods belong to the class of series solutions methods. Galerkin proposed a method to solve partial differential equations (PDE) that were too difficult to be solved analytically. This method turned out to be very useful to analyze complex PDE to find approximated solutions addressing the main concerns at the time: what residuals were to be used and how to be sure that the solutions actually converged. Galerkin was focused on the approximation to the solution by proposing a set of linearly independent test functions that are orthogonal to the residual. This method was tested first with a biharmonic equation and the results were published in 1915 [
5]. Because of the flexibility of Galerkin’s approach, nowadays most of the known methods fall in this category, adopting the name of the contributor who developed a particular application [
6]. The method of harmonic balance is another method belonging to the category of series solutions methods. In this case, the solution is proposed as the summation of exponential terms that capture the dependence with the amplitude, frequency and harmonics or sub-harmonics. Lately, this method has been used in several mechanical applications such as a bio-inspired work [
7] or the study of nonlinear vibrations of a thin, polar-orthotropic circular plate [
8]. The method of harmonic balance is a well-established method for the analysis of, for example, the power–frequency response to illustrate the effects of geometrical nonlinearity [
9]. However, in the literature, the application of the method of harmonic balance and other methods to analyze the behavior of nonlinear systems subjected to FRT has been mostly using numeric approaches to particular study cases (see, for instance, [
3,
10,
11,
12,
13]), and solving equations with the methods described previously can be challenging since the analytic approach may not be solvable and the numerical one does not provide an exact solution. To the best of our knowledge, there are no reported works on the general analytic solution for FRT.
In this paper, we present an analytic method for FRT with applications to nonlinear dynamical systems that is an improvement over the series solution methods available. This method provides a structured procedure to obtain information of nonlinear systems subjected to periodic inputs for the cases where the output is stable and others where the system is not at equilibrium. Hereby, we present some criteria to identify different modes of behavior such as the small amplitude oscillatory (SAOs) and the medium amplitude oscillatory (MAOs) that arise in several FRT applications. In the method reported here, we work with the invariant submanifold of the problem that is equivalent to the harmonic balance series solution. The recursive relation obtained for the coefficients in our analytical method simplifies traditional approaches to obtain the solution with the harmonic balance series method.
  2. Preliminary
Consider a pair of cascade interconnected systems of the form (see 
Figure 1).
      
      where 
, 
, 
, 
, and 
, 
 are locally Lipschitz in 
. 
 and 
 represent the time derivative of 
x and 
z, respectively. The state variables 
z of system (
2) can be seen as an input vector of system (
1); therefore, system (
2) is called the driving (or master) system of the tested (or slave) system (
1), which is considered to be input-state stable. According to the input-state stability theory of dynamical systems [
14], the tested system input-state is stable if for 
 the unforced system (
1)
      
      is asymptotically stable, i.e., that 
 as 
, and for a input 
 with finite supreme norm 
, it holds that
      
      where 
 is a class 
 function (a continuous function 
 is said to belong to the class 
 if it is strictly increasing and 
 [
15].) and 
 is a class 
 function (a continuous function 
 is said to belong to the class 
 if, for each fixed 
s, the mapping 
 belongs to class 
 with respect to 
r and, for each fixed 
r, the mapping 
 is decreasing with respect to 
s and 
 as 
[
15]).Then, given the properties of the class 
 function 
, it holds that 
, i.e., that the behavior of 
 synchronizes with the behavior of 
. This implies that the state variable, 
, reaches an invariant submanifold, 
, that depends exclusively on the behavior of 
, i.e., any trajectory 
 of the system (
6) is the image through the mapping 
 of a trajectory of the driving system. Note that the mapping 
 is in general an immersion, i.e., the rank is equal to the dimension of 
z that satisfies the partial differential equation
      
In particular, if the driving system (
2) is Poisson stable, i.e., its dynamical behavior has persistent oscillations with maximum amplitude 
A, then 
, and the tested system (
1) might also tend to a Poisson stable behavior in the invariant submanifold, with maximum amplitude bounded by 
. This result can be applied to the analysis of FRT, as shown in the following section.
  3. Frequency Response Analysis
Consider the input-state stable dynamical system
      
      where 
 is the state vector, while 
 is the input vector and 
 is a continuous function. In particular, in traditional FRT, 
z has a linear oscillatory behavior with the following dynamics
      
      where 
 is the frequency of oscillation. The behavior of 
 and 
 depends on the initial conditions and has the general form: 
 and 
, where 
A and 
 are the amplitude and phase of oscillation.
Notice that the oscillator (
7) and (
8) is Poisson stable and is a particular case of system (
2), while system (
6) is similar to system (
1) assuming input-state stability; therefore, 
 will reach an invariant submanifold, 
, that satisfies a partial differential equation analogous to Equation (
5), i.e.,
      
      where for the oscillator (
7) and (
8), in particular, the time derivative of 
 is
      
For instance, consider a simple pendulum swinging under gravity and frictional viscous damping, whose dynamical behavior can be represented by the following equation
      
      where 
 is the angular position, 
m is the mass concentrated in the tip, 
l is the length of the pendulum, 
g is the gravity acceleration, 
 is the friction coefficient and 
T is the applied torque. For simplicity purposes, let us define the following dimensionless variables 
, 
, 
, 
 and 
, then the dynamical model becomes
      
Notice that when 
, the steady-state is given by 
 and 
; therefore, 
, for 
, and given the stability properties, the only stable equilibrium points are 
, for 
. Then, if the pendulum is submitted to an oscillatory torque: 
, where 
 is the first element of the state vector of the oscillator (
7) and (
8), for small frequencies, after a transient behavior that depends on the initial conditions, the pendulum will approach to an invariant submanifold 
, which satisfies the following pair of nonlinear coupled partial differential equations: 
In the following section, a method is proposed to solve this class of equations.
  4. Analytical Matrix Method
One procedure to find periodic solutions of nonlinear systems is the method of harmonic balance [
15]. This method assumes that the solution can be represented by a linear combination of sinusoids (a Fourier series) of the form:
      where 
 is the main frequency with harmonics of the form 
, for 
. The problem reduces to find the frequency and Fourier coefficients 
. In the present work, an analogous procedure is proposed, namely, it is assumed that each element of the invariant submanifold, 
, that satisfies Equation (
9), has the following form
      
      where 
 are undetermined coefficients. Then, the problem reduces to find the correct value of these coefficients, that can be found using the quadrature method, where the proposed solution (
16) is substituted in Equation (
9) to reduce the problem to find coefficients 
. This solution can be rewritten in matrix form as
      
      where
      
      and
      
Now it is possible to state the problem to be solved in the present work.
Problem 1. Given the tested system (6) submitted to a oscillatory regimen by the driving system (7) and (8), after the transient period, system (6) reaches a invariant submanifold, , described by the invariance partial differential equation Then, the problem to be solved is to find an analytical series solution with the form (17) that satisfies Equation (20).  Notice that instead of using a harmonics series, such as Equation (
15), in the proposed series solution (
16), the driving variables, 
 and 
, are elevated to integer powers and the use of the matrix form (
17) allows simplifying the analysis and solution in a more systematic way. The following two definitions are instrumental for developing the proposed solution method.
Definition 1. Given two square matrices  and  with the same dimension  (N could tend to infinity), the product operator is defined as where the elements of matrix P are while E is a vector of dimension , whose elements are  Definition 2. Given the square matrix  of dimension , the differential operator is defined as where the elements of matrix D are while the elements of matrix P are defined in Equation (22).  To substitute the proposed solution (
16) in Equation (
9), it is necessary to compute the time derivative of 
; given that 
 is the sum of coefficients of the form 
, with 
, its time derivative is
      
      and considering the dynamics of 
z, given in Equations (
7) and (
8), it holds that
      
Notice that the sum of exponents of 
 is 
 and, given the linear dynamics of 
z, the sum of exponents of 
 is preserved and equal to 
. Following this structure, the time derivative of the infinite series (
16), or its matrix form (
17), is presented in the following Proposition.
Proposition 1. Consider the square matrix  of infinite dimension with constant coefficients, associated to a function of  and  defined as where vectors  and  are defined in Equation (18). The time derivative of  is where  is the operator described in Definition 2.
 Proof.  Notice that the derivative of vector 
 with respect to 
 is
        
        where
        
        is defined in Equation (
25). Therefore, the partial derivatives of 
 with respect to 
 and 
 are
        
On the other hand, notice that
        
        where
        
        is defined in Equation (
22); therefore, the time derivative of 
 is
        
Finally, with Definition 2, it holds that , concluding the proof.    □
 Equation (
6) contains nonlinear term; therefore, it might have products of finite or infinite series of 
 and 
. For instance, the product 
 where 
 can be expressed in matrix form as
      
      where
      
      while the procedure to compute the product of two series is presented in the following proposition.
Proposition 2. Consider the square matrices  and  with infinite dimension, associated to functions  and  of  and  defined as the product of these functions is where the operator  is described in Definition 1.
 Proof.   is a scalar function; therefore, Equation (
41) is equivalent to
        
        and considering Equation (
34), the previous equation is equivalent to
        
Notice that, using matrices 
, defined in Equation (
23), 
 can be expressed as
        
        therefore
        
        and finally, using Equation (
34) again, it holds that
        
From here it is possible to define matrix
        
        as described in Equation (
21), in order to rewrite Equation (
43) as
        
        concluding the proof.    □
 Finally, the following theorem presents the conditions to find the oscillatory invariant submanifolds.
Theorem 1. Given the input-state stable differential Equation (6), with  and , and the oscillatory dynamics defined in Equations (7) and (8), assume that  reaches an invariant submanifold that satisfies Equation (20). Then, if function  can be expressed as series of integer powers of the elements of x and z, the solution of each element of vector π can be proposed to be an infinite series of integer powers of  and , i.e.,  Proof.  The system of differential equations defined in Equation (
20) is
        
Assume that the elements of the invariant submanifold 
 can be described as in Equation (
45), then, according to Proposition 1, their time derivatives are
        
        where 
. Using this time derivative, the elements of Equation (
20) become
        
        for 
. If functions 
 can be expressed as series of integer powers of the elements of 
x and 
z, then using identities given in Equations (
37) and (
42) in Proposition 2, it can be shown that functions 
 are equivalent to
        
        and the differential equations can be transformed to the form
        
        where 
 is a matrix that depends on the values of coefficient matrices 
, defined in Equation (
46). Independently of the values of vectors 
 and 
, Equation (
47) holds if
        
        then the problem reduces to finding coefficients 
, 
, 
, by solving Equations (
48) to obtain the solution given in Equation (
45).    □
 By looking at invariant Equation (
47), we can infer that the matrix Equation (
48) depend on the applied frequency 
; therefore, the values of coefficient matrices 
 also depend on this frequency. Once these coefficients (
46) have been determined, it is possible to analyze the system’s behavior in the frequency domain. Since 
 and 
, then
      
Notice that the trigonometric properties: 
, 
, and 
, allow verifying that solution (
49) is equivalent to the proposed solution of the method of harmonic balance (
15). On the other hand, the series (
49) is convergent if the amplitude is small enough in comparison with the values of coefficients 
 that are fixed by the applied frequency. Therefore, we can define the following oscillations modes:
Definition 3. Consider the norm of the coefficients of series (49) with the same order: . The possible oscillations modes are the following: -  SAOs 
- If  for  and , then system (9) is in Small Amplitude Oscillations mode (SAOs); therefore, the terms with order two or higher are negligible, and solution (49) reduces to . 
-  MAOs 
- If  for , then system (9) is in Medium Amplitude Oscillations mode (MAOs) and additional terms of higher-order in solution (49) must be included to describe its oscillatory response correctly. 
-  LAOs 
- Finally, if the series is not convergent, i.e.,  as n increases, then system (9) is in Large Amplitude Oscillations mode (LAOs) and solution (49) is not longer suitable to describe the behavior of system (9). 
 Since the current contribution is focused on the analysis of SAOs and MAOs modes, the LAOs analysis will be left to a future contribution.
  6. Discussion
For the study case of the series solutions for a simple globally asymptotically stable model, 
Figure 2 shows that the analytical solutions truncated up to the first-order and third-order are unable to reproduce the behavior, while the analytical solution truncated up to seventh-order satisfactorily describes the system behavior. Notice that, by looking in Equation (
54) at the coefficients of the series and considering that 
 and 
, it holds that the first-order term (
) is proportional to 
, while the third, fifth and seventh-order terms (
) are proportional to 
, 
, and 
, respectively. Following this sequence, the terms of order 
, for 
, are proportional to 
. Thus, the convergence of series (
54) is correlated to the convergence of coefficient 
 with respect to 
k, then, without considering the constant coefficients in series (
54), an approximated limit to guarantee the convergence of series is that the relation frequency–amplitude must satisfy 
; if this relation is satisfied then we can say that the system is in SAOs mode when 
 or MAOs mode when 
, respectively. In 
Figure 2, 
 and 
 were selected, which satisfies the aforementioned relation, while 
Figure 5 shows the predictions for the same frequency but with an amplitude 
, which violates the relation 
, and therefore, the analytical solution Equation (
54) is no longer suitable for describing the dynamical behavior because, in this context, the system is in LAOs mode. To make the analysis in LAOs mode, it would be necessary to reconsider the assumption 
 made to solve Equation (
A1), which actually has an infinite number of solutions of the form 
; therefore, if 
, it would appear a term of the form 
 that is equivalent to 
. Therefore, a new constant term would appear, modifying the equation 
 of system (
53), i.e., instead of 
, it would be 
, and now 
 could be different to zero and a function of the amplitude. The complete analysis of LAOs mode will be left to a future contribution.
Notice that for the well-known problem of a simple pendulum that oscillates under sinusoidal potentials, the resonance characteristics are difficult to find by solving the problem analytically; hence, the numerical approach is the method of choice [
16]. However, we were able to find the analytical solution for the study case of the series solutions for a simple pendulum. We found that when the system behaves almost as in the SAOs regime (
Figure 3), there may be deviations from the linear behavior and additional third-order terms must be included to accurately reproduce the system’s behavior.
Figure 4 depicts the system in a MAOs regime with more pronounced nonlinear characteristics, where coefficients up to eleventh order must be included to correctly reproduce the system’s behavior.
 Figure 6 shows that for 
, 
 becomes constant and the steady-state solution of Equations (
11) and (
12) is given by the equations 
 and 
; therefore, 
, where 
n may be equal to 
. However this solution is only feasible if 
, because given the domain of sine function, if 
, this class of steady-state is not longer feasible and the pendulum reaches a permanent regime where 
 keeps growing, while 
 oscillates around a constant value, i.e., the pendulum rotates persistently. Consequently, solution (
60) is not longer valid to describe this behavior, because according to Theorem 1, the system described in Equation (
12) is not longer input-state stable. An example of this behavior is shown 
Figure 6. In this case, the system is not longer in a SAOs or MAOs regimen.
 To analyze the approximated regions in which SAOs, MAOs, or LAOs hold for the angular position of the pendulum, it is possible to use Definition 3 in the coefficients 
, given in Equation (
A6). 
Figure 7 sketches these regions as a function of the applied frequency and amplitude. In particular, the boundary between SAOs and MAOs regions was selected to have a maximum 1% variation with respect to the linear behavior, i.e., where 
 and 
 for 
, while the boundary between SAOs and MAOs regions approximates the points at which solution (
60) is not longer suitable to describe the oscillatory behavior because 
 for some 
. As can be seen, for low frequencies and for amplitudes approximately bigger than 1, the pendulum is in LAOs mode, while for high frequencies the SAOs mode predominates regardless of the amplitude. Particularly for frequencies around 1, the boundaries between the three regions appears at smaller amplitudes, because this frequency is the natural frequency of the pendulum (given the numerical value of the damping parameter, the eigenvalues of the Jacobian around equilibrium are approximately 
), and the driving force is producing larger resonances. Another resonance region is approximately at frequencies between 0.1 and 0.4, associated with the damping constant 
. Finally, notice that the diagram in 
Figure 7 has some similarities with the typical amplitude ratio Bode plot for a second-order damped system, which for comparison purposes was superimposed on the diagram (yellow line), suggesting that this class of diagram may be a valuable tool for frequency response techniques of nonlinear systems.