Current Status Investigation and Predicting Carbon Dioxide Emission in Latin American Countries by Connectionist Models

: Currently, one of the biggest concerns of human beings is greenhouse gas emissions, especially carbon dioxide emissions in developed and under-developed countries. In this study, connectionist models including LSSVM (Least Square Support Vector Machine) and evolutionary methods are employed for predicting the amount of CO 2 emission in six Latin American countries, i.e., Brazil, Mexico, Argentina, Peru, Chile, Venezuela and Uruguay. The studied region is modelled based on the available input data in terms of million tons including oil (million tons), gas (million tons oil equivalent), coal (million tons oil equivalent), R ew (million tons oil equivalent) and Gross Domestic Product (GDP) in terms of billion U.S. dollars. Moreover, the available patents in the ﬁeld of climate change mitigation in six Latin American countries, namely Brazil, Mexico, Argentina, Peru, Chile, Venezuela and Uruguay, have been reviewed and analysed. The results show that except Venezuela, all other mentioned countries have invested in renewable energy R&D activities. Brazil and Argentina have the highest share of renewable energies, which account for 60% and 72%, respectively.


Introduction
The continuous rise of carbon dioxide emissions is a major cause of global warming. Global warming is one of the biggest and probably most difficult environmental, social and economic threats that the world has faced so far in the recent century [1]. Increasing carbon dioxide emission into the atmosphere is one of the main reasons for global warming with adverse environmental effects such as sea level rise, floods, droughts, etc. During the 20th Century, the average temperature of the Earth increased by 0.6 degrees, and it is estimated that it will increase 1-5 more degrees in the next century [2]. According to the studies conducted by the World Meteorological Organization, the average global temperature in 2015 was the highest one ever recorded. Based on research, a combination of El Niño streams and global warming resulting from human activities has led to the highest recorded The major distinction that gives superiority to renewable energy is the lower amount of GHGs (greenhouse gases) emission in comparison to typical fossil fuels. Different techniques and equipment are employed in energy systems for electricity production or other possible energy conversion targets such as heating or cooling, etc. [18][19][20][21][22][23].
Geothermal, solar and wind are the most practical sources of renewable energies, which have been developed vastly in recent years [24][25][26][27][28][29][30][31][32]. These sources are utilized in various applications such as power production, proving heating loads, freshwater production systems, etc. [33][34][35][36][37][38]. The amount of GHGs emission can be reduced by either increasing the portion of renewable energies in the total energy consumption or by improving the operational efficiency of energy-conversion systems. Various investigations have concentrated on optimization and enhancement of energy-conversion systems to reach more desired operating statuses [39][40][41]. One of the powerful tools that is highly utilized in the modelling process of the energy-conversion systems is machine learning. A recent method based on machine learning was introduced, Least Squares Support Vector Machine (LSSVM), which has been vastly used in modelling of engineering systems [42][43][44][45][46][47][48][49][50][51].
Applicable and fit data are required to have an accurate model. In this investigation, the input of the designed model was fuel consumption (i.e., oil, natural gas, coal and possible renewable energy sources) of countries in South America and the GDP of each studied country in the period from 1990-2016. The amount of carbon dioxide emission in each year was defined as the output of the study. Here, all of the influential items that were considered by other researches are gathered in order to give an inclusive model. Moreover, the most highlighted novelty of this study is proposing a mathematical model to estimate the amount of carbon dioxide emission of different countries in the region of South America and Mexico. The precision of the introduced model is examined and evaluated by statistical methods including relative error and R2 (R-squared).
Furthermore, patents can provide visual information about the future of the technology and the amount of investment in a policy. Accumulated numbers of published patents by a country not only determine the industrial orientation, but also demonstrate the future program of that country in a specific domain. Therefore, the mentioned countries are investigated in terms of the published patents in the technologies for mitigating greenhouse gas emission. In this paper, all patents that have been published by Brazil, Chile, Argentina, Uruguay, Mexico, Peru and Venezuela are extracted and analysed. Finally, the top five published patents by the mentioned Latin countries for mitigating CO 2 emission technology are indicated.

Methodology
Based on Ahmadi et al. [52,53], applying some non-population optimization methods such as Levenberg-Marquardt (LM) and Simplex Simulated Annealing Algorithm (M-SIMPSA) is not practical since these approaches are not able to handle the SVM techniques since the nonlinearity factor is high. Therefore, GA, which is a population-based optimization method, was employed in order to assess the two major factors of γ and σ 2 . In addition, a fitness function was used in the external optimization process, i.e., the Mean Squared Error (MSE) of estimated data. In order to specify the most optimum result of the fitness function, the optimization process was repeated several times. Here, the LSSVM machine learning technique was hybridized with GA, PSO, ICA and GAPSO to model an accurate estimation of CO 2 production in Latin American countries. Each methodology is briefly discussed in the following.

Least Squares Support Vector Machine
Suykens and Vandewalle introduced LSSVM for the first time in 1999 and applied it to the primary kind of SVM, in order to specify the regression and function. Over-fitting issues were recognized as a challenging situation when dealing either with conventional SVM or feed-forward neural networks. Therefore, the LSSVM is proposed to surmount this specific barrier. Here, Xi, Rew, and GDP were defined as the input values of the problem, and Yi was the objective output. The time series of Xi consisted of oil (million tons), gas (million tons oil equivalent) and coal (million tons oil equivalent). Rew and GDP were measured as million tons oil equivalent and in billion 2005 U.S. dollars, respectively. Overall, the LSSVM nonlinear function can be demonstrated as follows [54][55][56][57][58][59]: (1) f illustrates the relation between input variables and the objective function of CO 2 production (million tons). Input variables were oil (million tons), gas (million tons oil equivalent), coal (million tons oil equivalent), R ew (million tons oil equivalent) and GDP (billion 2005 U.S. dollars). w represents the weight vector (m-dimensional); φ maps x into the characteristic vector (m-dimensional); and b states the bias [54][55][56].
Employing the Lagrange multipliers α i in order to change the previous limited problem to an unlimited problem is a strong and efficient tool to determine the optimization problems that are stated in Equation (2) [55][56][57][58][59]: As stated in the investigation of Karush-Kuhn-Tucker (KKT), considering the stated variables including w, b, e and α and also performing the partial derivatives of Equation (4), the optimum condition can be expressed as follows [55][56][57][58][59]: Therefore, the linear equations are specified as [55][56][57][58][59]: In Equation (5), Y, Z, I and α are defined as: . . , 1] and α = [α 1 , . . . , α 1 ], respectively. By applying the kernel function of K(X, X k ) = φ(X) T φ(X k ), i = 1, . . . , m, the discussed LSSVM regression is formulated as follows [55][56][57][58][59]: The Radial Basis Function Kernel (RBFK) is typically utilized in regression errors. This function is stated as follows [55][56][57][58][59]: In Equation (8), σ 2 indicates the squared bandwidth. This parameter should be calculated during any optimization procedure, such as the GA optimization method, in order to have a potent optimization approach. The Mean Squared Error (MSE) of the results of the LSSVM optimization approach is defined as the target function. The MSE of the results of the LSSVM method can be expressed as follows [60][61][62][63][64][65][66][67]: where CO 2 illustrates the amount of CO 2 emission (million tons) and subscripts rep/pred and meas denote predicted and measured amount of CO 2 emission, respectively. The number of samples from the original population is indicated by ns. In this investigation, the evolved model of the LSSVM method, which was previously implemented by Pelckmans et al. [59] and Suykens and Vandewalle [60], was applied. In general, Equation (10) simply states the optimization issue:

Genetic Algorithm
The preliminary stage of the Genetic Algorithm (GA) procedure is to form the primary population. Then, every stage is precisely assessed to form a statistical fitness function. In the following, every stage should be evaluated to be compatible. The "global best satisfactory" result, which is the result with a range of acceptable error, ended the algorithm, and then, the parameters were extracted and reported. When the global best satisfactory result was not achieved, weaker individuals were opted to be deleted in the next stage. In the next stage, in order to decrease the error, mutation and cross-over operations were applied. This stage is then called the "evaluation fitness" [68][69][70][71]. The process of the genetic algorithm LSSVM is demonstrated in Figure 1.

Particle Swarm Optimization
At the commencement of Particle Swarm Optimization (PSO), pattern arbitrary locations and velocities are used to initialize the primary population. The particle's fitness is then performed by utilizing the statistical function. The desired factors are extracted, when the particle's fitness rate meets the stopping norms. In the cases that the desired fitness rate is not achieved, in order to fulfil the requested rate, the particle's velocity and locations are varied under specific conditions. In the first step, the global best factor should be evaluated and used to monitor how the particle fitness is in comparison to the global best. An update to the pertinent factors of the prime particle is needed when the fitness is greater than the global paramount fitness. At last, subsequent particles should be assessed again by changing the direction to the 2nd stage [73][74][75][76]. The schematic process of the PSO-LSSVM is illustrated in Figure 2.

Hybrid PSO and GA
This hybridization is started by arbitrarily forming the primary population and its following assessment. The process is stopped when the regulated values of the amount of produced error of the best individual is achieved. If the regulated values are not met, then the procedure continues to reach toward the closest acceptable range by utilizing the PSO algorithm. In this algorithm, the number of leaders are increased, tournament selection is carried out based on the chaining processes, mutations and cross-over operations are applied and, finally, the fresh offspring are generated. New populations with improved characterizations of elites and offspring are generated by merging the effects of these two methods with each other [77,78]. The box chart of the Hybrid GAPSO-LSSVM method is shown in Figure 3.

Imperialist Competitive Algorithm
Imperialist Competitive Algorithm (ICA) follows a similar process as GA to solve optimization problems. However, ICA is based on a human socio-political organization and evolution strategy instead of the biological evolution of GA. ICA creates several social domains, transferring a number of colonies to their related domains. Then, the colony's cost of an empire should be evaluated. A variation in the locations of domains and colonies is needed when the colony's cost is high for a specific domain. After that, the cost of all domains is computed in the next step. In the following phase, the greatest colony of the greatest domain owns the colony with the lowest potential. Afterwards, all domains without colonies should be cut. The next level is to examine the breaking sets to monitor the amount of satisfaction to cease the algorithm [79,80]. The schematic diagram of the ICA-LSSVM method is demonstrated in Figure 4.
The LSSVM Lab 1.8 free toolbox and GA Toolbox of MATLAB R2009a were used to optimize the GA-LSSVM problem. In addition, other optimization techniques including ICA, PSO and hyper-PSOGA were coded and run in MATLAB software in order to determine the hyperparameters of the LSSVM. When the dataset was identified and collected, the model was formed and the input parameters into the model and the target output of the model were specified. Based on the reported data from the mentioned literature, the amount of carbon dioxide emission (million tons) was selected as the output of the LSSVM model and forecasted.
The collected real data were classified into two subclasses: 80% of the total data, which was 68 data points or 55 data lines, were employed in the training process; the other 20% of the data established the test and validation process to verify the recommended method.
The RBF was opted for this problem, since there were few variables in this optimization problem and also the 1st-rate total performance of the RBF kernel function. Based on the literature, the RBF is in agreement with other kernel functions and also is highly practical [81,82]. In the process of modelling the problem utilizing the LSSVM with RBF kernel function, the feature of parameters γ and σ 2 is a fundamental task (see Equation (8)). These parameters have a noticeable role in verifying the acceptance of the LSSVM approach used. Here, the regularization factor is indicated by γ, and σ denotes the kernel sample variance [83] (see Figure 5).

Patent Analysis
Is there any helpful technology for mitigating greenhouse gas emission? What are the technological roles of South American countries in mitigating greenhouse gas emission? One of the reliable approaches to denote the technological profile of countries is patent analysis. Patents have a technical value, determining the market potential [84,85]. The economics of a patent can be an indicator of the market tendency of that technology. The patent provides useful information for the business developer, policy makers and researchers. To extract this information and visualize it, analysing the patent is required [86]. Patent analysis, besides showing the technology trend, can demonstrate countries' orientation on the specific technology sector. The total published patents in the specific technology by a country may indicate is movement in that specific industry. Additionally, one of the interesting results of the analysis of the patent can be the indication of a hot area in technology. Indeed, the specific groups of technology used by a country are found by this analysis. Therefore, what are the technological approaches of these countries for mitigating CO 2 emission? Have they matured in their technology? To estimate their technological approaches, investigating patents could be helpful. To find the technology status, the technological life cycle is curved. As was said earlier, patents were used as data for this investigation. Patents can be assumed as one of the most reliable indexes for estimating technology development. Patents, by stimulating the economy and technological innovation, motive firms to develop the technology [79,81]. Patents play a climatic role in highlightingbetter technology life cycles and each step through innovation, technology development to market growth [78]. The data were extracted from the SaaS online database, Patentispiration (Patentispiration.com). Finding the best approach for mitigating CO 2 emission technologies that are employed by the mentioned countries was determined by investigating the obtained patents. Thus, the exactness of the extracted patent is the key factor of this investigation. Since searching patents by keywords may not be an efficient method, the combination of keywords and patent codes significantly increased the data accuracy. In the first place, interviewing of experts was done to refine the main keywords. Then, the patents were searched by keywords, which are shown in Figure 6. These keywords were candidate from the result of interviewing. After searching by keywords, the extracted patents were screened to find key codes. Key codes refer to CPC (Cooperative Patent Classification), which organizes the patents into nine classes. Searching patents by codes averts extracting irrelevant patents; therefore, this approach certifies the accuracy. For better insight into the technology categories, the hierarchy of CPC of the patent survey is presented in Table 1.

Results and Discussion
The values of σ 2 and γ for the studied methods including GA, PSO, ICA and hybrid GAPSO were obtained and are listed in Table 2. In addition, sensitivity analysis was performed on the numbers of the introduced digits of the mentioned variables. The predicted obtained values through the GA-LSSVM model in comparison to the real amount of measured carbon dioxide emission for South America is depicted in Figure 7a. In order to obtain the best linear fit line between the predicted and real amount of CO 2 emission, a correlation coefficient of 0.9965 resulted in the GA-LSSVM. The closeness of the coefficient to the unit demonstrates that the forecasted and measured data points were similar. The scatter plot of the amount of forecasted CO 2 emission through PSO-LSSVM vs. the measured data is illustrated in Figure 7b. Applying the PSO-LSSVM method resulted in the correlation coefficient of 0.9957 for the best linear fit line between the predicted and measured values of CO 2 emission. It can be seen than the calculated correlation coefficient was closer to unity in PSO-LSSVM than the GA-LSSVM method, and it can be stated that the precision of PSO-LSSVM model was higher and the estimations close enough to the measurements of the carbon dioxide emission. The regression plot of the obtained results from the HGAPSO-LSSVM method and related actual amount of carbon dioxide emission is demonstrated in Figure 7c. The regression of the predicted results through the ICA-LSSVM model in comparison to actual measured data of the carbon dioxide emissions is depicted in Figure 7d. As can be clearly understood, based on the values of the correlation coefficient, the ICA-LSSVM approach gained the lowest amount among all other models, including PSO-LSSVM, GA-LSSVM and HGAPSO-LSSVM.
It was concluded that among all the discussed methods, the HGAPSO-LSSVM technique was more accurate based on the obtained correlation coefficients. In addition, the relative deviation of the calculated results through the discussed methods vs. corresponding oil production (million tons) is demonstrated in Figure 8. It can be seen from Figure 8a that the obtained results from the GA-LSSVM model deviated from actual measured carbon dioxide emission values by 20%. This means that the deviation of the obtained results by the GA-LSSVM model from the real measured amount of carbon dioxide was about 15%. It can be understood from Figure 8b that the deviation of the obtained results through the PSO-LSSVM model was in the range of 22%. It can be stated that the error of estimations from the PSO-LSSVM model was between the error lines of −22% and +22%. The relative error graph of the estimations based on the HGAPSO-LSSVM model in comparison to the actual measured data to carbon dioxide emission is depicted in Figure 8c. It can be clearly seen from Figure 8c that the highest error of the HGAPSO-LSSVM model was in the range of 12%. The relative deviation results of ICA-LSSVM model are shown in Figure 8d. It can be seen from Figure 8d that the highest deviation of the ICA-LSSVM approach from actual CO 2 emission was −22%. The relative error distribution shows that the HGAPSO-LSSVM method had dominance in comparison to the other discussed LSSVM models. Overall, two dependable statistical benchmarks including Average Absolute Relative Deviation (AARD%) and MSE were applied to determine powerful intelligent techniques among the recommended LSSVM models.   Figure 9a. In addition, Figure 9b depicts the comparison between the resulting MSE of different LSSVM models. It can be seen from Figure 9 that the HGAPSO-LSSVM technique is an accurate and precise method for estimating CO 2 emission in gas injection processes.  The obtained programming codes and the required instructions are straightforwardly accessible and will be shared readily with others. Therefore, people can easily use the model to re-calculate all of our results and may predict carbon dioxide emission for any circumstances.
The patent analysis result of the technology trend for mitigation of CO 2 shows that Brazil with 5885 patents is the leader in mitigation of CO 2 emission technology so far. All the published patents of the South American countries are illustrated in Figure 10. Mexico has published 2644 patents and after Brazil is the second pioneer country. According to the extracted data, which were patents in this case, only 101 patents have been published by Uruguay. The trends of published patents by these countries from 2007-2017 are illustrated in Figure 11. What can be noted in the countries' trends is that Chile and Peru have accelerated working on mitigating technologies. The top patent main technology groups that indicate movement of these countries to mitigate CO 2 emission will be discussed further. The country with a modest behaviour would be Venezuela. Furthermore, Mexico can be considered as the next most modest country in published patents. In fact, the published patents by country in the specific technology determines R&D activities and expenditure directed toward mitigating CO 2 emission. Brazil and Mexico have the biggest economies in Latin America, which determines their remarkable role in CO 2 mitigation. Although the CO 2 emission from Brazil and Mexico is not the highest one, their policy and program to enhance mitigation action are considered globally [87]. Argentina has the fourth biggest economy in Latin America. In spite of being a relatively small CO 2 emitter, Argentina has made efforts to mitigate CO 2 emission recently. Peru is one of the countries that ratified the Paris agreement. The annual rate of CO 2 emission of Peru in 2016 was 2.67%, which determines 0.68 metric tons CO 2 , increasing from 1997-2016 [88]. As is obvious in Figure 11, Uruguay patent trends have not increased or faced a considerable change during the last decade. About 80% of CO 2 in Uruguay was produced by the agriculture sector in 2013 [89]. According to its GDP, Venezuela has the seventh largest economy in Latin America. The main source of CO 2 emission is energy in Venezuela. The total produced CO 2 by Venezuela in 2014 was 6.03 metric tons [89]. Chile is another important Latin American country with a high GDP. The total CO 2 emission by Chile in 2016 was 5.45 metric tons [90]. To find out the technological orientation of the mentioned Latin countries, the main technology groups of each country's patents are illustrated by Figure 12, which considers the technologies for mitigating CO 2 emission published as patents by the mentioned Latin American countries, except Uruguay. The reason for considering Uruguay as an exception is the low number of patents. Only 45 patents have been published by this country, which demonstrates that the major activity by Uruguay is to promote renewable energies. Being a developing country and less industrialized, and not significant, does Uruguay have an influence on CO 2 emission? However, the approach of Uruguay to reduce CO 2 is promoting renewable energies. The energy sector in Uruguay contributed more than 90% of CO 2 emissions, and the ambition is to reduce this by 29% by 2025 [91].  Figure 12a demonstrates the top five patent technology scopes that have been published by Brazil since 1970. Energy generation through renewable energy resources and technologies for the production of fuel of non-fossil origin, which is 60%, is the lion's share of technologies for mitigation of CO 2 emission. Production of fuel from non-fossil fuel origin includes biofuel technologies. One of the programs for achieving 10% reduction of greenhouse gasses by 2028 is Brazil's biofuel mix stimulation from the current 20% to 28.6% [92]. In fact, Brazil's key strategies include a deforestation action plan and developing renewable energies. The other approach in order to reduce CO 2 emission involves the industrial sector: modifying the efficiency of processes [93]. Mexico as the second biggest economy in the region, besides developing a renewable energies program, concentrates more on specific regulatory tools in the industrial sector. As is obvious in Figure 12b, only 13% of the top five technologies are allocated to energy generation through renewable energy resources. Mexico is the thirteenth largest steel producer in the world, which produces more than 14% CO 2 [94]. For that reason, the four other areas of technologies in which more efforts are made in the R&D activities are the metal and related chemical industries. Therefore, the main program for mitigation of CO 2 emission in Mexico would be improving the energy efficiency. Regarding Figure 12c, which demonstrates the technology sector in Argentina with respect to published patents, energy generation through renewable energy resources accounts for more than 70% of the patents. Road transport of goods or passengers and arrangements for handling mechanical energy structurally associated with dynamo-electric machines are the next highlighted ones. Their target that the net emissions must not exceed more than 483 MtCO2eq is divided into three main sectors, which are energy, forest and transport. The main scenario for Argentina to mitigate CO 2 is renewable energies' promotion [95]. Indeed, developing renewable energies has gained the majority of funds in Argentina through the renewable energy auction program of Argentina (RenovAr). As a result of the first round of this program, the total wind and solar capacities are 3469 MW and 2813 MW, respectively [96]. The top five patent technologies that have been published by Chile from 1970-2017 are illustrated in Figure 12d. Being the world's largest exporter of copper [97], the main published sector of technology by Chile is the industrial sector. Although expanding the renewable capacity and transition from coal to non-conventional resources for energy supply comprise the main action plan of Chile in order to reduce CO 2 emission, enhancing the efficiency of industrial processes has received high attention in their R&D activities. Regarding Figure 12e, Peru's technology sectors for published patents are somewhat similar to Brazil. The energy generation through renewable energy sources accounts for 36%, while the four other sectors have the same shares when combined into two groups. Forestry, energy and agriculture are the three main greenhouse gas sources in Peru. The main energy sources in Peru for electricity and transportation are gas power plants and imported oil and gasoline. Renewable energy can be the best opportunity for addressing electricity demand, and the transportation sector demands can be met by low-carbon technologies [98]. The auction plan for developing renewable energies by Peru is illustrated in Figure 13. Despite other mentioned Latin American countries, in the top five published patents technology sectors, there is no share of energy generation through renewable energy sources in Venezuela in the published patents. The lion's share of the technology sector in the published patents by Venezuela is technologies related to metal processing, which account for 36%, according to Figure 12f.

Conclusions
In this investigation, two artificial intelligence techniques were discussed to estimate the CO 2 emission. The LSSVM and evolutionary methods have been used to predict the CO 2 emission precisely. The information used here was gathered from the accessible data in the literature. A comparison was made between the estimated amounts of CO 2 from the optimization techniques and the measured actual data of carbon dioxide emission. Based on the recommended methods, the following major conclusions have been extracted:

1.
All of the discussed optimization approaches showed agreement for forecasting the amount of carbon dioxide emission. However, HGAPSO-LSSVM demonstrated a more accurate result and showed a higher reliability and compatibility. In addition, under specific circumstances with restricted field information, the significance of these methods was highlighted more than other predicting techniques.

2.
The HGAPSO has the potential to be integrated with other evolutionary algorithms in order to optimize its parameters and additionally enhance its strength and accuracy.
Moreover, the patents related to climate change mitigation were evaluated in six Latin American countries, namely Brazil, Mexico, Argentina, Peru, Chile, Venezuela and Uruguay. The results showed that, except Venezuela, all other mentioned countries have invested in renewable energy R&D activities. Brazil and Argentina have the highest share of renewable energy, which accounts for 60% and 72%, respectively. Author Contributions: All authors contributed equally to the paper.
Funding: This research received no external funding.