Abstract
Energies of the atomic and molecular orbitals belonging to one and two atom systems from the fourth and fifth periods of the periodic table have been calculated using ab initio quantum mechanical calculations. The energies of selected occupied and unoccupied orbitals surrounding the highest occupied and lowest unoccupied orbitals (HOMOs and LUMOs) of each system were selected and used as input for our artificial intelligence (AI) software. Using the AI software, correlations between orbital parameters and selected chemical and physical properties of bulk materials composed of these elements were established. Using these correlations, the materials’ bulk properties were predicted. The Q2 correlation for the single-atom predictions of first ionization potential, melting point, and boiling point were 0.3589, 0.4599, and 0.1798 respectively. The corresponding Q2 correlations using orbital parameters describing two-atom systems increased the capability to predict the experimental properties to the respective values of 0.8551, 0.8207, and 0.7877. The accuracy in predicting materials’ bulk properties was increased up to four-fold by using two atoms instead of one. We also present results of the prediction of molecules for materials relevant to energy systems.
1. Introduction
The volume of information that is available for any given chemical or physical system is becoming increasingly unmanageable. We are literally in danger of being “swamped” by this profusion of available data. Artificial intelligence (AI) is one of the only techniques by which we have any chance of sorting through this voluminous amount of data and extracting meaningful information about the system under study. An artificial neural network is one of the more powerful techniques of artificial intelligence. Being able to predict a material’s bulk physical characteristics before attempting synthesis would be very advantageous. By choosing a target bulk property, a researcher can determine combinations of atoms yielding that desired property using predictions based on small nanocluster systems. Predicting the physical properties of molecular systems would also be very beneficial, especially in systems relevant to energy production. Most molecules and atoms can be modeled and optimized with software such as Hyperchem 5.01 [1] and the Gaussian09 [2] quantum mechanical program. The scope of this research is to use the output data from a program such as Gaussian09 to train the artificial intelligence (AI). The training set consists of an input calculated with Gaussian09 and selected bulk properties as the output. This training set is used to teach our AI software correlations between the calculated input parameters and the bulk properties of a material. Once a correlation is found, the AI is able to predict the bulk material properties from any atom or combination of atoms composing the material, provided the energies of the atomic or molecular orbitals are known from ab initio SCF/DFT calculations.
2. Artificial Intelligence
Artificial intelligence (AI) via artificial neural networks (ANNs) consists in parallel types of computing systems. The main structure of ANNs is a highly interconnected network of simple processing elements known as nodes. A popular configuration for a supervised feed-forward back-propagating neural network is three main layers. The first layer, called the input layer, receives all of the input data. Next, there is a hidden layer; its function is to perform the desired mapping between the input and output layers. Finally, there is the output layer, where the network gives its predicted value. There are generally multiple input nodes connected to multiple hidden nodes connected to one or more output nodes (see Figure 1). In our study, we use the lowest 20 unoccupied molecular orbitals and the highest 20 occupied molecular orbitals that surround the orbital gap. Each input node’s value is adjusted by a weight before reaching the hidden layer where all products of inputs multiplied by weights are summed. This sum is transformed by a nonlinear activation function, which scales the summation to an appropriate nodal value. If the hidden node’s value is less than a specified threshold value, it will remain inactive and does not contribute to the next layer. However, if the hidden node’s value is greater than the threshold value, it will be adjusted by another weight and summed with all other adjusted hidden nodes that met the threshold criteria. The new summation will be transformed to represent the output layer, where the predicted value is compared to the correct value. These errors are back-propagated, before weight adjustment, through the network to the input layer. This forward and backward training process continues until a convergence is achieved. It is this feature of propagating the error backwards through the network which is responsible for the network being referred to as a backpropagation neural network.
Figure 1.
Architecture of an ANN backpropagation network with three active layers.
Our AI software is not programmed in the conventional sense. This software uses a large amount of data as examples and works by trial and error. They are known as supervised networks. This trial-and-error phase is called training. The most popular general type of neural network uses the training of a three-layer, feed-forward, and fully connected network of nodes and incorporates a backpropagation algorithm. For additional information, see references [3,4,5] and references therein.
3. Methods
The molecules that were analyzed were designed and modeled using HyperChem 5.01 molecular modeling software. This program was used to construct the atoms in the periodic table beginning with element 19 through 35 and 37 through 53. Spin multiplicity for the lowest ground state, as determined from term symbols, was used for geometry optimizations on these elements. Gaussian09 was then used to perform Density Functional Theory (DFT) single-point energy calculations on the structurally optimized atoms. In the Gaussian09 program, the LanL2DZ basis set and the B3PW91 functional was used since most of the elements that were analyzed were slightly heavy in nature.
The LanL2DZ basis set incorporates parameters that account for the relativistic effects of these heavier elements.
Gaussian09 was used to perform Self-Consistent-Field Molecular Orbital (SCF/MO) calculations in order to calculate the energies of the lowest unoccupied atomic orbitals [LUAO] and the energies of the highest occupied atomic orbitals [HOAO]. After the Gaussian09 calculations were completed, data were collected for the lowest unoccupied atomic orbitals and the highest occupied atomic orbitals [3,4].
The artificial intelligence program employed was Nets [6], developed by the Artificial Intelligence Division of the NASA Johnson Space Center. The AI modeling created using Nets was utilized to discern correlations between chemical and physical properties of bulk materials and HOMO and LUMO energies. Once the network was taught how to respond to a set of specific examples, those weights were stored. The network was then tested for the accuracy of predictions. The backpropagation network processed the inputs and compared the resulting outputs to the experimental values. Output errors were propagated back through the system, causing the system to readjust the weights. The process is repeated over and over, continually tweaking the weights until the desired results are achieved. For additional detail on these methods, see references [5,7,8,9]. The first set of results involved modeling the bulk properties of atomic systems. The second set of results involved the prediction of bulk properties of various molecules, including molecules involved in energy systems. For clarification, when we refer to one-atom systems, we will use LUAO and HOAO. However, when discussing multi-atom systems, we will use LUMO and HOM.
4. Results
Atomic Systems
The predictive value of the models formed were represented by plotting the experimental value of the property on the horizontal axis and the ANN predicted value on the vertical axis. The optimal prediction would yield a best fit line with Q2 = 1.0 and slope = 1.0. Q2 is a measure of the data’s correlation. Statistical revisions have been performed on each plot by removing any prediction made in error greater than two times the standard deviation of the error population. The results depicted in this section are the result of Neil Mitchell’s Master’s thesis [10].
Figure 2a,b represent the ability of the ANN to predict the bulk materials’ first Ionization Potential based on data describing one- and two-atom systems from which the bulk materials are composed. Figure 2a represents the predictions made using orbital data obtained from a single atom of the corresponding bulk material. Figure 2b represents the predictions made using orbital data obtained from two atoms of the corresponding bulk material, modeled as homogeneous diatomic molecules. Note that the Q2 value for the single-atom modeling was 0.3589. The value of Q2 for the diatomic system increased significantly to 0.8551.
Figure 2.
Experimental vs. predicted first ionization potentials for (a) single-atom series and (b) diatomic series.
Figure 3a,b represent the ability of the AI to predict the melting point of bulk materials using orbital data obtained from atoms and homogeneous diatomic clusters. Figure 3a represents predictions made using atomic orbital parameters. Figure 3b represents the predictions made using diatomic orbital parameters. A similar increase in Q2 was also observed for the melting point predictions. The single-atom series yielded a Q2 of 0.4599 in contrast with the diatomic series melting point predictions which exhibited a Q2 of 0.8207.
Figure 3.
Experimental vs. predicted melting points for (a) single-atom series and (b) diatomic series.
Melting points and boiling points are purely physical properties and are not as apparently related to the electronic properties of the atom as ionization potential. However, a significant ability to predict the melting and boiling points was observed.
Figure 4a,b represent the ability of the AI to predict the boiling point of bulk materials using orbital data of atoms and homogeneous diatomic clusters. Figure 4a represents boiling point predictions made using single-atom orbital parameters resulting in a Q2 of 0.1798. Figure 4b represents the boiling point predictions made using diatomic orbital parameters with a Q2 value of 0.7877.
Figure 4.
Experimental vs. predicted boiling points for (a) single atom series and (b) diatomic series.
5. Summary
5.1. Atomic Systems
The ability to predict bulk properties of materials using parameters from very small “chunks” is very beneficial. The parameters used to train the artificial neural network can be obtained very accurately and inexpensively by using DFT calculations on these small atomic systems. The same calculations are computationally very expensive or impossible for large bulk systems. The procedures outlined in this paper allow researchers to decrease costs while increasing their capabilities to make predictions.
The modeling of chemical and physical properties of bulk systems using artificial intelligence has been successfully accomplished. Energy values for the atomic and diatomic HOMOs and LUMOs for each of the 34 elements composing the fourth and fifth periods of the periodic table were used as input for an artificial neural network. The experimentally determined first ionization potentials, melting points, and boiling points characteristic of the homogeneous bulk materials composed of the 34 elements were found to correlate with the orbital inputs. Up to a four-fold increase in the predictive capability of this technique was observed by going from a single-atom to a diatomic system to obtain the orbital parameters. We anticipate a continued increase in predictive ability in going to slightly larger systems while maintaining the computational advantages of using small clusters. Our future goal is to apply our procedure to molecular systems; we would like to determine the minimum number of molecules it would take to predict their bulk properties.
5.2. Molecular Systems
In this part of our study, we trained our AI software on 66 different molecules on which the corresponding boiling point (B.P.), melting point (M.P), refractive index (R.I), density (D), and dipole moment (D.M.) were provided and then asked for “trained” AI to predict the molecular weights for an additional four compounds. We used the learning algorithm technique of feed-forward backpropagation using software developed by NASA and modified by us [11]. Figure 5 illustrates the AI network used in the training and prediction of molecular weights (M.W.) from the physical properties of melting point (M.P.), boiling point (B.P.), density (D.), refractive index (R.I.), and dipole moment (D.M.).
Figure 5.
Topology of network used in simulation of molecular weights (M.W.) of various molecules where M.P. is melting point, B.P. is boiling point, D. is density, R.I. is refractive index, and D.M. is dipole moment.
6. Results
Molecular Systems
The results shown in this section are the result of Ashish Soman’s Master’s thesis [12]. We trained on 66 compounds and tested on 4 compounds at a time. The compounds used for training and testing are listed in Table 1. The five physical properties used for training are melting point (M.P.), boiling point (B.P.), density (D), refractive index (R.I.), and dipole moment (D.M.). These properties were found in the 76th edition of the CRC Handbook [13]. We varied the compounds used for training and testing purposes. Table 1 shows the molecular weights for the 70 compounds used for training and testing. Table 2 shows the results of training the neural network on organic compounds 1 through 14, 16 through 23, 25 through 47, 49 through 63, and 65 through 70 in RUN 1. Compounds 15, 24, 48, and 64 were used to test the training of our network.
Table 1.
Molecular weights of various molecules.
Table 2.
Results of predictions of molecular weights.
In RUN 2, the neural network was trained on organic compounds from 1 through 12, 14 through 27, 29 through 55, and 57 through 69 with testing on compounds 13, 28, 56, and 70. In this case, the percent error for the molecular weight ranged from 0.55% to 7.00% with an average error of 3.61%.
In RUN 3, the neural network was trained on organic compounds from 1 through 6, 8 through 18, 20 through 51, 53 through 67, and 69 and 70 with testing on compounds 7, 19, 52, and 68. It was noted that the percent error for the molecular weight ranged from 2.04% to 7.96% with an average error of 4.42%.
In RUN 4, the neural network was trained on organic compounds from 1 through 7, 9 through 22, 24 through 32, 34 through 56, and 58 through 70 with testing on compounds 8, 23, 33, and 57. In this case, the percent error for the molecular weight ranged from 0.32% to 5.00% with an average error of 3.59%.
In RUN 5, the neural network was trained on organic compounds from 1 through 29, 31 through 44, 46 through 59, and 61 through 69 with testing on compounds 30, 45, 60, and 70. In this case, the percent error for the molecular weight ranged from 7.18% to 17.63% with an average error of 11.30%.
In RUN 6, the neural network was trained on compounds from 1 through 5, 7 through 21, 23 through 49, 51 through 54, and 56 through 70 with testing on compounds 6, 22, 50, and 55. In this case, the percent error for the molecular weight ranged from 0.80% to 5.81% with an average error of 3.71%. It should be noted that the overall average error in prediction of the molecular weights of the six runs reported was 4.97%. A correlation plot is shown in Figure 6. Table 3 shows the experimental values used for the AI training [13].
Figure 6.
Cross-validation plot of the results of prediction of molecular weights.
Table 3.
Physical properties of the organic compounds.
Another point of importance is that we made numerous runs in which we eliminated one or more of the physical properties from the input data. The results of all these runs were that we were unable to produce accurate predictions of the molecular weights. It required all five of the physical properties to produce results with an average error below 5%. We should also mention that we made many additional runs besides the six reported. For brevity, we included only the results of runs with the smallest average error, with the largest average error, and four runs with intermediate average error. Lastly, molecular weight was chosen to train on although in principle other properties could have been used for this training. Convenience and availability were the primary reason for this choice.
Also, prediction of the molecular weight is important because it is a characteristic property of a compound which can be deduced from the molecular formula. In this project, attempts were made to make an association of such a molecular property with boiling point, melting point, density, refractive index, and dipole moment which are all bulk properties of a compound.
Also, due to the limited resources of the library and the lack of a wider database, only 70 compounds were used for training and testing purposes. Attempts were also made using molecular weight in the training set and predicting melting point or boiling point. In either case, it could predict the corresponding properties, but the results were not that good. One cannot predict the dipole moment or refractive index based on the training of the remaining five physical properties. The reason for this might be that the numbers on which the network was trained were very large as compared to the testing data.
Again, it is not mandatory to use 28 nodes in the hidden layer. Since the network was trained on 66 organic compounds, it is always better to keep the number of nodes in the hidden layer, less than half the number of data sets. If there are a greater number of nodes in the hidden layer, say 40, 50, or 60, there is a possibility that instead of trying to make an association by learning, it will try to memorize the pattern. We want the network to learn and then generate an implicit functional relationship between the input data (five physical properties) and the output (molecular weight), instead of memorization.
7. Summary
Molecular Systems
We found that a neural network could successfully make an association between the five physical properties of melting point, boiling point, density, refractive index, and dipole moment and the molecular weight of the various organic molecules. We also noted that it was not able to make this association when one or more of these properties were eliminated from the input data. This implies that there is a functional relation between all of these properties and molecular weight. We also believe that hidden within these properties there is probably geometrical information about these molecules (i.e., conformational or spatial isomers). Future studies will attempt to extract this information.
Also, by analyzing the weight space, we feel that we can come out with some mathematical relationship between those five physical properties and molecular weight. If we are able to do so, one can go backwards and solve for any one of the physical properties if the other five are known. For additional details on an introduction to and the theory of neural network computing, see references [14,15,16,17,18].
Table 2 presents the results of the AI predictions of the molecular weights from the five physical properties (melting point, boiling point, density, refractive index, and dipole moment).
The following cross-validation plot (Figure 6) illustrates the results of the prediction of the molecular weights of various organic molecules using the physical properties of melting point, boiling point, refractive index, density, and dipole moment.
NOTE: In Table 3, the pressure at which the B.P. was determined is about one atmosphere for all the compounds. The density is a relative to water, otherwise it has the dimensions g/mL. A superscript indicates the temperature of the liquid and a subscript indicates the temperature of water (i.e., 4 °C) to which the density is referred. Compounds for which the superscripts are not mentioned are the ones whose density measurements were carried out, when the temperature of the liquid is 20 °C. The refractive index is reported for the D line of the spectrum. The temperature of determination of this physical property appears as a superscript [13].
The significance of these values may involve some ambiguity because of the possibility of different conformations or spatial isomers.
Funding
This research received no external funding.
Data Availability Statement
Data are contained within the article.
Conflicts of Interest
The author declares no conflict of interest.
References
- Hyperchem, version 5.01; Hypercube, Inc.: Waterloo, ON, Canada, 1994. Available online: http://www.hypercubeusa.com/News/PressRelease/NewPolicyOct1997/tabid/397/Default.aspx (accessed on 29 August 2024).
- Frisch, M.J.; Trucks, G.W.; Schlegel, H.B.; Scuseria, G.E.; Robb, M.A.; Cheeseman, J.R.; Scalmani, G.; Barone, V.; Mennucci, B.; Petersson, G.A.; et al. Gaussian, version 09; Gaussian, Inc.: Wallingford, CT, USA, 2009.
- Haykin, S. Neural Networks: A Comprehensive Foundation, 2nd ed.; Prentice Hall: Hoboken, NJ, USA, 1999. [Google Scholar]
- Käser, S.; Vazquez-Salazar, L.I.; Meuwly, M.; Töpfer, K. Neural network potentials for chemistry: Concepts, applications and prospects. Digit. Discov. 2023, 2, 28–58. [Google Scholar] [CrossRef] [PubMed]
- Sumpter, B.G.; Getino, C.; Noid, D.W. Theory and Applications of Neural Computing in Chemical Science. Annu. Rev. Phys. Chem. 1994, 45, 439–481. [Google Scholar] [CrossRef]
- Shelton, R.; Baffes, P.T. Nets Back-Propagation; version 4.0; Software Technology Branch, NASA, Johnson Space Center: Houston, TX, USA, 1989. Available online: https://ntrs.nasa.gov/api/citations/19920012191/downloads/19920012191.pdf (accessed on 29 August 2024).
- Darsey, J.A.; Sumpter, B.G.; Noid, D.W. Correlating Physical Properties of Both Polymeric and Energetic Materials and their Organic Precursors of Polymers Using Artificial Neural Networks. Int. J. Smart Eng. Syst. Des. 2000, 2, 283–298. [Google Scholar]
- Tu, X.; Geesman, E.; Wang, K.; Compadre, C.; Buzatu, D.; Darsey, J. Prediction of the partition coefficient of dillapiol and its derivatives by use of molecular simulation and artificial neural networks. Chim. Oggi 2002, 10, 51–54. [Google Scholar]
- Lane, T.R.; Harris, J.; Urbina, F.; Ekins, S. Comparing LD50/LC50 Machine Learning Models for Multiple Species. J. Chem. Health Saf. 2023, 30, 83–97. [Google Scholar] [CrossRef] [PubMed]
- Mitchell, N. A Systematic Approach for Developing Feed-Forward/Back-Propagating Neural Network Models for Predicting Bulk Chemical and Physical Properties of Transition Metals. Master’s Thesis, University of Arkansas at Little Rock, Little Rock, AR, USA, 2007. [Google Scholar]
- Baffer, P.T. NNETS Program; version 2.0; Johnson Space Center Report No. 23366; NASA: Houston, TX, USA, 1989.
- Soman, A. Application of Artificial Intelligence to Chemical Systems. Master’s Thesis, University of Arkansas at Little Rock, Little Rock, AR, USA, 1993. [Google Scholar]
- Lide, D.R. CRC Handbook CRC Handbook of Chemistry and Physics, 76th ed.; CRC Press: Cleveland, OH, USA, 1977. [Google Scholar]
- Eberhart, R.C.; Dobbins, R.W. Neural Network PC Tools. In A Practical Guide; Academic Press, Inc.: London, UK, 1990. [Google Scholar]
- Hert, J.; Krogh, A.; Palmer, R.G. Introduction to the Theory of Neural Computing; Addison-Wesley Publishing, Co.: Redwood City, CA, USA, 1991. [Google Scholar]
- Wasserman, P.D. Neural Computing Theory and Practice; Van Nostrand Reinhold: New York, NY, USA, 1989. [Google Scholar]
- Zurada, J.M. Introduction to Artificial Neural Systems; Weit Publishing Company: New York, NY, USA, 1992. [Google Scholar]
- Darsey, J.A.; Noid, D.W.; Wunderlich, B.; Tsoukalas, L. Neural-net extrapolations of heat capacities of polymers to low temperatures. Makromol. Chem. Rapid Commun. 1991, 12, 325. [Google Scholar] [CrossRef]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).