Cloud Point Temperature of Thermoresponsive Systems: A Predictive Approach in Data Scarcity Conditions

Abstract

In this study, we employ machine learning techniques to improve materials in data scarcity conditions. In particular, we focus on the prediction of the cloud point temperatures of polymer–water systems with thermoresponsive behavior. We compare a model trained directly on the available data with a model based on representations learned through an encoder–decoder model, in turn pre-trained on a larger dataset to generate molecular fingerprints. Our results demonstrate that the embedding-based model significantly outperforms the direct model in predicting the cloud point temperature under the data limitations imposed by rigorous curation. This approach highlights the potential of domain-informed representation learning to tackle complex materials science problems with limited data.

1. Introduction

Thermoresponsive polymers represent an increasingly important class of “smart” materials, capable of experiencing abrupt and reversible physical transitions in response to temperature variations [1,2]. These polymers, particularly those displaying lower critical solution temperature (LCST) and upper critical solution temperature (UCST) behaviors, have attracted substantial interest in diverse fields, ranging from drug delivery and biomedical engineering to sensors, coatings, and smart textiles [3,4,5,6,7,8,9,10,11,12]. The trademark of these polymers is their ability to reversibly transition between soluble and insoluble states in response to temperature changes, driven by a thermodynamic phase separation process, in turn related to complex intermolecular and intramolecular interactions including hydrogen bonding, hydrophobic interactions, and changes in their chain topology [13,14,15,16,17]. In particular, LCST polymers are soluble below a certain critical temperature and become insoluble when heated beyond this point, while UCST polymers are soluble above a specific temperature and phase-separate below it [18]. This thermoresponsive characteristic has significant potential in the design of stimuli-responsive drug delivery systems, in which the polymer solution can circulate freely at physiological temperature but undergo phase separation to release therapeutic agents at “target” sites [19]. Furthermore, such temperature-induced changes in polymer solubility and conformation can also be exploited in smart coatings and bioseparations, thus providing enhanced control over material properties and interactions. Despite considerable research, poly(N-isopropylacrylamide) (PNIPAM) has remained the benchmark polymer for thermoresponsive applications due to its critical temperature being close to physiological conditions [20]. However, PNIPAM has several limitations, including concerns about biocompatibility, phase transition hysteresis, and pronounced sensitivity to end-group [21]. As a result, research efforts have diversified towards alternative thermoresponsive polymers with finely tunable response profiles. For example, polymers such as poly(N-vinylcaprolactam) offer enhanced biocompatibility, while poly(oligoethylene glycol methacrylates) are characterized by minimal hysteresis and greater control over molecular end-groups. Additionally, polyphosphazenes and polyphosphoesters have gained attention due to their structural versatility and exceptional biodegradability [19].

An important area of investigation within thermoresponsive polymer materials involves understanding how chemical structure and environmental conditions influence LCST and UCST transitions. As in many other polymer properties, structural attributes such as hydrophilic–hydrophobic balance, polymer topology, side-chain chemistry, and molecular weight [22,23,24] critically determine the temperature responsiveness of polymer systems [19,25]. On the other hand, purely environmental factors (like solvent quality, salt concentration, and pH, among others) further modulate these transitions. For instance, increasing salt concentrations can dramatically alter cloud points by affecting hydration shells around polymer chains, thereby modulating polymer–solvent interactions [26]. As a result, determining critical solution temperatures experimentally is not trivial due to the number of variables to control and also due to the complexity (and variability) in the measurement methods. Some of the most popular experimental approaches include turbidimetry, differential scanning calorimetry (DSC), and dynamic light scattering (DLS) [11,27,28]. However, significant variations arise due to the subjective definitions of cloud points and differences in turbidity thresholds, heating rates, and polymer concentrations [29,30]. These methodological challenges have been reported, and substantial discrepancies resulting from different operational definitions of cloud points, ranging from the onset of turbidity (approximately 10% opacity) to the inflection point on the turbidity curve (approximately 50% opacity), were noticed [26].

Given these experimental challenges, predictive modeling has become a critical area of advancement, offering computational tools to estimate critical solution temperatures from molecular structures and descriptors. Quantitative structure–property relationships (QSPR), artificial neural networks, and polymer-specific fingerprinting approaches have been effectively employed to streamline polymer design processes, thereby reducing time and resource consumption [19,31,32,33]. For example, some research has focused on developing linear topological and quantum-chemical-based QSPR models, achieving high prediction accuracy [19]. These results evidence the effectiveness of quantum chemical norm descriptors, which capture subtle molecular interactions crucial for predicting thermoresponsive behaviors. Moreover, machine learning techniques, employing molecular fingerprints such as the Morgan fingerprints [34,35,36,37], have significantly enhanced predictive capacities. These approaches efficiently represent polymer structures for machine learning by capturing compositional and topological data derived from routine polymer characterization techniques and can achieve high prediction accuracies (and therefore show the potential of ML-driven approaches for rapidly screening novel polymer candidates). However, predictive models must navigate a critical trade-off between dataset size and experimental data quality [38]. Larger datasets can enhance model generalizability but often can also compromise detailed molecular characterization, potentially limiting prediction accuracy for specialized polymer–solvent systems. Conversely, highly curated datasets help to increase prediction accuracy but at the expense of reducing applicability to broader contexts. Thus, meticulous descriptor selection, rigorous data curation, and experimental validation remain essential to developing robust, generalizable models.

In this study, we address the challenge of predicting cloud point temperatures for aqueous homopolymer solutions under conditions of severe data scarcity. We compare two modeling strategies: (a) a direct model trained solely on a curated set of SMILES (simplified Molecular Input Line Entry System) strings and number-average molecular weights (Mn) and (b) a transfer learning approach based on embeddings generated by an encoder–decoder model pre-trained on a larger dataset of organic compounds (logP data). The latter approach exploits learned representations of chemical structure to enhance predictive capacity. By evaluating and contrasting both models, we aim to demonstrate the substantial advantages of embedding-based transfer learning over conventional direct prediction in low-data regimes and to highlight the potential of such techniques for virtual screening of novel thermoresponsive polymers without the need for costly and time-consuming experimental efforts.

2. Methods

In this section we describe the data curation, the model architectures, and the other methodologies employed to develop a neural network capable of predicting cloud point temperatures in dilute water–homopolymer systems. In particular, we outline our direct prediction approach and a data-scarcity-tolerant architecture, which uses the knowledge of a previously trained encoder–decoder network (originally optimized on logP data [38]) to generate chemical structure embeddings (fingerprints). We then detail the integration of these fingerprints into a model capable of predicting cloud point temperatures, along with our evaluation strategy. The general procedure is illustrated in Figure 1.

2.1. Data Collection and Preprocessing

We employ a raw dataset of measured cloud point temperatures for various polymer solutions, accompanied by polymer-specific parameters (such as molar mass, end-group types, and polymer concentrations) and solution conditions (pH, salt composition, among others). We perform a general curation process by using an automated pipeline that removes empty or inconsistent data and keeps entries with the relevant variables (such as SMILES structure of the repeating units and number-average molecular weight of the polymer). This selection also ensures that the models will be trained on the chemical structure of the materials, so that the approach remains flexible enough to be extended to more-complex systems in the future. In order to ensure a consistent and high-quality dataset, a subsequent specific curation is executed. During this specific curation stage, the dataset was refined to include only homopolymers, and entries were filtered by grouping together all cloud point temperature measurements corresponding to a fixed SMILES string and Mn. From these groupings, only the data corresponding to the most diluted solutions (that is, the lowest concentration recorded) were retained (i.e., values ranging from 0.05 to 0.00025 wt%). This decision ensures that the recorded cloud point temperatures reflect the polymer’s intrinsic thermoresponsive behavior at maximum dilution, with minimal interference from any concentration effects. While these filtering criteria result in a highly reliable and internally consistent dataset, they also impose a severe data scarcity condition due to the exclusion of multiple measurements and non-homopolymeric entries.

As usual, the resulting dataset was randomly partitioned into training, validation, and test sets, with the training data used to optimize the model parameters, the validation set employed for hyperparameter tuning and to trigger early stopping, and the test set reserved for external performance evaluation. All molecular structures were represented as SMILES strings, a standard text-based format for encoding molecular information [39,40]. During preprocessing, the SMILES strings were tokenized at the character level, therefore allowing for sequence processing by the models. The obtained sequences were in turn padded to a uniform length (equal to the maximum length in the dataset).

2.2. Direct Model

We defined a reference by training a model directly on the dataset. In this direct approach, several neural networks were trained to predict cloud point temperatures from the curated polymer data (homopolymer SMILES with their associated Mn).

Our network architecture comprises a feed-forward design with an input layer corresponding to the size of the tokenized SMILES vector. This is followed by an embedding layer with several hidden convolutional layers and a fully connected layer (which enable the network to learn hierarchical representations while gradually compressing the feature space). Each hidden layer employs non-linear activation functions (LeakyReLUs) to capture complex interactions among the input features. The training was performed using the Adam optimization algorithm to minimize the mean squared error (MSE) between the predicted and experimental cloud point temperatures. Given the limited size of the dataset produced by our strict curation strategy, a key aspect of the training protocol is the use of early stopping based on the validation set performance, which together with the use of a dropout strategy helps prevent overfitting. Finally, hyperparameter optimization via grid search was used to fine-tune model parameters.

As mentioned, by training a model directly on the curated dataset, it acts as a reference benchmark and provides some insight into the intrinsic relationship between polymer structure and thermoresponsive behavior. Figure 2 shows a schematic picture of the direct model architecture.

2.3. Data Embedding (Encoder–Decoder Model)

To obtain low-dimensional (and yet chemically meaningful) representations of the polymers, we employed an encoder–decoder architecture. In this way, this model learns to understand chemical structures, therefore allowing any subsequent model to just focus on the property at hand, thus resulting in a much better performance than that of the direct model. The encoder–decoder was trained on a large logP dataset, with the aim of capturing important aspects of the chemical structure, molecular solubility, intermolecular interactions and hydrophobicity. As shown in the schematic picture in Figure 3, the architecture is composed by two Long Short-Term Memory (LSTM) networks [41]: (a) the encoder processes the tokenized SMILES sequences and converts them into a latent vector that captures the chemical structure features (it is composed by an LSTM layer with 256 units); (b) the decoder reconstructs the original SMILES sequence from this latent representation (also an LSTM layer with 256 units that takes the latent vector as input); and (c) intermediate dense layers with ReLU activation functions [42] were used to refine the latent representation before feeding it into the decoder.

In order to optimize the difference between the input and reconstructed SMILES sequences, during the training process we employed sparse categorical cross-entropy as the loss function [43]. We trained the model on 14,500 molecules (validation on 800), and we optimized it using the Adam optimizer [44] (learning rate of 0.001 and a batch size of 1000). As before, we employed early stopping on validation loss to prevent overfitting. The validation loss closely followed the training loss curve, therefore suggesting a good generalization to previously unseen data and reaching a minimum validation loss of approximately 0.5 after 208 epochs.

2.4. Model Based on Embeddings

We developed a model for predicting cloud point temperatures from the embeddings generated from the encoder described in the previous subsection. In this step, the tokenized SMILES, the fixed-size embedding vectors and the corresponding Mn are fed to a deep neural network. Optimization is performed using the Adam optimizer, and training is terminated using early stopping based on validation loss stagnation. Data normalization and scaling steps were applied to the input features to ensure consistency and improve training efficiency. Finally, the hyperparameters were optimized via grid search. Figure 4 shows a schematic picture of the architecture.

2.5. Model Evaluation

The models’ performance was evaluated during training on the validation set, and later on an independent test set, using several standard metrics: root mean squared error (RMSE), mean relative error (MRE), mean average error (MAE), mean average percentage error (MAPE), and coefficient of determination (R²). In addition, visual diagnostic tools including hexagonal binning cross-plots of experimental versus predicted cloud point temperatures were used to inspect the distribution of errors and to validate the model’s generalization capabilities. We then compared our results with our reference direct model, observing that the integration of the embeddings with the deep neural network yielded robust cloud point predictions, even under conditions of very limited training data.

3. Results and Discussion

In this section, we compare the effectiveness of our models in the task of predicting cloud point temperatures of diluted water–homopolymer solutions. Our aim is to understand how our embedded representations perform under data-limited conditions and how they compare with traditional direct approaches. We start by describing the direct model approach, and then we focus on the more advanced embedding-based model results.

3.1. Direct Model

A key challenge in developing a robust direct model for predicting critical solution temperatures is the pronounced scarcity of reliable data. Even though several LCST and UCST cloud point data appear in the literature, the diversity of conditions (like pH, molecular weight, salt content, among others) is immense, and therefore systematically measured, high-quality data remain limited. This poses a substantial obstacle to these direct machine-learning pipelines, which require large, balanced datasets to effectively learn the underlying structure–property relationships. After curating and preprocessing the dataset of polymer–water pairs, nearly 150 samples remained, from which only 115 were employed in the training process: 80% training (92 samples)/20% validation (23 samples). A typical training trajectory is shown in Figure S4.

Moreover, all the trained direct models exhibited only moderate-to-low performance. While certain architectures and hyperparameter configurations seemed promising, their results consistently fell short of delivering stable and accurate cloud point temperature predictions across the full range of samples. Error metrics such as MAE and MAPE (calculated directly in °C scale to avoid masking deviations) remained relatively high (see Table S1). Qualitatively, the models showed limited extrapolation ability beyond the training region, often producing large prediction errors for the polymer chemistries and molecular weight inputs within the data. These findings reinforce the hypothesis that data deficiency, rather than model design alone, is the prime limiting factor in direct modeling efforts (resources are going into learning the chemical structure and the correlation with the property, and the data is simply not enough for that).

Considering the reduced dataset’s complexity (i.e., since the dataset includes only homopolymers, it basically removes any variability due to copolymer structure, crossed interactions, among others) the model performance is low. Focusing on these simpler systems allows a more direct correlation between polymer structure and the observed cloud point temperature behavior. However, the small number of experimentally measured samples for homopolymers and the limited variety of chemical structures still lead to insufficient coverage of feature space, and consequently, poor predictive performance. These results highlight that even under simplified conditions, a purely direct prediction approach struggles without a robust, large-scale dataset and comprehensive coverage of polymer chemistries.

3.2. Embedding-Based Model

Since it basically allows building upon a learned representation of polymer structures, embedding-based models offer a substantial advantage for predicting the polymer properties. In this study, the encoder-based fingerprint strategy circumvents the need for the predictive model to extract all molecular features directly from raw descriptors or simplified molecular input lines (SMILES). Instead, the encoder is initially trained on a broader set of polymer structures, generating latent-space representations (or “embeddings”) that capture the main chemical features. This latent representation effectively reduces the dimensionality of the problem, distilling key structural motifs, molecular weights, and other relevant chemical characteristics into a more manageable space. Figure 5 shows the results of hyperparameter optimization during grid training, where most of the trained models widely outperform their equivalent direct model counterparts.

Compared with direct modeling, where the network must simultaneously learn both the features and their relationship to the cloud point temperatures, the encoder-based approach provides a form of inductive bias that guides the subsequent prediction model toward more-relevant features. These compressed, chemically meaningful fingerprints allow the downstream network to focus on the property of interest without expending resources on “understanding” chemical structures, complex intra- and intermolecular interactions, among other variables. This separation of representation learning from property modeling is particularly valuable in the context of data scarcity: the encoder can be trained on a much broader set of data, using structural information beyond the relatively small subset for which high-quality cloud point temperature measurements are available.

As shown in Figure 6, our evaluations reveal that the encoder-based model significantly outperforms the direct model. On the one hand, training trajectories converge faster and to lower losses (MAE), while, on the other hand, other metrics such as RMSE, MAPE, or R² consistently improve. These results reflect the more robust and stable predictions derived from these learned fingerprints. Figure 6a shows the Predicted vs. True values obtained from the best model during training (model 7 in Table S2). These results are also reflected in the histogram of deviations shown in Figure 6b. Despite the dispersion in the experimental values, the deviations are well below 10 °C. The same results are observed in the validation set, in Figure 6c and Figure 6d, respectively. Finally, the same trend can be observed in the independent test set (see Predicted vs. True values in Figure 6e).

As shown by the different chemical structures in the dataset (see Figure 7 for the test set examples), the model exhibits an enhanced ability to handle variations in polymer chemistry and molecular weight, with fewer outlier predictions even in these underrepresented regions of structure space. Another noticeable advantage arises from how the encoder captures global and local structural features in a single representation. By enforcing a shared latent space during training, the encoder effectively “clusters” similar polymer motifs, allowing the predictive model to recognize analogies among polymer structures that may appear quite distinct based on simple descriptors (e.g., repeating units, functional groups, or side-chain lengths). This kind of chemical cognition is especially valuable in predicting cloud point temperatures, a property that depends on complex, multifactorial interactions among polymer structure, solvent polarity, and hydrogen-bonding capabilities. When compared with the results of the direct model (with MAPE values of no less than 30%), the superiority of the encoder-based approach (MAPE values of about 14%) highlights a broader principle in data-driven materials science: domain-informed representation learning can often mitigate the inherent limitations of small or noisy datasets. The direct model suffers from insufficient training examples to capture the breadth of structure–property relationships, leading to comparatively erratic and less accurate predictions.

These findings underscore the power of representation learning in the chemical and materials domains, particularly for complex, data-scarce problems like predicting phase behavior in polymer–solvent systems.

4. Conclusions

This work presents a comparative analysis of the effectiveness of direct and embedding-based models for predicting the cloud point temperatures of dilute homopolymer solutions in water. We observed that the scarcity of high-quality data significantly limits the performance of direct models, which struggle to learn the complex structure–property relationships. In contrast, embedding-based models, which use previously learned knowledge about polymer structure representations, offer a substantial advantage. Our results demonstrate that the models based on an encoder pre-trained to generate fingerprints consistently outperform the direct models in predicting the cloud point temperature. This is attributed to the encoder’s ability to capture both global and local structural features, allowing the predictive model to recognize analogies between polymer structures. The superiority of the embedding-based approach highlights the importance of domain-informed representation learning to mitigate the inherent limitations of small or noisy datasets in materials science. In conclusion, this study highlights the power of representation learning in the chemical and materials domains, particularly for complex, data-scarce problems such as predicting phase behavior in polymer–solvent systems.