Automating Air Pollution Map Analysis with Multi-Modal AI and Visual Context Engineering

Abstract

The increasing volume of data from IoT sensors has made manual inspection time-consuming and prone to bias, particularly for spatiotemporal air pollution maps. While rule-based methods are adequate for simple datasets or individual maps, they are insufficient for interpreting multi-year time series data with 1 h timestamps, which require both domain-specific expertise and significant time investment. This limitation is especially critical in environmental monitoring, where analyzing long-term spatiotemporal PM_2.5 maps derived from 52 low-cost sensors remains labor-intensive and susceptible to human error. This study investigates the potential of generative artificial intelligence, specifically multi-modal large language models (MLLMs), for interpreting spatiotemporal PM_2.5 maps. Both open-source models (Janus-Pro and LLaVA-1.5) and commercial large language models (GPT-4o and Gemini 2.5 Pro) were evaluated. The initial results showed a limited performance, highlighting the difficulty of extracting meaningful information directly from raw sensor-derived maps. To address this, a visual context engineering framework was introduced, comprising systematic optimization of colormaps, normalization of intensity ranges, and refinement of map layers and legends to improve clarity and interpretability for AI models. Evaluation using the GEval metric demonstrated that visual context engineering increased interpretation accuracy (defined as the detection of PM_2.5 spatial extrema) by over 32.3% (relative improvement). These findings provide strong evidence that tailored visual preprocessing enables MLLMs to effectively interpret complex environmental time series data, representing a novel approach that bridges data-driven modeling with ecological monitoring and offers a scalable solution for automated, reliable, and reproducible analysis of high-resolution air quality datasets.

1. Introduction

Heightened environmental concentrations of fine particulate matter (PM) result in numerous health-related complications, including respiratory [1,2,3,4,5,6,7] and cardiovascular [1,3,4,8,9,10,11] issues, as well as increased mortality rates [12,13,14]. PMs are classified by their size, with the most commonly measured PM concentrations being those smaller than 10 $\mathsf{\mu }$m and 2.5 $\mathsf{\mu }$m, the latter of which can infiltrate deeply into the lungs [15]. Low air quality (AQ) is associated with low-income areas or nations with poor economic conditions, suggesting that air pollution (AP) reflects socioeconomic disparities, as well as disparities between urban and rural areas, with urban settings demonstrating elevated AP levels relative to rural areas [16,17,18,19].

In light of significant evidence linking increased AP to adverse health effects, the World Health Organization (WHO) and the United Nations Sustainable Development Goals (SDGs) established specific objectives, guidelines, and initiatives to combat AP. The WHO released the 2021 global AQ guidelines and established revised thresholds for annual and daily PM concentrations, reducing them from the 2005 edition of the guidelines [20,21]. Likewise, SDGs 3.9 and 11.6 aim to reduce pollution-related health and mortality risks, as well as urban environmental impacts, especially in terms of AQ and waste [22].

Advancements in sensors have reduced the entry cost associated with monitoring AP, thus providing more information for data-driven policy and regulation purposes. Conventional sensors employed by governmental or regulatory agencies can impose a considerable financial strain when attempting to monitoring AP on a larger scale or at a fine resolution. Low-cost sensors can potentially fill that gap in order to efficiently monitor AP on a broader scale and/or lower resolution [23] at a substantially reduced cost. Low-cost sensors have been shown to achieve an acceptable accuracy compared to regulatory-grade AQ monitoring sensors when a correction equation is applied [23,24,25], but the data from low-cost sensors should be interpreted with caution [26].

The development of novel analysis, processing, and interpretation workflows for both regulatory-grade and low-cost sensors is particularly beneficial in the era of artificial intelligence (AI). When dealing with a large volume of low-cost sensor data, manual inspection and interpretation of spatially dependent AP data is susceptible to bias and reliant on the domain-specific expertise of the researcher, in addition to being labor-intensive.

Large language models (LLMs) have shown several promising examples of their applicability to AP data. For instance, MLLMs trained on peer-reviewed literature in atmospheric sciences have effectively been utilized to deliver regulatory information, data analysis, and management recommendations for AQ data, as demonstrated by AirGPT [27]. MLLMs were also utilized in a comprehensive literature review regarding the effects of climate change on global AQ with notable precision [28]. Additionally, MLLMs were applied on AQ data during the Los Angeles wildfires in January 2025, successfully generating health recommendations, policies, and summary reports [29]. Beyond text-based applications, a wide array of AI techniques, particularly machine learning, have been applied in AQ forecasting, yielding promising results on AQ datasets [30,31,32,33,34] and references therein.

Despite the broad applicability of MLLMs in AQ research, previous studies have focused primarily on text-based datasets. To the best of our knowledge, no research has systematically investigated the utilization of MLLMs on image-based spatial AQ maps or analyzed the impact of visual context design on the interpretive potential of MLLMs. Considering the positive results of utilizing AI and MLLMs for AP data, it was considered worthwhile to examine this research gap by testing the capabilities of MLLMs in the interpretation of AP maps, particularly PM_2.5 maps. Both commercial and open-source MLLMs were applied, with map optimizations such as colormap adjustments, intensity normalization, and layer refinement to enhance interpretability. As in classical modeling, where neglecting key spatial parameters can lead to misinterpretation [35], careful visual preparation of maps is critical for reliable MLLM analysis. The possible outcomes of this research could provide a basis for the automation of AQ interpretability where large data streams are present, thus decreasing labor-intensive processes susceptible to researcher biases. The findings of this research may be extrapolated beyond AQ data to other environmental and ecological datasets, particularly those utilizing high-resolution data and substantial volumes of data, where automation can enhance interpretation, automate labor-intensive processes, and mitigate bias.

2. Materials and Methods

2.1. Data and Maps

The dataset used for the MLLM analysis originates from Kraków (Poland) and its surrounding areas (see Figure 1), covering the period from 00:00 on 1 January 2022 to 00:00 on 1 January 2023. These data represent outputs from forecasting models developed in previous pipeline studies [36,37]. The dataset consists of hourly mean PM_2.5 values. With regard to preprocessing, the raw data provided by Airly (www.airly.org (accessed on 1 Novermber 2025)) were used with minimal intervention: only linear interpolation of occasional missing values was applied, cross-checked with nearest-neighbour sensor consistency and with the nearest governmental reference station. No additional cleaning, outlier removal, or bias correction was performed, as preserving the natural variability of the measurements was an important part of the analysis. The dataset was scaled (normalized) solely to ensure numerical stability of the models. For the MLLM experiments, we utilized both historical observations and forecasted values. The distinction between observed and predicted data was secondary, as the primary objective was to evaluate the MLLM’s general capability to comprehend and process the underlying data patterns. In total, 52 low-cost sensors (LCSs) provided by Airly (https://airly.org/ (accessed on 1 Novermber 2025)) were employed. These sensors operate based on the principle of light scattering. Although LCS units generally exhibit higher bias and uncertainty compared to reference-grade monitoring stations, which typically rely on gravimetric methods, numerous studies have reported strong agreement between their measurements and reference data under a wide range of environmental conditions [38,39]. Other research has evaluated the performance of these sensors in detecting smog episodes and in assessing the influence of various environmental factors on prediction accuracy [36]. Furthermore, the effects of meteorological conditions and local topography on sensor readings have been extensively investigated using explainable artificial intelligence (XAI) methods [40]. Kraków was selected as a representative area for studying AP due to its location within a moderate climate zone characterized by pronounced seasonal weather variability. The city is situated in Poland (European Union), where coal combustion during the winter months remains one of the primary sources of deteriorating AP [41], a phenomenon that shares many similarities with urban environments in China [42], India [43], and other regions facing comparable challenges. At the same time, Kraków represents a unique case study, as it is an isolated urban area that has implemented a complete ban on the use of fossil fuels for domestic heating. This transition is occurring under the influence of European Union legislation enforcing strict air quality standards, making Kraków an exceptional natural laboratory for assessing the effectiveness of environmental policies and the dynamics of air pollution under evolving regulatory frameworks.

In the current study, an additional pipeline was developed to generate spatial distribution maps of PM_2.5 concentrations. Both forecasted and observed values were interpolated using ordinary kriging [46,47] with the following semi-variogram parameters: sill = 5.0, range = 0.6°, nugget = 0.1; linear model. The interpolation was performed on a regular grid with a resolution of 0.02°. A single global variogram model was applied, and Kriging was performed using the same time-invariant parameters estimated from the combined dataset. This global approach may smooth episodic variations in the autocorrelation structure and attenuate local extremes, which could reduce the visibility of sharp gradients and local hotspots for MLLM interpretation. However, this is acceptable, as the primary objective of the study is to assess the overall capability of MLLMs to interpret spatial patterns in PM_2.5. The resulting Kriging grids were visualized as maps representing the spatial distribution of PM_2.5 concentrations.

2.2. Multimodal Large Language Models

These visualizations were used to assess how effectively multimodal large language models and commercial models can interpret such data. Multimodal models are models capable of processing and integrating multiple modalities such as text, images, audio, and others. In this study, the modalities used were images and text. Typically, such architectures consist of three main components: a visual encoder, a connector or projector, and MLLM (Figure 2).

In the Cambrian-1 study by Tong et al. [48], it was shown that MLLMs often face difficulties with the visual component, relying primarily on the language model for interpretation and reasoning. This observation proved highly valuable for the present research and informed several subsequent design decisions. During the experiments, two open-source MLLM architectures were evaluated: Janus-Pro-7B [49] and LLaVA-1.5-7B-HF [50]. For comparison with commercial models, GPT-4o [51] and Gemini 2.5 Pro [52] were also tested.

All models utilized the identical prompt template. Inference was performed with a temperature of 0.3 and default top-k settings. Results were averaged across multiple generations to ensure stability.

2.3. Visual Context Optimization for MLLM Interpretation

To improve the interpretability of PM_2.5 spatial maps by MLLMs, a dedicated visual optimization process was implemented. The procedure focused on the selection of color palettes, scaling methods, and normalization techniques that enhance contrast and semantic consistency between visual and textual modalities.

The distribution of PM_2.5 concentrations (Figure 3) was first analyzed to identify the characteristic range of values. More than 95.7% of all observations were below 50 µg/m³, while extreme values were rare. Based on this finding, a nonlinear power normalization (PowerNorm) was applied to the input data to improve contrast in low-value regions. The transformation was defined as [53]:

$$c^{\prime }={\left({\displaystyle \frac{x^{*}-v_{min}}{v_{max}-v_{min}}}\right)}^{\gamma },\gamma =0.4,v_{min}=0,v_{max}=289.$$

(1)

where $c^{\prime }$ is the normalized concentration value, $x^{*}$ is the original (pre-normalized) PM_2.5 concentration, $v_{\min }$ is the minimum value used in normalization (here: 0), $v_{\max }$ is the maximum value used in normalization (here: 289), and $\gamma$ is the exponent controlling the degree of nonlinearity (here: 0.4). The parameter $\gamma =0.4$ was determined experimentally through visual inspection. It was selected to enhance contrast within the predominant low-concentration range, as linear scaling resulted in insufficient visual differentiation.

Alternative $\gamma$ values were evaluated but proved less effective in highlighting spatial patterns.

For $\gamma <1$, the transformation expands the lower part of the value range, resulting in stronger differentiation between small concentrations while compressing higher values. This adjustment improves visual readability in the most frequent range of data (0–50 µg/m³). To ensure intuitive interpretation both by MLLMs and by human users familiar with air quality indicators, a color palette inspired by Air Quality Index (AQI) scales was adopted (Figure 4). The gradient transitions from green → yellow → orange → red → violet/burgundy, reflecting increasing pollution levels. This color semantics supports fast perception of risk gradients and aligns with common visual conventions used in air quality communication. A fixed maximum scale value of 289 was applied across all maps to maintain stable and comparable visualization between temporal and spatial scenarios. Values exceeding this limit were clipped to the highest color class, indicated with an upward triangular end on the color bar. Since such extreme observations constituted less than 0.02% of the dataset, this clipping did not introduce bias into hotspot detection. This approach prevents inconsistent scaling that could lead to misinterpretation by MLLMs when comparing different map contexts.

2.4. Evaluation

Two complementary evaluation approaches were applied to assess the interpretability of the generated visual–textual outputs: manual subjective evaluation and automated G-Eval scoring.

2.4.1. Manual Subjective Evaluation

A manual qualitative assessment was conducted to capture aspects of model perception that are difficult to express numerically. The evaluation focused on the following key capabilities of multimodal large language models:

• Temporal reasoning—recognizing and describing changes over time between consecutive visualizations (e.g., sequences of PM_2.5 maps);
• Identification of spatial extremes—detecting and describing hot spots (high-concentration regions) and cold spots (low-concentration regions);
• Recognition of spatial gradients—interpreting gradual attenuation or intensification of PM_2.5 concentrations and identifying the general direction of dispersion;
• Tracking of PM_2.5 cluster displacements—detecting movement or shifting of pollution concentration clusters between time steps, indicating dynamic atmospheric transport or meteorological influence.

Each of these aspects was analyzed through direct visual comparison and examination of model-generated descriptions, with attention to whether the model correctly captured environmental dynamics and spatial relations.

2.4.2. Automated Evaluation with G-Eval

To complement the manual analysis, an automated quantitative metric based on G-Eval was used. G-Eval [50] is a large-language-model-based evaluation framework designed to assess text generation quality with strong correlation to human judgments. It employs a chain-of-thought (CoT) reasoning process combined with a form-filling scoring paradigm. The evaluator first generates intermediate evaluation steps from predefined task instructions and criteria (e.g., clarity, coherence, consistency), and then outputs a continuous score between 1 and 5. Instead of using discrete ratings, the probability of each possible score $p\left(s_i\right)$ is extracted from the MLLM’s output and combined into a weighted continuous metric:

$$\mathrm{score}=\sum _{i=1}^{n}p\left(s_i\right)\times s_i.$$

(2)

where $p\left(s_i\right)$ is the probability assigned to score $s_i$, and $s_i$ represents each discrete rating option (e.g., 1–5).

This method provides fine-grained, continuous evaluation scores that more accurately capture subtle differences between generated responses. In this study, the G-Eval metric was applied to assess the textual interpretations of PM_2.5 spatial maps, their relevance to observed map features, and the accuracy of spatial reasoning. The evaluation criteria focused on verifying the correctness of identifying key measurement points (hot-spots and cold-spots). The dataset used for this evaluation consisted of 100 manually annotated PM_2.5 maps, serving as high-quality reference data.

Theorem 1. 

In the correct answer, the following sensor IDs (hot-spots) must be included: [Hot-spots]. Additionally, the following sensor IDs (cold-spots) must also be included: [Cold-spots]. If a required point (e.g., 99) is replaced with an incorrect one (e.g., 77), a stronger penalty should be applied. If a required point is simply missing (no ID provided), a smaller penalty should be applied.

3. Results

The open-source models evaluated in this study, Janus-Pro-7B and LLaVA-1.5-7B-HF, exhibited significant difficulties in handling visual context. Their spatially grounded responses were often inconsistent and demonstrated limited spatial understanding. Moreover, older versions performed poorly on tasks requiring Optical Character Recognition (OCR), specifically the recognition of numerical sensor ID labels visible on the map images. For instance, when prompted with “List me ID of point with the most pollution on map?”, LLaVA-1.5-7B-HF responded: “The point with the most pollution on the map is located at approximately 1.33, 0.48, and 0.52. This point has a high concentration of red dots, indicating a significant amount of pollution in the area.” Quantitative assessment via G-Eval confirmed these limitations, with open-source models achieving negligible scores (averaging 2.5%). However, error analysis reveals that this underperformance stems primarily from OCR hallucinations rather than a lack of visual attention. While the models often correctly located high-pollution regions visually, they failed to transcribe the specific sensor IDs required by the ground truth. This indicates that for current open-source MLLMs, the bottleneck lies in fine-grained text recognition (OCR) rather than general spatial reasoning.

After the evaluation of open-source models, the focus was shifted toward commercial MLLMs, which were assessed under controlled and systematically varied visualization conditions. Unless stated otherwise, all results presented in this section were obtained using the G-Eval. Commercial models were tested on four differently prepared PM_2.5 spatial maps (Figure 5), designed to assess the impact of visual context and preprocessing on model interpretation:

(i)

Map I—a correctly prepared map after context engineering, using a customized color scale inspired by the AQI scheme (see Section 2.3, Visual Context Optimization for MLLM Interpretation);

(ii)

Map II—a correctly prepared map with an additional overlaid shapefile;

(iii)

Map III—a raw, unprocessed map rendered with a variable color scale (Vardis), corresponding to near-default plotting parameters;

(iv)

Map IV—a raw Vardis map with an overlaid shapefile.

Figure 5. (a) Map I: Context-engineered map (AQI color scale) (b), Map II: Context-engineered map + shapefile overlay (c), Map III: Raw map (Vardis color scale) (d), Map IV: Raw map (Vardis color scale + shapefile overlay).

Figure 5. (a) Map I: Context-engineered map (AQI color scale) (b), Map II: Context-engineered map + shapefile overlay (c), Map III: Raw map (Vardis color scale) (d), Map IV: Raw map (Vardis color scale + shapefile overlay).

The experiments indicate (

Table 1. Average G-Eval Scores by Map Variant.

Map Variant	Map Type (Color Scale)	Shapefile Overlay	G-Eval Score
Map I	Context-engineered (AQI)	No	0.381
Map II	Context-engineered (AQI)	Yes	0.253
Map III	Raw map (Viridis)	No	0.288
Map IV	Raw map (Viridis)	Yes	0.274

) that adding a shapefile overlay strongly shifts model attention toward the overlay, leading to misinterpretation of hot and cold spots. The overlay darkens the color field, prompting the model to infer higher concentrations where a human reader would immediately recognize an added vector layer. With a well-prepared map, the model attains an average G-Eval of 0.38 with a shapefile overlay, and performance drops to 0.253, reducing image-analysis accuracy by over 33.6%. Crucially, visual context aligned with human perception also benefits the models: consistent color normalization combined with an AQI-like, high-contrast color scale—a palette widely associated with air-pollution risk—improves interpretability for both humans and MLLMs. This human–model alignment arises because the AQI palette foregrounds monotonic risk cues (e.g., yellow → orange → red) that models readily associate with textual descriptions of elevated PM_2.5, yielding higher G-Eval scores and improving hotspot/cold-spot detection by up to 32.3%. Moreover, an AQI-like palette likely biases the model even at the language-component level by activating learned linguistic priors that link yellow→orange→red with “higher risk,” tightening the mapping between text tokens and the spatial signal. See scores in Table 1.

Manual tests were also conducted for Temporal Reasoning, Identification of Spatial Extremes, Recognition of Spatial Gradients, and Tracking of PM_2.5 Cluster Displacements. These tests showed that visual context engineering substantially improves performance on all tasks, especially when models receive more than one kriging map. For cluster tracking over time, a fixed color scale and consistent normalization kept the model aligned: with a single, stable legend, the model could compare panels directly and did not lose track of clusters. Under variable scales, the model often flagged locations that merely shifted position on the legend rather than exhibiting true changes in concentration—failing to capture the actual cold/hot spots. With multiple maps, the model sometimes confused the colormap semantics (e.g., interpreting “warm” as “cold” and vice versa), an error that disappeared when the maps were prepared with consistent scaling, normalization, and explicit context cues.

4. Discussion

In this study, 52 low-cost sensors were deployed across Kraków and its surrounding areas to investigate the potential application of MLLMs for the spatial analysis of PM_2.5 air pollution in 2022. Kraków was chosen as a reference city. Solid fuel burning is completely banned within the city. However, the country still relies heavily on coal for energy, resulting in high air pollution levels, especially in winter. Given this energy mix and geographical location, the city provides a robust evidence base and a valuable reference point for other low- and middle-income countries to assess the effectiveness of air quality improvement policies implemented by the European Union. The results of this study confirm that the use of MLLMs, such as GPT-4o and Gemini 2.5 Pro, can significantly enhance the interpretation of spatial distributions of PM_2.5 air pollution. A key factor contributing to this improvement was visual context engineering. This proved to be a critical component determining the quality of model interpretation, rather than merely a supplementary step in data preprocessing. Visual context engineering refers to the deliberate design of visual data, such as maps and satellite imagery, enabling multimodal models to effectively understand spatial relationships and geographic patterns. In practice, this means that the way a map is represented its color palette, scale, proportions, and layer structure can substantially influence the model’s ability to interpret spatial information accurately. In this work, techniques such as colormap optimization, scale normalization, and layout standardization contributed to a noticeable improvement in model accuracy. Additionally, contextual map overlays (including administrative boundaries, emission zones, and transportation networks) allowed the models to better identify local anomalies and gradients in PM_2.5 concentration. Optimizing the visual context increased the accuracy of model responses by 32.3% (relative improvement). Furthermore, it improved the coherence and interpretability of the generated explanations. This enhancement led to more stable outputs under varying input conditions, which is crucial for reliable application of MLLMs in environmental data analysis. It is important to emphasize that the effectiveness of MLLMs is highly dependent on visual consistency. Even minor changes in color scale or projection can alter the model’s interpretation, underscoring the need for standardized visual preprocessing in multimodal environmental research. Future research may also focus on automatic map tuning and generation of appropriate color scales tailored to the specific characteristics of the analyzed problem, using Generative Adversarial Networks (GANs) and other machine learning approaches. In summary, visual context engineering should be considered a key component of multimodal analysis, bridging data visualization, geoinformatics, and machine learning. Proper preparation of visual input data is essential for achieving high-quality, interpretable outputs from MLLMs in environmental applications.

5. Conclusions

This work proposes a new methodological direction for map interpretation that leverages MLLMs to extract, represent, and reason about complex spatiotemporal relationships. Unlike traditional approaches based on visual inspection or static statistical methods, the proposed framework enables scalable, data-driven interpretation of spatial phenomena, facilitating the discovery of latent patterns and causal relationships that remain hidden or difficult to detect through conventional analysis. The experiments demonstrate that the quality and consistency of visual context are crucial for accurate spatial map interpretation by MLLMs. The open-source models exhibited limited spatial understanding and struggled with OCR-related and visually grounded reasoning tasks, confirming their immaturity in visual analysis. It should be noted that a quantitative comparison (e.g., G-Eval) between these open-source models and commercial counterparts was deemed methodologically unsuitable. Preliminary tests revealed that open-source architectures exhibited a fundamental perceptual limitation in resolving small sensor ID numbers (OCR failure). Since the models could not correctly perceive the input data, evaluating their subsequent spatial reasoning capabilities would yield misleading results. Therefore, we report these limitations qualitatively rather than through metrics that conflate visual acuity with reasoning logic. In contrast, commercial MLLMs achieved notably better results, though their performance was highly dependent on the preparation and clarity of the input maps. Adding a shapefile overlay significantly distorted interpretation, as the models tended to focus on the vector layer rather than the underlying color field, leading to systematic misjudgment of PM_2.5 concentration levels. The highest performance was obtained when using a consistent, AQI-inspired color scale, which improved both human and model interpretability and increased accuracy by over 30%. Stable legends and uniform normalization enabled models to reliably track spatial and temporal changes, reducing confusion caused by inconsistent colormaps or scaling. Well-structured visual context is crucial in enabling reliable spatial analysis with MLLMs, especially in environmental monitoring and assessment focused on particulate matter. While current models demonstrate competence in basic map interpretation, they exhibit systematic and reproducible errors when visual cues are ambiguous, inconsistent, or poorly standardized. This highlights the need for continued advancements in both model architecture and visualization protocols to ensure accurate, high-precision geospatial analysis, which is essential for informed environmental decision-making and policy development.

6. Limitations and Future Work

While this study focuses on a single city, sensor network, and pollutant (PM_2.5), future studies should include other urban areas, diverse chemical regimes, alternative map types (e.g., wind fields, NO₂ plumes), and additional pollutants, including gaseous species, to assess the generalizability of our findings. Further work could also explore the performance of different MLLMs, examine alternative visual context designs, and evaluate robustness across multiple years, seasons, and spatial resolutions. Such extensions would provide a more comprehensive understanding of how MLLMs can interpret environmental maps in varied real-world settings.