Utilizing Geoparsing for Mapping Natural Hazards in Europe

Abstract

Natural hazards exert a detrimental influence on human survival, environmental conditions and society. Historical hazard events have generated a broad corpus of literature addressing the spatiotemporal extent, dissemination or social responses. With regard to quantitative analysis based on information locked within verbose text, the release of such information from the narrative format is encouraging. Natural Language Processing (NLP), a technique demonstrated to be capable of automated data extraction, provides a useful tool in establishing a structured dataset on hazard occurrences. In our study, we utilize scattered textual records of historical natural hazard events to create a novel dataset and explore the applicability of NLP in parallel. We put forward a standard list of toponyms based on manual annotation of a compilation of disaster-related texts, all of which were references in an authoritative publication in the field. The final natural hazards dataset comprised location data, which referred to a specific hazard report in Europe during 1301–1500, together with its geocoding result, year of occurrence and detailed event(s). We evaluated the performance of four pre-trained geoparsing tools (Flair, Stanford CoreNLP, spaCy and Irchel Geoparser) for automated toponym extraction in comparion with the standard list. All four tested methods showed a high precision (above 0.99). Flair had the best overall performance (F1 score 0.89), followed by Stanford CoreNLP (F1 score 0.83) and Irchel Geoparser (F1 score 0.82), while spaCy had a poor recall (0.5). Then we divided natural hazards into six categories: extreme heat, snow and ice, wind and hails, rainstorms and floods, droughts, and earthquakes. Finally, we compared our newly digitized natural hazard dataset to a geocoded version of the dataset provided by Harvard University, thus providing a comprehensive overview of the spatial–temporal characteristics of European hazard observations. The statistical outcomes of the present investigation demonstrate the efficacy of NLP techniques in text information extraction and hazard dataset generation, offering references for collaborative and interdisciplinary efforts.

1. Introduction

Natural hazards, such as droughts, floods and many other weather events, have played a distinctive role in society throughout history [1,2]. According to Munich Re’s “NatCatSERVICE” database [3], the direct economic losses caused by natural disasters amounted to EUR 580–640 billion from 1980 to 2022. A close relationship has been demonstrated between natural hazards and geographical, topographical and meteorological factors [4]. For example, the Alpine region is frequently subject to natural disasters caused by snow, ice and floods, owing to its high altitude [5]. Thus, extracting and analyzing the information about the space and time of natural hazards is a contributing factor to ensuring timely responses.

Up till now, natural disasters have resulted in a substantial corpus of publications about their historical spatial–temporal spread [6,7,8], exemplified by the records of weather-induced hazards in China and Europe. Given the abundance of documentary evidence (written and iconographic sources), it is possible to conduct a comprehensive analysis across a broad geographical area, extending back in time. The conversion of original narrative hazard texts into quantitative data is typically required for their use in catastrophic analyses. It is imperative to employ data mining techniques to enhance the efficiency of information extraction from raw textual data, given that prior manual-based workflows were both time-consuming and irreproducible.

With the significant progressions in machine learning algorithms, the automated identification and extraction of unstructured data from text through Natural Language Processing (NLP) has become a subject of particular interest. Internationally, NLP-driven studies based on social media have set a precedent for the design of analogous studies [9,10]. These studies carry out integrated analysis based on crowdsourced data on hazard occurrences (e.g., floods [11], earthquakes [12]). Ye et al. [13] developed an NLP workflow, incorporating text classification, topic modeling and geoparsing methods, to build a structured account of the data and map scientific literature on natural disasters and human health. These attempts cast light on the interdisciplinary approach in hazards research, releasing the information that has long been locked in a narrative format.

The NLP field has been enriched by the integration of various state-of-the-art algorithms. Based on these algorithms, automated workflows are prioritized for tasks such as breaking down text into individual words (word tokenization), identifying entities (named entity recognition, NER), analyzing the syntax (parsing) and quantifying the relationship (e.g., similarity) between entities. Geoparsing is a domain-specific NER focusing on the extraction of place names (often called toponyms) from a certain text [14]. Theoretically, the identification result is linked with a geographical entity compiled in a geographical information dataset such as the GeoNames placename database [15], which provides geocoding services.

Over the past few years, considerable progress has been made in developing digital databases of disaster events based on historical information [8,16]. On a regional scale, for example, the East Asian climate database REACHES constructed by Wang et al. [17] makes full use of Chinese official and local chronicles. It translates the written materials into content that contains information about time, location and type of events, highlighting the opportunity for precise climate interpretation dating back to the Middle Ages.

The relationship between the second plague pandemic (which caused a significant loss of life in late medieval Europe) and climate change has attracted widespread scientific interest [18]. Focusing on the linkage among ecological, biological and archeological evidence, the Black Death Digital Archives project (http://globalmiddleages.org/project/black-death-digital-archive-project, accessed on 27 September 2023) seeks to provide a new interrogation of traditional sources of historical information. With the aid of the NLP technique, our work here contributes to the ongoing efforts with a case study on digitizing hazard-related records across Europe during the period 1301–1500 and visualizing these events based on location data. The input of the digitization process is a series of written materials (chiefly books and articles) referenced by Pfister’s treatise “The Palgrave Handbook of Climate History”. The application of different geoparsing tools with regard to the extraction of place names was compared. Finally, we accomplished the merging of our novel natural hazard dataset with a well-established electronic database by Harvard University, which we screened and geocoded, to emphasize the merits of deriving information from a more extensive corpus. The objective of this research endeavor is to generate an improved, georeferenced dataset concerning historical natural hazards, and to provide a foundation for future studies on spatial–temporal analysis of hazards based on the great wealth of documentary data available and automated data mining tools.

2. Materials and Methods

2.1. Data Source and Preprocessing

Concentrated on historical climate variations and extremes, Pfister’s treatise provides compiled information primarily from secondary literature instead of original sources.

Table 1. List of works that compile information on historical natural hazards in Europe. Some of the works listed are cross-referenced by the others.

Author	Temporal Coverage	Hazard Type	Category	Reference
Lamb	1000 B.C.–1850	Meteorological hazards and geological hazards	Book	[19]
Le Roy Ladurie	1000–1950	Meteorological hazards and geological hazards	Book	[20]
Pfister	1300–1400	Low temperature	Journal article	[21]
De Kraker	1400–1953	Droughts, wind, storms and floods	Journal article	[22,23,24]
Van Engelen	1000–1900	Floods	Book section	[25]
Camenisch	1399–1498	Freeze, frost, droughts and floods	Journal article	[26,27]
Rohr	1441–1590	Floods and earthquake	Journal article and book	[28,29]
Jäger	1250–1900	Wind and snow	Book	[30]
Pribyl	1256–1448	Freeze, rain, droughts and floods	Journal article and book	[31,32]
Titow	1209–1350	Floods and droughts	Journal article	[33]
Bell	950–1500	Sea ice, floods and droughts	Journal article	[34]
Ogilvie	1200–1430	Sea ice	Book section	[35]
Brandon	1340–1444	Heat, severe cold, floods and droughts	Journal article	[36]
Schuh	1300–1400	Rainstorm and droughts	Journal article	[37]
Huhtamaa	1100–1500	Heatwave, cold, frost, snow, rainstorms and droughts	Journal article	[38]
Brázdil	974–1500	Hail, rainstorms, snow, floods and droughts	Book	[39]
Kiss	1307–1507	Floods and droughts	Journal article	[40,41]
Camuffo	853–1985	Freeze	Journal article	[42]
Bauch	1432–1433	Freeze, frost, wind, rainstorm and earthquake	Journal article	[43]
Telelis	803–1470	Heatwave, cold winter, freeze, snow, rainstorm, floods and droughts	Book section	[44]
Haldon	300–1453	Heatwave, cold winter, hail, snow, rainstorms, floods and droughts	Journal article	[45]

presents a summary of the selected literature, which compiles the places and times of historical natural hazard events. The majority of these texts correspond to the treatise’s 22nd chapter, in which the major characteristics of the climatic conditions during the European Middle Ages are discussed. These texts contain either overall findings about natural hazards or specific information (historical anecdotes) on a particular event (e.g., a poor harvest) that is attributed to natural hazards in a given year. The original literature was obtained from various internet sources, including documents downloaded from websites or collected by cyber crawlers, and was stored in a PDF data format.

A schematic workflow of our analysis based on a compilation of literature is shown in Figure 1, and comprises the following components: (1) data preprocessing, including automated literature Optical Character Recognition (OCR) followed by manual spelling checking; (2) manual establishment of standard list of toponyms; (3) comparison of geoparsing performance among four automated tools; (4) finalization of natural hazards dataset based on manual linkage of detailed hazard information with places of natural hazards and on geocoding of toponyms; and (5) categorization and mapping of natural hazards. The following paragraph delineates the preprocessing procedure in detail.

In the first preprocessing step, we performed OCR using Adobe Acrobat Pro. The scanned OCR version of text contained embedded structural elements like page numbers, end-of-line hyphenations, headers and footnotes. We processed text data cleaning by removing these elements and checked the data file for recognition errors (e.g., unrecognized special characters, misjudgment of “o” and “0”). Meanwhile, we manually calibrated word spelling arising from writing habits and misspellings. For the implementation of automatic geoparsing, we corrected the misaligned text and removed names in parentheses and numbers in brackets, which were references on most occasions and thus were insignificant for syntax analysis. After OCR and preprocessing, the broad corpus of literature on natural hazards was converted into 21 plain text files, reducing the number of characters from 963,471 to 858,825. The alteration in character length was primarily attributable to the elimination of superfluous content, including author citations and reference lists, which were irrelevant to the research subject.

2.2. Establishment of Standard Location List and Natural Hazard Dataset

In order to efficiently extract information from different sources, separate textual data was compiled together to form a single running text file. Two experts on paleoclimatology annotated the preprocessed text, respectively, using the annotator tool INCEpTION (version 37.1) [46], with a focus on the location toponyms. The definition of the locations refers to [47], including natural feature names such as “the Baltic Sea” or administrative place names like country, city, region or county names. Given the multilingual texts, toponyms in English, German and French were all included. We then evaluated the two versions of the annotations and created a joint agreement document. This list of toponyms, a total of 4413 (representing 1.4% of all word tokens), was a combination of all location entities, regardless of whether the geographical location was linked to a hazard event. This standard list was used for the evaluation of toponym NER performance among text mining algorithms.

The standard list was the basis of the final dataset of hazard occurrences in Europe during the period 1301–1500. Through this, we conducted an evaluation to ascertain the relevance of each toponym to a specific hazard event that meets the above criteria. If the linkage was unclear, we decided for each case by referring to the context (sometimes even the inserted tables and diagrams) in the original text. We manually extracted the year and hazard event(s) corresponding to the toponym and linked the source of the information (i.e., author of the source literature) as well.

Finally, we used the geocoding Application Programming Interface (API) service provided by GeoNames (https://www.geonames.org/export/web-services.html, accessed on 22 May 2025) to retrieve the attached geographic information for each location and only kept the best match. All batch geocoded results were mapped through WGS84 projection to detect mismatched locations. Questionable results or confusable toponyms were checked individually by referring to the original literature or experts’ experience. Entries that could not be identified automatically were marked as questionable at first. Their coordinates were then looked up manually using Google Maps (https://www.google.com/maps, accessed on 13 June 2025). We divided all geocoding results into three categories, including places (cities, towns, villages, districts, municipalities and historic sites), administrative units (countries, states, provinces and other local divisions), and regions (historical regions, geographical regions, and natural features such as islands, seas, mountains, valleys, lakes and lagoons). For further information on the spatial extent of geographic entities, we manually extracted the bounding box coordinates for administrative units and regions based on Google Maps and calculated the diameter of the bounding box. The batch geocoding experiment was performed in May 2025 using a dedicated Python script (version 3.7).

2.3. Evaluation of Toponym NER Performance

For toponym identification, we chose the following geoparsing tools with pretrained models for English, German and French: Flair NLP [48], Stanford CoreNLP [49], spaCy [50] and Irchel Geoparser [51]. A technical comparison of the geoparsing tools is shown in

Table 2. Basic information on the four selected geoparsing tools.

	Flair	Stanford CoreNLP	spaCy	Irchel Geoparser
model	BiLSTM, CNN, Transformer	CRF	CNN	rule-based/machine learning
programing language	Python	Java	Python (Cython for speed)	Python
open source	yes	yes	yes	partial
community support	active and growing open-source community	large academic and developer user base	large and actively growing open-source community	small community
training datasets	CoNLL-2003, OntoNotes	CoNLL-2003	TIGER, WikiNER	built-in gazetteers covering millions of place names
tasks	tokenization, NER, text classification	tokenization, NER, text classification	tokenization, NER, text classification	location recognition
language support for NER	English, German, French, Arabic, Danish, Spanish, Dutch, Ukrainian	English, German, French, Arabic, Chinese, Hungarian, Italian, Spanish	English, German, French, Chinese, Danish, Spanish, Dutch, Croatian, Finnish, Ukrainian	English, German, French, Chinese, Danish, Spanish, Dutch, Croatian, Finnish, Ukrainian
license	MIT	GNU GPL v3	MIT	MIT

. Technically, the original texts had to be transformed into tokens through word-embedding algorithms before performing NER to obtain the toponyms. All four tools support plain text as input and provide part-of-speech (POS) tags, a common syntax analysis used to identify and label tokens. The NER for Irchel Geoparser relies on the word tokenization result by spaCy, and the output result only contains toponyms together with the corresponding location attributes (including feature type, country name, and so on), but not the POS tagging of the whole paragraph. For a better comparison, we carried out the NER experiment using the pre-trained models trained on a smaller corpus (en_core_web_md, 31 MB) for spaCy, and a larger corpus (en_core_web_lg, 382 MB) for Irchel Geoparser. Flair, Stanford CoreNLP and spaCy use distinct algorithms for tokenization. Different tools may treat words in special forms (e.g., “1473-74”, “twenty-five” or “B.C.”) as one or two tokens. Consequently, the total number of tokens is different, which has a slight influence on the identification of toponyms with complex spellings. For adequate comparison, we generated a unified text dataset by mapping the entity identification results to the tokens of the standard list (defined in Section 2.2). In some cases, the geographical entities could not be fully recognized, for example, “Washington Irving Island” could be identified as “Washington” or the full name. We had a tolerance for partial matches when calculating the performance matrix. After token mapping, the tokenized text was re-categorized into “toponym” or “other” for all four approaches. At the entity level, with the standard and predicted location entity list, we selected the Precision, the Recall, the F1 score and the Matthews Correlation Coefficient as NER performance evaluation indicators.

$$Precision={\displaystyle \frac{TP}{TP+FP}}$$

(1)

$$Recall={\displaystyle \frac{TP}{TP+FN}}$$

(2)

$$F1={\displaystyle \frac{2\ast Precision\ast Recall}{Precision+Recall}}$$

(3)

$$\begin{array}{c}\hspace{1em}\hspace{1em} Matthews Correlation Coefficient\hfill \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} ={\displaystyle \frac{TP\ast TN-FP\ast FN}{\sqrt{\left(TP+FP\right)\left(TP+FN\right)\left(TN+FP\right)\left(TN+FN\right)}}}\hfill \end{array}$$

(4)

where TP, FP, TN and FN denote the number of true positive samples, false positive samples, true negative samples and false negative prediction samples, respectively.

The Flair NLP Python library (version 0.15.1), Stanford CoreNLP Python library (version 3.9.1), spaCy Python library (version 3.8.7) and Irchel Geoparser Python library (version 0.2.2) were all downloaded and accessed from the Python command line using the Pip standard package manager (version 25.1). The French, German Stanford CoreNLP java library (version 2018–09–04) was downloaded from the Stanford NLP Github Page (https://stanfordnlp.github.io/CoreNLP, accessed on 26 July 2025).

2.4. Natural Hazard Data Description and Spatial–Temporal Analysis

In this study, we also collected observed evidence from an electronic database “Western European Climate from Written Sources, Reports, ca. 1300 to ca. 1500” (https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/K0AJD7, accessed on 13 June 2025) digitized by the Center for Geographic Analysis, Harvard University. Harvard’s data, which is mainly composed of information derived from Alexandre’s treatise [52], compiles narrative accounts of places, countries or regions that experienced climate events in a given year. Some of the written historical evidence requires further interpretation (e.g., “Expensiveness of wine around 11/11”, or “Locusts devoured the wheat crop”); thus, we only kept the records that undoubtedly show that a place in Europe experienced natural disaster events during the period 1301–1500. Subsequently, we employed the same approach described in Section 2.2 to geocode all locations.

For the purpose of analyzing the temporal and spatial distribution of natural hazards, we merged the two datasets by year and coordinates. We then calculated the proportion of complete matches among all observations of both datasets and plotted all natural hazard events using Qgis software (version 3.34).

2.5. Natural Hazard Categorization

Finally, the hazard types were identified using a rule-based method, which has high accuracy when classifying a small sample of texts that contain obvious feature words in each category. Since hazard reports in the merged dataset are concise and succinct summaries of the event in most cases, the summarized feature words can thereby be treated as triggers for each event type. The specific procedure was as follows: we divided the natural hazards into the following six groups by extracting the keywords: extreme heat, snow and ice, wind and hail, rainstorms and floods, droughts, and earthquakes. The common trigger words (

Table 3. Trigger words for natural hazard types.

Natural Hazard Type	Triggers in Textual Data
Extreme heat	Severe heat, great heat, hot
Snow and ice	Severe cold, very cold, heavy snow, snowstorm(s), (strong) freeze, frozen, heavy ice, ice and snow, hoarfrost
Wind and hail	Windy, hail, hailstorm(s), wind force n (n > 5) bft
Rainstorms and floods	Heavy rain, abundant rain, very rainy, continually rainy, ceaseless rain, flood(s), storm flood, thunderstorm
Droughts	Drought(s), severe drought, low water level, low water stage
Earthquakes	Earthquake(s), ground shake

) were defined after randomly sampling 10 records (5 for each dataset) for each hazard type. Consequently, the record was classified according to whether there was a trigger word for a particular hazard type that appeared in the text. Sometimes one particular record in the dataset contained more than one trigger word that belonged to different categories, and we classified it into a mixed type.

3. Results

3.1. Standard Location List

Of the 4413 identified location entities in the standard location list, only 350 toponyms referred to a hazard observation report in a specific year. The other 4063 toponyms were mentions of the places where the evidence shows no significant potential for hazard interpretations (e.g., “In 1349 when the Black Death raged in England, the weather remained very wet”) or positions of various kinds of climate-related phenomena. The automatic geocoding algorithm matched 92.5% of all entities, while the rest could not be geocoded using the GeoNames API service and had to be identified manually. For instance, the geolocation of valleys and passes in the Alpine region proved challenging. Meanwhile, 32.5% of the batch-geocoded coordinates required manual adjustment or correction. Over half (65%) of all the errors stemmed from imprecise outputs for regions (i.e., a deviation from the centroid of the bounding box). This was followed by mismatches due to homographs (21%), mismatches between historical place names and modern coordinates (7%), and error types that remain unidentified (7%). In the final dataset, the number of places, administrative units and regions accounted for 45%, 30.4% and 24.6%, respectively. Of the three types of location entries, the automatic geocoding tool demonstrated the highest level of accuracy (89%) in matching places. The bounding box diameter for administrative units and regions was found to be 495 km and 390 km, respectively.

3.2. Toponym NER Performance Evaluation

The NER experiments were conducted using a two-step approach, i.e., vectorization followed by tagging, for all four geoparsing tools. The SpaCy NLP library (Python/Cython-based) integrated built-in word vectors and was known for its rapidity [53,54]. Consequently, the execution time for spaCy and Irchel Geoparser (<5 min) was less than for Stanford CoreNLP (Java-based, 9 min) and Flair NLP (Python-based, 16 min).

Generally, the proportion of location entities in the identification result (precision) was extremely high (above 0.99) for all four geoparsing tools (

Table 4. Performance of different geoparsing tools after mapping to a uniform tokenization.

	Precision	Recall	F1 Score	Matthews Correlation Coefficient
Flair	0.997	0.98	0.89	0.89
Stanford CoreNLP	0.996	0.79	0.83	0.83
spaCy	0.992	0.50	0.62	0.64
Irchel Geoparser	0.995	0.85	0.82	0.82

). The Flair NLP library showed the highest recall (0.98, i.e., 98% of the location entities in the standard list were captured), followed by Irchel Geoparser (0.85), Stanford CoreNLP (0.79) and spaCy (0.5). As indicated by the F1 score and the Matthews Correlation Coefficient, the overall NER performance of Flair, Irchel Geoparser and Stanford CoreNLP yielded promising results (above 0.8 for both indicators), and the Flair NLP library had a slight advantage over the latter two. While the performance of spaCy was poor, with an F1 score (0.62) and Matthews Correlation Coefficient (0.64) far below the other three tools.

3.3. Description of Natural Hazard Datasets

The final natural hazard dataset based on the literature mentioned in Pfister’s work (hereafter called “the Pfister dataset”) contained 673 records of hazard observations during the period 1301–1500. According to the output of geocoding, there were 157 unique locations. Data points in the Pfister dataset were particularly concentrated in central Europe, north of the Apennine Peninsula and the coastal areas on both sides of the English Channel (Figure 2).

As is visualized in Figure 3, in terms of detailed information on natural hazard events, the top ten words with the highest frequency of occurrence in the Pfister dataset were as follows: “cold” (213 times), “flood” (134 times), “severe” (133 times), “drought” (104 times), “rain” (74 times), “abundant” (51 times), “frost” (47 times), “snow” (46 times), “wind” (36 times) and “hot” (22 times). The majority of these records (669) corresponded to meteorological hazards, while records corresponding to geological hazards were limited, and all of these were earthquake data.

Compared with the Pfister dataset, Harvard’s digitization work (hereafter called “the Harvard dataset”) had a considerably larger number of data points. The screened version of the dataset contained 3091 observations (534 unique locations) with detailed information about the disaster event, the year, and the place name. In total, 92.5% of the locations were geocoded automatically, and 7.5% had to be localized manually. There was an overlap between the entries of the two datasets, and the duplicate data points accounted for 7.4% and 1.7% of the Pfister data and the Harvard data, respectively.

3.4. Spatial–Temporal Characteristics of Natural Hazards

Both datasets provide an overview of precisely localized places with natural hazard occurrences inferred or documented during the period 1301–1500. The Pfister dataset indicates that, over the 200-year period, there were 29 years in which no disasters occurred. However, the Harvard dataset reveals that only three years were disaster-free. Both datasets indicated that Europe witnessed several years of peak natural hazards, including 1305, 1322, 1342, 1343, 1374, 1432, 1491 and 1496 (Figure 4).

Figure 5 shows their spatial distribution in detail. The Pfister dataset provided supplemental information (623 new observations across 153 unique locations) in comparison to the Harvard dataset. These additional data displayed a sparse spatial distribution, primarily in southern Britain, the eastern Alpine regions, eastern Hungary, the Balkans, western Russia, Spain and Iceland. Mainz had the largest count of hazard occurrences (108), followed by Prague (88), Parma (86), Paris (82) and Florence (67). After hazard data categorization and summarization, the ten countries with the highest number of records were Germany, France, Italy, the Netherlands, Czechia, Belgium, Austria, Switzerland, the United Kingdom and Poland (Figure 5). Over the 200-year period, the most common natural hazards for these countries were rainstorms and floods, accounting for 35.2–67.9% of all natural hazards. Furthermore, considerable disparities were observed among the selected countries. A larger proportion of snow and ice disasters (over 40%) seemed to be documented in Switzerland, Austria and Italy, while extreme heat and droughts were reported more in the United Kingdom. Additionally, Czechia was the only country that suffered a lot from wind and hail.

Our result reveals no discernible transition in the geographical distribution (or focus) of European natural hazards during the period 1301–1500. Still, from the initial half of this period to the latter, a decline in the total number of reports was evident for Italy (from 283 to 79) and Austria (from 87 to 72) due to a lack of data falling within the period 1445–1475. This lies in the structure of both the Harvard and Pfister datasets. On the contrary, the total number of reports appeared to have increased in eastern Germany and Czechia, especially for places like Magdeburg, Dresden and Prague. This can be partly explained by a notable surge in flooding reports in this district during the years 1427, 1433, 1485 and 1496.

4. Discussion

Our toponym NER experiment has demonstrated the capability of NLP approaches (i.e., geoparsing/geocoding tools) in extracting georeferenced locations in unstructured texts, which contributes to the digitization and compilation of literature-based natural hazard data. The automatic approach was superior to manual annotation in terms of speed, with an average completion time of less than half an hour, as opposed to several days. The ability of location entity recognition varied among different NLP algorithms. Flair NLP had a better overall performance (high F1 score and Matthews Correlation Coefficient) over Stanford CoreNLP, spaCy, and Irchel Geoparser for texts in English, French and German. This open-source library generates contextual string embeddings for sequence labeling, which capture the meaning of a word based on its surrounding text [55]. As the geoparsing tool with the second-best overall performance, the Java-based Stanford CoreNLP toolkit was trained using the same dataset (CoNLL 2003) as Flair, but adopted a Conditional Random Field (CRF) model architecture instead of neural networks (BiLSTMs, CNNs and Transformer models). The Stanford CoreNLP and spaCy showed unbalanced precision and recall, indicating they were both experts in detecting real locations and avoiding non-locations, while a considerable proportion of locations remained undetected. The latter tool missed nearly half of the real locations. The Stanford CoreNLP performed somewhat better with English and French texts than with German texts. The Irchel Geoparser has been designed to process toponym identification using the same NLP pipeline as spaCy, so the result of our experiment indicates that the size of the model has a significant influence on capturing named entities buried in texts. Generally, the F1 score (0.89) of Flair fell in the range (0.88–0.96) claimed by the official documentation [56], and the F1 score (0.82) of Irchel Geoparser was a bit lower than evaluated by the author of the spaCy library (0.86 for en_core_web_lg NER pipeline) [57]. The recall of Stanford CoreNLP (0.79) and spaCy (0.5) on the compilation of the literature was also comparable to previous experiments on modern text corpora (0.48–0.89 and 0.49–0.75) [53,58,59,60]. The extensive range may be attributable to the utilization of disparate text inputs and model selections across studies.

Based on our findings, false positives (not real locations identified) and false negatives (missed locations) arose from the automated geoparsing process. The geoparsing results yielded a number of non-locations, including directional nouns, abbreviations of proper nouns and fine-grained places such as buildings, farms, vineyards, POIs and glacial deposits. On the other hand, real locations that were easily missed included historic site names and unofficial names of geographical locations, such as the Mediterranean countries. Of all the geoparsing tools, Flair demonstrated a superior aptitude in addressing the aforementioned challenges, especially for the extraction of data from texts with historical place names. The Irchel Geoparser was a noteworthy contender for the generation of datasets with detailed location information, facilitated by its capacity for direct geocoding result output.

In this specific instance of data mining, the automated NLP workflow demonstrates its efficacy in resolving location information in unstructured text. The chosen NLP libraries all use learning–based approaches that capture the association of texts and location references. Word-embedding (capture of semantic relationships between words) models seem to be a useful tool in detecting location entities by their similarity in the text [61]. However, these pre-trained tools have been trained solely on contemporary texts and thus possess limited capacity to recognize uncommon spellings and old place names, and to deal with OCR errors as well. One possible enhancement would be the retraining of geoparser libraries on a large corpus of historical texts. Currently, the training of a custom deep-learning model based on word vectors has the potential to leverage advanced neural networks designed for specific NLP tasks. This approach could also balance the false positive and false negative rates, which remained a concern for all tested libraries. Sun et al. [62] used a three-stage NER pipeline (i.e., pretraining, feature extraction and decoding) for the recognition of natural hazard-related information such as hazard category and location data in domain scientific papers. The optimal named entity recognition model, namely the XLNet-BiLSTM-CRF model, has been trained on an annotated corpus free of texts irrelevant to natural hazards, and outperformed other results for geographical location recognition (F1 score = 0.93), natural hazard category recognition (F1 score = 0.92) and research method recognition (F1 score = 0.92). This pioneering work in the field of disasters sets a benchmark for the recognition of location entities in clean texts relating to natural hazards. Recently, the emergence of newly developed location recognition techniques (e.g., hybrid approaches fusing neural networks) [63,64] calls for annotated textual datasets for model training. Our dataset here presents a reusable training set that comprises descriptions of historic natural hazards for the future improvement of ontology recognition.

Based on a collection of the domain literature, our newly digitized Pfister dataset, together with the geocoded version of Harvard’s climate dataset from written sources, has yielded novel insights into the continental history of climate. The synthetic analysis employs novel materials and interdisciplinary approaches, and corresponds with the special issues on international methods in climate reconstruction and impacts from archives of societies that have been produced by the Past Global Changes Climate Reconstruction and Impacts from the Archives Societies working group (CRIAS) [65]. The temporal distribution of our natural hazard records (1301–1500) coincides with the outbreak of the Black Death (occurring in the middle of the 14th century), thus providing an opportunity to evaluate the social vulnerability to the twin disasters of climatic change and epidemics as suggested by previous studies [66,67,68].

It should be noted that both the Pfister and Harvard datasets likely have inherent limitations caused by the nature of the data. Both datasets are subject to spatial bias, with a concentration of records originating from central and western Europe, where source texts (local chronicles, treaties, private weather diaries, etc.) are easier to access. Nonetheless, some regions, especially the broad districts in eastern and southern Europe, are infrequently referenced by sources, a phenomenon that may be attributable to divergent writing practices across cultures or constrained accessibility. As a result, the small number of entries for these places does not necessarily represent a devoid of natural hazard occurrences. In addition, the final datasets are presented as structured narrative texts collected from raw documentary records, and the entities from scattered historic sources are compiled based on inconsistent screening criteria by different authors. Such criteria are pertinent to the questions of how to include or exclude information as well as the severity of the disaster. The hazard information may be problematic, which suggests users should check the original literature sources. So a rigorous data screening and source verification (e.g., cross-checking the confusable records) are recommended depending on the research objectives and expected outcomes. As a special case, the spatial scales of the locations in both datasets are far from homogeneous, ranging from small villages or historic sites to colloquial areas spanning hundreds of kilometers. Therefore, further quantitative analysis works better with selected data of justified location type, and a check for coordinates and the diagonal of the bounding box, which provides a feasible estimate of an entity’s spatial extent, is typically required before modeling. We then suggest noticing the shift in location coordinates over time, which occurred occasionally when certain geographical regions experienced disintegration or consolidation throughout the long history. The coordinates of these regions provided in our datasets derive from a modern place name gazetteer and are thus approximate. This phenomenon calls for future detection of geographical displacements based on maps that indicate the change in borders of historical regions.

5. Conclusions

Based on a collection of textual records of natural hazard events, this paper constructs a georeferenced dataset of European disaster observations during the period 1301–1500. The dataset has the potential to be used for exploring the intertwined effect of climate change and other factors, including, but not limited to, agricultural practice, plague outbreaks and human migration [69]. Four location entity recognition models, namely Flair, Stanford CoreNLP, spaCy and Irchel Geoparser NLP libraries, are utilized to conduct toponym recognition tasks on the corpus. Given the feasibility of the annotated corpus, the performances of these pretrained models are compared. Finally, the Flair model outperforms the other three with a considerably high recall (0.98).

Combining disaster data from different sources constitutes a valuable asset for historical catastrophology and improves the spatial and temporal representativeness of the data. Generally, our newly digitized Pfister dataset can serve as a supplement to Harvard’s existing database, especially for places in eastern and southern Europe. It is evident that there have been numerous occurrences of natural hazards throughout the 200-year period, notably in 1305, 1322, 1342, 1343, 1374, 1432, 1491 and 1496.

Our newly compiled structured dataset, which will benefit further quantitative analysis, may be regarded as a foundation upon which additional data are superimposed to construct a continental or global natural hazard database. Rich documents from different sources, a considerable number of which are available online in the era of open data, need to be scanned and OCR-coded. Meanwhile, it is imperative to acknowledge the critical importance of data consistency in the domain of data integration studies. Quantitative analysis continues to pose challenges and requires a cooperative effort, with a focus on transcending cultural disparities in future endeavors.