Digitization of Museum Objects and the Semantic Gap

Abstract

This article examines the “semantic gap” in the digitisation of museum collections—the divide between human-comprehensible representations of artefacts and machine-readable data structures. Drawing on a comparative analysis of national museum databases from Latvia, Estonia, and Finland, the study explores how material objects are transformed into digital surrogates and the challenges of creating interoperable, searchable, and meaningful datasets. Key obstacles include inconsistent metadata standards, linguistic variability, and differences in classification systems, which hinder aggregation and transnational analysis. Case studies of temporal, material, and image-based metadata reveal how human-oriented descriptions—rich in nuance, context, and uncertainty—often resist direct computational translation. The research shows that while digital formats offer powerful opportunities for aggregation, search, and reinterpretation of heritage at scale, this flexibility comes at the cost of reducing object-specific richness. The paper argues that the value, or “aura,” of digitised objects lies in their potential for connectivity and cross-institutional integration, achievable only through metadata standardisation and thoughtful design. Understanding digitisation as a culturally embedded process can help bridge disciplinary perspectives and improve future museum data infrastructures.

1. Introduction

Since the digital turn, the saying “the past is a foreign country” [1] has acquired a new layer of meaning. The past without smartphones and the World Wide Web feels like distant history, and a digital representation of the past has become a precondition for its relevance. When everything is a click away, non-digital testimonies of the past, accumulated by archives and museums, risk becoming detached, irrelevant, inaccessible, and, consequently, forgotten. Therefore, one could say that the digitisation of museum collections creates a link between the pre-digital past and the digital present in more than one sense.

Digitisation is challenging because material traces of the pre-digital past, diverse in materials and shapes, have to be represented digitally via a transformation carried out in a complex interaction among hardware, software, licences, people, and protocols [2]. It is neither automatic nor purely technical [3,4]. It requires creative translation of each cultural artefact from a format that is accessible, informative, and meaningful for a human being to a digital representation readable by a machine. In other words, it requires bridging a semantic gap between human and machine perception of the world.

The ‘semantic gap’ in computer science refers to a ‘gulf between the rich meaning and interpretation that users expect from [computer] systems’ and ‘the shallow, low-level features that the systems actually compute’ [5]. IT developers and computer scientists traditionally have used the term to point out difficulties they must overcome in designing IT solutions and focus on teaching a machine to understand human-created content. From a humanities point of view, the term can be used in even broader sense—in understanding digitization as a “culturally loaded” process [6] (p. 15)—in which pre-digital containers of information, such as museum artefacts, inventory ledgers, card catalogues, paper labels, etc., are translated into digital data [7]. In this process, digitizers face multiple semantic gaps that must be overcome to transform material cultural memory accumulated over time in museum collections into data that fits the requirements of contemporary digital technologies.

Considering the growing pervasiveness of digital technologies in all areas of contemporary societies, future museums will be increasingly concerned with conserving the so-called born-digital objects [8] (p. 185). Still, for now, the creation and curation of digitised objects—digital representations of material historical objects that museums collected before the digital era—are just as important. Created either through fast mass digitisation projects that aim to represent digitally entire collections of millions of objects [9], such as the EU heritage database Europeana or the national museum databases examined in this paper, or slow digitisation initiatives focused on specific objects or collections (e.g., the Electronic Beowulf project at the British Library [10]), these digitised objects, also called digital surrogates [8,11,12,13,14,15], data doubles [16], or digital twins [17], can be viewed as translations of material historical artefacts into a digital format.

Due to their numerical nature, digital objects have properties that distinguish them from their material counterparts. From a humanist point of view, the most notable are modularity or the ability to be divided into parts without losing the identity of the whole; transcodability from one format to another (e.g., from audio to visual); and variability or the ability to exist in an indefinite number of copies [18]. These properties that make it possible to process, consume, and manipulate digitised cultural objects differently from their material counterparts have triggered the development of whole new disciplines, such as the digital humanities [19], computational museology [20], cultural analytics [21], and others.

Digital copies should not be viewed as something that would replace the originals. In rare cases, digital technologies allow to create digital surrogates that might provide even more information than the material original, some of the most notable cases being the digital unwrapping of the Mummy of King Amonetop I [22] or the virtual reading of otherwise unreadable En-Gedi scrolls [23], but the majority of digital representation of material artefacts created to this day are relatively shallow and consist of one or several images and basic metadata. Without addressing aspects of material reality that can hardly be digitised—such as smell or touch—the very nature of the digital format, composed of a finite number of digits, means that digital objects are finite compared to continuous material objects. Manovich long ago illustrated this finitude, asking us to imagine colour transitions in an oil painting that are necessarily scalar and, therefore, can theoretically be ‘cut’ into an almost infinite number of pieces, or in other words, with the development of new technologies, can be a source of an endless amount of information. At the same time, its digital copy captures a certain level of detail that, once created, cannot be increased without returning to the original material [18].

Moving beyond strictly visual properties, Hansen (2015) remarks that any quantification is necessarily a reduction in “diverse, complex, qualitative, often localised phenomena into abstract, countable quanta” [16] (p. 125). The database logic of the digital format [24] strips an object of its unique context and privileges its relational properties [4]. From this perspective, a digital copy often is a simplified and finite version of its material counterpart, whose unique properties are partly replaced with simplified relational ones.

In the literature on the digitisation of GLAM (galleries, libraries, museums, and archives) collections, the potential loss in digitisation of material collections into digital format, or what can be called a semantic gap, is addressed through Walter Benjamin’s notion of the aura [11,13,25,26]. In his famous essay “The Work of Art in the Age of Mechanical Reproduction” [27], Walter Benjamin (1965) suggests that through mechanical reproduction, artworks lose their aura—the embeddedness in tradition, space, and time, and a distance necessary for meaningful individual engagement. Some have argued that a digital surrogate lacks an aura, because it is a “sanitised, distancing and disengaging artefact” with no substance or location, and that their immunity to degradation and openness to infinite reproducibility that inhibits their ownership only increases the feeling of disengagement [26]. Others point to digital immateriality as a cause for the loss of aura and argue that digitization often captures just the content of historical photographs [11] or documents [28], leaving out their material qualities and conditions that might be crucial for their appraisal and interpretation.

Owens’ (2018) frameworks for preservation [29] provide a more nuanced and structural view of where the value of various historical artefacts lies and in which cases it is essential to capture their materiality. He distinguishes between artifactual, informational, and folkloric frameworks for preservation. Among these three, artifactual preservation values the object for its historical contiguity. Therefore, in this case, authenticity and materiality are crucial. In contrast, informational preservation values the object’s content, but the medium is insignificant. In this case, the content can be transferred to another medium—material or digital—without significant loss of value [29]. For example, Jane Austin’s diary would have both artefactual and informational value. Its informational contents would be transferable to a new medium without a significant loss. However, its worth having been touched by Jane Austin can hardly be transferred or copied to another medium. Finally, folkloric preservation aims to capture lived knowledge. This can be performed through media and content variations, where any specific artefact is preserved only as an example. In this case, Jane Austin’s diary would be valued as an example of eighteenth-century note-taking practices.

Overall, one could argue that the potential loss of an object’s aura or meaning through the digitisation of a historical artefact can be reduced by carefully constructing the digitised copy by combining images with metadata, where the choice and depth of metadata derive from the preservation rationale. If properly constructed, a digital copy of an individual object, even if lacking a smell or a feel, would contain enough information to capture the value of the original. That means a digital copy of a material object does not necessarily have to replicate the original material. Instead, it has to enrich and amplify the value of its original material through iterative interaction with users of the copy across institutional and international boundaries [11,13]. Burns (2017) writes: ‘The embodiment of the physical object is carried into the digital file through selective and reliable digital capture, robust descriptive and technical metadata, and wide dissemination. Through these processes, Benjamin’s “aura” is not only retained […] but it is preserved and amplified to an extent that the singularity and originality of the physical archival photograph depends almost entirely upon the existence of its digital copies’ [11] (pp. 7–8). In other words, the value of digitised objects or their aura is not in mimicry of the material original but in their potential multivocality, connectedness, and transgression of institutional and international boundaries.

It is important to note that a detailed and in-depth digitisation of individual digitised objects does not automatically result in such connectivity. To utilise the potential connectedness and multivocality, it is crucial to ensure structure, clarity, and metadata consistency to allow us to relate and connect objects within the collections and across institutional and national boundaries. Following database logic, the relational properties of digital objects are more important than the unique properties of each object [4]. Only if such relational properties are built can the digitised objects acquire an aura and enrich the aura of their material counterparts. Even if every object in a collection is meticulously described, without standardisation of features based on which objects can be connected and ordered, the collection resembles a digital pile that, increasing in size, becomes hard or even impossible to navigate. Digitisation or translation of artefacts into objects with relational properties is first and foremost ordering, where one has to “ignore their uniqueness and regard them as typical members of a particular class of objects” [30] (p. 3) to emphasise certain features of an object and ignore the rest.

Since the beginnings of modernity, maintaining order has been a prerequisite of a well-managed museum, but pre-digital and digital order function differently. Museums most often acquire objects already organised in a collection, which has been “assembled with some degree of intention (however slight) by an owner or curator who believed that the whole was somehow more than the sum of its parts” [31]. After the acquisition, curators categorise, classify, and organise objects, adding additional layers of meaning. In the pre-digital order, each thing had one place. This kind of order resembles a tree that starts with a few broad categories that divide into subcategories like branches. Each object can be located as a single leaf on this tree [32]. Maintenance of such an understanding of order rests upon the human ability to ignore unfitting pieces of reality or the flexibility of mind [30]. Pre-digital order rests on the assumption that the world can be organised in one best way, where each thing has its natural place, and this illusion of order can be maintained because, when necessary, humans flexibly recategorise things and redraw boundaries.

In digital format, the fit between the messy reality and a desired order is achieved by different means. Each object does not have to have one place. “Leaves” can be organised in unlimited ways [32]. Because of numerical representation, the ordering of digitised objects can be partly automated. Digital machines can look through, count, and add much faster than humans, which opens up the possibility of much quicker and more efficient classifying and organising. However, for digitised objects to be connectable, they should share features based on which digital orders can be created. The features should be standardised and expressed consistently. If, on an individual level, an object is described inconsistently, the flexibility of digital ordering is lost. This new connectivity does not happen automatically. It must be created. A challenge in digitising a museum object is ensuring it is connectable with other objects within the same institution, across institutional boundaries, and even transnationally. The semantic gap between a non-digital and digital format manifests most prominently upon fitting analogue traces of the past in digital orders and balancing each object’s individual and relational properties.

Multiple insightful studies have examined the gains and difficulties in translating pre-digital into digital format through interviews and discussions with museum practitioners, IT specialists, and users of the digitised collections. Some of these studies have illuminated semantic gaps between the users of the digitised data and the museum practitioners [33] or between information management and the humanities [34]. This study contributes to this literature by taking a somewhat ethnographic approach. It examines three datasets of digitised museum collections from a user perspective—a perspective of a memory studies scholar interested in an empirical overview of museum collections on a national and transnational level. Difficulties and challenges faced while using the two databases are used to illuminate the semantic gap between human-comprehensible and machine-readable data that must be overcome upon translating the material cultural memory into a digital format.

2. Methodology and Data

The observations that form the empirical basis of this paper were made as part of a broader digital memory project, where data from three national aggregated databases of museum collections were used to provide an empirical insight into “storage memory” [35,36] on a national and transnational level. While the results of this project are reported elsewhere [37], this paper reflects on the actual analysis process. It uses challenges and difficulties in data analysis to shed light on the semantic gap between the material museum artefacts and information stored in pre-digital catalogues, ledgers, inventory books, and their digitised counterpart.

In memory studies, storage memory refers to the traces of the past, such as books, documents, historical artefacts, etc., collected by memory institutions, that remain stored away and are never checked out or displayed. In the case of museums, typically about 90% of the objects in museum collections remain in storage and are never displayed in any exhibition [38]. Spread out across thousands of institutions, documented in thousands of paper catalogues and inventory lists, in the pre-digital era, museum storage could hardly be studied beyond the bounds of each institution. Digitization of museum catalogues promised the uniting of dispersed containers of institutional data into a vast connected digital archive and, at least theoretically, opened up the opportunity to study empirically material cultural memory across institutional and national boundaries.

The project aimed to test if this promise works in practice—to examine actual databases of digitised museum collections and attempt to generate a national and even transnational level overview of sociomnemonic densities [39] of different historical decades in storages of Northeastern European museums, as well as explore other potential ways to overview this what memory scholar would call “storage memory” on a national and transnational scale. Because the aim was to gain a national and international level overview, three databases that aggregate museum collections on a national level were selected—the national aggregator of museum collections in Latvia (the Catalogue of National Museum Collection—Nacionālā Muzeju Krājuma Katalogs or NMKK), a similar national aggregated database in Estonia (Estonian Information System for Museums—Eesti Muuseumide Infosüsteem or MuIS), and in Finland (FINNA).

NMKK is a database of Latvian museum collections run by the Cultural Information Systems Center (Kultūras informācijas sistēmu centrs or KISC), a governmental agency in charge of information management for Latvia’s cultural sector. It contained digital copies of about 1.4 million objects at the time of analysis. MuIS is run by the Estonian National Heritage Board (ENHB), a government agency responsible for managing and protecting cultural heritage in Estonia, parented by the Ministry of Culture. It was launched in 2010, at the time of data collection in 2021, was undergoing a significant restructuring, and contained data on 3.7 million objects across 72 museums. FINNA is a search service run by the National Library of Finland. It brings together materials from Finnish archives, museums, and libraries and was created as a part of the National Digital Library Project (2008–2017) of the Finnish Ministry of Education and Culture. At the time of analysis, it contained digital copies of about 1.7 million objects across the storage of 73 museums and museum consortia.

The analysis of the three databases was carried out in consecutive steps. First, the web interfaces of all three databases were examined, and data accessibility was assessed. Second, the whole dataset of the smallest of the three databases—NMKK—was harvested, cleaned, and analysed to explore possibilities for an aggregated national-level view of storage memory. Third, potential categories for an overview on a transnational level were identified, and the analysis was narrowed down to physical objects. Finally, a selection of data was acquired from FINNA and MuIS to aggregate a transnational overview. Data harvesting, aggregation, cleaning, and exploration were performed using Excel for Mac 16.78.3, Open Refine 3.6.0, and Python 3.12.9, and detailed fieldnotes on the process of analysis and the obstacles were made throughout the process.

The subsequent analysis is based on these fieldnotes and uses difficulties met in the process and imperfections observed in the digitised data to point out the semantic gap that the digitizers of the museum collections have to overcome. It is important to note that the observations were made from a specific user perspective, from a perspective of a memory scholar and digital humanist with above-average skills in Excel and Open Refine, and a basic background in Python, but without in-depth knowledge of digital collection curation and management or database programming. The observations reflect the existing usability of the data from this perspective at the time they were accessed, i.e., looking at the situation as it is, not as it could or should be.

3. Results

3.1. Web Interfaces and Accessibility

For any digital data to be usable, it has to be accessible. In an ideal world, the web interfaces of the digitised collections would allow browsing through lists of objects and searching and filtering the collections by multiple parameters. Because it is hard to design a web interface that would accommodate the needs of all potential database users, open access to structured computer-readable data is crucial [40].

From all three databases, only FINNA had a web interface that allowed any aggregate insight into the data. A sidebar in the FINNA provided an opportunity for a faceted search—a more accessible option for a non-expert user than a regular search. Faceted search allowed one to select specific values from a list of object types, museums, topics, regions, years of manufacture, and other parameters, revealing the total number of results corresponding to each selection. In combination with the API access, this provided a national-level aggregate view of FINNA data and allowed for filtering and accessing the actual data for further insight.

The web interfaces of NMKK and MuIS provided a minimal view of the data. In NMKK, the user could narrow data by museum, collection, and object type, in MuIS—by museum and collection. NMKK and MuIS provided an advanced search function that allowed a search by many parameters but did not specify possible values, making the search almost inaccessible for a regular user. In the case of NMKK, the searchability of data was severely limited by inconsistencies in data structures and formatting that were revealed during analysis and will be reported later in the paper.

From the three databases, only FINNA allowed open access to the dataset behind the web interface through an API service. NMKK did not provide any direct access to data. MuIS had an API service, but its documentation was available only in Estonian. When documentation was requested, the database administrators offered to prepare the necessary dataset instead of providing instructions on how to use the API.

3.2. Aggregation on a National Level—NMKK

In all three databases, digital representations of objects were shallow—each object represented by an image (or in rare cases with several images) and supplementary metadata. Not all objects had images, and the metadata elements varied. Some objects had only a title and an ID number. Others had very detailed information—a detailed description, information on its size, the materials it is made of, its author, the geographical location it originates in, and others.

Only the smallest of the three databases—NMKK—was examined to attempt data aggregation on a national level. The most inhibiting factor was a lack of metadata fields shared across the whole collection.

Even though the web interface of NMKK made an impression that it is an integrated database, the analysis of the actual data behind it, received from database administrators in December 2019, revealed that each of the 127 museums uses different metadata categories to describe their objects. There are very few categories shared across the museums. The data was received in 127 CSV files, one for each museum. These were aggregated into a single CSV file, cleaned, and explored using Open Refine. Combined in a single data table, it contained 438 metadata fields for each object. Figure 1 illustrates the use of various metadata fields across the museums.

As shown in Figure 1, the number of fields that the museums use to describe the objects varies from 100 to 10. Of all 438 metadata fields, 332 were used by a single museum. Only 10 fields were used by almost all 127 museums. These fields were as follows:

-

Name of Object;

-

Object ID;

-

System Number;

-

Museum holding the object;

-

Address of the Museum;

-

Date of Entry in the System;

-

Name of the collection the object is part of;

-

Type of Collection (primary or supplementary);

-

Type of Object.

Some discrepancies in the use of the fields could be explained by the vast differences between the objects that museums collect. For example, a museum of nature might use a field called “Date of Collection” to document when the museum harvested a specimen. In contrast, a museum of art might extensively use the “Date of Creation” field to report when a specific artwork was created. Similarly, the former would have no use for a field “Author”, while this field would be crucial for the latter. Nevertheless, a content analysis of the metadata fields showed that numerous fields are used to record the same information. For example, information on the historical period or date was recorded in more than 40 fields, and many museums use one or a few.

These discrepancies in field usage illuminate the semantic gap between human and computer data readability. A museum practitioner or a researcher could access the information on each object by browsing and viewing objects one by one and being able to recognise the data if it is labelled as “Date”, “Other Date”, “Date (Approximate)”, “Date of Production”, or “Year of Production”. In contrast, any attempt to aggregate data or search for objects from a specific year or historical period did not work. For data to be computer-readable, these numerous fields had to be translated into a single field, sorting out multiple and contradictory values. This creative side of the translation process will be discussed further in the section on transnational aggregation of time data.

In the case of NMKK, the only field used by all museums that could serve as a basis for the aggregation was “Object Type.” Figure 2 shows an overview of all objects in NMKK by object type.

The aggregation based on “Object Type” was easy because its values were standardised and based on a hierarchical classifier. Because the field “Object Type” apparently had been a mandatory field to be filled when entering a new object in the database, and because it was based on a hierarchical classifier and the museum practitioners entering the information had to select a type from a drop-down menu rather than type in text, like in the case of dates, the data in this field was clean and well-structured and could be successfully used for aggregation.

Unfortunately, object type could not be used for data aggregation across the three databases because MuIS did not have a standardised classifier for object types, and FINNA had a different classifier. At the same time, this data field was used to narrow down transnational analysis to a specific kind of object—physical objects.

3.3. Aggregation of Data on the Transnational Level

Due to limitations in computational resources, the comparative analysis with FINNA and MuIS could not be carried out on all datasets, and the transnational analysis was narrowed down to physical objects. The rationale behind this decision was twofold: First, this type of object could be identifiable in FINNA and MuIS. FINNA had a type “Physical objects” in its classifier. Second, the size of the combined dataset of about 1.5 million objects did not exceed the computing capacity of the author. Third, this type of object seemed pertinent to a study focused on museums, physical objects rarely being part of the collections of other memory institutions.

FINNA data on all physical objects (about 358,000) were acquired in March 2021 through the FINNA API service (Application Programming Interface—a programming interface or a set of protocols that enables software applications to communicate with each other). MuIS data on physical objects (about 325,000) were requested from the Estonian National Heritage Board and received in April 2021. Because the overall project focused on calculating the mnemonic density of different historical decades, the primary category for aggregation was date. Another category considered was materials—a category available in all three databases. Finally, because all three databases contained images, an attempt was also made to generate an overview based on images.

3.4. Aggregating Based on Dates

The most apparent connecting element for historical objects in museum collections is the historical period to which each object belongs. Depending on the object, it might be the year it was produced or, in some cases, the period it was originally used. Using metadata on time, we can organise objects by year, decade, or century and thereby compare how densely various periods are represented or how many historical artefacts date back to each historical period. Figure 3 illustrates differences in mnemonic density [39] of decades from the 16th to the 20th century based on the number of objects in NMKK, MuIS, and FINNA (for more detailed analysis, see [37]).

Selecting a specific period allows us to explore other elements of objects. For example, we can learn what kind of photos were taken in particular decades, compare fashion styles of the various periods, or examine what tools were used. To order objects this way, we need consistently formatted, machine-readable metadata that associates the object with a specific year or period.

In the case of time, the framework for comparison is not a problem—the dates can be arranged in chronological order. Language differences do not hinder analysis because a date can be and usually is expressed numerically. Nevertheless, the format in which dates are encoded might vary. Apart from cultural variations in formats, e.g., American (mm.dd.yyyy) vs. European (dd.mm.yyyy) format, there are plenty of possible variations within each culture, such as the use of a slash, dash, or dots to separate date, month and year, linguistic instead of a numerical expression of the month, the positioning of the year in the beginning instead of the end, and others. All these variations might not make data incomprehensible for a human, but they present a challenge for a machine.

Exploration of data in NMKK lead to some observations that might apply specifically to NMKK, and some that characterise documentation of historical artefacts in general. First, most objects cannot be dated to a specific year. At best, their production or use time could be dated with a period. In some cases, the object is associated with a historical event that has lasted for several years or decades, e.g., ammunition used in World War II, and therefore is dated to that period. In most cases, the date in a period format is kept because of uncertainty. An exact production year is unknown, and the date is given as an approximation. Such approximate dates can be as broad as a century or, in the case of archaeological objects, even multiple centuries.

Second, uncertainty about the date is an integral part of information about the object and is often expressed linguistically in diverse formats that make the data machine-unreadable. This was especially true for NMKK data. The following variations in the wording were typically used about the date in NMKK:

-

‘around’ a particular year;

-

‘the beginning’, ‘the middle’, or ‘the end’ of a decade or a century;

-

‘The first half’ or “the second half”, or a decade or a century;

-

A specific ‘quarter’ of a decade or a century.

These expressions are meaningful and valuable for a human reader, but are not readable by a machine. They show how flexible human categories are and how pre-digital metadata documentation in card catalogues has tolerated this flexibility. To be digitised, such flexible formulations of time cannot be copied to the digital format. They have to be reinterpreted. For example, dating an object “around 1983” into a computer-readable format requires deciding how many years before and after 1983 “around” refers to. This uncertainty is even more apparent when dealing with a decade or a turn of the century. Should “around 1900” still be translated to ‘1899–1901’ or perhaps the turn of the century requires a more extended ‘around’ period?

NMKK data also had purely descriptive “dates”, such as the following:

-

‘The Early Metal Age’;

-

‘Times of Alexander I’ and “Times of Alexander II”;

-

‘After the money reform of 1422–1426’;

-

‘The 12th–14th century by the classification of E.Mugurevics’.

All of these formats of dates are meaningful, but even for humans, they require additional knowledge. The first is easily translatable—the Early Metal Age refers to the period from 2000 BCE to 300 CE. The second is more questionable—does it refer to the period Alexander I lived (1777–1825) or the time of his rule (1801–1826)? The third is even more challenging. Technically, after an event, it should be translated to a period that starts with the event and lasts till today because the end date is not specified, but we can assume that a museum curator would be able to date the specific object more precisely than that. The last adds unnecessary information for the machine, but is probably essential for the curator.

Third, the analysis also revealed how the digital format can amplify a human mistake. For example, the Finnish Prison Museum (FPM) in FINNA digitally holds many objects from 2000 to 1900, years before the Common Era. These include coat hangers, pencil sharpeners, pots, and other everyday objects that could not have been used in ancient history. In the user interface, a person can find information about the object that states it was created, ‘käyttöaika-1993 viimeistään arviolta’ (i.e., ‘service life-1993 at the latest estimate’—according to Google Translate), which seems reasonable. However, if this interface is used to search for objects from the 1990s, this part of the FPM collection does not appear in the search results. Instead, one can find it among objects from the era before the beginning of the Common Era by adding ‘-’ in front of the year. A human reader would dismiss a tiny line in this context as a dash, but the machine treats it as a minus sign and categorises the object as one from before the Common Era. If this were a single mistake, it would not make much difference. But suppose it is a consistent mistake of a hardworking human typist or a careless programmer. In that case, it can misplace an entire collection of objects (in this case, 632 objects) about 4000 years back in history.

Translating information on dates recorded in card catalogues and ledgers to digital metadata is not a technical process. It involves separating information on the date from other valuable information, such as the degree of uncertainty about it, the methodology used to determine it, and others. Also, the digital format requires much more meticulous wording, where a simple dash can displace and make digitally inaccessible a whole collection.

3.5. Aggregating Based on Materials

An element that could serve as a basis for aggregation is the material each object was made of. Aggregation by material might help explore material culture. For example, from a cultural studies point of view, it would be interesting to study the cultural representation of various materials, say, the use of birch or oak in Latvian, Estonian, and Finnish museum collections. An overview of the materiality of museum collections might also be of some value from a collection management and preservation point of view.

In MuIS, the metadata on materials is based on a standardised classifier, and it is easy to create an aggregate overview (see Figure 4).

Aggregating data on materials was much more cumbersome in the case of MuIS and FINNA, where this metadata field was not standardised. In both cases, data on the material is expressed linguistically, so the aggregation of NMKK and FINNA data requires translation between Latvian and Finnish. At the same time, translation of material values is less demanding than translation of other linguistically expressed elements, such as title, description, or type, because there is a much smaller degree of variation in possible values. The materials an object can be made of are not unlimited, and most objects in both databases are made of a relatively narrow range of materials.

From a data processing point of view, translation constitutes a lesser problem than grammar, which makes some languages harder to digitise than others. Latvian and Finnish are synthetically inflected languages, i.e., the endings of words change to express different grammatical categories—tense, case, voice, aspect, etc. This means the same material expressed as an adjective and a noun would result in two different words. For example, ‘wood’ translated as a noun in Latvian would be ‘koks’, in Finnish—‘puu’, but translated as an adjective in Latvian would be ‘koka’, in Finnish—‘puinen’. Both forms might be used to determine the material of an object. Materials were predominantly expressed as nouns in FINNA data, but in NMKK data, using both nouns and adjectives was relatively common. This discrepancy would not represent a problem for a human reader—both grammatical forms are equally meaningful and mentally lumped together—but each grammatical form is treated as a separate entity for a machine reading.

Second, even though linguistically expressed, materials can theoretically be ordered—the vast diversity of materials can be categorised into larger groups, such as paper, wood, metal, textile, glass, stone, skin, ceramics, plastic, and digital. There are some categories used in both FINNA and NMKK that are clear: birch, pine, and oak would be classified as wood; cotton, wool, and flax—as textile; silver, gold, copper, brass, tin, steel—as metals; and then separate categories would be glass, bone, skin, and stone. Other categories are not that clear-cut. For example, synthetic fabrics like polyamide and polyester can be classified as “plastic” or “textile”. Because the vast majority of museum objects come from the pre-plastic age, the museums generally use one generic category and do not distinguish between various kinds of plastic. Most likely, in a hundred years, the classification of plastic in museums will be much more elaborate and include PET, PP, HDCP, PS, PVC, and others.

Third, it is apparent that the meaning of various materials is contextual. For example, a typical value of material for an oil painting is “Oil on canvas”. “Canvas” can undoubtedly be categorised as a textile. On the other hand, it is seldom specified what kind of fabric the specific canvas is made of. It could be made of cotton, linen, hemp, or polyvinyl chloride. These distinctions are currently considered essential for clothing but not crucial for artworks, but in the future, that might change.

Finally, many physical objects are composed of several materials, making analysing this element more difficult. In both NMKK and FINNA, data on the materials of these combined objects differ in their level of elaboration. In some cases, only a few materials are listed, e.g., a painting is described as “oil, canvas” without reference to the material of its frame. In others, all materials are listed. In still others, the use of each material in the composition of the object is specified. In some cases, even though the material is the same, e.g., “bottom: wood/board, bottom: wood/wood, sides: wood/wood, sides: wood/wood”. This highlights another problematic issue while transitioning from human-comprehensible data to machine-readable data. Some objects can be easily categorised into neat categories of “paper”, “metal”, or “wood”. Others, however, are made of multiple equally essential materials. For example, a doll, made at the end of the 19th century, that has a porcelain head, a fabric body, full clothing made of cotton, lace, wool, and leather, and even an embroidered handkerchief (Doll ca. 1897, K10273:2, FINNA) cannot possibly be reduced to any few machine-readable categories for material without losing a part of important information.

In summary, translating information on materials recorded in card catalogues and ledgers to digital metadata involves dealing with the specifics of language and the diversity of materials. Some of these challenges can be solved by developing and applying classification standards. Still, it is clear that in digitising objects composed of multiple materials, a compromise between the detailed description of the specific object and its relational properties will be an issue. Even when digitised, metadata on the material will have to be continuously updated due to the development of new materials, like synthetics and plastics.

3.6. Aggregating Images

An image of the artefact is the first that comes to mind when we think of a digitised object. Considering the availability of image capture technologies—cameras and scanners—it is also easy to create. On the other hand, using images in meaningful ordering and classifying artefacts is far from straightforward due to the semantic gap between human and computer vision. Where a human sees an image of a vase or a painting, a computer recognises a grid of monochromatic squares or pixels that can be ordered by saturation and colour, but hardly by the semantic meaning of the image. Analysis of colours and saturation can be used for the meaningful ordering of artefacts, e.g., the change in colours in works of an artist in different periods of his life [21], but in the case of images pooled from many sources, as is in the case of FINNA and NMKK, this way of ordering has severe shortages. The same object, digitised with a different capturing technology or using different parameters, can result in very different colours and saturation, limiting the applicability of this kind of ordering.

For the past decade, the development of computer vision technologies has opened the possibility of ordering images by their semantic meaning. An essential step towards this development was the creation of ImageNet—a publicly accessible dataset of manually annotated images—that, since its creation, has been used for training visual object recognition software. Currently, the dataset contains about 14 million pictures classified in 1000 categories. Since 2010, software developers have competed in an annual challenge called The ImageNet Large Scale Visual Recognition Challenge, whose solution will be able to classify images with greater accuracy [41]. Between 2010 and 2015, there was remarkable progress in applying convolutional neural networks (CNNs) for image recognition.

Even though these software models are openly accessible to any user and are widely used in numerous areas of everyday life, from automatic tagging of photos by iPhone to traffic regulation and medical diagnosis, the application of machine vision to museum collections is just beginning. Trained on sets of images people post on the internet, these intelligent algorithms are better at recognising numerous breeds of dogs and cats than distinguishing an ornamental plate from a coin. Recently, museums have been fine-tuning pre-trained models on specific cultural datasets to help with labelling artworks in their collections [42], to identify persons in historical photograph, or to aid art scholarship in the analysis of subtle aspects such as structures of brush strokes or colour scales [42,43].

Figure 5 shows a machine vision of the 19th century in FINNA and NMKK data. The visualisation was created using a pre-trained convolutional neural network, VGG-19, invented by the Visual Geometry Group (VGG) at the University of Oxford in 2014. The algorithm has plotted images by their semantic similarity—the more semantically similar two images seem to the machine, the closer they are plotted. Images seen as semantically identical are plotted on top of each other. On the right side, there are three zoomed-in sections of the plot.

The algorithm has accurately recognised chairs and clothing, but has lumped coins, plates, medallions, and other round objects together. This suggests that, in the case of such databases as FINNA and NMKK, machine learning algorithms for visual recognition can serve as a tool for exploration rather than classification. The semantic gap between machine vision and human understanding can open up new creative interpretations, but it cannot be a reliable management tool. To diminish the gap and develop reliable algorithms for image-based object classification, museums must train algorithms using their collections. Because each museum’s collection might be too small for such purposes, collaboration among several museums might be necessary [44]. Aggregated databases, like FINNA and NMKK, might be helpful for such cooperation in the future.

4. Conclusions and Discussion

The “digital turn” notion emphasises the difference between the digital present, its overabundance of information, fluidity, connectivity, and the pre-digital past, with institutional fragmentation, hierarchies, and access limitations. As with any transition and transformation, the reality lies somewhere in between. The old ways and structures exist along with the new, and it is much more sensible to talk about a range of new constellations than about a clear-cut between the old and the new. Sometimes, a close look at the in-between tells more than a comparison of then and now. Digitised museum collections, examined in this paper, lie between the new digital reality and the old pre-digital one. The remains of the old patterns of information within the new digital order allow us to highlight the difference between the two and to see the semantic gap between a human’s and a machine’s understanding of the world.

Museums preserve information about the past embodied in cultural objects and paper-format documentation. Digitising these traces of the past opens up possibilities to explore and oversee the accumulated cultural memory as it has never been possible. It allows the creation of multiple new orders—to see new patterns, such as the combined mnemonic density of Finnish and Latvian heritage, the representation of various materials in museum collections, and even a machine view of a particular period or nation. All these digital orders open possibilities for a fresh interpretation of the past and allow us to see it outside the framework of established categories and classifications.

At the same time, this newly acquired flexibility of digital ordering comes with a loss of flexibility and context at the level of each particular object. Objects must be digitally captured consistently and standardised to be connectable in new, meaningful orders. Closely examining the digitised data shows that the human way of characterising reality is contextual and flexible. Flexibility in the expression of time is used to compensate for uncertainty. Regarding materials, flexibility and contextuality allow us to represent the complexity of reality. Digital ordering provides flexibility on the aggregate level at the cost of standardisation on the level of particular objects.

One effective strategy for addressing many of the challenges identified in this study is the adoption of controlled vocabularies for key metadata fields such as object type, material, and date. Controlled vocabularies or standard descriptive vocabularies—structured, standardised lists of accepted terms—can reduce inconsistencies, facilitate cross-institutional aggregation, and improve machine-readability without entirely sacrificing descriptive nuance (see [45] on application in archaeology). When applied consistently, they enable the creation of more interoperable datasets, support multilingual interoperability through term mapping, and enhance both searchability and analytical possibilities across national and transnational heritage platforms.

The current study used digitised data to grasp possibilities for digitised ordering. It focused on the imperfections and flaws of digital infrastructures to shed light on the semantic gap between pre-digital and digital information. A better understanding of this gap might improve communication and foster understanding between the IT developers who design databases and IT applications and museum practitioners, curators, and conservationists who use these databases. A better understanding of digitisation as not a technical but a culturally transformative process would allow the development of a more efficient data infrastructure and lose as little as possible of its aura in this process of mechanical reproduction.

From a scholarly perspective, this paper contributes to the literature on cultural heritage digitisation by offering an empirically grounded, cross-national comparison of metadata practices in museum collections. By approaching the problem from a user’s point of view, it provides a rare account of how semantic gaps manifest in real analytical workflows, complementing more technology-driven or curator-focused studies. The findings highlight not only technical shortcomings but also the cultural and epistemological dimensions of digitisation, showing how decisions about classification, metadata, and standardisation influence the interpretive possibilities of heritage collections.

5. Limitations and Future Studies

The study is not without limitations. It focuses on three national databases from a specific geographical region, which may limit the generalisability of findings to other cultural or institutional contexts. The analysis reflects the state of these databases at a particular moment in time, and changes in technical infrastructure, policy, or practice may have since altered their usability. Furthermore, the study adopts a perspective informed by memory studies and digital humanities; a more technically oriented approach or a broader survey of user experiences might reveal different dimensions of the semantic gap.

Building on these findings, future research could expand the scope to include other types of heritage institutions, such as archives and libraries, or explore more diverse linguistic and cultural contexts. Future research might use other digital tools or explore other features to explore the digitised data; for example, analysing the timestamps of each digitised object would allow the creation of a genealogy of digitisation. Also, it might be interesting to learn if curators and museum practitioners find the digital ordering of their collections informative, valuable, and engaging. In addition, collaborative projects between IT developers, curators, and researchers could be studied as experimental testbeds for developing shared metadata standards that balance machine-readability with the preservation of contextual richness.