Datasets for Material Culture Studies: A Protocol for the Systematic Compilation of Items Held in Private Hands

Abstract

Crowdsourcing has been widely used in cultural heritage research, but mainly from an institutional perspective. Research into items of material culture often requires the researcher to examine specimens held in private hands. The dispersed nature of such holdings, primarily as collectable and, thus, tradeable objects, requires different techniques for identification and access than that for items held in museum collections. Crowdsourcing data from online discussion groups and online marketplaces, coupled with snowballing, represents a very powerful tool. This paper discusses the relevant parameters and provides a protocol for the systematic compilation of items held in private hands.

1. Introduction

‘Crowdsourcing’ [1] has become a widely accepted tool in academic research [2], primarily for collecting primary, general or specific observational datasets (e.g., iNaturalist [3], FrogID [4]) or for transcribing or converting data from analogue to digital records [5,6,7].

In the heritage profession, crowdsourcing has been used to generate observational and geo-referenced datasets of physical objects [8,9] and sites [10,11], to capture intangible values and cultural memories attached to heritage places [12,13], for identification of no longer extant places depicted on historic photographs [14] as well as for data transcription and verification projects [15,16,17,18]. In addition to the productive outcomes, cultural institutions have seen crowdsourcing as a means for public outreach and engagement of the wider community [15,16,19].

Crowdsourcing is a useful and powerful tool where academic research relies on the analysis of objects of material culture held in private hands, be they collectibles or heirlooms. Indeed, for some research into material culture, public collections are either inadequate or do not exist. At present, crowdsourcing of these data occurs on an ad hoc basis, guided by the relevant researcher’s interests and abilities.

This protocol was developed to ensure the systematic and structured compilation of data sets that are comprised of items held in private hands. It is based on the author’s multi-year experience in crowd-sourcing information, which implies that a certain amount of self-citation is unavoidable while exemplifying issues.

Objects of material culture held in private hands can be providers of primary data, where the object itself if the focus of study. Examples are trophy fish [20], cast sculptures and figurines [21] or pottery decorated with specific motifs [22]. Objects of material culture are more commonly a means to an end as providers of secondary data. Here the object contains information that illuminates a research question. Examples are items that can provide pictorial evidence of events and places (e.g., commercial postcards [23]) or human behaviour (e.g., tourist photographs [24]), to illuminate strategies of subliminal marketing (e.g., promotional QSL cards [25]) or production processes [26] or to supply dated genetic or morphometric samples (e.g., taxidermized Murray Cod [20] or Iberian Lynx [27]).

The connection between objects and secondary data can also be less obvious. In order to understand the dimensions of decision-making of German colonial administrators in Micronesia, for example, it was necessary to understand the actual mail-run times, especially in the period before reliable postal steamer services could be established [28]. A methodologically independent data set could be developed based on actual postal envelopes, which, in keeping with late nineteenth and early twentieth century custom, carried a postal stamp on the front of the envelope―cancelling the postage stamp and dating the receipt of the letter by the postal service―and an arrival stamp on the verso―dating the receipt of the letter by the destination post office and the day of delivery. The dataset could be compiled through a call to philatelists who not only collect stamps but also entire covers [29,30].

At present there appears to be no consolidated methodology for compiling such datasets. This paper will advance a replicable and reportable protocol for the identification of data sources with suggestions for the solicitation of further details.

2. Protocol Design

This protocol has been developed based on examples in the literature and on the author’s multi-year experience.

2.1. Nature of the Data Sources

Common to research into research material culture held in private hands is that the data sources are, by and large, widely distributed and that their curation is not stable. As the objects are collectible and thus tradeable entities, the data set is somewhat fluid. Indeed, there is no guarantee that objects accessed at the time of the research project will remain accessible in the future, either because the owners have changed their mind regarding access, or the object has been passed on to new owners―who either may be unknown to the previous owner (if the item was sold at a trade fair, for example) or who may be disinterested in the research.

While the documentation of an object is research project specific, the inherent fluidity implies that each act of documentation should be carried out on the premise that the object will never be seen again. This requires that documentation of the object be holistic and follow standard museum documentation parameters [31,32].

2.2. Locating Data Sources

Since material culture held in private hands is widely distributed, searches will, in most instances, only return single objects. Specialized collectors (see below) can function as node points, not only connecting the researcher with other collectors, but often also providing access to multiple specimens.

Initial data acquisition occurs via systematic online searches using text-based search logic on the general World Wide Web using multiple search engines (all accessed 10 February 2023), such as Google (www.Google.com), Microsoft Bing (www.bing.com) and DuckDuckGo (duckduckgo.com). Unless an item of material culture is country-specific, it advisable that searches are carried out in the key languages, i.e., English, French, Spanish and German, to be augmented on a case-by-case basis by Italian, Arabic, Japanese and Chinese, to capture the majority of mentions. It is important to appreciate that the items of material culture may be mentioned both in passing and as the focus of a web page. Unlike systematic literature reviews in academic research, where more or less standardized keywords can be applied, personal web pages are created without much or any editorial control and thus will feature a plethora of terms. Therefore, the search process must be iterative, re-running searches using newly discovered terms or observed common spelling mistakes. Where searches include digitized text of printed historic materials, infelicities in the optical character recognition system, which can generate obscure renderings, need to be considered (e.g., Phoenix canadensis instead of P. canariensis, [33]).

The main search engines for the World Wide Web use ranking algorithms to sequence the results for search terms. The results sequencing is influenced by the type of algorithm used but also by preferential tweaking of the outcome in terms of internal page linking and externalities, such as advertising [34,35,36,37]. As some of these change over time, search results will not be consistent over time.

As collectible items are tradeable items, they can be systematically searched for via online auction houses, such as eBay USA (www.ebay.com, accessed on 10 February 2023) and its international offerings (e.g., www.ebay.fr or www.ebay.de, accessed on 10 February 2023), CataWiki (www.catawiki.com, accessed on 10 February 2023), Delcampe (www.delcampe.net, accessed on 10 February 2023), MercadoLibre (mercadolibre.com, accessed on 10 February 2023), Yahoo Auctions Japan (auctions.yahoo.co.jp, accessed on 10 February 2023), Hood (www.hood.de, accessed on 10 February 2023), Invaluable (www.invaluable.com, accessed on 10 February 2023), Ricard (www.ricardo.ch, accessed on 10 February 2023), TradeMe (www.trademe.co.nz, accessed on 10 February 2023) and Collector’s Mark (thecollectorsmark.com.au, accessed on 10 February 2023); as well as associated sales platforms, such as Amazon (www.amazon.com, accessed on 10 February 2023), Etsy (www.etsy.com, accessed on 10 February 2023), Facebook Marketplace (www.facebook.com/marketplace, accessed on 10 February 2023), eBay Kleinanzeigen (www.ebay-kleinanzeigen.de, accessed on 10 February 2023), Kijiji (www.kijiji.ca, accessed on 10 February 2023) and Gumtree (www.gumtree.com.au, accessed on 10 February 2023). In addition, there are specialist sites, such as Zentrales Verzeichnis Antiquarischer Bücher (www.zvab.com, accessed on 10 February 2023) or AbeBooks (www.abebooks.com, accessed on 10 February 2023).

Image aggregator sites, such as PicClick (picclick.com, accessed on 10 February 2023) allow ready access to images contained in the listings, and through these access to the original listings themselves. In parallel to text-based searches, image searches can be carried out using the same search terms. Once images have been identified, similar objects that may not have been adequately described in the text of a web page can be located using reverse image matching software provided as part of the search function of Google (lens.google.com, accessed on 10 February 2023) and Bing (www.bing.com/images, accessed on 10 February 2023). PicClick and, in particular, Bing, which seems to have a ‘deeper memory’, may also return archived images to which the links have gone stale. Pasting a stale URL into the Internet WayBack Machine (archive.org/web/, accessed on 10 February 2023) may prove fruitful in some instances, especially where the images do not relate to expired online auctions.

As the reverse image search algorithms employed by the search engine providers differ in their approaches, they will return different results; not only when submitting the same image to different engines, but also when images of the same object are submitted that were taken from slightly different angles. Likewise, different results will be returned for multi-coloured objects, depending on the colour combinations. Consequently, it is advisable to submit images of variations of multiple objects.

While most auction houses delete expired listings after 60 or 90 days, some national versions of eBay, for example, tend to retain at least the text of the listing and the vendor’s contact details. Thus it has proven to be a fruitful approach to paste the title of the auction listing (which is provided as part of Bing’s image listing) as a phrase into the search bar. Historic listings of Japanese auctions (going back to 2012) can be located via AucFan (aucfan.com/search1, accessed on 10 February 2023).

Online/social media discussion platforms have allowed like-minded individuals to form common-interest communities hosted on sites such as Facebook (www.facebook.com, accessed on 10 February 2023), Flickr (www.flickr.com, accessed on 10 February 2023) or Reddit (www.reddit.com, accessed on 10 February 2023). A plethora of tightly moderated and, on occasion, highly specialized groups for all things collectible exists on Facebook, some of which are open and some require moderator approved membership. As with all online discussion groups, these communities are populated by active individuals who post frequently, and a much greater number of ‘lurkers’ who observe and take in the information.

Online discussion platforms are also useful in cases where the objects are widely distributed, valuable either to an individual or a group of people, but in themselves not collectible. In this case, calls for the identification and location of these objects can be posted on conceptually related social media discussion platforms resulting in direct nominations or participation in linked online questionnaires functioning as data collectors (e.g., groups of recreational fishers collecting data on taxidermized Murray Cod [20]).

Snowballing

Given the potentially widely dispersed and, to some extent, fluid nature of the data set, snowballing is an integral part of the data collection process [38,39,40]. This entails direct communication with vendors, not only about an object currently offered, but also about past sales and, in the case of collectors selling off surplus items, communication about items in their collections that are not for sale or that they once owned. In the ideal scenario, relationships are developed where a collector, or a commercial trader (see below) becomes a ‘champion’ of the project and draws on their own contacts and networks with fellow collectors or traders, to increase the project’s reach.

2.3. The Nature of Information Providers

The owners of material culture held in private hands can be grouped into four discrete categories: collectors, commercial traders, incidental traders and owners of heirlooms. It is important to understand these as they have different motivations and thus may respond differently to information requests.

Collectors are commonly distinguished from hoarders by the existence of an implicit or explicit collections framework, which sets out, in the collector’s mind, which item is ‘in’ and what is ‘out’. Thus collectors will focus on kinds of objects from an era (e.g., Art Deco), a place of origin (e.g., Australian-made pottery), related to a town or location (e.g., Bristol or Normandy), a motif (e.g., figurines of cows), a manufacturer (e.g., Nikon cameras) or an author (e.g., editions of Charles Dickens’ works). Depending on the item of material culture researched, collectors may hold one of two items as part of a thematic collection or may hold multiple items as part of a specialist collection. Common to collectors is a profound interest in the history and context of the objects they are collecting, with a desire to learn more. Given their interests, collectors are not only very likely to favourably respond to requests but are also happy to facilitate a snowballing of the enquiry by making referrals to fellow collectors. As caveat, it should be noted, however, that the more valuable the collectible object, the more reserved collectors will be to initial approaches. In addition to general interpersonal dynamics (introvert vs extrovert personalities), some collectors may be less forthcoming with information; in particular if they question the motives of the researcher, are ambiguous about the research undertaken or are secretive about their holdings because the nature or provenance of some items may be perceived as disreputable. Consequently, while these collectors may be aware of the research being undertaken and may have noted the call for specimens for assessment, they may choose to remain on the side lines and not volunteer information, data or specimens.

Hoarders on the other hand will, for example, ‘collect’ any older camera without distinction of age, type or manufacturer and will accumulate multiple copies of the same item without any intent of researching variants. Hoarders tend to exhibit only a superficial interest in the history and context of the objects and thus are generally less likely to engage. Moreover, hoarders often do not know whether they actually have the specific sought-after object, frequently resulting in protracted communications which, in the end, often prove fruitless and futile.

Moreover, any collectors are protective of the knowledge they have accumulated over the years and are generally reticent to share information with people with whom they have no relationship. This can be overcome if other collectors ‘vouch’ for the researcher, or by the researcher developing ‘street credibility’ in the specific collection area. Rather than through academic publications, such credibility can be best established by active and knowledgeable participation in the relevant online discussion groups and by formal contributions to newsletters and other publications by relevant collector groups (for example, see the laying the groundwork for the above-cited Micronesian case [41,42]).

Commercial traders tend to be, by and large, supportive of information requests, in particular about objects that serve as primary data sources. The increased knowledge about the object(s) themselves that a trader derives from the interaction with the researcher benefits them in future interactions and negotiations with prospective purchasers. Auction houses also tend to be supportive, but their freedom to act may be potentially curtailed by the fact they do not own the object in question but merely act as brokers between the owner and a prospective buyer. In general, however, auction houses tend to be supportive in forwarding requests to the object’s owners.

Incidental traders are individuals who sell surplus objects via on-line marketplaces. Some of these engage in small-scale trading of items allied to their own collecting focus and thereby fund acquisitions for their own collections. Others treat incidental selling as a ‘side hustle’ to augment their income. The former, as collectors themselves, understand the motivations for research, even if it does not relate to their own collection focus, and are thus likely to countenance, if not support the research request. The latter are often unresponsive to requests, or if they engage in communication, tend to be reluctant to provide the information sought. Looking at the range of offerings of incidental traders often allows one to separate the two groups. Those who treat incidental selling as a ‘side hustle’ tend to have a wide range of objects unconnected by a common thread, other than being collectible in general. Collectors tend to trade within the same conceptual framework as their collection, which is unsurprising, as the centre of their knowledge base resides there, and thus have commonalities to their offerings.

Owners of heirlooms tend to be individuals who own an object for sentimental reasons. While they tend to be very knowledgeable as to the provenance of the item, they may not be very forthcoming with information for fear of invasion of privacy or safety of the item. The nature of the approach matters in this regard, with online approaches being more difficult, in a climate of online fraud.

3. Protocol Flow

In the ideal scenario, the process has five phases: scoping, searching, initial contact, intensification with snowballing and, finally, feedback. The protocol flow has been set out in Figure 1.

In the scoping phase it is critical to identify and then clearly define the sampling frame. Are the items of material culture under study geographically confined? If so, the searches can be restricted to specific languages and should include relevant, geographically restricted search engines. Care needs to be taken, however, to ensure the identification of the sampling frame is appropriate for the object in question. While some items are largely geographically confined [23], others may have become gift and souvenir ware with multi-national distribution [21]. Indeed, some items, such as tourist photographs at popular landmarks (e.g., San Marco in Venice) actually functioned as proof of ‘presence and achievement’ and were intentionally distributed to the individuals’ global networks. Consequently, a study into such real photo postcards sourced samples from all continents, bar Antarctica [24].

The search phase comprises text-based and image-based searches in tandem, both of web pages and online marketplace offerings. These searches are an iterative process and thus connected by feedback loops. If the marketplace platform offers that function, care needs to be taken to search not only active but also sold offerings. In some instances, sales platforms also include completed listings, where items went unsold but were not re-offered by the vendor. Depending on the search results of web pages and online marketplaces, it may be necessary to revisit the definition of the sampling frame and widen the geographic confines.

The longer the sampling period, the more comprehensive a dataset will be, not only due to additional items appearing on online sales platforms, but also due the snowballing effects derived from contacts (see below).

Akin to systematic literature reviews [43,44], the selection of combination of search terms is central to the success of sampling. It must be kept in mind, however, that both webpages and online marketplace listings are written both by specialists―professional traders and serious collectors―and by rank amateurs. It is the texts written by the latter that may contain unusual appellations and descriptive terms for an object, as well as terminology that may be plain wrong, politically incorrect or both. While text-based searches may miss these pages or listings, image-based searches are likely to find them. It thus behoves the researcher to include such ‘unprofessional’ terminology in iterative searches, being cognizant at the same time that doing so will bring up a high proportion of irrelevant material.

While text-based searches return relatively consistent results, reverse image searches are likely to return different results based on the angles and colouring of the object. Consequently it is advisable to iteratively submit images of variations of multiple objects.

Both the scoping and search phase need to be formally documented and reported in the methodology section of the respective study, setting out geographic scope, search engines used, online marketplaces accessed as well as the search terms or search term combinations, languages and iterative processes (if any) that were employed.

An initial contact phase follows where vendors of identified items, or authors of web pages showing or discussing relevant items, are contacted directly. As noted earlier, if the vendors are incidental traders without specific interest in the objects, the communication will cease after information about the specific item has been solicited and obtained. Researchers should be cognizant that some vendors may be unwilling to support the project and will refuse to provide data. Where the vendor is a commercial trader or a collector selling duplicates, surplus and allied items, it is prudent to intensify the contact and communication.

The intensification phase aims at deepening contacts to gain admission to the trader’s/collector’s networks. These are commonly fellow traders and collectors for whom the initial contact may provide a reference, as well as news groups and community discussion groups (for example on Facebook) where collectors of specific items and classes of items exchange information, help with identification and facilitate inter-groups sales. Membership and activity in these groups has a snowballing effect, as most collectors are members of several similar―and, at times, competing―groups with different network configurations. Formal posts on these groups soliciting information effect both direct responses in the discussion/comment threads as well as direct, behind the scenes communication―a mode often favoured by ‘lurkers, who do not want their items, or their collections hobby to be publicly known.

Any such posting to a group needs to be mentioned in the methodology, whereas specific contextual information that goes beyond generalities should be attributed to the source and formally referenced as a personal communication, with the contributor(s) also listed in the acknowledgments. Given that some collectors freely provide the information, but prefer to remain anonymous, any such formal inclusion and referencing should be agreed upon with the contributor.

Although not part of the data collection process per se, feedback is the final phase that must not be overlooked. It is not only critical for developing ‘street credibility’ among specific groups of collectors, but also for maintaining a reputation. It must never be underestimated that there are ‘nodes’ in the universe of collectors, people who have well-defined but eclectic interests and who straddle multiple collection areas. They act as conduits of both good and bad reputation. To maintain reputation, it is imperative that the findings of a study are being fed back to the collector community that furnished the raw data. Open access publication allows one to disseminate the findings in full through various channels, including linking in discussion threads in online groups. Where the resulting study is published in a subscription-model journal, and where the full text is hidden behind a paywall, it behoves the study author to synthesize the key findings in generally intelligible language and post them in a discussion thread in one or more online groups.

4. Conclusions

A considerable number of items of material culture are held in private hands, primarily in the form of collectable and, thus, tradeable objects. The dispersed nature of these holdings, coupled with the fact that the existence of such collections is often not widely known, makes it difficult for researchers to access these items in a comprehensive fashion. The identification of such holdings has been, in the main, serendipitous and unsystematic, relying on word-of-mouth. Employing the technique of crowdsourcing data from online discussion groups and online marketplaces, the preceding protocol has provided a workflow for the systematic and replicable identification and compilation of items held in private hands. Given that collectors are generally protective of their items, however, and may choose not to make some or all of their holdings publicly known, any crowdsourced data collection following the preceding protocol will be comprehensive but not holistic.