We begin this section with an explanation of the relevant numismatic terminology necessary for the understanding of the present article and the related literature, then categorize and describe in detail the most important (practically and technically) challenges in the field, and summarize the progress to date in addressing these.
An important consideration in numismatics regards the condition of a particular coin. As objects that are a millennium and a half to three millennia old, it is unsurprising that, in virtually all cases, they have suffered damage. This damage was effected by a variety of causes. First and foremost, as most coins were used for day-to-day transactions, damage came through proverbial wear and tear. Damage was also effected by the environment in which coins were stored, hidden, or lost, before being found or excavated—for example, the moisture or acidity of soil can have significant effects. Others were intentionally modified, for example, for use in decorative jewellery.
The amount of damage to a coin is of major significance both to academic and hobby numismatists. To the former, the completeness of available information on rare coins is inherently valuable, but equally, when damaged, the type of damage sustained by a coin can provide contextual information of the sort discussed earlier. For hobby numismatists, the significance of damage is twofold. Firstly, a better-preserved coin is simply more aesthetically pleasing. Secondly, the price of the coin, and thus its affordability as well as its investment potential, are greatly affected: The cost of the same issue can vary by 1–2 orders of magnitude.
To characterize the amount of damage to a coin due to wear and tear, as the most common type of damage, a quasi-objective grading system is widely used. Fair (Fr) condition describes a coin so worn that even the largest major elements are mostly destroyed, making even a broad categorization of the coin difficult. Coins of Very Good (VG) grade have most detail worn nearly smooth around the central areas but still visible on the periphery. Fine (F) condition coins show significant wear with many minor details worn through, but the major elements are still clear at all of the highest surfaces. Very Fine (VF) coins show wear to minor details, but clear major design elements. Finally, Extremely Fine (XF) coins show only minor wear to the finest details. Examples are shown in Figure 1
2.3. Practical Applications
One of the features of numismatics which makes it an interesting domain for the application of computer vision and machine learning lies in the number and diversity of specific problems that it presents. Many of these directly correspond to challenges faced by experts or hobby collectors, though some new work introduces innovative challenges which are only possible with the use of technology (we shall elaborate on this shortly). The key problems, few of which can be considered anywhere near solved, include the following:
As implicitly explained in the previous section, specimen matching
refers to the problem of determining if the same coin specimen in two images is the same, i.e., if they show the same actual physical artifact. There are several important applications of this task. For example, it can be used to determine the provenance of a specific coin or to track its value across time as it is sold and passed on from one collector onto another. Importantly, specimen matching can also be used to automatically monitor massive volumes of coins sold on non-traditional auction web sites, such as eBay, and to track stolen coins. The key challenges for specimen matching lie in differential appearance effected by different illumination conditions, camera settings (e.g., aperture, focus, and exposure), clutter, scale, and viewpoint [19
In contrast to specimen matching, issue matching
refers to the problem of determining if the coins shown in two images are of the same issue, i.e., if they contain the same semantic content and are of the same denomination (e.g., denarius, anotoninianus, follis, sestertius). This task is the first and probably the most commonly performed one by any numismatist; colloquially put, it answers the question “What is this coin I’ve got?”. In addition to all of the aforementioned challenges outlined in the context of specimen matching, in issue matching, a major challenge of a semantic nature emerges: Recall that issues are identified by the corresponding semantic contents, which can exhibit both stylistic variability (e.g., due to different die engravers), appearance change due to physical damage or chemicals in the environment, or die wear, to name but a few; see Figure 2
. Recalling from the previous section that the number of different issues of Roman Imperial coins exceeds 43,000, it is not difficult to see why issue matching is inherently an extremely difficult problem [15
]. In addition, such a high number of classes makes it all but practically impossible to obtain an annotated gallery of exemplars of all (or most) issues [18
is a classification problem which, as the name suggests, is concerned with the determination of the denomination of a coin. Denarii, antoniniani, sestertii, ases, and dupondii are examples of the most common denominations of the Roman Imperial period before the economic crisis of the third century. Some of these are shown in Figure 3
. The knowledge of a coin’s denomination can be useful as a step aiding in issue matching or in its own right for monitoring market trends (types of coins being sold, price changes, etc.).
Most Roman imperial coins feature a portrait (all but universally in profile, and usually facing right). Most often, this is the current emperor, sometimes their predecessor (as commemoration following their death), and also frequently their spouse. The recognition of this individual is one of the first things that a numismatist will do in the process of identifying a coin, i.e., it is a step in the process of issue recognition. Within the scope of computer-vision-based analysis of ancient coins, issuing authority recognition
started attracting attention following the realization that tackling issue recognition is a far more difficult challenge than anticipated at first. Hence, the attempts to apply generic object recognition algorithms waned in popularity, and instead, the focus shifted towards the use of more domain-specific knowledge, the recognition of the depicted person being an obvious choice. Thus, the challenge of legend readout
concerns the recognition of the legend inscription. So far, it has received little attention from the computer vision community [13
] despite its utility to numismatists. In large part, this is likely a consequence of the difficulty of the problem: Legends are abounding in fine detail and are prone to damage, with letters easily confused with one another, or indeed a damaged letter with a legend break.
The legend on an ancient Roman imperial coin is an interesting semantic element. Some parts of it contain, in essence, the same information as the motif they encircle. For example, on the obverse, the legend almost invariably explicitly names the issuing authority shown on the coin—in Figure 4
a, it begins with the ‘AVRELIVS’, which refers to Marcus Aurelius. Thus, this information can be used to aid in the process of issuing authority recognition or, with reference to the reverse, in the interpretation of the corresponding motif. However, the legend also contains some information which is generally not contained elsewhere. For example, the legend often contains the consular year of the issuing authority, such as ‘COS III’ (third consular year), which allows for the precise dating of the issue and its disambiguation from other issues otherwise identical to it.
We have already discussed issue matching as probably the most important and pervasive problem in automatic ancient coin analysis. A major and indeed fundamental problem with the existing approaches which rely on visual matching of images, as highlighted in Section 2.1
, is that the number of classes in this classification problem is enormous, exceeding 43,000. This is not only a technical challenge, but also a practical one: It is virtually impossible to obtain gallery samples of such a high number of issues or indeed anything even close to that number. Yet, this was only recently explicitly recognized in the literature [18
]. Thus, recently, an alternative approach was first put forward, as well as the first promising steps towards its implementation. The idea is very much akin to what a human numismatist does: Interpret and understand
the semantic content [21
] of a coin (hence, semantic analysis
), and then use this semantic description for matching against textual reference entries [22
]. Thus, the visual matching problem is eventually turned into a text-matching one. This work is still in its early stages, but highly promising results have already been reported using a deep-learning-based framework capable of automatically learning salient concepts and the range of their artistic depiction variability [20
Considering the size of the global ancient coin market, it is hardly surprising that it is an attractive target for fraud. Unlike most other ancient artifacts (e.g., highly ornate pottery, helmets and other armor, swords, etc.), for the most part, ancient coins are medium-value collectables. This makes it cost-ineffective to individually authenticate all but a small number of more expensive specimens. Yet, the high volume of sales makes forgery a lucrative business. Despite this major practical significance, interestingly, the task of automatic forgery detection
has not been explored in any published work to date. What makes this observation even more surprising is that the problem is technically quite interesting. In particular, the novel challenge lies in the new kind of intra-class variability within the class of forgeries. This variability emerges as a consequence of different methods used to produce fake coins. While a thorough discussion of this topic is beyond the scope of the present article, the simple example in Figure 5
will serve to illustrate the gist of it. Specifically, compare an authentic example of a silver denarius of Clodius Albinus in Figure 5
a with the three forgeries in Figure 5
b–d. The first of the latter, in Figure 5
b, is good in style, and was likely produced from a casting mold, itself made from an authentic specimen (as a ‘negative’ thereof). The lack of authenticity is given away by the casting sprue at 10–11 o’clock looking at the obverse, the relief pattern around the legend (especially on the reverse; it is highly unlike that of struck coins), and the surface of the coin (impressions of small casting bubbles). In contrast, the forgeries in Figure 5
b,c are poor in style, mostly likely made from modern molds, and readily recognizable as being produced by casting and not striking. How this wide inter-class variability can be learned is an open question and arguably makes the problem one of novelty detection.
Recall that ancient Roman imperial coins were minted by striking a blank coin placed between hand-carved dies [23
]. This is in contrast to casting, which was used briefly during the Republican period, as well as later in the production of medallions, probably due to their much larger size (often in excess of 50 g). Being able to tell if two coin specimens of the same issue were made using the same dies, i.e., die matching
, is of much interest to research numismatists (and much less so to hobby collectors), because, for example, this allows for the inference of migratory patterns of peoples, trading routes, etc. Die matching can also assist in the fight against high-quality forgeries, some of which were struck in modern times but using ancient dies or copies thereof [24
]. To the best of our knowledge, die matching remains an entirely unexplored challenge in the realm of automated ancient coin analysis.
2.4. Research Effort to Date
As noted in the previous section, most research on the application of computer vision and machine learning in the domain of ancient numismatics focused on the problem of issue recognition, or, more specifically, visual issue matching. Within this body of work, in terms of technical underpinnings, visual matching based on local features (chiefly SIFT [25
]) dominates the literature [14
]. Though highly successful in a wide variety of object recognition tasks [27
], these approaches were quickly found to perform very poorly in the context of the problems of interest herein, showing some success only in highly controlled conditions, i.e., when changes in illumination are small or non-existent, when images are devoid of clutter, and when the coins are canonically oriented. This is highly unrealistic in practice: Assumptions of limited photometric variability do not hold, and the removal of clutter (segmentation) is difficult, as is geometric registration [19
]. An illustration of just some of the challenges is shown in Figure 6
In hindsight, the disappointing performance of local-feature-based methods ought not to be surprising. Firstly, ancient coins do not possess discriminative textural information [31
]. Textural variability is a confounding factor. Rather, appearance variation emerges from geometry (3D) of coins and, thus, the manner in which light is reflected off them. Thus, in terms of local appearance, most coins look alike—the absence of the use of their geometric relationships is crucial. Driven by this insight, the best-performing local-feature-based method builds compound features in the form of directional histograms centered at automatically detected interest points [14
]. Thus, both local and distal appearances are integrated, and the geometric relationship is captured. Nevertheless, though significantly surpassing the performance of the existing method at the time, even this method failed to demonstrate practically useful matching rates.
Driven in part by the lack of success of what may be termed ‘conventional computer vision’ approaches on the one hand and the groundbreaking achievements of deep-learning-based methods on the other, much like other recent cultural-heritage-focused computer science work [33
], more recent efforts in automatic ancient coin analysis have turned their attention to the use of neural networks. Thus, Schlag and Arandjelović [17
] proposed a VGG16 deep-neural-network-based algorithm for issuing authority recognition, and demonstrated outstanding performance on three large corpora of data. Aslan et al. [16
] used a pre-trained ImageNet, adapted to the domain using transfer learning, on a small data set of Roman republican coins with lesser success. The deep learning algorithms of Cooper and Arandjelović [18
] and Anwar et al. [36
] both focus on the semantics of motifs depicted on coins, the former on Roman imperial and the latter on Roman republican coins—the problem which we already noted as being extremely promising in terms of practical significance, and most interesting from the technical viewpoint.
Related work, not falling under the umbrella of computer vision as such nor machine learning, includes the acquisition of 3D scans of ancient coins [37
]. This body of research is closer in spirit to efforts on the digitization and visualization of cultural artifacts [40
], including temporal modeling [42
] and hyperspectral imaging [45