Earlier research has demonstrated the potential of computational object retrieval for art images; while these mainly detected objects, such as dogs, horses, figures or gestures, the present article uses identical models to find artworks. In this way, exhibition histories for artworks can be recreated. The present study demonstrates this, using exhibition photographs provided by the MoMA. Once other museums release comprehensive collections of installation photographs, this task can be significantly enhanced; allowing more detailed and encompassing statements about where and when artworks were shown, among which other artworks and if this varied for different museums. By connecting these exhibition views with additional digitized sources, such as other photographs and auction catalogs, one significantly extends the potential of computer-based methods, also for provenance research.
2.4.1. The Detection of de Chirico’s Nostalgia of the Infinite (c1912–c1913)
A large tower dominates Giorgio de Chirico’s (1888–1978) metaphysical painting ‘The Nostalgia of the Infinite’ (c1912–c1913, oil on canvas, MoMA). Sunlight illuminates the image and cuts it in two halves; the place in front of the tower is deserted with the exception of two figures, whose silhouettes are still visible but soon might disappear. The oil painting was executed in 1912/13 and first sold to the Parisian art dealer Paul Guillaume (1891–1934) in 1918 and was eventually acquired by the MoMA in December 1936 through the Galerie Bonaparte in Paris—as the provenance entry on the museum’s website states. The interface was used to search for de Chirico’s work in the MoMA’s exhibition images; the painting was selected in an installation photograph from the ‘Painting and Sculpture from the Museum Collection’ show, which was held between 23 October 1940 and 12 January 1941 (Figure 6
). The archival material for the exhibition also holds a ‘Master Checklist’, where all included works are listed, a press release and four installation photographs.
In the query image, de Chirico’s painting is shown next to ‘Parade’ (1930), an oil painting by American artist Peter Blume (1906–1992); the latter was acquired in 1935 and has been a part of the museum’s collection ever since. Blume confronts the viewer with an industrial landscape: the painting is dominated by a surreal factory, positioned along a diagonal line; the sky appears to threaten the scenery, emphasized by contrasting clouds of white and dark color. In the right foreground, we see a man holding a knight’s armor; he is about to leave the image’s reality, thus entering the viewer’s world. The armor suggests his metamorphosis into a mechanical being. Blume used different tones of white and gray and highlights in red and yellow to create his surreal landscape. His works are often linked to Surrealism, which ultimately validates his connection to de Chirico. For the search, the user marked de Chirico’s work with a bounding box; after initializing the search, results are presented in another window, visualized in rows of eight in decreasing similarity (Figure 7
): evidently, algorithms were able to retrieve de Chirico’s ‘The Nostalgia of the Infinite’ (1912–13) correctly. His work was found in four other exhibitions, correct retrievals are included in the first three rows of the results’ page. Information about exhibitions can be concluded from the metadata, which is provided for every single image and saved on the interface; once results are given for a search query, this information can be accessed through the system. Besides the 1940 show, ‘Nostalgia of the Infinite’ was included in ‘Paintings, Sculpture, and Graphics form the Museum Collection’ (2 July 1946, until 12 September 1954), ‘Fantastic Art, Dada, Surrealism’ (7 December 1936, until 17 January 1937), ‘Permanent Collection Exhibition’ (29 March 1972, until 21 April 1980) and the Giorgio de Chirico solo exhibition (6 September until 30 October 1955). The confident findings encourage to look at individual exhibitions in more detail and study in which artistic context the image was shown and if that changed over time. As stated, the query painting is exhibited in the ‘Painting, Sculpture, and Graphics from the Museum Collection’ exhibition; the installation photograph shows it alongside Blume’s ‘Parade’; unfortunately, this is the only link the image reveals, since other artworks are not visible. Because of the relatively small set of photographs for this show (only four), other photographic records provide no further information about other artworks, with which de Chirico’s painting was presented. However, the ‘Master Checklist’ Museum of Modern Art, New York
) lists both paintings in the section ‘Magic Realism’; thus, it can be assumed that other works in this category hung in close proximity or indeed formed a group. In this case, ‘The Nostalgia of the Infinite’ would have been displayed alongside Blume’s ‘Parade’, de Chirico’s ‘Toys of a Prince or Evil Genius of a King’ (oil on canvas, 1914–15), Dalí’s ‘The Persistence of Memory’ (oil on canvas, 1931) and ‘Portrait of Gala’ (oil on canvas, 1935), Max Ernst’s ‘The Nymph Echo’ (oil on canvas, 1936), Richard Oelze’s ‘Expectation’ (oil on canvas, 1936), Pierre Roy’s ‘Danger on the Stairs’ (oil on canvas, 1927–28) and ‘Agricultural Conference’ (oil on canvas, c1930), Yves Tanguy’s ‘Mama, Papa is Wounded!’ (oil on canvas, 1927) and Henri Rousseau’s ‘The Sleeping Gypsy’ (oil on canvas, 1897). Creation dates for individual works reveal that most have been painted during the late 1920s and 1930s, with Rousseau’s Gypsy being an exception and that all are linked to Surrealism. In this way, de Chirico’s work was mainly presented alongside contemporaries and precursors. The exhibition introduced him as a forerunner of Surrealism—as such he was viewed by Surrealist artists, who celebrated his mystical, deserted architectural views. Regarding style, de Chirico’s artwork was presented within a homogeneous group of works; assumingly it was not the intention of the curator to display contrasting images in order to highlight differences. However, minor semantic contrasts can be found within the group. Although artists’ nationalities and connections to specific art movements and groups varied, all images exemplified a strong preference for figuration, depicted dream-like worlds and used bright colors—with the exception of Tanguy, who included abstract elements. As an example, the urban landscapes of de Chirico or Blume were opposed to Max Ernst’s nature-dominated representation of the nymph or Rousseau’s depiction of a sleeping gypsy in the desert. Depictions of natural fertility were contrasted against images of desertedness—as seen in the paintings of de Chirico or Dalí.
Four years prior to the Museum Collection exhibition, the painting was included in the ‘Fantastic Art, Dada and Surrealism’ show, which presented an extensive overview of one of the two most important art movements of modernity. At the time of the exhibition’s opening, Dada was already superseded, while Surrealism was internationally established and at a height; Surrealist groups in various European cities and numerous exhibitions, such as the ‘International Surrealist Exhibition’ held in the New Burlington Galleries in London in 1936, exemplify this. The exhibition at the MoMA presented a great number of works, dating back to the sixteenth century, such as Hans Baldung Grien’s ‘Seven Horses Fighting in a Wood’ (woodcut, 1534). De Chirico and his paintings (in total, 10 oil paintings and 16 drawings were shown) were again linked to a Surrealist tradition. Doing so, the show of 1936/1937 already picked up, what would be visible in the ‘Museum Collection’ exhibition in 1940. However, in the section, where de Chirico’s work was shown, organizers exclusively placed him among his own works. An installation shot shows ‘The Nostalgia of Infinite’ next to ‘Toys of a Philosopher’ (1917), ‘The Duo’ (1914–15, MoMA) and ‘Mystery and Melancholy of a Street’ (1914) (see photograph 1). All paintings were executed around the same time and are representative of de Chirico’s classical metaphysical style, showing deserted cityscapes, mannequins and stark contrasts of light. At least within this spatial section, the painting was not positioned within a greater art historical context, but remained positioned within de Chirico’s oeuvre—creating a homogeneous group of works. However, if we consider the entire exhibition, links to the past and to forerunners of modern art movements were certainly emphasized. The next search result links to the ‘Painting, Sculpture, and Graphic Arts from the Museum Collection’ exhibition, which again included de Chirico’s work and opened its doors in 1946; the more permanent show remained on display until 1954. For the first time an exhibition was staged at the newly remodeled galleries on the second floor, where all except for two were used for showcasing the museum’s collection of paintings Museum of Modern Art, New York
). Alfred H. Barr Jr. (1902–1981) was responsible for the hanging and organization of the exhibition: “Installing the painting galleries has been a problem of compression. The new space, now permanently set aside on the second floor, has made available three additional galleries for the Museum Collection, yet there is even now room for only 120 paintings. […] In the remaining galleries varieties of lyric fantasy, dream realism, and factual painting include the work of Klee, Masson, Graves, Tanguy, de Chirico Dix, Sheeler, and others” Museum of Modern Art, New York
Indeed, the installation photograph (see photograph 2) shows two of de Chirico’s works— ‘The Nostalgia’ and ‘The Evil Genius of a King’—on one wall, opposing Tanguy’s abstract painting ‘Slowly Toward the North’ (1942) and Ernst’s ‘Two Children Are Threatened by a Nightingale’ (1924). Similar to the ‘Museum Collection’ show in 1940, the exhibition of 1946 showed de Chirico’s work within a Surrealist tradition. This might indicate that the exhibit—maybe museum collection shows in general—were arranged in homogeneous groups, representing established styles, themes, or artists, rather than in a contrasting manner. This is further validated when looking at other installation photographs of the same exhibition: an image shows a room dedicated to post-Impressionist artists, such as Cézanne and van Gogh, whose ‘Starry Night’ (1889, oil on canvas) is visible in the right corner. This observation is validated by previous remarks on the presentation of de Chirico’s work in shows by the MoMA, which also elaborated on the homogeneity of displays. Scholars have indeed noted that historically group or mono-graphic exhibitions4
were organized chronologically, geographically or according to themes; this structure represented the Western art historical canon and is visible in the mentioned shows, which include the Italian artist’s works. It was not until the 1990s, with the reflexive turn, that traditional practices were criticized and new curatorial approaches were taken: new forms of representation were established, which were interdisciplinary, offered a critical perspective on canonical knowledge and created new contexts for artworks. Also museum collection shows aimed to showcase the quality and diversity of the collection and announce the museum’s identity—this was at least one main purpose of exhibits until the 1990s Grießer
). A collection thus had a specific purpose, being aesthetically, economically or because of personal taste, and was gathered accordingly; also, collections seldom remained static, but changed and increased over time. By exhibiting their collection, the MoMA publicly proclaimed their identity and thematic focus. It must be added however that these shows only displayed a small number of the collection, which were additionally curated and picked according to a specific pre-defined purpose Simmons
). The ’Master Checklist’ (Museum of Modern Art 1940) of the 1940 show, presenting the museum’s collection, reveals that various artists and styles were included, however, most of them were European; this was representative of the zeitgeschmack
. Almost twenty years after the previous show ended, the MoMA staged another exhibition dedicated to its extensive permanent collection (Permanent Collection, 29 March 1972 until 21 April 1980). Again, de Chirico’s image of the infinite tower was displayed; an installation view (see photograph 3) displays it next to ‘The Anxious Journey’ (1913, oil on canvas) and ‘Gare Montparnasse’ (1914, oil on canvas), both painted by the Italian artist. The subsequent walls show Rousseau’s ‘Sleeping Gypsy’ (1897, oil on canvas) and two other paintings, which were not identifiable. However, most likely they are from an earlier period, possibly around the late nineteenth century, because of their stylistic similarity to the works of Odilon Redon. Unfortunately, no archival material is given for this exhibition, so no further information was provided. Exhibiting de Chirico in close proximity to Rousseau resembles the arrangement in the ‘Painting and Sculpture from the Museum Collection’ exhibition (1940). Interestingly, both shows displayed the museum’s own collection but were held under different directors, namely Barr and Richard Oldenburg (1972–1995). For the show in 1940, Barr was again responsible for the hanging, for the exhibition in 1972, it is unclear whether Oldenburg was actively engaged in the installation or if it was the sole responsibility of one of the MoMA’s curators. However, since both displayed works of the museum’s collection, it can be assumed that similar installation practices and forms of representation are employed; not least, because exhibits, dedicated to the collection, had specific aims and purposes, namely to promote the museum’s taste and present masterworks, which testify to the collection’s quality. Other installation photographs of the ‘Permanent Collection’ exhibition (1972) illustrate that rooms were dedicated to Cubism (Braque, Picasso, Roger de la Fresnaye), Expressionism (Kokoschka), ‘Der Blaue Reiter’ or Abstract Art. Artists were mixed and works grouped together according to specific styles. In conclusion, the show presented an overview of the major (European) art movements and some of the most prominent works within the MoMA’s collection. While de Chirico’s ‘The Nostalgia of the Infinite’ was repeatedly shown in group exhibitions so far, the next search result assigns it to a show, which was held in 1955. Entitled ‘Giorgio de Chirico’ (6 September until 30 October 1955), it was solely dedicated to the Italian artist. An exhibition shot (see photograph 4) displays the oil painting ‘The Enigma of a Day’ (1914, oil on canvas) to its left and ‘Ariadne’ (1913, oil on canvas) to its right. This artistic context is highly similar to the ‘Fantastic Art’ exhibition (1936), organized by Barr, where de Chirico’s painting is also exhibited among his own works. The show gathered twenty of his most famous paintings, mainly from his early metaphysical period, visualizing popular characteristics, such as abandoned city streets, mannequins and dark/bright contrasts. The show was directed by James Thrall Soby, curator of the MoMA, and installed by Margaret Miller (Associate Curator of Painting and Sculpture), under the directorship of Rene d’Harnoncourt (1949–1968) Museum of Modern Art, New York
2.4.2. The Detection of Balthus’ portrait of Joan Miró and his Daughter Dolores (1938)
To demonstrate the consistency of the algorithm, the interface was used to search for Balthus’ (1908–2001) portrait of ‘Joan Miró and his daughter Dolores’ (1938, oil on canvas, MoMA). The Spanish artist is portrayed sitting on a chair with his daughter, who stands between his legs; their gazes directly confront the spectator, captivating him with their undivided attention. The room, in which the scene takes places, seems empty, almost abandoned. The Paris-born artist chose brownish, very muted colors for his double portrait. Again, the interface developed by the group was used to search for the portrait; the same process as described for the example of de Chirico was triggered: the 1940 exhibition of the museum’s collection includes an installation photograph, where Balthus’ work is visible (Figure 8
); a bounding box was used to mark the painting and then the search was initiated. Similar to de Chirico’s search, the algorithm performed well and detected the artwork four times, as can be seen on the results’ page, where retrievals are shown in declining similarity (Figure 9
). The first three images in the first row show Balthus’ work, followed by an image of another artist and then again followed by the portrait of Miró and his daughter. How incorrect retrievals might be interpreted will be discussed at a later stage in this paper. As stated, the first result is from the 1940 exhibition of the ‘Museum Collection’. To its right, Christian Bérard’s ‘Jean Cocteau’ (1928, oil painting) was exhibited, the left painting could not have been identified. Cocteau is depicted in half-length and parallel to the viewer; he is dressed in an orange shirt, set against a brown background. A loose brush stroke and application of paint suggest a short execution time of the painting. The second search result stems from the same exhibition; the painting was found in another exhibition photograph (see photograph 5). The photograph reveals additional links to other works; the image of Miró and his daughter was shown alongside Franklin Chenault Watkins’ ‘Boris Blai’ (1938, oil on canvas); Blai is shown in half-length, with his body turned to the left and directly looking at the viewer. Similar to Bérard, the artist used muted brownish tones, which he applied very loosely to the canvas. The show of 1940 embedded Balthus’ work within a group of portraits—the ‘Master Checklist’ reveals that the group also included Oskar Kokoschka’s ‘Portrait of Dr. Tietze and his Wife’ (1909, oil on canvas) and ‘Self Portrait’ (1913, oil on canvas) Museum of Modern Art, New York
). The group presented a mix of nationalities, where the Expressionistic style and the genre were constants tying works together. This corresponds to historical exhibition practices, where artworks were organized and displayed in homogeneous groups Grießer
Balthus’ image was again included in the exhibition entitled ‘Portraits from the Museum Collection’ (4 May–18 September 1960) twenty years later. The corresponding installation shot provides further information about the artistic context in which the painting was displayed (see photograph 6). To its right, Balthus’ portrait of André Derain (1936, oil on canvas) was hung, followed by two works by the French artist Christian Bérard ‘On the Beach—Double Self Portrait’ (1933, oil on canvas) and ‘Jean Cocteau’ (1928, oil on canvas). On the far right, the photo shows the portrait of ‘Boris Blai’ (1938, oil on canvas) by Watkins. The exhibition thus showed a group of works, which are very similar in terms of date and style. All paintings were created in the 1930s—with the exception of Bérard’s portrait of Jean Cocteau—and painted in an Expressionistic style. The group is almost identical to the one in the ‘Museum Collection’ exhibition of 1940, where Bérard’s Cocteau and Watkins portrait were also shown alongside the portrait of Miró. The last result shows Balthus’ work in the ‘Modern Masters: Manet to Matisse’ exhibition (5 August–28 September 1975) (see photograph 7). The retrieval of this result is surprising since the painting in the photograph is highly distorted by the perspective and small in scale. However, the algorithm was able to detect it, which testifies to the efficiency of the developed interface. The shot includes a large number of paintings and reveals the following works (from left): Balthus’ ‘The Mountain’ (1936–37, oil on canvas), Yves Tanguy’s ‘Fear’ (1949, oil on canvas), Matisse’s ‘Large Interior in Red’ (1948, oil on canvas), Giacometti’s ‘Peter Watson’ (1953, oil on canvas) and ‘The Apple’ (1937, oil on canvas), Miro’s ‘Maternity’ (1924, oil on canvas) and ‘Diana’ (1931, oil on canvas) by German painter Paul Klee. The exhibition catalog, which is available online, provides additional information: “[the exhibition] surveys almost a century of European painting which starts with Manet in 1861 and closes soon after the death of Matisse in 1954. The selection has been conceived in eight chapters: Impressionism; Post-Impressionism; Matisse […]; Expressionism […]; Cubism […]; the “painted dream”, which in this exhibition refers to painters of fantasy before, during, and after Surrealism; portraits, a personal predilection […] and last […] ten painters of the School of Paris […]” Museum of Modern Art, New York
). Groups displayed the full variety of art, suggesting an interconnected reading of the paintings—Balthus’ portrait is seen among Surrealist imagery, abstract as well as figurative paintings. Whereas previous exhibitions mainly exhibited Balthus’ work within a portrait tradition, the present show places it within a less homogeneous group, including different styles, motifs and artists’ nationalities. Assumingly, the aim was to present the variations known in modern art, resembling more modern approaches, which less reflected canonical knowledge Grießer
). Previous exhibitions mainly focused on presenting the image among other portraits; ‘Modern Masters’, however, enlarges the artistic context in which the query is shown.
While the interface is able to detect the query image in the set of exhibition photographs correctly, the search process also retrieves results, which are not identical to the query, but regarded as similar by the algorithm. These additional images are not relevant for the exhibition history of the specific artwork, but might suggest new and interesting links. General observations can be obtained from the results (Figure 9
): other images mainly consist of portraits, which are characterized by a central alignment of a figure, a simple background or style, similar to Balthus’ Expressionism. The first row shows Paul Cézanne’s ‘The Bather’ (c1885, oil on canvas), presently in the MoMA’s collection; although the painting was executed more than fifty years before the query image, its style and general composition bears resemblance to Miró’s portrait. The bather is shown in the center of the image, accentuating a vertical line, which divides the painting almost in equal parts. This is comparable to Balthus’ portrait, where Miró and his daughter also emphasize strong vertical lines, while the transition between floor and background additionally creates a distinct horizontal line. The background, consisting of different shades of blue and a rocky landscape, is simple and also suggests a horizontal line. Because of its reduced content and central position of the figure, the painting is easy to comprehend and Balthus’ image eventually inherits the same qualities. Cézanne’s image of the bather was exhibited in the 1929 exhibition dedicated to van Gogh, Gauguin and Seurat; an installation shot reveals that it was shown among other portraits, including Cézanne’s ‘Portrait of a Man in a Blue Cap’ (or ‘Uncle Dominique’, c1866, oil on canvas) and ‘Harlequin’ (1890, oil on canvas), Paul Gauguin’s ‘Melancholic’ (1891, oil on canvas), followed by Cézanne’s ‘Self-Portrait with Beret’ (1890, oil on canvas). The show thus presented the male figure of the bather among other portraits, mainly by Cézanne. By identifying Cézanne’s work as similar to Balthus, the algorithm detected an image, which is not only formally similar, but was also displayed among a very homogeneous group of works—an obervation, which has been made for Balthus’ portrait also. The motif of the bather was again detected in Picasso’s ‘Bather’ (1908–09) as shown in the second row, last image on the right. Although executed in a Cubist style and holding a white towel in his left hand, the figure, its composition and background indicate a strong link to Cézanne’s male figure, which was painted almost twenty years prior. Again Picasso’s work displays similar qualities to the paintings of Cézanne and Balthus; it is thus comprehensible why the interface would suggest a similarity between the three paintings. The third image in the second row shows Henri Matisse’s ‘Portrait of Michael Stein’ (1916, oil on canvas), which was included in the ‘Henri Matisse Retrospective’ in 1993, where it was presented among a group of portraits. In contrast to the painting by Balthus, which showed a full-length figure—the present work portrays Stein in half-length. He is positioned parallel to the viewer, his suit and background are painted in different shades of brown. While prior works bore greater resemblance to Balthus’ work, it is less obvious, why the algorithm detected the portrait. When contrasting both images, it is evident that both artists used a similar color palette –mainly brown tones–, style and facial expression. Comparing exhibition photographs highlights that Matisse’s work was embedded within a similar group of artworks than Balthus’ painting. Curators and directors of the MoMA— in this case Richard Oldenburg for ‘Henri Matisse: A Retrospective’, 1992 and Alfred H. Barr for ‘Cézanne, Gauguin, Seurat, van Gogh’, 1929 and ‘Painting and Sculpture form Museum Collection’, 1940—thus preferably hung works of identical genres together. The result page shows another work by Matisse, namely his oil painting ‘Nude with a White Towel’ (1902–1903), which depicts a female nude standing next to a chair in a brightly colored room. The motif is more similar to Cézanne, Picasso and Balthus’ Miró than the portrait of Stein, because it shows a figure in full-length, depicted at a central position in the image. Concluding, it is noticeable that most of the retrieved artworks are full-length portraits, with the exception of Matisse’s ‘Michael Stein’ (1916), characterized by a central alignment and similar in style to Balthus’ portrait; most works presented a reduced visual language and results were considered to be similar to the query image because of formal and semantic qualities.
The second search, based on the user’s evaluation of the results, leads to improved results and possibly to new findings and connections. In the case of Balthus’, a second round was initialized to demonstrate the usability of the interface, based on the user’s feedback; the four detected paintings by Balthus were marked as positive, others in red (Figure 9
). Results revealed that detections were rather similar to the first round and did not improve significantly, a first search already provided satisfying results (Figure 10
). However, it is noticeable that a new work appears: the second row now features Vincent van Gogh’s ‘L’Arlésienne’ (Mme. Ginoux) (1888), which was displayed in the van Gogh exhibition (1935–1936). Again, the algorithm’s detection indicates a strong connection to Balthus’ work, because both are portraits and similar in terms of central alignment and formal qualities.