The Early History of Visual Virtuality with Two Eyes

Abstract

Visual virtuality can be seen as involving the processing and perception of pictorial images. The production of such representations has a longer history than speculations about their perception. Pictorial images of objects and scenes lack two dimensions present in their subject matter—depth and motion. Instruments to investigate stereoscopic depth and apparent motion were invented in the early 1830s. Wheatstone devised stereoscopes and conducted experiments with them; Plateau and Stampfer independently crafted devices for presenting sequences of slightly different patterns that created the impression of motion. Wheatstone later proposed how moving and stereoscopic images could be combined. Thereafter, interpretations of virtual depth and motion perception became more concerned with central processing rather than being based on geometrical optics.

1. Introduction

Historical accounts of virtuality typically begin in the twentieth century and are concerned with computer-based developments thereafter [1,2,3]. An exception to this tendency can be found in Grasnick’s [4] survey which commences with cave art and progresses through the changing artistic styles to present-day stereoscopic displays. Thus, a broad definition of virtuality includes pictorial representation and that is the position adopted in this article with respect to binocular vision. Pictures are strange objects but they are rarely considered as objects—typically framed patterns on a flat surface. It is the spatial pattern, the pictorial image, that is of interest not the surface on which the marks are made. Pictorial images can represent some other object or event that is not present. One of the distinguishing features of our species is the reliance on virtuality in general (language) and visual virtuality in particular (pictures). The production of pictorial images, be they spatial or symbolic, separates us from other species. Spatial images, like rock art, paintings, photographs and computer graphics, correspond to marks made on a surface that represent some other thing. The distinguishing features of viewing pictures of a scene and seeing the scene itself are that the former lacks the depths and motions that are a fundamental part of the latter. Pictures are indirect references (allusions) to objects that we cannot see or to events we cannot experience—they are virtual. Much the same applies to the motions we perceive. We have evolved to see objects in motion or to detect our own motion with respect to objects or to compensate for them. Nonetheless, much of the motion we perceive is not of this type, it is not real but apparent and occurs as a consequence of the rapid display of static images. The motion is synthesised in our brains as a consequence of such sequences. Symbolic images, like written letters and words, can represent categories of things but also states of being, like emotions. When two slightly different views are presented separately to two eyes the situation is more complex as is the task faced by the brain in processing them. Nonetheless, we solve these problems continually in our daily lives both for the natural three-dimensional world and for paired pictures of it [5].

Visual virtuality is the term employed here to distinguish between the visual perception of three-dimensional objects and two-dimensional representations of them. Humans are continually confronted with this distinction because many of our visual interactions with the world are virtual. We are taught the equivalence of pictures of objects and the objects themselves from an early age and the learning is so successful that we rarely question the equivalence. At one time the medium was the printed image whereas now it is more commonly a digitised image on an electronic devise. Moreover, the latter is likely to be dynamic rather than static—the images move rather than remain still as in a print on paper. Despite the digitization of images there remains the age-old distinction between objects in space and representations of them on a flat surface.

Thus, the issues at the centre of visual virtuality are not new, as they address some ancient questions regarding visual perception. One of the oldest of these is the contrast between monocular and binocular vision. Although we see the world as single with one eye and with two, that singular world is seen in more subtle depth with two eyes than it is with one alone. For much of the long history of investigations on vision, seeing with one eye was considered superior to that with two. In large part this was a consequence of the theory that visual spirit passed from the brain to the eyes. According to this view, the visual spirit was more concentrated when it was confined to a single eye. Allied to the belief of the central locus of the visual spirit was the theory that light emanated from the eye rather than received by it. The contrasting theories of the nature of light were decided largely by optics. In 1589 Porta [6] likened the eye to a camera obscura and declared that this is “how vision is made” [7] (p. 365). A few years later Kepler [8] described image formation in the eye and Scheiner [9] dissected the eyes of numerous animals and provided an illustration of the gross anatomy of the mammalian eye with accurate curvatures of the surfaces of the lens and with the nasal exit of the optic nerve. This was a turbulent time in the history of binocular vision, too. Two approaches to how we see a single world with two eyes were proposed; one was based on suppression and the other on fusion. In 1593, Porta [10] advanced a seductively simple idea: we only see with one eye at a time and the other is suppressed. It was based on an easily observed phenomenon—binocular rivalry. When different patterns are presented to each eye (like separate pages from a book) they are not combined but compete. Porta stated that the competition favours one eye alone, usually the right eye. Two decades later, Aguilonius [11] proposed that the two eyes cooperated resulting in fusion of signals from similar parts of each eye. Correspondence between parts of each eye was not a novel idea; it can be traced back to Ptolemy in the second century and Alhazen in the eleventh [12,13] but it was refined and illustrated by Aguilonius. The refinement continued apace thereafter, culminating in the Vieth-Müller binocular circle [14,15] in the early nineteenth century. With fixation by both eyes on one point, a circle can be drawn that passes through the optical centres of both eyes and the fixation point. The theory proposed that all points on the circumference of this binocular circle stimulate corresponding points and are seen as single; any point not lying on the circle will be seen double. This was the dominant theory for how we see the world as single with two eyes until the stereoscope was invented [5].

On the basis of simple observations, Leonardo da Vinci disputed the theory that one eye was better than two and he wrote about this contrast in his numerous notebooks [16,17]. Moreover, his creations of virtuality on canvas remain unsurpassed. When comparing vision with one eye or two, he stated: “A Painting, though conducted with the greatest Art and finished to the last Perfection, both with regard to its Contours, its Lights, its Shadows and its Colours, can never show a Relievo equal to that of Natural Objects, unless these be view’d at a Distance and with a single Eye” [18] (p. 178). That is, the perception of depth is incomplete in a painting, unlike that for near objects in a scene viewed with two eyes. Despite appreciating the distinction, Leonardo was not able to act on his insight because no instrument for confirming it was available.

2. Stereoscopes

About 300 years after Leonardo (in the early nineteenth century) the situation was radically changed by the invention of the stereoscope by Wheatstone [19]: two slightly different representations of an object (equivalent to the left and right eye views of an object) when viewed in a stereoscope, created the impression of solidity quite different from that seen by either eye alone. The stereoscope (Figure 1) was announced to the public at a meeting of the Royal Society in London on 21 June 1838. Wheatstone is shown together with his illustration of the mirror stereoscope; it was with this instrument that he carried out most of his studies on stereoscopic depth perception. It is also possible to combine two adjacent pictures by under- or over-convergence, and this technique had been adopted to observe binocular rivalry [5]. Rivalry occurs when radically different images are presented to each eye. Wheatstone was able to combine slightly different images in this way to see stereoscopic depth but found that many other people could not do this easily. It is for that reason he invented the stereoscope.

The stereoscope transformed the ways pictures were perceived. It was not the only invention that had enormous implications for visual virtuality at that time. Six months later, in January 1839, two different means for making photographic images were announced to the public. First, Daguerre [20] described his metal-based positive method, followed quickly by the report of a paper-based negative process by Talbot [21]. Daguerre treated silvered-copper metal plates to make them sensitive to light which after exposure produced single positives which were called daguerreotypes. Talbot coated paper with silver chloride in darkness and when it was exposed to light in a camera a negative image was formed; positives could be produced by contact-printing the negatives. The benefits of photography for stereoscopy became apparent. Not only could paired photographs be seen in stereoscopic depth but photography was also an aid to the science of binocular vision: the precision required to draw stereoscopic diagrams could be bypassed. Wheatstone remarked: “What the hand of the artist was unable to accomplish, the chemical action of light, directed by the camera, has enabled us to effect” [22] (p. 7). Conventional photographs contain perspectival clues to the relative separations between different structures which can be amplified stereoscopically (Figure 2). That is, rather than viewing a single photograph with two eyes, two photographs taken from slightly different lateral positions (like the views from the left and right eyes) are combined by means of a stereoscope. The stereoscopic photograph in Figure 2, which is called an anaglyph, uses a very simple method of separating the images presented to each eye—colour. In addition, the monocular images are presented beneath the anaglyph so that adjacent images can be combined by over- or under-convergence of the eyes.

When announcing the stereoscope in 1838, Wheatstone [19] only described his model based on reflection of the left and right eye pictures (called half-images). This was so despite the fact that refracting (prism) stereoscopes were made for him in 1832 [23]. The prism model, together with his pseudoscope for reversing disparities, were both illustrated in his second memoir on binocular vision published in 1852 [22]. Prism stereoscopes were more compact and portable than mirror models, and the one devised by Brewster [24] became popular, in large part because of the burgeoning market in stereoscopic photographs. Many variations on refracting models were patented in the following years [25,26,27]. Projecting two cross-polarised half-images onto silver-coated screens and viewing them through similarly polarised Nicol prisms was introduced by Anderton [28]. The technique was more widely adopted when sheet polarizers were manufactured in the 1930s [4]. Employing coloured half-images and viewing them with similarly coloured filters in front of the eyes was explored by Rollmann [28] and D’Almeida [29]; they were given the name anaglyphs by Ducos du Hauron [30]. The popularity of anaglyphs increased with developments in printing stereoscopic drawings and photographs and they were used in movie making. Recent improvements in the bandwidths of colour filters and computer software for separating the half-images [31] has resulted in the adoption of a general standard of red/left eye, cyan/right for viewing stereo-photographs. Constructed stereograms do not have this constraint.

3. Stereoscopic Viewing

The initial stimuli for stereoscopic viewing were outline drawings of geometrical forms (Figure 3). Wheatstone [19] was aware that any drawing of an object would carry some monocular clues to its solid structure. Accordingly, he tried to avoid the use of monocular aids for conveying the impression of depth or distance so that only disparity was operating. To this end, he presented outline drawings of geometrical objects, without any shading (Figure 3). He realised that this did not remove all the monocular cues to depth and that critics would claim that the perceived depth could be seen with one eye. The most convincing evidence would be to produce a stereogram in which depth could only be seen as a result of binocular combination. Wheatstone’s Fig. 20 was the only drawing of a solid structure that included shading.

However, one stereogram (Fig. 11 in Figure 3), consisting of lines of dots, yielded depth without monocular recognition. That is, there is no indication in either the left or right patterns alone of the depth that is seen in their binocular combination. Depth disappears as soon as one eye is closed. The significance of this simple stereogram was not appreciated by Wheatstone. He made it to show that stimulation of non-corresponding points can result in single vision as well as depth, in contradiction to predictions from the Vieth-Müller circle. The stereogram can be considered as a precursor of the dot patterns to be developed later [32]. An illustration based on Wheatstone’s Fig. 11 is shown in Figure 4. Rather than a single row of dots, there are several rows with the dots increasing in horizontal separation from left to right creating the impression of a surface in depth. Note that variation in one physical dimension (lateral separation) results in a change in two perceptual dimensions—size as well as depth.

Wheatstone’s depth effects involved the whole surface whereas subsequent stereoscopists sought to create stereograms that yielded depth in an enclosed region within a display. We associate such enclosed stereoscopic depth effects with the random-dot patterns devised by Julesz [33] who used a computer to produce them. Small squares were in an orderly array of around 100 × 100 cells; the randomness involved assigning each cell as black or white. Once an array was formed a region within it (usually a central square of around 10 × 10 cells) was displaced laterally. This was presented to one eye and the original unchanged array to the other. Depending on the direction of displacement the square appears either closer to or farther from the surrounding corresponding cells. With this stereogram Julesz demonstrated stereoscopic depth of the square independently of recognising a square in either eye alone. He appreciated the novelty of the technique as well as the opportunities it provided for exploring binocular vision. A simple random-dot stereogram is shown in Figure 5.

Julesz produced a wider range of computer-generated carrier patterns than random dots. In his book Foundations of cyclopean perception [34], the stereograms were presented as anaglyphs as well as stereopairs for crossed or parallel viewing. Anaglyphs and viewers were provided for those readers who had difficulty with free-fusing the black and white stereograms. Anaglyphs do have an advantage over conventional paired stereograms insofar as the combined monocular views are superimposed, and that can result in designs that have visual appeal beyond the stereoscopic depth they conceal. This is much more evident when more complex patterns than random dots carry the disparities that can yield stereoscopic depth [5].

Pseudo-random patterns can be produced by hand in several ways, and they were applied to examine stereoscopic depth perception long before Julesz [33]. Various methods have been adopted for creating displays that are sufficiently irregular to conceal the displacement of some region within them. These include drawing or scribbling on paper; scattering small objects (like the paper discs from hole-punches) on a flat surface; dots of either similar or different sizes have been most commonly used. Nearly all employed photographing the patterns before modifying them. The novelty of Julesz’s system was that more systematic variations in stereograms could be made with computers. When Cajal was a medical student in the 1870s, he experimented with concealing messages in stereograms; the carrier patterns were drawn lines, dots or squiggles. He did not describe his technique in print until years later [35,36]. His technique was to draw a large pattern on a board with words written on glass in front of the board using the same pattern. The separated patterns on glass and board were photographed with a binocular camera and the two half-images were viewed in a stereoscope; the written words then could be seen in depth. Cajal produced stereograms very similar to Julesz’s random-dot stereograms [32]. The distinguishing feature between them is that Julesz appreciated their significance whereas Cajal did not. Cajal was more interested in the crypotographic possibilities offered by such concealment.

In 1919 Mobbs made two dot-patterns in which the letter L was visible in depth when combined binocularly by over- or under-convergence. Dots of different sizes were drawn on a printed surface and then reproduced on a transparent surface placed over it, adding dots in the shape of the letter L; the two patterns were then photographed separately and mounted side-by-side [37]. Paired patterns of white dots on a black background were produced by Kompaneysky [38]; when viewed binocularly the face of Venus could be seen in depth. Kompaneysky’s stereogram was made for free viewing.

Aerial stereoscopy was also a stimulus for Julesz [33] to make random-dot stereograms. A few years earlier, Aschenbrenner [39] was similarly stimulated by his work on revealing camouflaged objects from aerial stereoscopic photographs and he devised a novel means of simulating this: “A large quantity of black and white discards from a paper punch were thoroughly mixed together to assure an even mixture of distribution. The mixture was strewn on a large surface so that the density completely covered the surface. A single photograph was taken of the surface which is the background image of the stereogram. One set of letters was cut from a copy of the image, superimposed onto the background image, and photographically recorded. The letters were then displaced laterally by 1½ mm on the background photographic image and a second photographic record was produced. (The original photographs are 9 cm by 12 cm and are separated by 24 cm)” [40] (p. 1491).

Aschenbrenner’s stereogram was made for viewing with an optical stereoscope and the side-by-side half-images are reproduced in Shipley [40]. Aschenbrenner’s method of spreading small black and white discs over a larger surface satisfies the requirements of carrier patterns for deployment in stereograms: the photographed texture was made up of small elements of regular size irregularly distributed so that no overall distinctive features emerged. A small displacement of a region within it is difficult to detect monocularly but is the source of its appearance in stereoscopic depth. Black and white half-images of the other early stereograms can be found in Howard and Rogers [41] and Blundell [42]. Anaglyphs derived from all the stereograms can be seen in Wade [32].

Differences between the views from each eye were reported in antiquity but they were interpreted as providing a problem for binocular single vision and not a solution to depth perception. They could not be incorporated into theories of vision until the invention of the stereoscope [43,44]. The significance of the stereoscope is in demonstrating that the small disparities in the projections to each eye result in depth perception not double vision. Wheatstone also appreciated that stereoscopic depth perception was a consequence of processes in the brain rather than in the eye. At that time little was known about neural activity in the visual areas of the brain, but more recent research has supported Wheatstone’s theory [41].

One stereoscopic observation of note was reported shortly before the stereoscope was invented; it consisted of seeing repetitively patterned surfaces in false depth planes. In 1813, Blagden [45] observed his marble chimney-piece, which was fluted vertically. When he viewed it with two eyes but under-converged so that laterally separated flutings were aligned he saw the surface singly but apparently more distant than the physical surface; it also appeared larger. When he over-converged the surface looked nearer and smaller. Thus, the sign of convergence influenced the apparent distance and size of the fluting. In 1844 Brewster [46] reported a similar observation with repetitively patterned wallpaper. Its stereoscopic significance was evident to him, and he used depth based on convergence as evidence against Wheatstone’s [19] cognitive theory of stereoscopic depth perception [23]. Because Brewster illustrated the effect with pictures of patterned wallpaper it was subsequently called the wallpaper illusion. It can be seen in Figure 6, which is composed of an array of small portraits of Brewster. When equivalent but laterally separated patterns are combined binocularly, they seem suspended in the plane of convergence.

The wallpaper illusion involves patterns with regular and even horizontal separations between the elements. The introduction of slight variations in the separations along rows results in the visibility of more complex depth planes because aspects of disparity processing become involved. This can be seen in Figure 7 where there are systematic differences in the sizes and separations of the portraits of Brewster. Rather than seeing a single depth plane there are multiples. The surface no longer is seen as flat but stepped in depth. This is the principle employed in autostereograms [47,48], but they are now generated by computer rather than by hand.

The attraction of autostereograms is that they can be seen without the aid of any viewing device; they involve dissociating convergence from accommodation. Such uncoupling of a normally linked association can be difficult for some people to achieve, which in turn increases the likelihood that depth will not be seen in them.

Brewster referred to his prism model as a lenticular stereoscope because it was made from a single biconvex lens that was cut in half and mounted with the edges towards one another; they then acted as both magnifiers and prisms [49]. The term is now used to refer to a method of printing. Pictures in thin, vertical strips are alternatively printed beneath thin linear cylindrical lenses. Different pictures can be presented beneath the lenticulars so that movements of the head result in the apparent movement of the pictures. Strips of stereoscopic half-images can alternate so that each eye receives one of the two half-images. Viewed from a suitable distance the images from one set of strips are visible to one eye and those from the alternative strips to the other eye [4,5]. As with autostereograms, the attraction of lenticular devices is that the optical separation between the left and right eye images does not require an additional viewing device; they do, however, require quite precise positioning of the viewer’s head to align the images.

4. Moving Images

Depth perception deriving from head and eye movement has a longer history than depth from retinal disparity. What we would now call motion parallax was clearly described over 300 years ago [50]. For example, Rohault [51,52] outlined how movements of a single eye could determine judgments of distance: “If we make use of but one Eye, we can know the Distance of an Object, provided we move from one Place to another” [52] (p. 252). An even more precise account was given by La Hire [53] who used the term ‘parallax’ to indicate how near objects can be distinguished in depth: “The parallax of objects is what we use most to recognize distance, but the eyes must change place to recognize which of the two objects is the nearer” [53] (p. 236). Two eyes are mentioned in the quotation but the analysis applies to the effects for a single eye. La Hire provides clear accounts of the direction of displacement of objects either side of the object fixated and the link between the direction of displacement and the relative depth perceived. Wheatstone [19] went further and suggested that those with only one eye derived much of their perception of depth from motion parallax due to head movement: “A person deprived of the sight of one eye sees therefore all external objects, near and remote, as a person with both eyes sees remote objects only, but that vivid effect arising from binocular vision of near objects is not perceived by the former; to supply this deficiency he has recourse unconsciously to other means of acquiring more accurate information. The motion of the head is the principal means he employs” [19] (p. 380). Thus, the importance of motion parallax was appreciated but it was considered to be a substitute for binocular stereopsis when the latter was not available. In natural vision both stereopsis and motion parallax work in tandem, the former when the head is still and the latter when it is moving. The effects of head movement and depth can be readily seen in Figure 5. When the stereoscopic square looks more distant then lateral head movement result in it appearing to move in the opposite direction; when it appears nearer, movement of the head results in motion in the same direction.

The stereoscope was one of many instruments invented at around the same time to extend experimental investigations of vision [54,55]. Not only depth but also motion could be manipulated in laboratories of vision and attempts at combining stereopsis with virtual motion were made soon after [56]. The main participants in the early developments of apparent motion stereoscopy are shown in Figure 8. Wheatstone was pivotal in the developments and inventions and he was a catalyst in forging the alliance between stereoscopy and moving images. The phenomenon of visual persistence is at the heart of a series of inventions which found different ways of expressing its operation.

Young [57] illustrated the persisting patterns reflected from silvered paper attached to vibrating piano strings. Wheatstone [58] was stimulated by this phenomenon to devise an instrument rendering the motions of vibrating rods visible to the naked eye; it was called a kaleidophone or phonic kaleidoscope. Roget [59] described and interpreted the patterns seen when the spokes of carriage wheels rotate behind vertical railings. The thaumatrope or wonder turner was invented by Paris [60]. Faraday [61] viewed counter-rotating cogwheels and noted that when one wheel was aligned with the other stationary shadows were seen. In order to investigate it experimentally he constructed a simple arrangement of cut-out sectored discs and illustrated the patterns seen with it. Babbage [62] noticed that both sides of a spinning coin could be seen together. Talbot determined the apparent brightness of a rapidly rotating sectored disc with variable proportions of black and white [63]. Most of these instruments are illustrated in Wade [56].

The early devices rendered moving stimuli apparently stationary Whereas the later ones reversed this relationship: making a sequence of slightly different stationary images of a moving object appear to be in motion. Faraday was the catalyst for nurturing this idea: “The eye has the power, as is well known, of retaining visual impressions for a sensible period of time; and in this way, recurring actions, made sufficiently near to each other, are perceptibly connected, and made to appear as a continuous impression” [61] (p. 210). After reading Faraday’s article, Plateau [64] and Stampfer [65] constructed very similar devises that could synthesise motion from a sequence of briefly presented pictures, each differing slightly from its predecessor. Plateau called his instrument a phenakistiscope or fantascope, and Stampfer’s was called a stroboscopic disc (Figure 9). Plateau described how the instrument was used: “A cardboard disc pierced along its circumference with a certain number of small openings and carrying painted figures on one of its sides. When the disc is rotated about its centre facing a mirror, and looking with one eye opposite the openings … the figures are animated and execute movements” [64] (p. 305). Stampfer [65] described it in similar terms: “The principle on which this device is based is that any act of vision which creates a conception of the image seen is divided into a suitable number of single moments; these present themselves to the eye in rapid succession, so that the ray of light falling on the change of the images is interrupted, and the eye receives only a momentary visual impression of each separate image when it is in the proper position” [66] (pp. 499–500). The rotating discs were marketed commercially and became very popular throughout Europe. Many variants on the theme were made in the following decades and they often incorporated sequences of photographs; these instruments were the precursors of cinematography [4,67,68,69].

5. The Combination of Stereoscopic Vision and Motion

Wheatstone was at the forefront of developments in both virtual depth and virtual motion. He was well aware of the advantages photography offered for stereoscopy and, in a letter to Plateau in 1849 [68], he explored the possibility of combining stereoscopic photographs with apparent motion. Wheatstone did share interests in visual persistence with Plateau who passed on the suggestion to Duboscq, a Parisian optical instrument maker. Brewster enlisted Duboscq’s assistance in making and marketing his lenticular stereoscopes [5,23,70]. Plateau wrote: “One could go still further, taking advantage of an idea communicated to me by M. Wheatstone, which consists of combining the principle of the Stereoscope with that of the Phenakistiscope. Thus figures simply drawn on paper will be seen indisputably in relief and moving, and in this way will present, in a complete manner, all the appearances of life. This will be the illusion of art brought to its highest degree. M. Wheatstone has conceived obtaining, by means of photography on paper, two pictures of an object by placing the daguerreotype in two different positions, such that the two images have the necessary relationship between themselves” [68] (p. 238).

Duboscq attempted to add virtual depth to virtual motion in 1852 [71] as did Claudet, independently [72,73,74]. Claudet was a student and then partner of Daguerre and improved the daguerreotype process. Later, he opened the first daguerreotype studio in London and became recognised as a scientist as well as a photographer [75]; he took a stereoscopic daguerreotype of Wheatstone and his family in 1854 [23,76]. Duboscq announced his stereofantascope or stereoscope with movement, later calling it a bioscope. He combined the stereo-pairs by means of prisms so that they were presented simultaneously. Claudet also combined stereo and motion but he used a shuttering device for creating the stereoscopic effect. An article in La Lumière of May 1852, stated: “M. Claudet informs us that he has constructed a stereoscope in which one can see a person moving, for example a lady working with a needle and making all the necessary movements, a smoker with his cigar moving in and out of his mouth while exhaling smoke, people who drink and toast one another in the English way, steam engines in motion, etc. M. Wheatstone, on his side, without knowing about the device proposed by M. Claudet, seeks to resolve the same problem, and in a few days the mechanisms proposed by the two physicists will be published. M. Wheatstone and the inventor of the phenakisticope (Plateau) have been struck for several years by the possibility of applying stereoscopic principles to the effects of the phenakisticope, but at present have not been successful.” [77] (p. 88). A patent for the instrument was lodged by Claudet in 1853, but there is little evidence that Wheatstone continued with his attempts until over a decade later [56]. For Claudet, virtual motion was readily visible but virtual depth was less convincing. He presented the half-images in rapid succession. In retrospect he noted that he had “constructed his instrument in such a manner that by means of a slide with one hole he can, by moving it rapidly in a reciprocating horizontal direction, shut one lens while the other remains open; and in continuing that motion, while one eye sees one of the two pictures, the second eye cannot see the other picture” [73] (pp. 9–10). Since the motion was controlled by hand, and therefore the timings would have been variable, this could have been the reason why motion was more easily seen than stereoscopic depth. Claudet’s method is a precursor of the electronic shuttering systems that have been employed more recently [42]. The instrument was not produced commercially. He commented favourably but cautiously on Duboscq’s system: “M. Duboscq made some ingenious attempts in this direction, but not entirely satisfactory” [73] (p. 9). Claudet seems to have been a more astute observer than Duboscq. For example, he noted that with his alternating vision technique it was not possible to distinguish which eye was being stimulated at any one time. Many novelties were added to the instruments for combining depth and motion in the following decades [68,69,78,79], but Duboscq and Claudet were the pioneers. They might not have been successful but they did whet the appetites of both scientists and the public for seeing virtual motion in virtual depth.

6. Eye Movements and Stereoscopic Vision

Arguments about the involvement of eye movements in stereoscopic depth perception are as old as the stereoscope itself. The earliest, and most vitriolic, attack on Wheatstone was mounted by Brücke [80] and it was more concerned with binocular single vision than stereoscopic depth perception. He stated: “There are such good reasons for the theory of the identical points of the retinas, and from it arise such important conclusions for the theory of vision, that it is of the greatest interest to investigate how it can be defended against Wheatstone’s attack” [80] (p. 459). The essence of Brücke’s argument is that only a small (foveal) region of an object is clearly resolved at one time and the eyes move rapidly over an object with perception of it based on an integration of these fixations and the convergence of the eyes. He maintained that in the case of solid objects or stereoscopic images variations in convergence determine the depth perceived. Thus, Brücke proposed a combination of rapid eye movements and visual persistence as the basis for stereoscopic depth rather than the stimulation of non-identical (or disparate) points on the retina.

Brücke’s argument had been anticipated by Wheatstone [19] who reported that stereoscopic depth could be perceived with afterimages (which are stabilised on the retina). On repeating this experiment Brücke reported that he did not experience depth; he even questioned Wheatstone’s ability to maintain steady fixation. The issue was resolved by Dove [81] in the year Brücke’s article was published; Dove illuminated stereoscopic pairs with an electric flash and saw depth. Volkmann [82] reached the same conclusion by presenting paired stimuli briefly in a tachistoscope. Thus, the argument that stereoscopic depth was dependent on rapid changes in eye position could not be sustained. This does not exclude the possibility that eye movements, particularly those involving changes in convergence, do play a part in stereopsis. Julesz [33] noted that the emergence of depth in random-dot stereograms takes longer than in more conventional stereograms. The impression when viewing the former is that the eyes are hunting for correspondences in the patterns and this could reflect small variations in convergence. Such rapid changes in binocular convergence present considerable measurement challenges.

7. Summary and Discussion

Visual virtuality involves the visual perception and representation of space. Some of the earliest evidence of human cognition derives from incisions on cave walls, often representing animals. Perceiving objects in space accurately is essential for survival; it would seem that representing some of those objects is, too. Three-dimensional as well as two-dimensional representations were produced—both sculptural and pictorial works. Visual virtuality is more concerned with the latter. Over the centuries, different styles for representing three-dimensional space on a two-dimensional surface emerged and one of the most successful was linear perspective, invented in the early fifteenth century, which captured visual angles from a single viewpoint. It was literally a one-eyed representation of space, lacking the binocularity we experience in natural vision, as Leonardo da Vinci so eloquently described [18]. Indeed, he referred to the distinction between representation and perception as artificial and natural perspective, respectively. Thus, pictorial art has long grappled with the contrast between two- and three-dimensional perceptions of space.

Theories of space perception have displayed similar concern with seeing the world with one or two eyes, but they relied more on notions of brain function than on observation, as well as on theories of the nature of light itself. For many centuries, it was argued that the visual spirit coursed from the ventricles to the eyes where light emanated resulting in vision. The visual spirit was more concentrated when passing to one eye than two so it was contended that monocular vision was superior to binocular. At the beginning of the seventeenth century, Kepler [8] provided an account of optical image formation in the eye and Scheiner [9] described and illustrated the gross anatomy of the mammalian eye. These, together with Porta’s [7] equation of the optics of the eye with that of a pinhole camera supported a reception theory of vision; external light was received by the eye rather than produced in it and projected outward. These developments focused interest on projections to a single eye rather than to two. Porta [6] went so far as to propose that we see with one eye alone, even when both are open. Aguilonius [11] proposed the contrary theory that there is fusion of the signals from corresponding points in each eye. It is this theory that was amplified in the early nineteenth century [14,15].

The situation was transformed by the invention of the stereoscope [19] and the experiments Wheatstone carried out with it. Binocular vision was not only different to monocular vision, but it added a dimension to it—depth from disparity. Wheatstone’s theory of stereoscopic vision was also transformational; it proposed that stereoscopic depth perception was a consequence of processes in the brain based on signals from the eyes. Stereoscopic representations of objects in space remained problematical due to the difficulty of producing stimuli devoid of monocular depth clues. Julesz [33,34] provided a solution with computer-generated random-dot stereograms: stimuli could be constructed in which depth could only be seen with stereoscopic viewing and not by either eye alone. Julesz added that the stereograms he constructed did not correspond to any solid objects that would be encountered in the natural world. With natural vision of a three-dimensional scene, depth can also be derived from motion parallax when the head moves. Virtual motion, that is the visibility of motion generated by a succession of stationary images, was demonstrated at about the same time as the invention of stereoscopes [64,65]. Attempts to combine virtual depths and motions were made in the 1850s [71,72] but their success was limited.

We perceive the natural world in much the same way as our ancestors did despite the many differences in the objects that surround us. Perhaps, the most significant changes are associated with the virtual worlds we encounter. Perception has not changed but representation has. Displays can now be digital, dynamic and interactive. Do the historical developments outlined above have implications for contemporary research on visual virtuality? Has the increased understanding of depth and motion perception assisted in virtual representations of the world? Probably so, if the problem of visual virtuality is seen as trying to restore the perception of three-dimensionality from two-dimensional input. However, this is only a problem if the input to each eye is considered as two-dimensional like a picture projected onto a screen. There is no two-dimensional image on the retina, only neural activity that varies with the intensity of the light striking receptors and the speed of transduction and transmission to more central sites [83,84]. This is an issue for visual perception as well as for visual representation. Paradoxically, the invention of the stereoscope hastened the use of pictorial stimuli (albeit paired) in laboratory studies of perception [54]. If the aim of visual science is to understand how we perceive objects in space then it will not be advanced by investigating pictures of objects.

8. Conclusions

The language used to describe phenomena associated with perceived depth and motion has changed relatively little over the centuries but the technological innovations for investigating them over the same period have been enormous. Nonetheless, the perceptual issues separating the virtual from the real remain much the same.