Augmented Reality and Perception of Analogue and Digital Images and Maps ^†

Abstract

The idea of combining information from a database with information derived directly through the senses is termed augmented reality. Augmented reality (AR) is that organically, in real-time and dynamically overlaying virtual images created by computers and other information on real environment which the observer sees. And when the observer moves in real environment, virtual information changes according to the movement, just like those virtual information truly exists in real world. AR is used with the mobile devices with GPS and WiFi connections. The user gives the system information to identify the desired destination, and the system then generates sufficient information. The disadvantages reveal themselves when there is no WiFi connection or there is no space in the storage of mobile device to maintain the image of the whole map. Then the traditional ways of recognition of localization must be used, most often with analogue (paper) map or image. The presentation of coding of some data and metadata on an image or a map (both in digital and analogue form) permits inter alia the inclusion of e.g., paper map or analogue image into the chain of digital devices use. Some solutions, remarks and comments concerning functioning of the digitally augmented (printed) map within the information society are presented.

1. Introduction

Geographers uses metaphors. Johannes Vermeer’s famous painting The Geographer has been the subject of such metaphor of the importance of virtual reality: “On the table in front of the figure is a map, taken to represent the geographer’s window on a part of the world that happens to be of interest. But the subject figure (Geographer) is shown looking out of the window, at the real world, perhaps because he needs the information he derives from his senses to understand the world as shown on the map” [1] (pp. 286–287). Printed (paper) maps and analogue photos (images) are yet important but nowadays there is prevailing use and rising challenge of growing use of mobile (digital) ones. Technology influenced also into traditional understanding of map definition in geography and cartography.

2. Digital Map and Its Usefulness in Information Society

Maps, as spatial notations of the reality surrounded us, are linked to the process of civilization changes since the earliest times. The problem of map ontogenesis is the question of human-milieu relation. Optimal solution of this issue results from integrated human structures: somatic, psychological and mental, revealing during the human actions in the environment, including also the creative and useful role of vision of the map. So, map is oriented information entity based on unity of three concepts: system (target), model (information) and image (transmission), tria iuncta in uno [2]. Taking into account that the relation between map and human action is causal, the content of definition of map according to A.Makowski is as following: “Geographic map is a systemic model-image information entity, mapping space-time situations in context of human intentional action in adopted georeferenced system coordinates.” [3].

All maps recently are instances derived from geospatial databases (Figure 1).

Almost all map instances characterize interaction and feedbacks to geospatial database. They make possible update of the digital maps, but only hardcopy i.e., printout of map is finished product and seems completely separated from mother, geospatial database. Update of hardcopy needs restart of the whole production process, and the feedback with geospatial database is limited or does not exist. Interaction of hardcopy with digital devices is minimal.

Classic processing of spatial data in Geographic Information Systems (GIS) involves several stages: fieldwork and acquisition of spatial data, archiving (management and editing), spatial analysis procedures (e.g., generalization), geovisualization (display on demand) and (in case of need, final) printout.

People still buy and use printed (paper) maps, despite of the digital technologies expansion, like they still buy and read newspapers, books or display their scanned images on digital devices. The popular on-line advices from experienced travelers are e.g.,: “The separate problem is fact, that during the travel with support GPS device, the large number of users come to conclusion, that paper map may (and absolutely should in any case), calmly lie in a backpack. The GPS information completely support orientation in the hardest terrain. Even if the digital map is not very detailed in the device or presents only the waypoints.” [4].

• If cartographical paper map yet is not the niche product, one should at least attempt to diagnose the several reasons of this situation [5]:
• First, habits, shaped during the curricular and extracurricular education i.e., formal and tacit knowledge of printed maps, atlases and guidebooks.
• Second, lack of WiFi or GPS signal (regardless of reason).
• Third, lack of knowledge and skills which make possible generation on demand of digital (professional) maps by the users from accessible sources in net and ever more widely available software and geolocation tools.
• Fourth, psychological human mechanism, i.e., belief in “durability” of printed image, which is available any time without additional equipment.
• Fifth, hardcopy, i.e., printed map documents the state of real world in defined point of time and may be the proof (according to law) to any public or individual decisions, actions, projects and claims.
• It can be enumerated the several (omitted above) other reasons, e.g., maps become historical documentation and present also aesthetical values as the works of art.

3. Augmented Reality

The definition of augmented reality (AR), avoiding limiting to specific technologies, concerns the systems involving three characteristics [6]:

• combines real & virtual [elements],
• interactive in real time,
• registered in 3D.

AR is dynamic overlay of digital, virtual images and information concerning real environment, which is observed. Images and virtual information change constantly while observer is moving. AR means to integrate synthetic information into the real environment and means overlay of spatially-registered computer graphics over a live image of the real-world. The range of applications of AR include (inter alia, [7]):

• geolocation using GPS and Geographic Information Systems (GIS),
• geonavigation, especially marine and air traffic,
• geovisualizations of interactive maps, i.e., displaying and interactive analysis of terrain, interactive 3D maps and landscape visualizations,
• Building Information Modeling (BIM): visualization and modeling of buildings and urban fabric,
• support for visitors: displaying tags and labels of observed objects,
• simulations: flight and drive simulators,
• virtualization of conferences,
• entertainment and education.

The newest service supported by Google is e.g., searching by images. Service is related to geolocation using images of terrain objects i.e., search places by images.

Two different augmented reality display concepts exist to superimpose graphics onto the user’s view of the real world; video see-through and optical see-through head-mounted displays (HMD) have been the traditional output technologies for augmented reality applications for almost forty years. The quite new is spatial augmented reality approach (SAR). SAR is new display paradigms exploit large spatially-aligned optical elements, such as mirror beam combiners, transparent screens, or holograms, as well as video projectors. In many situations, SAR displays are able to overcome technological and ergonomic limitations of conventional AR systems [8].

Milestones of AR have been presented in several publications [8,9,10,11,12]. All of them pointing out reference to diagram depicting reality-virtuality continuum by Paul Milgram and Fumio Kishino [13] (Figure 2). There are two types of mixed reality i.e., augmented reality (AR) and virtuality (AV), which are in the scope of the interest of geographers and cartographers.

4. Geographic Information System versus Augmented Reality

The progress in Information, Communications and Technology (ICT) sector has brought rapid development of AR coexisting with GIS technology on portable devices. GIS and AR processing chain involves gathering of: specific user request, display parameters concerning scale, symbology and metadata of requested map and users position (location) and orientation.

Combination of GIS and AR supports both indoor as well as outdoor exploration of spatial data. Augmented virtuality (AV) is related to exploration of geospatial data directly from real or virtual (digital) world and may be defined as augmented map (AM), which implies the extensible use of map as the base element of searching of information. Augmented reality (AR) as the interactive tool makes possible gathering additional information during exploration of natural environment or anthropogenic (e.g., urban) milieu can be, in turn, defined as augmented territory (AT).

These types are designed and implemented into mobile applications (e.g., GPS navigation) [8].

4.1. Mobile AR and Paper Maps

So far combination of GIS and AR predominantly is related with mobile (digital) maps and there are few other practical applications except navigation. Ann Morrison (et al.) experimented with augmented reality (AR) using a ‘magic lens application’ over a paper map using smartphone. “Maps are one of the main application categories for mobile AR. The focus is in augmentation of physical maps with useful and interesting real-time information. Paper maps have a large static surface and AR can provide a see-through lens without forcing the user to watch map data only through the small ‘keyhole’ of the display” [14] (p. 1).

Some findings points out, that type and layout of physical map undergone augmented reality operation (augmented map) affect its efficiency while using mobile devices (through spatial pattern analysis and ways of users’ interaction). “The maps act as mediation objects for multimodal discourses providing resources such as a context and facilitation for embodied communication.”[15] (p. 3).

Some other examples are related to geocaching—outdoor recreational activity using GPS receivers (often in smartphone) and e.g., its latest, new instance Pokémon Go application, globally available since 2016. But its root can be derived from orienteering—(formal) olympic discipline, which requires from users navigational skills using (specially prepared, without labels) maps and compass while moving in unknown terrain at speed, during the race.

Yet another example of augmented maps, not exactly, but almost paper, are commercially available portable document format (PDF) maps offered on the market (e.g., Avenza Maps).

4.2. Map Context and AR Technology in Details

Since some time, each year, Gartner, Inc. identifies and reports top strategic technology trends and hype cycle of emerging technologies on the market. In 2016 the company pointed virtual reality and augmented reality on 4th position after intelligent things (3rd) and intelligent applications (2nd) on the list, enumerating the importance of the IT technologies in 2017. Simultaneously, AR has been pointed as the IT technology, which crossed the peak of inflated expectation and is closer to maturity. Designers, programmers and users got rid of illusions of AR and the technology will enter into the next stages of enlightenment and (real) productivity [16,17].

Scientific approaches differentiate of AR technologies into two main types: marker based AR (prevailed indoor applications) and marker-less/gravimetric AR (used in outdoor applications). Marker-based AR uses template markers or 2D barcodes (e.g., QR-codes, Datamatrix, etc.). “This reality type uses visual marker known as fiducial or AR card to determine center, orientation and range of coordinate system. The system detect marker, identify and calculate pose of the object with help of computer vision techniques. (…) Marker-less AR utilizes GPS (geopositioning), compass and other related sensors. Augmented reality browsers help users to navigate between POIs (Point of Interests) appear on camera view for exploring AR contents based on location and context.” [12] (p. 115).

Simultaneously, authors are pointing that, there are challenges of mobile augmented reality related to [12]:

• Image capturing capabilities of smartphones camera, which are dependent on lighting conditions.
• Energy consumption, which remained an open challenge for smartphones.
• Access a large amount of data of AR applications over networks for locating/navigating point of interests; any downtime of network access may harm instant response to users.
• Accuracy of sensor information, which is a vital component for indoor and outdoor navigation.
• Lighting Conditions, with which technologies are struggling, to remain readable in bright outdoor conditions.
• Technology Adoption Life Cycle rising from the fact, that people resists change to adopt new operating mechanism for handling devices and learning new methods of interactivity and realism.

On the other side, some significant voices warn that growing volume of digital data would might disappear as the result of sudden crash or be wiped out in result of crime or war. “…digital data of crucial importance to archivists and future historians are potentially under threat from deletion, corruption, theft, obsolescence and natural or man-made disasters. (…) We’re only just beginning to understand how important this data is and what the consequences might be if we lost it.” [18]. Maps as the one the valuable and important product of human culture, science and economy are especially vulnerable to mentioned above threats. Other instances derived from geospatial databases (as described on Figure 1), as well as all the digital images (photos) can be similarly treated.

5. Intelligent Augmented Digital and Paper Map

Paul Levinson in his book, The soft edge a natural history and future of the information revolution wrote: “(…) an information technology will survive to the extent that it satisfies human needs better than its rivals. (…) First, the ongoing evolution of new media, usually with profound unintended consequences, means that the rivals of any given medium are often in flux. Second, a given medium usually performs a variety of tasks, which results that it may well best its rivals in one area, and fail against the same or a different set of rivals in another area.” [19] (p. 185).

The idea concerning intelligent augmented analogue map (or analogue image) is based on opinion, that classic map (or image/photo) would have advantages not only in critical situations (blackout, no mobile or WiFi coverage or no GPS signal for any reason). Mobile device application could reveal and interpret especially prepared markers (or RFID tags) hidden in the analogue map. This set of hidden tags included into symbology and legend of map lets using the map (or image) in classic, traditional way and not disturbing human perception. Simultaneously, these tags expand volume of information contained on the printed, paper map (or photo, e.g., they let reveal metadata).

From the point of view cartographer/geographer, it needs to prepare additional, steganographic information layer of map, overlaid on basic, printed content, thematic layers and calibrated with markers, which make possible its augmenting. The mix of known technologies can be used, to include the intelligent, augmented paper map into digital world, inter alia:

• digital and classic steganography,
• augmented reality, software applications,
• fiducial markers,
• mobile devices with different sensors, GPS, etc.
• access to GPS and WiFi signal,
• (last but not least) analogue (printed) paper map (or image/photo).

This way augmented paper (printed) map (or photo) would become (more) intelligent thing included into the chain of digital devices use, and does not loosing its primarily function. Other side is mobile intelligent application.