Augmented and Virtual Reality Evolution and Future Tendency

: Augmented reality and virtual reality technologies are increasing in popularity. Augmented reality has thrived to date mainly on mobile applications, with games like Pokémon Go or the new Google Maps utility as some of its ambassadors. On the other hand, virtual reality has been popularized mainly thanks to the videogame industry and cheaper devices. However, what was initially a failure in the industrial ﬁeld is resurfacing in recent years thanks to the technological improvements in devices and processing hardware. In this work, an in-depth study of the different ﬁelds in which augmented and virtual reality have been used has been carried out. This study focuses on conducting a thorough scoping review focused on these new technologies, where the evolution of each of them during the last years in the most important categories and in the countries most involved in these technologies will be analyzed. Finally, we will analyze the future trend of these technologies and the areas in which it is necessary to investigate to further integrate these technologies into society.


Introduction
Augmented reality and virtual reality are technologies that have been under research for several years [1]. Even so, there are several products that have been developed in that line and that are accessible to the general public [2,3].
However, due to the society's needs and variations, these technologies have stagnated in certain areas. Therefore, it is important to know the research evolution they have had in recent years and, thanks to that, to study the current trend in order to anticipate the areas in which they will be applied in the coming years.
First, it is convenient to define the concepts of augmented reality and virtual reality in order to better understand the subject of this work.
These terms are part of the concept of "virtuality continuum" defined by Paul Milgram and Fumio Kishino [4]. This term describes a continuum that goes from reality itself to virtual reality generated by a computer. Within the virtual continuum, we find the subset of mixed reality, which has been defined as everything between reality and a totally virtual environment (see Figure 1).
We can find several definitions of Virtual Reality (VR), but maybe the most global and inclusive way of defining it is as follows: "A Virtual Reality is defined as a real or simulated environment in which a perceiver experiences telepresence", written by Jonathan Steuer [5]. This is the chosen definition because it put apart the technology's implications, and in this way, there is no need to specify any Head Mounted Displays (HDM) or globes and we can focus on techniques and applications to try to figure out what is the path that the technology is following.
In the same way, we can define Augmented Reality (AR) as a technique to show extra information over the real world. With this definition, there is no need to talk about specific hardware but we can specify techniques and applications and focus on technology development [6]. Although the technologies of AR and VR have been under development for a long time [7], we can say that it has recently begun to leave the laboratories, thanks in large part to the improvement of the computing capacity and the lowering of the devices [8]. Nowadays, augmented reality and virtual reality can be used in a mid-range smartphone [9]. Although, for more immersion, it would be more interesting to use devices that are more sophisticated, although a little more expensive.
However, this technology still has a small user niche and a small field of application based primarily on games. In this work, we investigate these technologies' evolution over the last years, which aspects have evolved more, and the different fields that have been benefited from this evolution.
The health sector is one of the fields that have adopted these immersive technologies to help patients, providing training processes for phobias treatments and even for surgery simulations. There are also several scientific studies that demonstrate that virtual reality helps patients reduce stress and anxiety levels in the face of pain perception [10,11]. In the field of education and industry, they stand out as very powerful tools for learning and training, achieving a more efficient, interactive, and participatory learning.
The main goal of this work is try to answer these questions: What has been the evolution of AR and VR until now? and What is the trend that AR and VR will have in the coming years? As said before, AR and VR have experienced an evolution in recent years, but it is important to analyze this evolution in depth. Are AR and VR more popular than before? Nowadays, prestigious companies are investing in these technologies.
In order to answer the first question, the most relevant works in the last years will be analyzed and the information obtained will be used to get the conclusions about the global evolution of these technologies and the evolution in the different fields and regions. For the second question, the publications over the last years will be examined in detail to check the status of these technologies (peak of research, maturity, production, etc.), and finally, videogame publications regarding these technologies will be studied to know its application to this particular field.
There are two other questions we wanted to answer: Has anything been investigated about AR/VR about collaborative interactions to share its information? and Is there any kind of protocol to manage collaborative systems of augmented reality or virtual reality? There are some works inside the videogame industry that use collaborative information sharing (like Pokemon Go as the most popular example), but it is important to know if these interactions and information communications are actually in use in other fields like industry or medicine, along with the protocols used by some popular devices as Oculus Rift. The aim of this study is to determine the communication standards and protocols used to transmit and store virtual information in order to use it as a first step to develop a new communication and storage protocol for collaborative environments that allows all devices to be integrated and that can be applied to all areas.
The rest of the manuscript is divided as follows: first, the methodology used for the study is presented. Next, the results obtained after the study are detailed and analyzed. Finally, the conclusions are exposed.

Methodology
The methodology used in this work corresponds to the classical scoping review process. The processing steps used for this work are shown in Figure 2.
First, we have to analyze the works and manuscripts of which the information is interesting for this review. Therefore, we need some basic information from these works: title, ISSN (International Standard Serial Number), topic, etc. Then, we have to choose the criteria used for the searching process. After that, each author independently uses the developed guide to search for relevant information inside the works/manuscripts. Next, all the information obtained is collected: the authors exchange their manuscripts, and the searching process starts again. This step is repeated until each author reviews all the manuscripts: in this work, there are three authors, so the total amount of manuscripts reviewed is divided by three and distributed over them. Each author finishes reviewing all the manuscripts after three iterations. At this point, all the information is extracted from the manuscripts, put in common, and mixed. The final review result will be shown in a summarizing table. Finally, this data will be analyzed in order to get the information that answers the questions indicated in the Introduction section.

Extracted Information
In this epigraph, the information extracted from the reviewed manuscripts is detailed. This information is essential in order to get the answers to the initial questions. After describing the most relevant information needed for the study, the searching step will be detailed.

Searching Phase
In order to obtain a large amount of good-quality works to analyze, only scientific papers published in international journals are taken into account. The main search engine used in this work is Google Scholar because it integrates works from several platforms, simplifying this step. If the information is not complete, Mendeley database is used to fill it.
The keywords used in the searching process are detailed below: Google Scholar is configured to search works and patents from the year 2000, so other cites like books or book chapters are not included. The information obtained is sorted by relevance and only English-written papers are taken into account. With this configuration, the firsts 200 entries are extracted, and to be used in this work, it must talk about augmented reality or virtual reality (or include some device that uses one of them).
After the first search process (papers from the year 2000), the results obtained are as follows: • Virtual reality: 1,490,000 results, with 22 papers chosen.
After analyzing superficially the results, the first search presents one conclusion: a big amount of the works obtained are obsolete (many papers are outdated), so several papers are not good enough for this work. After that, the searching process is redefined using a different configuration: the starting year is changed to 2010 in order to obtain more recent and relevant works.
After the second search process (papers from the year 2010), the results obtained are as follows: • Virtual reality: 322,000 results, with 29 papers chosen.
In this occasion, we obtain less entries than before, but we can pick up more information. Also, to get a more complete manuscript database for this work, two new searches are done using extended keywords: collaborative virtual reality and collaborative augmented reality. As detailed in the introduction section, for a future work, it is important to know if there is enough information about collaborative works that use AR or VR.
This final search obtains these results: Summarizing, all the searches obtain about 1200 potential papers and we select 189 papers after a first filter. Then, as detailed above, we only contemplate international journal works with ISSN, so we take out 26 of them. Finally, we a have a database of 163 papers (this information is shown in Table 1). As detailed previously, Table 1 shows that the first search was made without good criteria from 2000; that is why the searching configuration changed in the second search from 2010.

Results
In this section, the results obtained after analyzing the works detailed previously are presented. First, the number of publications evolution is studied and presented. After that, a deep data analysis is done using the geolocation of the authors during the work development and the main topics on which they are focused.

Manuscript Publication Evolution
After collecting the work database, we analyze the number of publications year by year in order to get the evolution on AR and VR fields. In the same way that this information selection is done, we start this analysis from the year 2000 and we use Google Scholar to get the data.
As we can see in the Table 2 and in Figure 3, nowadays, the popularity on this field is lower than before: this is maybe influenced by the low cost of the technology that made possible working with AR in middle tier smartphones and VR headsets, so these technologies came out from laboratories to the industry and, finally, to the people. To make this hypothesis stronger, we check out Gartner's hype of cycle [12]; this hype of cycle is known for estimating the emerging technologies and the time when these technologies will be in production. Gartner's hype cycle represents the evolution of interest in certain technologies, classifying them in five stages during their interest cycle (in this order):

•
Technology trigger: technology is starting. Gartner's hype cycle evolution for virtual reality and augmented reality technologies can be observed from the year 2000 until now in Figure 4. In this figure, the "x" axis represents the year and the "y" axis represents the Gartner's hype cycle stage where the technology is placed. The position in this axis is related to the importance inside Gartner's hype cycle.
As can be observed in Figure 4, the first apparition of augmented reality in Gartner's hype cycle is dated back to 2005. This technology starts in the "Technology trigger" section of the curve until 2010, when it reaches the section "Peak of inflated expectation". Augmented reality stays in the section "Trough of disillusionment" from 2013 to 2018. Finally, in 2019, it can be found again in the "Technology trigger" section but with the addition of the term "Cloud". This evolution indicates that, between 2010 and 2012, it was a very popular topic and, after those years, its importance diminished. However, currently, this technology has become interesting again thanks to the inclusion of cloud technologies, which allow computing in the cloud and provide ease of use to the end user. Virtual reality technologies appear in different ways inside Gartner's hype cycle, and all these related terms are taken into account to show its evolution in Figure 4. First, it can be found as "Head mounted display" inside the "Technology trigger" section in 2001. Its next occurrence is in 2007 as the term "Virtual environment/Virtual worlds" located in the "Peak of inflated expectation" section; it stays in this section until 2012. After this year, this term changes its section: in year 2013, it appears in the "Trough of disillusionment" stage. It is important to observe that, from 2013 to 2016, it appears as the term "Virtual Reality". Finally, in 2016, it reaches the "Slope of enlightenment" stage. Virtual reality disappears from the hype cycle in 2018, so it can be considered to be in the production stage (this fact will be demonstrated after the references analysis). Summarizing this first study, after year 2013, both technologies start a decay stage until 2019, when both almost disappear from Gartner's hype cycle. However, AR is getting a new life as an "AR cloud", which is a new paradigm that uses cloud technologies with augmented reality works, and it is important to get focus on that too. With this information, we should not expect that virtual reality will increase its importance in the next coming years; however, AR tendency describes a fresh start with the inclusion of cloud technologies, so we should expect an important increase in the works and references in this field in the next years.
The numerical data about the total number of publications in those years (not only the papers selected for this work) is shown in Table 2.
The data presented in Table 2 is represented graphically in Figure 3. In both representations, it is important to observe the tendency detected in Gartner's hype cycle for AR and VR: the works related to both technologies are decreasing in the last two years. In Gartner's hype cycle, its decay phase started in 2013-2014 and its consequences are observed in the works and citations four years later. Because of this relation, there is a real possibility that, in the next 3-4 years, there will be an increase in the number of works and publications related to augmented reality due to the increased importance of AR due to the cloud technologies. On the other hand, these technologies are very close to the game industry, so it is important to focus the attention in this sector and study if the same evolution can be observed in this field. To do that, a deep searching process is done using the most famous platforms and devices (like Steam or Oculus Rift), looking for games that use one of them at least. The only way to look for that is using the Steam platform because other platforms do not allow to sort by year or to look for certain keywords. The evolution of publications in the game industry obtained is shown in Figure 5. As can be seen, the developed games tendency (see Figure 5) is similar to the manuscript publications in virtual reality (see Figure 3). Game development reaches a peak in 2017 and then starts to decrease. Because of the improvement of virtual reality HDM, it is possible that this tendency will change and become stable in a few years.
To do this study in AR, the same problems are observed: Google Play and Oculus do not allow to sort information by year. Only some sporadic works in this area can be observed (like Pokemon Go), but too few works are obtained. Therefore, no tendency can be extracted from this information.

General Data Analysis
A general information study has been done above regarding manuscript publications in the last years. Now, using the methodology described in the previous section, the manuscript database, a subset of 163 manuscripts, is examined deeply in order to obtain more detailed information.
This study includes the country where the authors developed the publications and the field of application.

Countries
First, a study focused on where the publications come from is carried out. After this study, the regions that are researching and developing AR/VR technologies will be shown.
After analyzing the database, results are detailed in Figure 6 (top). In this figure, it can be observed that the results are shown in a global way and focus on USA and EU (the regions with more publications in AR and VR).
The results of the study show in Figure 6 (top) that the European Union (EU) is the region which invests more funds in these fields (34.4% of publications came from this region), the United States of America (USA) follows with 27.3% of the publications, and Australia can be found in third place. As can be seen, EU and USA take up more than 60% of the publications, so the study will focus on these two regions.
Now, we will focus on the areas in which these regions are investing. Later, the different topics will be analyzed in detail.
As we can see in Figure 6 (bottom), both regions have a lot of similarities about fields of publication. However, their main differences came from education and industry: EU made more efforts in industry than USA.
In Figure 7, it can be observed that most of the publications are carried out after 2010. We can see that the number of publication of papers decreases. However, this pattern is not the same as seen in Figure 3: in that case, the popularity decreases in 2017, but in this occasion, the peak of popularity is in 2010 and, after that, starts decreasing. This fact can be a consequence of the citation system: older papers have more references than newer ones, so Google Scholar shows first the most cited works; in this case, it gives more importance to those works.
Finally, we compare the number of publications of our subset year by year in Figure 7. After studying the general results year by year, they will be separated in the main topics where these technologies are being applied.

Topics
The main topics that have been taken into account are research and development (R&D), healthcare, education, and industry. The distribution of the selected works with respect to the selected topics is shown in Figure 8.
In Figure 8, it can be seen that the most developed topic is R&D with 34.4% of the works selected, which is quite logical because of the search engine used to select the works (Google Scholar, which is a platform mostly used for research works publications). However, 65.6% of the selected works belong to specific applications. In this section, three relevant topics can be observed: healthcare with 25.8%, education with 17.2%, and finally, industry with 9.8%.
We can analyze those results in the same way as above, using a year-by-year classification. These results are shown in Figure 9.  In Figure 9, some details belonging to each topic and its evolution are observed. R&D reaches its peak of popularity in 2010 and 2011, but it slowly decreases until 2016, when it reaches a small peak, and starts to decrease again. In healthcare, it also reaches popularity peak in 2010 and starts to decrease until 2017. In education, the popularity peak is reached in 2012 and 2013. Industry's publications are almost stable from 2010.
In the next subsections, these topics are studied deeply in order to find more information about them. Also, we will study its evolution in the EU and in the USA.

Research and Development
This topic is the most referenced. In this field, many specializations are covered but are not going to be represented because of the multiple fragmentations that can be obtained. For example, we can observe works about integrating VR on mobile devices [13], papers about issues in AR [14], works about pose estimation [15,16], works about how to use P2P (peer-to-peer) networks to make collaborative systems for AR [17], etc.
From the database used in this work, those manuscripts in which the main topic is related to research and development are divided in countries, and the results are shown in Figure 10.
Although several countries are represented in Figure 10, EU and USA cover 61.6% of the publications. The remaining 38.4% is very fragmented between Asiatic and Oceania countries.

B. Healthcare
As we see in 8, healthcare is the most popular field of application of AR and VR. First, we will analyze the topics studied.
In Figure 11 (top left), it can be observed that the most relevant subtopic is surgery, followed closely by psychology and rehabilitation. In the surgery field, there are several works for helping surgeons before or during surgery [18], training for surgery in order to minimize risks [19], or even some systems to introduce students to this field. In psychology, we found papers focused, for example, on phobias [20]. In rehabilitation, most of the works are related to helping stroke patients [21,22].
Focusing the attention in EU and USA regions (see Figure 11 (bottom)), the analyzed data shows that surgery and psychology are the most relevant topics: both cover almost 95% of the total publications in those regions. In the USA, psychology takes a bit more interest than surgery and rehabilitation is an important topic to be taken into account.
This study is extended to all the regions around the world to compare the results (see Figure 11 (top right)). As expected, EU and USA are the regions with more contributions.

C. Education
In this section, the main attention is focused on the topic of education. The process made for the previous topic is done for this one too, analyzing the results after that. Those results are shown in Figure 12.
The works related to these topics are divided in three categories: first, "Early" stage is related to works focused on the early stages of the education, up to 10 or 11 years old; "Middle" is used to denote works for high school until they are 18 years old; and finally, the category "High" is used for works applied to university studies. Figure 12 (top left) shows that the most present category is "Middle": related to that, we can find works focused on helping students with maths [23] or science [24,25], among others. In the category "Early", we can find works focused on teaching kids about science, too [26]. Finally, in the category "High", works are focused on applying complex teaching techniques like collaborations [27,28]. We also found works which used AR/VR technologies to teach languages [29] in more than one category.
The information obtained from Figure 12 shows that 50% percent of manuscripts found deal with working with students from middle school. Another important conclusion is that works with higher levels look more interesting than early levels (more citations).
In Figure 12 (bottom), it can be observed that EU is focused mainly on the "Middle" and "High" levels but, in USA, the "Middle" education category covers 63.3% of the works while the "High" category covers less than 10%.
Finally, observing Figure 12 (top right), the main regions focused on this topic are EU and USA, covering 60% of the contributions.

D. Industry
Finally, the industry topic is analyzed in the same way as the previous topics. The summarized information can be seen in Figure 13.
The information presented in Figure 13 is divided in three categories. The first and more popular one is "Maintenance", covering 50% of the studied works: in this topic, we can find techniques to maintain factories [30,31] or aircraft [32]. The second category is "Productivity", which is fundamental in modern industries in order to obtain better results and more efficiency in the daily industry; this can be done to help workers work with industrial robots or to even use some applications to make realistic 3D models and to share with colleagues [33][34][35]. Finally, the category "Training", with 12.5% of the publications, covers works focused on helping workers to learn how to use their equipment [36]. Figure 13 (bottom) presents the analyzed information from USA and EU. It can be observed that 71% of the EU's works belong to the "Maintenance" category. Therefore, the "Training" and "Productivity" categories remain in second place in the EU. On the other side, USA's works within this topic are focused on productivity and maintenance equally.
Finally, Figure 13 (top right) presents the total distribution by countries. As can be seen, the EU and USA cover 60% of the publications.

Conclusions
Results have been presented in different ways, focusing the information in countries and topics. In this way, it is easier to understand the future tendency of AR and VR technologies. First, all the results are summarized in Table 3. After this, the results and its tendency are analyzed and described. In Table 3, the information summary is analyzed for the main topics (R&D, healthcare, education, and industry) within the two most relevant regions (EU and USA). However, in this investigation, relevant works focused on other categories or topics have been found. For example, we also found papers about archaeology [121][122][123], architecture [124],videogames [125], construction [126][127][128], tourism [129][130][131][132], geography [17], or big data visualization [133].
Once the information has been presented and analyzed, it is important to return to the main objective of this work. In the introduction, four main questions were detailed and, after this work, we are able to answer them.
The first one was what has been the evolution of AR and VR until now? As have been detailed during this work, a lot of works focused on AR and VR fields have been developed. A huge amount of papers have been published involving countries around the world, but we can see that EU and USA are the main centers of investigation for these technologies. We can also see that most of the publications can be grouped in four topics: research and development, healthcare, education, and industry.
The second question presented was what is the trend that AR and VR will have in the coming years?
Using the information obtained from the total number of publications observed year by year and Gartner's hype cycle evolution tendency observed in the publications, AR and VR technologies are losing popularity. Researches in these areas will continue working on them, but the number of publications will decrease until they reach stability. In the videogame industry, the tendency observed is similar to the one presented in the other fields. One of the most downloaded and played games (Pokemon Go) is based on AR technology, but there are no popular games that use VR technology, although there is a small and stable niche market. AR technology does not have a popular tool to make videogames, but it is very common to find it in several works about surgery, industry, or education. We also can conclude that AR popularity will grow in the next years thanks to the evolution of cloud technologies and the possibilities that they can contribute to AR.
Third question was has anything been investigated about AR/VR about collaborative interactions to share its information?
There is not much researching projects focused on this topic, but we can find some like VirCA (Virtual Collaboration Arena) project, which is focused in robotic control [28,44,115].
The final question was is there any kind of protocol to manage collaborative systems of augmented reality or virtual reality?
Developers of VirCA project use Robotics Technology Middleware (RT-Middleware) to make their scenarios, and the interconnection is made using a VPN (Virtual Private Network). Apart from this, we did not find any protocol.
After answering the questions presented at the beginning of this work and after carrying out the analysis about publications in AR and VR technologies, it can be concluded that we are currently in a stage of decline of these technologies. However, there are certain areas where the trend is stable or slightly upward. In addition, thanks to the evolution of cloud technologies, it is very possible that the tendency will change in the coming years. We can infer that the interest slope observed in these technologies can change but will need the help of new technologies, such as 5G, which will make possible lighter and smarter devices, or AR cloud, which can help to compute the most complex information. Other technologies, like artificial intelligence paradigms or biosensors, are also suitable to be integrated with AR and VR applications but, as can be observed in Gartner's curve, only cloud technologies are linked to the augmented reality field. Moreover, AR applications need powerful hardware systems; thus, it will be normal that the first technology of all of them that joins AR is the cloud one.
About collaborative environments, only a few works are focused on creating such types of environments. In addition, there are virtually no standardized mechanisms to represent and send information from the virtual environment, which may be one of the reasons why there are no advances in collaborative environments. If future works focus on developing standards in this way, devices from different manufacturers and with different encoding could be integrated into a single virtual environment. Funding: This research was funded by the Telefonica Chair "Intelligence in Networks" of the Universidad de Sevilla, Spain.