- freely available
Future Internet 2017, 9(4), 60; doi:10.3390/fi9040060
Naomi wakes up in the morning as usual, then she unmounts her smart glass from a charging spot and wears it. The virtual agent character “Riki” begins talking to her. He looks like a strange monster whose size is small enough to be held by her. The character is not so unusual after wearing a glasses-type display on a daily basis. She is notified of the schedule and the weather today by him, and she is gradually waking up. He tells her, “Today is not hot, but it seems to be rainy; it will last until the evening, so you should bring your umbrella.” She also tells him that she recently had lost her umbrella. He offers to lend his umbrella delightedly. He may feel happy because he can help her. He has an umbrella in the virtual world, and there is a physical umbrella that is the same design as Riki. In addition to the umbrella, there are several of his own things in her house. One example is a ball toy that Naomi does not completely understand. He occasionally plays with it deftly, for example juggling with it. Looking at his playing, Naomi is healed for a while, so she has not regretted buying it for him. She also practices juggling a little, but she stops because she cannot juggle better. Recalling such a thing, she takes breakfast while watching the news displayed on a wall. After breakfast, she dresses in her favorite suit and goes to her office with Riki's umbrella.
3. Analysis of Current IoT Products and Services that Introduce Virtuality
3.1. Current IoT Product Development Landscape
3.2. Case Studies of Digital Services Introducing Virtuality
3.2.1. Virtual Aquarium
- The households are asked to continue their normal activities while daily tooth brushing patterns are recorded by the toothbrush attached the Cookie. Once the patterns have become stable over time, the study progresses to the next phase.
- Virtual Aquarium’s feedback components are introduced to the households’ bathrooms while the tooth brushing patterns continue to be recorded. Once the patterns have stabilized to a pre-determined degree over time, the study progresses to the final phase.
- The feedback components are removed while tooth brushing patterns continue to be recorded. Once the patterns have stabilized to a pre-determined degree over time, the data collection is concluded. Brushing records are analyzed to determine the effect of the feedback on the patterns.
- Association-based method: People who used this method associated an environment’s physical objects with the intended information to memorize. For example, when a participant had to memorize the description about Niger, she imagined water from Niger because the description suggests that Niger had an image of water; thus, she placed a corresponding marker near water bottles. Another participant said that she chose a place related to water because the marker’s texture looks like water.
- Category-based method: The participants categorized the information to be memorized and then placed the similarly categorized information spatially close to each other. For instance, the participants placed information about country names to be memorized in one location and placed population and demographic information in another location and then started memorizing the information.
- Sequence-based method: The participants memorized the information by arranging the information based on personally created rules. For instance, a participant sorted information, via their corresponding fiducial markers, in alphabetical order and then memorized all information.
3.2.3. Ambient Bot
4. Design Space for Virtuality-Introduced IoT Products
4.1. Preliminary Definition of the Design Space
- In this level, an IoT product displays information using a physical device. For example, Virtual Aquarium used a physical display. Since Visualizing Level 0 service employs a fixed display, the display needs to be deployed in places where the context introduced by virtuality is easily understood. For example, Level 0 is good at providing services for spatially limited activities, such as restrooms and bathrooms. In another example, another IoT product named Augmented Go provides functions to support the game of Go with a projector, and it displays the exercises of past games . Some attempts to extend the real world using displays and projectors have also been studied .
- In this level, an IoT product visually extends physical objects, and virtual objects can be arranged on the physical objects, which enables the virtual objects to be seamlessly introduced in the real world. We suggest that these services will be seen more often when it becomes commonplace to wear an HMD device, such as HoloLens, on a daily basis. Compared with Visualizing Level 0, if a wearable HMD, such as Google glass, is more likely to be used, it can improve mobility to exploit virtuality. Virtual objects can directly introduce virtuality; thus, the immersiveness can be enhanced.
- In this level, an IoT product displays virtual objects that are free from the physical constraints of the real world. For example, Ambient Bot can display the virtual character without the physical constraints of the real world. Therefore, the real space can be virtually enhanced through virtual objects seamlessly located in the real world. Eliminating the unnaturalness of the virtual objects that appear in the real world is a challenge to be overcome at this level. Ambient Bot approached this problem from the aspect of character design. If creators do not adequately design the reality of the introduced virtuality, users will not be able to believe the introduced virtuality, then the service may become meaningless.
4.2. A Preliminary Focus Group-Based Analysis
- We gave the participants a detailed explanation of the taxonomy of IoT products and the concept of Visualizing Level for approximately 30 min.
- We explained the details of the focus group for approximately 10 min. The participants discussed two IoT products proposed in Kickstarter: Static IoT and Customizable IoT. The goal of this focus group was to propose IoT products by discussing the Visualizing Level of each taxonomy of IoT products. Therefore, the participants were encouraged to propose at least six ideas.
- The participants started their discussion. The participants could ask questions about Visualizing Level and the taxonomy of IoT products.
- When the discussion of three Visualizing Levels of the two types of IoT products is completed, the focus group was terminated.
4.3. Design Space for Virtuality-Introduced IoT Products
- Worldview: in a virtual world, a designer decides what kinds of things have value in the world. In many cases, a deep story is not necessary in the virtual context. For example, in Virtual Aquarium there is a definition of the world that is the aquarium, and there is a worldview, where it is important to keep it beautiful.
- Individuality of a character: it is possible to add virtuality by virtual characters. By displaying the characters a designer can give virtuality, extend an IoT product in various directions, and use it as a rhetoric to change or persuade a user’s behavior.
- Game Mechanism: gamification also gives a type of virtuality. By introducing the gaming context, it becomes possible to use the product more positively and encourage users’ actions.
5. Future Direction: Incorporating Virtuality for a Community
Conflicts of Interest
- Allan, S.; Gilbert, P. A Social Comparison Scale: Psychometric Properties and Relationship to Psychopathology. Personal. Individ. Differ. 1995, 19, 293–299. [Google Scholar] [CrossRef]
- Da Silva Cameirão, M.; Bermúdez I Badia, S.; Duarte, E.; Verschure, P.F. Virtual Reality Based Rehabilitation Speeds Up Functional Recovery of the Upper Extremities After Stroke: A Randomized Controlled Pilot Study in the Acute Phase of Stroke using the Rehabilitation Gaming System. Restor. Neurol. Neurosci. 2011, 29, 287–298. [Google Scholar] [PubMed]
- Ip, H.H.S.; Wong, S.W.L.; Chan, D.F.Y.; Byrne, J.; Li, C.; Yuan, V.S.N.; Lau, K.S.Y.; Wong, J.Y.W. Virtual Reality Enabled Training for Social Adaptation in Inclusive Education Settings for School-Aged Children with Autism Spectrum Disorder (ASD). In Blended Learning: Aligning Theory with Practices, Proceedings of the 9th International Conference, ICBL 2016, Beijing, China, 19–21 July 2016; Cheung, S.K.S., Kwok, L., Shang, J., Wang, A., Kwan, R., Eds.; Springer International Publishing: Cham, Switzerland, 2016; pp. 94–102. [Google Scholar]
- Hock, P.; Benedikter, S.; Gugenheimer, J.; Rukzio, E. CarVR: Enabling in-Car Virtual Reality Entertainment. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA, 6–11 May 2017; pp. 4034–4044. [Google Scholar]
- Miyashita, T.; Meier, P.; Tachikawa, T.; Orlic, S.; Eble, T.; Scholz, V.; Gapel, A.; Gerl, O.; Arnaudov, S.; Lieberknecht, S. An Augmented Reality Museum Guide. In Proceedings of the 7th IEEE/ACM International Symposium on Mixed and Augmented Reality, Cambridge, UK, 15–18 September 2008; pp. 103–106. [Google Scholar]
- Billinghurst, M.; Kato, H.; Poupyrev, I. MagicBook: Transitioning between Reality and Virtuality. In Proceedings of the CHI ‘01 Extended Abstracts on Human Factors in Computing Systems, Seattle, WA, USA, 31 March–5 April 2001; pp. 25–26. [Google Scholar]
- PTC. ThingWorx IoT Platform. Available online: https://www.ptc.com/en/internet-of-things/technology-platform-thingworx (accessed on 30 August 2017).
- Lee, J.H.; Ku, J.; Cho, W.; Hahn, W.Y.; Kim, I.Y.; Lee, S.M.; Kang, Y.; Kim, D.Y.; Yu, T.; Wiederhold, B.K.; et al. A virtual reality system for the assessment and rehabilitation of the activities of daily living. CyberPsychol. Behav. 2003, 6, 383–388. [Google Scholar] [CrossRef] [PubMed]
- Maule, L.; Fornaser, A.; Tomasin, P.; Tavernini, M.; Minotto, G.; Lio, M.; Cecco, M. Augmented robotics for electronic wheelchair to enhance mobility in domestic environment. In Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); 10325 LNCS:22–32; Springer: Berlin, Gernamy, 2017. [Google Scholar]
- Ktena, S.I.; Abbott, W.; Faisal, A.A. A Virtual Reality Platform for Safe Evaluation and Training of Natural Gaze-Based Wheelchair Driving. In Proceedings of the 7th International IEEE/EMBS Conference on Neural Engineering (NER), Piscataway, NJ, USA, 22–24 April 2015; pp. 236–239. [Google Scholar]
- Pokric, B.; Krco, S.; Drajic, D.; Pokric, M.; Rajs, V.; Mihajlovic, Z.; Knezevic, P.; Jovanovic, D. Augmented Reality Enabled IoT Services for Environmental Monitoring Utilising Serious Gaming Concept. JoWUA 2015, 6, 37–55. [Google Scholar]
- Atsali, G.; Panagiotakis, S.; Markakis, E.; Mastorakis, G.; Mavromoustakis, C.X.; Pallis, E.; Malamos, A. A Mixed Reality 3D System for the Integration of X3DoM Graphics with Real-Time IoT Data. Multimed. Tools Appl. 2017, 1–22. [Google Scholar] [CrossRef]
- BMWi. Federal Ministry for Economic Affairs and Energy. Industrie 4.0. Available online: https://www.bmwi.de/Redaktion/EN/Dossier/industrie-40.html (accessed on 30 August 2017).
- Heng, S. Industry 4.0: Huge Potential for Value Creation Waiting to be Tapped. Available online: https://www.i40.de/wp/wp-content/uploads/2015/04/Industry-4.0-Huge-potential-for-value-creation-waiting-to-be-tapped.pdf (accessed on 30 August 2017).
- Inaizumi, R. The Era of Farming from Your Office has Come. Forbes Jpn. 2017, 38, 50–53. [Google Scholar]
- TongKe, F. Smart Agriculture Based on Cloud Computing and IOT. J. Converg. Inf. Technol. 2013, 8, 210–216. [Google Scholar]
- Zulkifli, C.; Noor, N. Wireless Sensor Network and Internet of Things (IoT) Solution in Agriculture. Pertanika J. Sci. Technol. 2017, 25, 91–100. [Google Scholar]
- Popović, T.; Latinović, N.; Pešić, A.; Zečević, Ž.; Krstajić, B.; Djukanović, S. Architecting an IoT-Enabled Platform for Precision Agriculture and Ecological Monitoring: A Case Study. Comput. Electron. Agric. 2017, 140, 255–265. [Google Scholar] [CrossRef]
- Cook, D.J.; Youngblood, M.; Heierman, E.O.; Gopalratnam, K.; Rao, S.; Litvin, A.; Khawaja, F. MavHome: An Agent-Based Smart Home. In Proceedings of the First IEEE International Conference on Pervasive Computing and Communications (PerCom 2003), Fort Worth, TX, USA, 23–26 March 2003; pp. 521–524. [Google Scholar]
- Qian, Z.; Wu, K.; Bi, Z. The Wireless Sensor Network of the Family Environment Monitoring System Research. In Proceedings of the 6th International Conference on Machinery, Materials, Environment, Biotechnology and Computer (MMEBC 2016), Tianjin, China, 11–12 June 2016. [Google Scholar]
- Das, A.K. A Secure and Effective biometric-based User Authentication Scheme for Wireless Sensor Networks using Smart Card and Fuzzy Extractor. Int. J. Commun. Syst. 2017, 1–25. [Google Scholar] [CrossRef]
- Pirbhulal, S.; Zhang, H.; E Alahi, M.E.; Ghayvat, H.; Mukhopadhyay, S.C.; Zhang, Y.; Wu, W. A Novel Secure IoT-Based Smart Home Automation System using a Wireless Sensor Network. Sensors 2017, 17, 69. [Google Scholar] [CrossRef] [PubMed]
- Niantic. Pokémon Go Celebrates One Year of Exploration and Discovery with Real-World and In-Game Summer Events. Available online: https://www.nianticlabs.com/press/2017/anniversary2017/ (accessed on 30 August 2017).
- Althoff, T.; White, W.R.; Horvitz, E. Influence of Pokémon Go on Physical Activity: Study and Implications. J. Med. Internet Res. 2016, 18, e315. [Google Scholar] [CrossRef] [PubMed]
- Caudell, T.P.; Mizell, D.W. Augmented Reality: An Application of Heads-Up Display Technology to Manual Manufacturing Processes. In Proceedings of the Twenty-Fifth Hawaii International Conference on System Sciences, Kauai, HI, USA, 7–10 January 1992; pp. 659–669. [Google Scholar]
- Holmgren, D. Design and Construction of a 30-Degree See-through Head-Mounted-Display; Department of Computer Science technical report TR; UNC Chapel Hill: Chapel Hill, NC, USA, 1992. [Google Scholar]
- Fischer, E.; Haines, R.F. Cognitive Issues in Head-Up Displays; NASA Ames Research Center: Moffett Field, CA, USA, 1980.
- Konica Minolta. Konica Minolta Develops the World’s First Automotive 3D Augmented Reality Head-Up Display. Available online: https://www.konicaminolta.com/about/releases/2017/0227_01_01.html (accessed on 30 August 2017).
- Fisher, S.S.; McGreevy, M.; Humphries, J.; Robinett, W. Virtual Environment Display System. In Proceedings of the 1986 Workshop on Interactive 3D Graphics, Chapel Hill, NC, USA, 23–24 October 1986; pp. 77–87. [Google Scholar]
- Oculus. Oculus Rift. Available online: https://www.oculus.com/rift/ (accessed on 30 August 2017).
- VIVE™. Discover Virtual Reality Beyond Imagination. Available online: https://www.vive.com/us/ (accessed on 30 August 2017).
- PlayStation. PlayStation VR. Available online: https://www.playstation.com/en-au/explore/playstation-vr/ (accessed on 30 August 2017).
- Cluster. Available online: https://cluster.mu/ (accessed on 30 August 2017).
- Kickstarter. Available online: https://www.kickstarter.com/ (accessed on 30 August 2017).
- Brixo. Building Blocks Meet Electricity and IoT. Available online: https://www.kickstarter.com/projects/1068475467/brixo-building-blocks-meet-electricity-and-iot?ref=discovery (accessed on 30 August 2017).
- Kickstarter. MODI: Create Anything You Want with Robotics of Things. Available online: https://www.kickstarter.com/projects/luxrobo/modi-create-anything-you-want-with-robotics-of-thi?ref=nav_search (accessed on 30 August 2017).
- Kickstarter. Wio Link—3 Steps. 5 Minutes. Build Your IoT Applications! Available online: https://www.kickstarter.com/projects/seeed/wio-link-3-steps-5-minutes-build-your-iot-applicat?ref=nav_search (accessed on 30 August 2017).
- Kickstarter. Omega2: $5 Linux Computer with Wi-Fi, Made for IoT. Available online: https://www.kickstarter.com/projects/onion/omega2–5-iot-computer-with-wi-fi-powered-by-linux?ref=most_funded (accessed on 30 August 2017).
- MiaLinkup. Mialinkup Connects You with Your Car Like Never Before. Available online: https://www.mialinkup.com/ (accessed on 30 August 2017).
- Kickstarter. Raksha SafeDrive—Your Safety Companion on the Go! Available online: https://www.kickstarter.com/projects/prasadpillai/raksha-safedrive-indias-first-smart-road-safety-pl?ref=nav_search (accessed on 30 August 2017).
- Raksha SafeDrive. Safe, Smart & Connected Car. Available online: https://www.rakshasafedrive.com/ (accessed on 30 August 2017).
- Kickstarter. Autonomous Cultivation Controller w/The Track Your Tree App. Available online: https://www.kickstarter.com/projects/iot-gardens/autonomous-cultivation-controller-w-the-track-your?ref=discovery (accessed on 30 August 2017).
- Kickstarter. Tittle X: The World’s First Ultra Compact IoT Golf Simulator. Available online: https://www.kickstarter.com/projects/1601126475/tittle-x-the-worlds-first-ultra-affordable-golf-si?ref=nav_search (accessed on 30 August 2017).
- Milgram, P.; Kishino, F. A Taxonomy of Mixed Reality Visual Displays. IEICE Trans. Inf. Syst. 1994, 77, 1321–1329. [Google Scholar]
- Nakajima, T.; Lehdonvirta, V.; Tokunaga, E.; Kimura, H. Reflecting Human Behavior to Motivate Desirable Lifestyle. In Proceedings of the 7th ACM Conference on Designing Interactive Systems, Cape Town, South Africa, 25–27 February 2008; pp. 405–414. [Google Scholar]
- Yamada, Y.; Irie, K.; Gushima, K.; Ishizawa, F.; Al Sada, M.; Nakajima, T. HoloMoL: Human Memory Augmentation with Mixed-Reality Technologies. In Proceedings of the 21th International Academic Mindtrek Conference, Tampere, Finland, 20–21 September 2017. in press. [Google Scholar]
- Gushima, K.; Akasaki, H.; Nakajima, T. Ambient Bot: Delivering Daily Casual Information through Eye Contact with an Intimate Virtual Creature. In Proceedings of the 21th International Academic Mindtrek Conference, Tampere, Finland, 20–21 September 2017. [Google Scholar]
- Kimura, H.; Tokunaga, E.; Okuda, Y.; Nakajima, T. CookieFlavors: Easy Building Blocks for Wireless Tangible Input. In Proceedings of the CHI’06 Extended Abstracts on Human Factors in Computing Systems, Montreal, QC, Canada, 22–27 April 2006; pp. 965–970. [Google Scholar]
- Kawsar, F.; Fujinami, K.; Nakajima, T. Prottoy: A Middleware for Sentient Environment. In Proceedings of the International Conference on Embedded and Ubiquitous Computing, EUC 2005, Nagasaki, Japan, 6–9 December 2005; pp. 1165–1176. [Google Scholar]
- Yates, F. The Art of Memory; Random House: London, UK, 2014. [Google Scholar]
- Suri, J.F. Thoughtless Acts?: Observations on Intuitive Design; Chronicle Books: San Francisco, CA, USA, 2005. [Google Scholar]
- Okoshi, T.; Tsubouchi, K.; Taji, M.; Ichikawa, T.; Tokuda, H. Attention and Engagement-Awareness in the Wild: A Large-Scale Study with Adaptive Notifications. In Proceedings of the 2017 IEEE International Conference on Pervasive Computing and Communications (PerCom), Big Island, HI, USA, 13–17 March 2017; pp. 100–110. [Google Scholar]
- Sakamoto, M.; Alexandrova, T.; Nakajima, T. Introducing Virtuality to Enhance Game-Related Physical Artifacts. Int. J. Smart Home 2014, 8, 137–152. [Google Scholar] [CrossRef]
- Sakamoto, M.; Nakajima, T.; Alexandrova, T. Enhancing Values through Virtuality for Intelligent Artifacts that Influence Human Attitude and Behavior. Multimed. Tools Appl. 2015, 74, 11537–11568. [Google Scholar] [CrossRef]
- Fandom Powered by Wikia. Candy. Available online: http://psychopass.wikia.com/wiki/Candy (accessed on 30 August 2017).
- Iwata, T.; Yamabe, T.; Nakajima, T. Augmented Reality Go: Extending Traditional Game Play with Interactive Self-Learning Support. In Proceedings of the 2011 IEEE 17th International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA), Toyama, Japan, 28–31 August 2011; pp. 105–114. [Google Scholar]
- Sakamoto, M.; Nakajima, T.; Alexandrova, T. Digital-Physical Hybrid Design: Harmonizing the Real World and the Virtual World. In Design and Semantics of Form and Movement; Koninklijke Philips Electronics: Amsterdam, The Netherlands, 2012. [Google Scholar]
- Ring, L.; Utami, D.; Bickmore, T. The Right Agent for the Job? In Proceedings of the 17th International Conference on Intelligent Virtual Agents, Stockholm, Sweden, 27–30 August 2017; pp. 374–384. [Google Scholar]
- Bartle, R. Hearts, Clubs, Diamonds, Spades: Players Who Suit MUDs. J. MUD Res. 1996, 1, 19. [Google Scholar]
|When users like to use Ambient Bot||Users like to receive news or weather forecasts when they wake up in the morning.|
|Users like to receive content when they have free time or take a rest and have nothing to do.|
|Users like to receive content when they are walking.|
|Users receives potentially necessary information when he/she is not conscious of it.|
|What type of content users like to receive from Ambient Bot||Users like to receive social media information from the jellyfish.|
|Users want to hear content from a lower-pitched human voice.|
|Users like to receive news content from a human-like character.|
|Users want Ambient Bot to aid in conversation with other people.|
|How users like to use Ambient Bot||Users want to receive various content without wearing and operating any physical devices.|
|Users only like to receive brief message content.|
|Users want to see text messages in a clearer manner. In some backgrounds, the clarity of the messages is hindered by their colors and positions.|
|Users want to choose the type of content that they receive according to their current situations.|
|Users expect to speak with a jellyfish.|
|Users expect that a jellyfish behaves with more jellyfish-like motions.|
|Visualizing Level 0||Visualizing Level 1||Visualizing Level 2|
|Group A||Static IoT|
|After the user performs eco-driving, he can check from the smartphone app that the island in the virtual world becomes beautiful||By using HUD or HMD, signs and pedestrians hidden in the building are sensed and displayed.||Navigation characters guide the user on HUD.|
|Providing a game that combines given modules to complete the target circuit and using it to study circuits.||Showing electronic current/ Creating a game stage with the blocks that is inspired by Mario Maker.||Objects created with blocks can be used in the virtual world as avatars.|
|Group B||Static IoT|
|Using HUD to replace conventional navigation/ Showing how much you are driving on the front passenger side.||Detecting the flow of the surrounding car and telling it to the driver/ Showing the weather of the destination by overlaying his view/ The window of the rear seat displays the beautiful scenery||Displaying movies outside the rear seat window/ Displaying books and movies to non-drivers who do not experience car sickness|
|Writing a line in the figure displayed on the display and reflecting the result in the physical world/ Providing a puzzle game/ Using block as UI||Displaying a completed circuit as a hint||None. It is hard to tie this product with Level 2 because it is bound to the physical world if it uses physical blocks.|
|Group C||Static IoT|
|An agent in the screen calls out when an accident happens and uses a touch screen to confirm the consciousness (to determine whether consciousness is interrupted)/ Displaying the accident map on a HUD.||Displaying vehicles hidden behind the scenes through HUD/ Overlaying the colors so that they can be recognized as young drivers and high accident rate in the car/ Showing road hazards to encourage drivers to slow down.||If the driver speeds, encouraging careful driving by inserting a virtual high-end Mercedes-Benz car in front of him/ Displaying a guide in front/ Displaying a virtual character that will speak from the front passenger’s side to encourage relaxation.|
|In order to prevent losing the moules, showing where the module is in the room in the application.||As people can see where the IoT device was installed, highlighting the location of the IoT device/ Making the electronic currents visible for debugging||A virtual person demonstrates how to use the block for the introduction of MODI/ Simulating by combining virtual modules. If the user can create something good, then he can buy the modules like e-shop.|
|Virtuality Level 0||Virtuality Level 1|
(Autonomous Cultivation Controller)
|Since the provided functions already achieved visualization of information, virtualization can be advanced by changing the Visualizing Level.||Adapting the idea of Virtual Aquarium to this / Customizing the character of the plant according to the user's preference and changing the state of the character according to the state of the plant/ Adding elements of gamification according to the result of pH level and watering timing.|
|When the user implements the function to visualize the data obtained from the sensor using a module to show a dashboard, he can display information virtually by designing Visualizing Level appropriately.||A fairy is in the module of Arduino, and she has preferences which modules she like. It also contains the virtual character and may include the world view why she exists. The user can know her world using Arduino modules adequately.|
© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).