Overview and Exploitation of Haptic Tele-Weight Device in Virtual Shopping Stores

Abstract

In view of the problem of e-commerce scams and the absence of haptic interaction, this research aims to introduce and create a tele-weight device for e-commerce shopping in smart cities. The objective is to use the proposed prototype to provide a brief overview of the possible technological advancements. When the tele-weight device is affixed over the head-mounted display, it allows the user to feel the item’s weight while shopping in the virtual store. Addressing the problem of having no physical interaction between the user (player) and a series game scene in virtual reality (VR) headsets, this research approach focuses on creating a prototype device that has two parts, a sending part and a receiving part. The sending part measures the weight of the object and transmits it over the cellular network to the receiver side. The virtual store user at the receiving side can thus realize the weight of the ordered object. The findings from this work include a visual display of the item’s weight to the virtual store e-commerce user. By introducing sustainability, this haptic technology-assisted technique can help the customer realize the weight of an object and thus have a better immersive experience. In the device, the load cell measures the weight of the object and amplifies it using the HX711 amplifier. However, some delay in the demonstration of the weight was observed during experimentation, and this indirectly altered the performance of the system. One set of the device is sited at the virtual store user premises while the sending end of the device is positioned at the warehouse. The sending end hardware includes an Arduino Uno device, an HX711 amplifier chip to amplify the weight from the load cell, and a cellular module (Sim900A chip-based) to transmit the weight in the form of an encoded message. The receiving end hardware includes a cellular module and an actuator involving a motor gear arrangement to demonstrate the weight of the object. Combining the fields of e-commerce, embedded systems, VR, and haptic sensing, this research can help create a more secure marketplace to attain a higher level of customer satisfaction.

1. Introduction

Smart cities are a decisive upheaval, and the perception of a smart city is akin to a rummage sale for the urban and developed vision in which diverse electronically progressive approaches, practices, and sensor networks are cast-off to collect data and manage assets within a city including transportation systems, water supply networks, traffic system, information system, power grids, and other community services by the local government [1,2,3]. The goals of smart cities are to improve the quality of life by using technologically advanced tools, techniques, and methods. Data visualization, modeling, management, and e-commerce sectors are essential fragments of smart cities, and planning professionals are looking for state-of-the-art real-time simulations. In this current “what if?” situation, impact analysis requires a huge number of resources, and both virtual reality (VR) and virtual stores are the potential tools to address the upcoming challenges in smart cities [4,5].

Developers have been working for many years to create new VR-based systems. However, due to the inclusion of haptic devices, sensor actuators, and improved realistic graphics in the last decade, the developed VR systems have undergone a technical evolution with improvements in telepresence and immersion [6]. A VR platform generates a computer-simulated environment that is fostered and assisted by the head-mounted display (HMD), which utilizes a manipulator in an immersive environment. VR is broadly used for cognitive purposes in various industries in the fields of engineering, architecture, and medicine [7,8,9]. It is assimilated with other fields and innovations as a supporting technique. For instance, surgeons who treat patients undergoing VR have gained admiration [10,11].

Software makers are also in the perspective of introducing virtual stores where consumers can enter the virtual stores and visualize the dimensions and usability of an item prior to purchasing it [12]. A virtual shopping store is a computer-simulated store where a handler can envisage, feel, and interact with the products and items in the store. Implementation of the virtual store can be a breakthrough in the e-commerce market. Other famous terms for virtual stores are cybernetic stores and computer-simulated 3D spatial stores [13]. In current e-commerce stores, one of the recorded fraudulent activities is the delivery of an ordered item different from that itemized in the e-commerce store [14,15,16,17]. Needless to say, a virtual store can spark a revolution in the e-commerce industry by not only altering the way of shopping but also by making it secure. Haptic devices make physical interaction possible between a human and a computing device by means of sensors and actuators. Humans can interrelate with haptic devices to deliver programming instructions and, correspondingly, schedules can be employed to extract essential information via a haptic interface [18,19,20].

The tele-weight device is evaluated in this project. Overall, the tele-weight device project is split into two sections. One part of the device measures and records the object weight ordered on the virtual store and sends the encrypted value of the weight as a short message service (SMS) over the cellular network. The extra end receives the encrypted SMS, decrypts it, and illustrates the weight materially using motor gear (actuator) motion. The objective is to improve the immersive shopping experience of the e-commerce user.

VR headsets (e.g., Oculus Rift, HTC Vive, etc.) provide an interactive 3D graphical experience to the wearer, but at present there is no way to interact physically with the running simulation. This study addresses the problem of creating interaction and introduces a tele-weight device that allows the weight to be shown materially. This is done to improve the immersive experience of the user while shopping in a VR store. Load cells are common devices used to measure the weight of an object and have been used in this research [21]. Load cells measure the weight of an object (usually of order 10⁻⁶ V), and the measured weight is amplified using an HX711 amplifier. Upon amplification, the obtained weight is transmitted to the receiver end where the e-commerce customer can realize the weight of the object to have a more immersive VR experience.

The objective of this research is to present a haptic device named “tele-weight” whose resolution is to enable consumers to feel the weight of the product ordered in the virtual shopping store. Moreover, the customers can feel the weight physically upon selecting an item in the virtual store. By having a weight-based assessment along with the displayed and delivered item, the customers will be able to identify the authenticity of the merchandise. For the prototype highlighted in this research, the weight is measured from one end and then the attained value is sent to the receiver end hardware with the help of the cellular module. This exploration work focuses on fashioning a prototype that can demonstrate weight realization in combination with the immersive ability to visualize the object. Weight realization, together with immersive VR, can enhance the shopping experience of users. To create a simple example, one set of the hardware is assumed to be in a warehouse in China while the other part of the device is observed to be in a technical store in Jeddah.

2. Literature Review

2.1. Smart Cities, Sustainability, and Inclusion of Virtual Reality

Smart cities are described according to the classification system model and established on six pillars: smart social and mobility system, intelligent urban management, smart economic environment, dedicated lawful structure, smart e-governance, and sustainability of lifestyle. Gómez et al. define a smart city to be a city occasioned from urban development exploitation sensors and scientifically unconventional tools and systems [22]. Azraff Bin Rozmi, et al. [23] state that sustainability deals with the intricate relationship between the survival of human life in accordance with changes in the economy, culture, government policies, and infrastructure in an urban milieu [24].

Proposals for smart cities are frequently rehabilitated, bearing in mind the cyclic process in harmony with the decision-makers views to implement pioneering solutions anticipated by connoisseurs and relevant mavens. Alessandro et al. endorsed that the sequence of a system with the connectivity of haptic devices, machine learning, VR, and artificial intelligence needs to be adopted while creating forthcoming verdicts in smart cities [25,26]. Gaffary, et al. [27] propose that the advent of VR and haptic devices can effectively assist urban planners in smart cities in decision making and introducing innovations. The interaction between the components in a smart city framework is shown in Figure 1.

According to Akhtar, et al. [28], VR systems will be an integral part of the smart city concept and might assist in replacing mass tourism with digital tourism. Paszkiewicz, et al. [29] term the usage of VR systems as the best option to teach industry 4.0 standards in engineering education. Nasralla [30] uses the term virtual smart cities and states that a sustainable VR patient rehabilitation system should be created using IoT technology and machine learning on time series to identify features that support sustainability and rapidly detect problems without interrupting rehabilitation services. Ospina-Bohórquez, et al. [31] suggest that the synergy between VR and multi-agent systems is a pathway to accomplish sustainability in a smart city framework.

2.2. Proprioceptive Haptic Devices

Rossi, et al. [32] revealed that haptic devices operate based on the instructions provided to them and work on the proprioceptive feedback topology. Proprioception, from the Latin terms “prospruis” (one’s own) and “căpio” (to attain), is the aptitude of the maneuver to obtain the activity information of the body. The authors further emphasize that the inclusion of proprioceptive feedback into the haptic interface can avoid the visual contact problem [33,34,35]. In their work, the researchers introduced a novel design termed “HapPro design” for the haptic device. The device is in the form of an arm that can be used on persons with disabilities to assist them in performing their tedious responsibilities. In designing the “HapPro”, both the linear and angular physical quantities are considered. The control strategy of the “HapPro” includes haptic feedback on the human EMG signal that helps control the soft hand placed on persons with disabilities. Experimental protocol and results present improvements in control with and without logarithmic mapping. With logarithmic mapping, efficiency is boosted from near 40% to around 80%, thereby doubling the improvement.

Sierra M, et al. [36] showed the application of the haptic interface in prosthetics. Shi, et al. [37] proposed the application of the haptic interface device and used both the purely mechanical system and the coupled hydraulics-based mechanism. The results indicate that the hydraulic and fluid flowing systems can be used in haptic feedback. Pistohl, et al. [38] show the control of a cursor on the desktop screen via EMG sensor signal with and without a proprioceptive feedback approach. The EMG signal is measured from the muscles of the left hand, and proprioceptive feedback is obtained from the right arm. The results showed that additional feedback is not required when the proprioceptive approach is used. However, in the present research, a haptic device is introduced that has the ability to assist customers in virtual stores since the weight of the object being ordered can be realized to have some influence on the perception of the product. This approach is useful to improve the satisfaction level of e-customers.

2.3. Concept of “Heaviness and Lightness”: Load Cell Development

The notion of “heaviness and lightness” was first deliberated in the 6th century by the ancient Greek philosopher Plato. According to Georgakopoulos and Quigg [39], Plato termed weight to be the natural propensity of an object, and this physical property of heaviness and lightness is associated with all objects on earth. Later, Euclid of Alexandria, renowned as the “father of geometry,” became the foremost Greek mathematician who described weight to be “the heaviness or lightness of substances relative to other substances and is measured by a balance.” Brain Bowers specified that in the 17th century, a major advancement was recorded when Galileo proposed a scheme to measure the weight of a stirring object. In due course, Galileo figured out that weight was proportionate to the extent of substance in an entity. In modern science, the weight (vector quantity; the direction toward the earth’s center) is thought to be the force resulting from gravitational action, whereby mass (scalar quantity, no direction) is equal to the matter quantity in an object. Sumit and Akhter [40] detailed that the primary maneuver to ration the weightiness of a body was christened as “Spring Balance” and was formed in 1770 by Richard Salter in Britain [41].

According to Ackerman and Seipel [42], in 1843 a British physicist named Sir Charles Wheatstone assembled a device dubbed as the “Wheatstone Bridge.” The created device was capable of computing an unknown resistance magnitude. Far along in the 19th century, strain gauges were industrialized. By means of the latest expertise at that time, the perception of fabricating a load cell with a Wheatstone bridge arose into training. Replacing one leg resistor in the Wheatstone bridge with piezoelectric substantial resulted in varying voltages that corresponded to the applied weight. Bar, et al. [43] depicted that “variation in weight resulted in voltages across the Load cell terminals” is the approach that is eliminated nowadays in almost every weight measuring device. As the object is placed on one of the bridge’s leg, the corresponding voltage fluctuates and the modification in voltage is amplified and processed in a microprocessor device across some typical scale to acquire the exact weight value of the object. This principle of load cell composition is used in the tele-weight device.

2.4. Virtual Reality Stores for E-Commerce

VR has recently become a trend in shifting technology. Over the last few years, it has been used for cognitive purposes. However, topical enhancements in half-life [44] and gaming engines have shifted the trend in fashioning VR stores to provide a better shopping experience. According to Hung, et al. [45], virtual stores are gaining in popularity and people are choosing to have a virtual shopping environment. Huang, et al. [46] suggested that considering all the impending trends and gaining customer satisfaction is imperative for the e-commerce market to meet errand requirements. Scarle, et al. [47] indicated that fraudulent activities are gaining a new level along with the trend.

One primary problem highlighted in this research is that the current virtual store experience is not immersive, and even with the utilization of haptic sensing e-commerce customers are unable to have a better overview of the object prior to purchasing. The key goal of this research is to introduce the idea of a tele-weight device that can enhance the user’s immersive experience by letting the customer realize the weight of an item ordered in the virtual store. The idea is implemented in the form of a prototype, and the relevant details and results are discussed in this work.

With the help of the tele-weight device, an approach is indicated to allow e-commerce users to realize the weight of an object to have a better immersive experience while shopping in VR stores. The tele-weight device can be used in future HMD devices to achieve a better immersive shopping experience.

This research aims to create a tele-weight device that can be mounted over the HMD display and can display the weight of the item from the virtual world to the user, thereby allowing the user to have haptic interaction with the objects from the VR series game simulation. To demonstrate the accuracy of the tele-weight device, a prototype is created and an experiment is performed to collect data. The difference between the sent and collected weight demonstrates the overall accuracy of the device.

3. Methodology and Analysis

3.1. Working Principle of the Tele-Weight Device

The original aim for the tele-weight device is to be affixed over the HMD display and show the weight of the object. From the virtual store perspective, the cart item weight can be demonstrated to the user and, in this way, the customer can realize and interact with the item’s equivalent weight in a haptic manner (Figure 2). The tele-weight device shows the weight of the item ordered by the customer in the virtual store. First, a user chooses the item, perceives it on the VR display, and then the sensor system takes command from the virtual store PC (sending end) and displays the equivalent weight (receiving end). This approach can also determine the shipping weight and cost prior to purchasing an item.

To determine the accuracy of this approach, a prototype comprised of two parts was created. At the receiving end side, a motor and gear arrangement was used to show the weight to the user. Therefore, the difference between the obtained weight and shown weight must be obtained to figure out the effectiveness.

3.2. Hardware Design for Performance Evaluation

The tele-weight haptic device prototype created for this research is divided into two portions. For the sake of simplicity, one side is labeled the sending end and the other side the receiving end. The circuitry at the sending end side (Figure 3) comprises a programmed set of Arduino family device, an HX711 amplifier chip, and a Global System for Mobile Communications (GSM) module [48]. The sending end circuit is programmed to send encrypted weight coded text messages over a cellular network, and these messages are received and decoded by the receiving (other) end. The circuit at the receiving end (Figure 4) involves a programmed (firmware installed) Arduino family processor, a custom-built motor driver, a Sim900A module, and a dc brushless motor with a gear chain mounted on the shaft. The projected archetype can demonstrate the mass equivalent of up to 5 kg. The weight of the object under 5 kg is sent in an encrypted format to the receiving side by pressing a button.

A limited weight displaying prototype is created because a weight over 5 kg can make the headset heavy, causing neck injury. Haptically feeling the weight of the item together with immersive visualization can allow the user to get a better idea about the in-hand (holding) experience of the item. This function not only enhances the customer’s shopping experience by adding sustainability in the approach but also allows them to interact physically with the equivalent load in their hands. For this project, the approach to evaluate the created prototype is shown in Figure 5. At first, a prototype is created and then experimentation is performed on the hardware in the form of repeated trials to obtain estimation regarding the confusion matrix, accuracy, coverage range, and time factor.

4. Results

A tele-weight device is proposed in this research. The objective of this device is to assist the customers in feeling and realizing the weight of an object. The customer can order any item on a VR store by visualizing the item, and during the process of visualization, the gear chain on the motor can make the customer feel the weight of the object at their hand. Tele-weight is a haptic device that can interact with the customer according to their actions on a computer. The sending end prototype (Figure 6) weighs the object, and the circuitry includes a weight sensor, a microprocessor, and a GSM message sending module. The message sending module sends the encrypted message, which is received by the receiving side (Figure 7) and decrypted. The corresponding weight is then shown accordingly.

Some degree of error was recorded in reading the measurement and in the demonstration end. A total of eleven trials were recorded, and the current prototype was observed to have some sort of error and thus requires tuning. Owing to the uniform distribution of the weight on the gear, inaccuracy in grams was recorded in the measurement. As the prototype system can show any value between 0 and 5 kg, the results of ten trials are shown in Figure 8.

As the sensors used in the prototype has a limit of 5 kg, therefore, trials were performed from 0 to 5 kg weight with a 0.5 kg decrease in each interval. In the first trial, the weight was set to be 5 kg at the sending end side and receiving end demonstrated a weight of 4.93 kg with 98.6% accuracy. The highest accuracy of 99.71% was achieved in the second trial where 4.5 kg weight was displayed with a positive error of 0.29% and a 4.513 kg weight value at the receiving end. The lowest accuracy of 96% was obtained on the tenth trial where a 0.5 kg weight was displayed on receiving end as 0.48 kg with a 4% error margin. The measured weight, displayed weight, and percentage error are shown for eleven trials and, from the Figure 8 results, it can be seen that the error never exceeded 4% for any of the individual trials. Another important thing to notice is that as the weight increases, the error value goes down comparatively.

5. Performance Evaluation of Haptic Tele-Weight

5.1. Performance Evaluation I: Confusion Matrix and Accuracy

This evaluation is based on the eleven trials considering the values from 0 to 5 kg with an interval increment of 0.5 kg. A confusion matrix is represented for Actual (A) and predicted (P). When all the trials are combined, the overall measured weight is 27.5 kg, and the overall weight demonstrated by the motor gear arrangement is 27.26 kg. The confusion matrix is as follows:

$$Confusion matrix=\begin{array}{cc}n=11& \begin{array}{cc}P:YES& P:NO\end{array}\\ \begin{array}{c}A:YES\\ A:NO\end{array}& \left[\begin{array}{cc}27.269& 0.231\\ 0.045& 0.186\end{array}\right]\end{array}$$

The overall observed accuracy for the experiment performed in eleven trials is

$$Accuracy=\frac{Obtained demonstrated values by gear chain from 11 trials}{Total measured weight from 11 trials}$$

$$Accuracy=\frac{27.26}{27.50}\times 100\%=99.89\%$$

The eleven trial values are ignored as they are at no load, whereas the relative index wise accuracy from the graph for the ten trials is

$$Accuracy \left[10\right]=\left\{ 98.7\%, 99.71\%, 98.5\%, 98.28\%, 99.16\%, 99.52\%, 98.95\%,99.33\%,98\%,96\%\right\}$$

The overall average accuracy from the relative method (index-based) of accuracy computation is 98.60%. Therefore, the accuracy is between 98.60% and 99.89% for loads under 5 kg.

5.2. Performance Evaluation II: Coverage Range

A generic architype was created to elucidate the enactment of the scheme. For the prototype, the warehouse was considered to be in China and the e-commerce VR store user was in Jeddah. The VR store user was interested in knowing the weight of an object before purchasing the item. Another possible reason for obtaining the weight is to estimate the shipment-related issues. However, the only target of the current research is improving the immersive experience. The item was in China, and the user can feel the object weight even with a distance of miles because the system uses cellular technology (GSM module), therefore making it independent of range.

5.3. Performance Evaluation III: Time Factor

The time factor is the time required to perform one maneuver (one complete process) and is calculated using Equation (1). Measurement time is the time required by the load cell circuit to measure the object weight and is denoted by T_m in this work (order of microseconds, µs). T_cellular is the time required by the cellular grid to deliver an encrypted message to the receiving end. The mean value of T_cellular is 1 s. Receiver display time (T_rs) is the time the receiver end circuit takes to decrypt the received message and show the weight in the motor gear arrangement.

$$T.F=T_{m }+T_{cellular}+T_{rs }$$

(1)

$$T_{rs}=0.1\times weight \left(in grams\right)$$

$$T_m=\frac{1}{No. of operations\times Clock Frequency \left(CPU\right)}$$

For the object having 1 kg of weight, the time factor (T.F) is

$$T.F=T_{m }+T_{cellular}+T_{r }$$

$$T.F=\frac{1}{8 \mathrm{MHz}\times 50}+1+0.1\times 1000=101 s=1.68 \min$$

The time factor varies with the weight of the object With a larger object, the time factor is higher. For the object with 1 kg of weight, the weight takes 1.68–2 min.

6. Conclusions

The concept of smart cities is evolving, and different governments are trying to transform their cities into smart policy cities using available advancements. One technology being adapted worldwide is VR, and with its recent progression virtual e-commerce stores are becoming more prevalent. Virtual stores allow the user to interact with the products, but there is no way to feel the weight of the object before it is purchased. In this research, a device termed as a haptic tele-weight is proposed that will allow virtual store customers to feel the weight of an object prior to their purchase. In this way, the customer satisfaction level is increased and the delivery of an unintended item can be prevented. The objective of this research is to propose, design, and implement a prototype of the tele-weight device that can be used to display the weight of the object at the remote end. The tele-weight device has two parts, a sending end and a receiving end. At the sending end, the circuit computes, encrypts, and transmits the object weight and then the receiving end decrypts and demonstrates the weight of the object. Another intention of the approach is to enhance the e-commerce customer’s level of user experience by allowing them to have immersive contact with the item in the virtual store.

A haptic tele-weight device is anticipated that utilizes the mobile cellular module to transmit encoded messages among its subparts. The accuracy and performance of the tele-weight device are marginally good, and an onboard microprocessor and circuit elements are used for communication. The crucial objective of this research is to improve the immersive experience of e-commerce shoppers. The computed time factor is not fixed in magnitude and depends largely on the weight of the object. The average accuracy of the created prototype is 98.60–99.89%.

The team of researchers working on this system is hopeful for the industrial version of the prototype. Many weight scales are available these days, but none complies with the VR store. In this research, a prototype of the tele-weight device is suggested to have a better user experience of immersive shopping. Many stockholders are interested in selling different products, but products in the shop often become inaccessible. This approach is handy and useful for the customer to have haptic immersive interaction with the item, allowing the weight to be placed on the hand of a person to have an improved pre-purchase experience. In this way, the trust of customers can be obtained as well. Moreover, every prospective customer can sense the weight and approximate the figure of the object (to have an estimation of the feeling of the item in the hand). A future recommendation is to create an industrial version of the prototype that works with the HMD.