Cooperative Obstacle-Aware Surveillance for Virtual Emotion Intelligence with Low Energy Configuration

Abstract

In this article, we introduce a cooperative obstacle-aware surveillance system for virtual emotion intelligence which is supported by low energy configuration with the minimal wasted communication cost in self-sustainable network with 6G components. We make a formal definition of the main research problem whose goal is to minimize the wasted communication range of system members on condition that the required detection accuracy with the given number of obstacles is satisfied when the requested number of obstacle-aware surveillance low energy barriers are built in self-sustainable network. To solve the problem, we have originally designed and implemented two different approaches, and then thoroughly evaluated them through extensive simulations. Then, their performances based on numerical outcomes are demonstrated with detailed discussions.

1. Introduction

For advanced smart cities, it is anticipated that 6G wireless systems beyond 5G communication and their relevant technologies will contribute to evolutional industrial and academic areas such as extended reality (XR), augmented reality (AR), virtual reality (VR) connected autonomous vehicles, humanoid robotics, intelligent internet of things (IIoT), mobile robots, Unmanned Aerial Vehicles (UAVs), haptics, remote surgery, medical applications, etc. [1,2,3,4,5,6]. By the unprecedented diffusion of intelligent devices and IIoT services, it is possible to accelerate artificial intelligence operations with minimal human interventions and to use 6G transceivers. Also, it is undoubtable that 6G will contribute to connected robotics, autonomous systems, multisensory systems, wireless brain-computer interactions, edge AI, distributed technology of blockchain, massive information management, device-to-device communication, localization, energy-efficient computing, reliable services, energy harvesting, etc. [7,8,9,10,11,12,13].

In order to accommodate numerous tasks in 6G wireless systems and environments, it is also expected that a cooperation of mobile robots and smart UAVs should play critical roles to complete various missions on the ground and in the air including borderline surveillance, epidemic prevention, traffic monitoring, harsh environment monitoring, rescue operation, ground delivery services, patrol services, emergent medical item services, etc. [14,15,16,17,18,19,20,21,22,23,24]. Also, the concept of virtual emotion was introduced by Kim et al. [25] with a consideration of security and surveillance issues. Given IoT device, mobile robots, UAVs equipped with wireless transmitter, receivers, the information with neutral, pleasure, sorrow, rage is derived through wireless signal and its reflection [26]. Then, the derived information is defined as virtual emotion. By the useful property of virtual emotion, the applicable areas and services cover emotion-based services, virtual emotion barrier, maritime transportation systems, epidemic prevention in public and private areas, etc. [27,28]. In particular, when the virtual emotion barriers using mobile robots and smart UAVs are built in maritime transportation area, the penetration of specific person with rage and anger emotion type can be detected by those barriers in the region of interests.

On the other hand, recent research topics of self-organizable network (SON) and self-sustainable network (SSN) have much interests from researchers in both academic and industrial areas [29]. Basically, the energy harvesting and energy storage play a key role in sustainable operations and tasks in SSN. If we deliberate on the applicability of virtual emotion detection into SSN, the obstacle-aware surveillance with low energy configuration by the cooperation of mobile robots and smart UAVs is highly necessary for intelligent virtual emotion surveillance. Therefore, the issue of how to achieve cooperative obstacle-aware surveillance with low energy consumption should be investigated thoroughly.

Based on the above motivations, the main contributions of this study can be summarized as follows.

• We introduce a cooperative obstacle-aware surveillance framework with low energy configuration using system components with mobile robots and smart UAVs in 6G self-sustainable network. The devised innovative system essentially creates obstacle-aware low energy surveillance barriers to provide the enhanced detection accuracy in self-sustainable network.
• Also, we make a formal representation of the main research problem whose objective is to minimize the wasted communication ranges or the squandering space of system components so that the secure surveillance with low energy configuration is accomplished.
• To resolve the defined problem, two different schemes are developed originally and are simulated through extensive experiments with various settings and scenarios. Then, their performances based on obtained results are evaluated with detailed discussions and demonstrations.

The remaining parts of this study have the following organization. The next section reviews the existing studies in regard to this paper. In Section 3, devised obstacle-aware surveillance framework using virtual emotion with low energy consumption is described as well as the main research problem is formally defined. In Section 4, we specify two different algorithms to resolve the problem. Also, in Section 5, the performance analysis based on numerical results is presented. Finally, this study is concluded in Section 6.

2. Related Works

The Human emotion classification and recognition were studied by many researchers. In [30], authors provided detailed analysis of the computational model of human emotion as well as compared the actions of computational type followed by medical environment. And, in [31], they had contributions for the effect of emotional expressions and quantitative features by tracking online participants or system members. Also, several approaches for emotion recognition have been introduced by a number of researchers. In [32], authors introduced a novel real time system for emotion derivation using mobile facial expression by smart phone’s camera. In [33], they devised the emotion recognition scheme with a consideration of multi-modal deep learning strategy. In [34], for emotion recognition and relevant applicable model, authors proceeded the novel affective computing study by evaluating ECG signal and its psychological features. Moreover, in [35], they focused on the study of channel state information according to activity or motion recognition using movement speed, state information estimation.

On one hand, the concept of barrier or barrier-coverage was initially introduced by Kumar et al. [36] to support secure surveillance into the square-shaped region so that the constructed barriers consisting of wireless sensors are able to detect any penetrations into the given area. And, they proposed novel sleep-wake up scheduling algorithms in order to maximize the system life time of the built barriers by alternatively applying those devised sleep-wake up scheduling schemes. In [37], authors proposed the basic idea of weak barrier coverage and presented low bound computation and estimation. And, in [38], authors developed a concept of partial barrier in event-driven environment and then, studied how to recover the partial barrier from failures due to energy depletion of wireless sensors. Furthermore, in [39], authors designed a barrier system using mobile sensors and provided an efficient handling method by estimated position errors and missed information.

On the other hand, the recent novel concept of virtual emotion was introduced by Kim et al. [25] and they provided an initial perspective how to apply barriers to detect virtual emotion information when people pass through the given area. And, in [28], authors designed the virtual emotion detection framework which supports two-way assisted virtual emotion detection such that the delay bound is satisfied when the virtual emotion is detected by two-way enabled virtual emotion barriers both horizontally and vertically. In addition, in [40], authors made a bridgehead for secure surveillance based on virtual emotion in maritime transportation stations with cooperative mobile robots and UAVs. With multiple number of divided sub-regions with different security priorities, they proposed several methods of how to construct differential security barriers for virtual emotion surveillance toward surveillance in maritime area.

3. An Architecture of Cooperative Obstacle-Aware Surveillance with Low Energy Configuration

Now, we design a cooperative obstacle-aware surveillance framework with low energy configuration by mobile robots and smart UAVs. Then, the core concepts and terms of the proposed framework are described in detail as well as the main research problem is represented.

3.1. System Settings, Assumptions and Notations

In this subsection, we define the essential system settings and assumptions which are considered in the proposed architecture as follows.

• For system members or system nodes, the mobile robots and smart UAVs participating in the proposed architecture with self-sustainable movements both on the ground and in the air. Every mobile robot and smart UAV is equipped with front detection sensor, rear detection sensor, camera, wireless equipment for transmission and reception.
• The detected virtual emotion is recognized in the proposed system as five types including happiness, neutral, sorrow, anger, rage [41]. For the purpose of security, the emotion type of anger and rage are considered in the proposed system.
• The virtual emotion is detected by system members which are equipped with wireless signal, reflection and recognition procedures [26]. And, the detection accuracy depends on the distance or the overlapped area of communication or detection ranges between two system members.
• The detected virtual emotion data can be reported or be transmitted to other system members for system updates and maintenance in self-sustainable network.
• The whole monitoring area is considered as square-shaped region and the obstacle also has quadrilateral or lozenge-shaped where multiple number of obstacles are included in the requested surveillance region.

Also, the basic notations with their brief descriptions are summarized in

Table 1. Notations.

Notations	Description
S	a square-shaped surveillance area
M	a set of system members
O	a set of obstacles
C	a set of system member communication ranges
T	a set of obstacle-aware low energy surveillance barriers
n	the number of system members in the architecture
p	the required minimum pair detection accuracy
q	the number of obstacles
r	the requested number of surveillance barriers
h	an identifier of surveillance barrier, where $h\le n,t_h\in T$
i	an identifier of member, where $i\le n,m_i\in M$
j	an identifier of member, where $j\le n,m_j\in M$
k	an identifier of obstacle, where $k\le n,o_k\in O$
$\epsilon$	the wasted communication ranges

.

3.2. Obstacle-Aware Low Energy Surveillance Barriers

Now, the essential terms and definitions are represented in this subsection.

Definition 1

(self-sustainable surveillance area with square-shaped obstacles). Suppose that a group of security mobile robots and a fleet of smart UAVs are deployed with initial random positions as system members. The self-sustainable surveillance area, referred as SelfSusSurveil, is square-shaped region where system members have capabilities to achieve surveillance tasks and to recover from impairments or failures through movements and 6G communications of system members. It follows that the detection hole due to those failures can be recovered by self-sustainable configuration. Also, the multiple number of obstacles that have square-shaped or lozenge-shaped space are included in SelfSusSurveil.

Definition 2

(obstacle-aware low energy surveillance barriers). Given that self-sustainable surveillance area with square-shaped obstacles, a group of mobile robots and a fleet of smart UAVs, the obstacle-aware low energy surveillance barriers, called as OaSLeBar, are barriers that are capable of carrying out unusual configuration due to obstacles. It follows that the OaSLeBar are built with low energy configuration by reducing the wasted communication ranges in cooperation with mobile robots and smart UAVs.

Figure 1 briefly shows an applicable example of the proposed architecture with 6G self-sustainable environment. As seen in Figure 1, there are four different 6G self-sustainable surveillance areas such as public buildings, private land. At each area with multiple number of square-shaped obstacles, the surveillance barrier is constructed by self-sustainable movements of mobile robots, smart UAVs, smart devices to detect any penetration from top to the bottom or from bottom to the top in the given area.

3.3. Problem Definition

In this subsection, we create a formal definition of the main problem that is to be solved as follows.

Definition 3

(MinSurveilEnergy problem). It is given that SelfSusSurveil area, a set of mobile robots and smart UAVs with heterogeneous communication ranges. The MinSurveilEnergy problem is to minimize the wasted communication ranges caused by obstacles such that the obstacle-aware low energy surveillance barriers are constructed through low resource configuration and cooperative formation of system members.

Formally, the MinSurveilEnergy problem is to:

$$\mathbf{Minimize}\epsilon$$

(1)

Figure 2 presents a brief explanation of the defined MinSurveilEnergy problem. As seen in Figure 2, it is given that a group of mobile robots and a fleet of smart UAVs with heterogeneous communication ranges are located within 6G surveillance region. Also, there are two square-shaped obstacles. Then, if we try to build three independent obstacle-aware low energy surveillance barriers within the given area, the middle-located obstacle-aware low energy surveillance barrier is normally built. However, the top and the bottom-side obstacle-aware low energy surveillance barriers have the wasted communication ranges and resources due to obstacles. The MinSurveilEnergy problem is to minimize these wasted communication ranges or the squandering space of obstacle-aware low energy surveillance barriers which are caused by obstacles.

4. Proposed Schemes

In this section, two devised different approaches are specified so that the outcome value is obtained to resolve the MinSurveilEnergy problem. Then, each scheme is presented with its input, output, procedures, descriptions.

4.1. Algorithm 1: Rapid-Construction

The first approach, referred as Rapid-Construction, is proposed to solve the MinSurveilEnergy problem. Then, its execution steps are specified as follows.

• Verify n number of system members with their communication ranges C.
• Identify a set of obstacles O and their positions within S.
• Create a set of surveillance barriers T and confirm the required r number of surveillance barriers, referred as OaSLeBar.
• The below iterations are implemented until the required r number of obstacle-aware low energy surveillance barriers are found.

  -

  From left side border to right side border of S, search for obstacle-aware low energy surveillance barrier through Edmonds-Karp max-flow algorithm [42].

  -

  If an obstacle-aware low energy surveillance barrier $t_h$ is found such that $t_h$ satisfies the pair detection accuracy p, then add it to the set of obstacle-aware low energy surveillance barriers T.

• Estimate the wasted communication ranges or areas between the given obstacles and the found obstacle-aware low energy surveillance barriers.
• Update the estimated areas as $\epsilon$ and return it.

Also, the pseudocode with implementation procedures is explained in Algorithm 1.

Algorithm 1 Rapid-Construction

1: Inputs: $S,M,O,C,n,p,r$   2: identify a set of system members M with their ranges C;   3: recognize a set of obstacles O in S;   4: set T = ∅;   5: while r number of OaSLeBar are not built in S do   6:     search for OaSLeBar $t_h$ satisfying p;   7:     if the valid $t_h$ is found then   8:         set $T\leftarrow T\cup t_h$;   9: calculate the wasted communication range between O and T; 10: set the estimated range as $\epsilon$; 11: return $\epsilon$; 12: Output: $\epsilon$

4.2. Algorithm 2: Pulling-Lift-Relocation

In order to solve the MinSurveilEnergy problem, the second approach of Pulling-Lift-Relocation is devised whose basic idea is to pull up or pull down of adjacent members with obstacles so that the wasted communication ranges by those relocations of system members in obstacle-aware low energy surveillance barrier are reduced. So, we specify the execution steps of Pulling-Lift-Relocation scheme as follows.

• Validate n number of system members with their detection ranges C.
• Within self-sustainable surveillance area S, generate a set of OaSLeBar T with the requested r number to be formed.
• Then, the following sub-steps are iterated while the given r number of OaSLeBar are built in S.

  -

  From left border line to right border line of S, seek a new OaSLeBar by [42].

  -

  If a new OaSLeBar $t_h$ is found on condition that $t_h$ meets the pair detection accuracy p, add $t_h$ to the set of OaSLeBar T.

• Recognize the system members which has the maximum overlapped communication range with obstacles and make every pair of system member and matched obstacle.
• Also, the below sub-steps are iterated for all pairs.

  -

  For each pair between system member $m_i$ and obstacle $o_k$, draw a virtual line between the center of system member and the center of obstacle.

  -

  Lift up or pull down the position of system member $m_i$ through virtual line to reduce the wasted communication range such that the connection and pair detection accuracy is maintained continuously.

• Calculate the current wasted communication ranges or areas for every pair between obstacles and system members in OaSLeBar.
• Update the estimated areas as $\epsilon$ and return it.

Furthermore, the pseudocode of the second scheme with formal implementation processes is specified in Algorithm 2.

Algorithm 2 Pulling-Lift-Relocation

1: Inputs: $S,M,O,C,n,p,r$   2: check a set of system members M with their detection ranges C;   3: identify a set of obstacles O in S;   4: set T = ∅;   5: while r number of OaSLeBar are not formed in S do   6:     seek OaSLeBar $t_h$ satisfying p;   7:     if the new $t_h$ is found then   8:         set $T\leftarrow T\cup t_h$;   9: recognize system members $m_i$ and obstacles $o_k$ with maximum overlapped range; 10: while r number of OaSLeBar are not constructed in S do 11:     draw a virtual line between the center of $m_i$ and the center of $o_k$; 12:     lift up or pull down $m_i$ through virtual line; 13: estimate the wasted communication ranges between O and T; 14: set the estimated range as $\epsilon$; 15: return $\epsilon$; 16: Output: $\epsilon$

5. Performance Analysis

In this section, the proposed schemes are evaluated based on numerical results through extensive simulations. For simulation settings, the number of system members consisting of smart UAVs and mobile robots is ranging from 300 to 450 as well as the square-shaped surveillance area size is considered at 2000 by 2000, 2200 by 2200, 2400 by 2400, 2600 by 2600, respectively. Also, the communication or detection ranges of system members is set up between 70 and 120. It is noted that all obtained results of $\epsilon$ are an average numerical value of 1000 different graphs and settings.

As the first group of experiments, Rapid-Construction scheme and Pulling-Lift-Relocation scheme are performed within the surveillance area size of $2000\times 2000$ as shown in Figure 3. In the performance graph, not only X-coordinate represents the minimum detection or communication range of system members but also X-coordinate stands for the wasted space or communication range of $\epsilon$. Figure 3a,b show the outcome of the proposed two algorithms if the graph deliberates on the number of system members n = 300 and n = 350 with the number of obstacles q = 4 in $2000\times 2000$ surveillance region. Also, Figure 3c,d present the performance comparison when the graph covers n = 400 and n = 450 with q = 4 in $2000\times 2000$ surveillance district. Then, as it can be demonstrated in Figure 3, the wasted space or communication range of $\epsilon$ increases as the minimum radius of system member increases in both Rapid-Construction algorithm and Pulling-Lift-Relocation algorithm. As a whole, it is verified that the performance gap or difference between Rapid-Construction algorithm and Pulling-Lift-Relocation increases when the minimum communication range increases the wasted space of $\epsilon$. Based on the outcome in applicable scenarios in Figure 3, we could see that Pulling-Lift-Relocation algorithm has better performance than Rapid-Construction algorithm consequently.

Secondly, to resolve the MinSurveilEnergy problem, Rapid-Construction scheme and Pulling-Lift-Relocation scheme are implemented according to various surveillance area sizes including 2000 by 2000, 2200 by 2000, 2400 by 2400, 2600 by 2600 as seen in Figure 4. Similar to the first set of simulations and cases, the X-Coordinate depicts the minimum communication range and the Y-coordinate means the wasted space or communication range of $\epsilon$. Figure 4a,b display the performance comparison of the devised Rapid-Construction approach and Pulling-Lift-Relocation approach if the spuare-shaped surveillance regions is considered as 2000 by 2000, 2200 by 2200. And, Figure 4c,d express the performance comparison based on numerical results when 2400 by 2400 and 2600 by 2600 are used as surveillance area size, respectively. As it can be identified in Figure 4, the wasted space or communication range of $\epsilon$ is increasing as the minimum radius of system member is increasing for both Rapid-Construction method and Pulling-Lift-Relocation method. Also, for all applied cases and scenarios, it is demonstrated that the performance difference between Rapid-Construction algorithm and Pulling-Lift-Relocation is increaseing when the minimum communication range is increasing for the wasted space of $\epsilon$. Hence, based on results in Figure 4, we could check that Pulling-Lift-Relocation scheme outperforms Rapid-Construction for all applied surveillance region sizes.

For the third group of experiment, Rapid-Construction scheme and Pulling-Lift-Relocation scheme are executed where the surveillance area has multiple number of obstacles q = 4, 5, 6, 7 within surveillance area of 2000 by 2000 as it can be seen in Figure 5. Similarly, the X-coordinate presents the minimum communication radius and the Y-coordinate shows the earned results by the proposed Rapid-Construction algorithm and Pulling-Lift-Relocation algorithm. Figure 5a,b signify the numerical outcomes when the given surveillance area covers the number of obstacles q as 4 and 5. And, Figure 5c,d show the numerical outcomes of $\epsilon$ when the requested OaSLeBar is built within surveillance area of 2000 by 2000 with the number of obstacles q as 6 and 7. If so, as it can be demonstrated in Figure 5, the wasted space or communication range of $\epsilon$ increases as the minimum radius of system member increases in both Rapid-Construction algorithm and Pulling-Lift-Relocation algorithm. On the other hand, it is checked that the performance gap for the wasted space of $\epsilon$ between Rapid-Construction algorithm and Pulling-Lift-Relocation increases as the minimum communication range increases. Finally, in third set of simulations, we could demonstrate that Pulling-Lift-Relocation scheme has better performance than Rapid-Construction in every scenarios and cases in the whole experiments.

6. Conclusions

In this article, we introduced the cooperative obstacle-aware surveillance framework for virtual emotion intelligence in 6G self-sustainable circumstance with obstacles. Then, we formally defined the main problem whose goal is to minimize the wasted communication ranges or resources such that the requested detection accuracy is satisfied and the required number of obstacle-aware surveillance low energy barriers are formed completely. To resolve the problem, two different algorithms are developed as well as their performances based on numerical results through expansive simulations are analyzed with demonstrations and discussions. As future works, we plan to study multi-purpose surveillance low energy barriers which fit with three-dimensional environment.