OkeyDoggy3D: A Mobile Application for Recognizing Stress-Related Behaviors in Companion Dogs Based on Three-Dimensional Pose Estimation through Deep Learning

Abstract

Dogs often express their stress through physical motions that can be recognized by their owners. We propose a mobile application that analyzes companion dog’s behavior and their three-dimensional poses via deep learning. As existing research on pose estimation has focused on humans, obtaining a large dataset comprising images showing animal joint locations is a challenge. Nevertheless, we generated such a dataset and used it to train an AI model. Furthermore, we analyzed circling behavior, which is associated with stress in companion dogs. To this end, we used the VideoPose3D model to estimate the 3D poses of companion dogs from the 2D pose estimation technique derived by the DeepLabCut model and developed a mobile app that provides analytical information on the stress-related behaviors, as well as the walking and isolation times, of companion dogs. Finally, we interviewed five certified experts to evaluate the validity and applicability of the app.

1. Introduction

Research on human pose estimation has progressed considerably through the application of deep learning. Although Kearney et al. [1] reported that researchers have traditionally focused on analyzing the pose estimations of human beings, similar research on animals has received increasing attention in recent years. DeepLabCut [2] and DeepPoseKit [3] have developed artificial intelligence (AI) algorithms for the identification of animal joints without the need for body sensors. These algorithms use software packages to save the location of animal joints, train artificial neural networks based on the saved data, and enable artificial neural networks to track the movements of animals.

Generally, researchers utilize a pose-estimation method that installs sensors on the animals’ bodies. Brugarolas et al. [4], Ahn et al. [5], and Kasnesis et al. [6] used inertial measurement units (IMUs) such as gyro and acceleration sensors to identify the conditions and behaviors of dogs. In these studies, the dogs sported wearable devices equipped with the aforementioned sensors, which enabled the researchers to recognize conditions and behaviors that could not be verbally described. However, certain animals might experience pain or discomfort when equipped with these sensors. Furthermore, this approach involves considerable effort. These obstacles hinder the creation of massive datasets of animal joint locations comparable to those available for human joint locations.

Haq [7] stated that behavioral disorders in animals such as companion dogs are quite similar to those experienced by humans, and the ratio of dogs suffering from depression is similar to that of humans. Depression is regarded as a type of a mental disorder, which can be classified as eating disorders and behavioral disorders in dogs. Notari [8] studied the effects of stress on the behavior and well-being of companion dogs. It was found that physical and psychological stress factors can lead to dysregulation in companion dogs, which might increase the risk of their physical and behavioral pathological symptoms. From these results, Notari [8] concluded that stress management for companion dogs is an essential area in veterinary medicine.

Cafazzo et al. [9] found that dog walking had a crucial impact on stress levels. The Royal Society for the Prevention of Cruelty to Animals (RSPCA) found that the longer the dogs were left alone, the more behaviors related to separation anxiety disorder (SAD) were observed [10]. Accordingly, the present study defined circling as a problematic behavior in dogs and developed a mobile application that identifies the stress levels in dogs by recording their walking times and time spent alone and by analyzing the problematic stress-related behaviors based on circling. To this end, an image processing technique was used that applies a deep learning algorithm to enable the mobile application to recognize the dogs’ behaviors without needing to place sensors on their bodies. The developed mobile app alerts dog owners to canines’ stress-related behaviors as well as their causes without sensors being placed on the dogs’ bodies.

This paper’s specific technical contribution is two-fold. Firstly, it is difficult to obtain a relevant dataset owing to the limited research on animal-pose estimation; therefore, this study contributed to preparing a dog joint position-based dataset by estimating three-dimensional poses from the multi-view RGB images. Secondly, previously, dogs were required to sport a wearable device with sensors for behavior analysis. However, our proposed approach predicts 3D poses from images showing 2D poses by using the VideoPose3D model, thus enabling the analysis of their behaviors without attaching any sensors.

The rest of this paper is organized as follows:

• “Related Work” introduces contemporary deep learning-based dog behavior recognition systems and dog disease diagnosis systems and establishes the theoretical understanding of behavioral disorders in dogs;
• “Architecture of OkeyDoggy3D” presents the roles and main results of the step-by-step algorithm used to develop this mobile application and provides a structural understanding of the mobile application;
• “Evaluation” discusses the quantitative and qualitative assessments of the developed mobile application;
• “Conclusions and Future Work” summarizes this paper, describes the main findings, and suggests future directions based on the limitations of this study.

2. Related Work

2.1. Canine Behavior Recognition Systems Based on Deep Learning

Petcube [11] is an interactive Wi-Fi-based companion dog care camera developed in the United States, which enables users to assess their dogs’ condition, and talk to or play with their dogs remotely from their smartphone. This camera also has a function that allows users to chat with veterinarians to help them handle general health problems in dogs. Many Petcube users have reported being able to instantly solve general health problems in their dogs by receiving professional support through the chat feature.

Furbo [12] is a pet-care camera developed in the United States that enables users to remotely give treats to their dogs. This product utilizes not only basic technology for detecting the movements of companion dogs but also AI technology to analyze the dogs’ behavior more comprehensively. For example, when a companion dog faces the camera as if taking a selfie, the AI technology sends a push notification to the owner’s smartphone. Furthermore, Furbo includes a Smart Dog Alert function, which sends a push notification to the owner’s smartphone when a stranger enters their home.

Using the Petcube camera, owners can capture and send images of their dogs to veterinarians. The Furbo camera monitors dogs in real time and notifies owners when their dogs perform active behaviors. However, Petcube and Furbo do not independently determine whether the detected behavior is problematic; owners and veterinarians need to analyze the behaviors shown within the captured images of the dogs to determine the presence of problematic behaviors. The approach developed in this study is capable of the automatic detection of problematic behaviors in images using deep learning, which is the main differentiating factor of this study.

2.2. Canine Disease Diagnosis Systems Based on Deep Learning

AlphaDo [13], a Korean pet-care solution development company, developed an AlphaDo AI engine based on deep learning applied to a large dataset of diseases in companion dogs. This engine analyzes the appearance and symptoms of companion dogs based on imaging. When sufficient data is accumulated, AlphaDo provides users with customized care services to help them prevent diseases in their dogs. AlphaDo offers three products: AlphaDo PET+, AI PetBody, and a pet urinalysis test kit. AI PetBody allows users to capture photos of the eyes, ears, teeth, and skin of their dogs, and use these images to analyze their dogs’ symptoms. The pet urinalysis test kit can be used by owners to determine whether their dogs are suffering from one or more of ten types of diseases.

TTcare [14], a mobile app developed in South Korea, automatically extracts target diagnosis portions from the photographs of companion dogs and analyzes six types of ophthalmologic diseases (corneal injury, ectropion, entropion, epiphora, corneal opacity, and hyperemia) using an AI algorithm based on Convolutional Neural Networks (CNNs). To facilitate these functions, the AI model applied in TTcare was trained on 7000 images of the eyes of companion dogs, including those of common dog breeds in South Korea, such as the Maltese, Poodle, and Shih Tzu.

The pet urinalysis test kit developed by AlphaDo enables users to detect the possibility of disease occurrence in their dogs and analyze changes in the dogs’ habits based on customized records. The TTcare app informs users whether their dogs suffer from any one or more of six types of ophthalmologic diseases by analyzing eye images, though the dataset is limited to common dog breeds.

As such, physical healthcare apps account for a high proportion of existing healthcare apps for companion dogs. By contrast, few mental healthcare apps for companion dogs have been developed. The novelty of the approach described in this study lies in the fact that it is a mental healthcare solution that detects problematic behaviors through deep learning and diagnoses and manages stress in dogs.

2.3. Behavioral Disorders in Dogs

Chung et al. [15] investigated the prevalence of ten types of problematic behaviors in companion dogs, which were classified generally and widely, by surveying 174 dog owners in South Korea. In the survey, 84.5% of the respondents stated that their dogs showed one or more problematic behaviors, the most common among which were excessive barking, inappropriate elimination, aggressive behaviors, fearfulness, and separation anxiety. Chung et al. [15] reported that male dogs showed excessive barking and destructiveness more frequently than female dogs, whereas neutered or spayed dogs exhibited excessive barking, aggression, and excessive activity. Puppies showed destructiveness more frequently, and fearfulness and separation anxiety less frequently, than adult dogs.

The National Institutes of Health (NIH) found that companion dogs suffering from seizure disorder show similar behaviors to those of people suffering from obsessive-compulsive disorder (OCD). From this finding, the NIH defined behaviors associated with seizure disorder as OCD behaviors. Luescher [16] classified the behaviors of dogs suffering from OCD, arguing that these behaviors are performed to reduce discomfort or prevent fear. Luescher [16] also stated that these behaviors were expressed regardless of the context, were repeated and exaggerated, and thus were evaluated to be abnormal.

Sherman [17] indicated that dogs suffering from SAD display behaviors to express pain that they suffer as a result of being separated or isolated from people to whom they feel attachment. Dogs that show symptoms of SAD should be treated immediately, as SAD weakens the ties between people and animals. When alone, dogs show completely contrasting behaviors to those that they exhibit with their owners. When owners do not have a deep understanding of problematic behaviors in dogs, they might assume SAD-associated behavior to be simply malicious. In this regard, Sherman [17] classified the behaviors of dogs suffering from SAD.

Bodnariu [18] analyzed behavioral and psychological indices relating to dogs facing stressful situations. The analytic results indicated that 50 in 58 dogs exhibit problematic behaviors in stressful situations, and these behaviors tend to be repetitive. It was also found that female dogs showed more problematic behaviors than male dogs. Bodnariu [18] classified the behaviors of dogs under stressful situations as normal or abnormal behaviors. Normal behaviors were further classified into behaviors caused by acute stress and those caused by chronic stress. Abnormal behaviors were categorized as displacement activities, stereotypical activities, and hallucinatory behaviors.

3. Architecture of OkeyDoggy3D

Figure 1 shows an overview of the OkeyDoggy3D pipeline. First, the 2D pose is estimated from the multi-view RGB videos using the DeepLabCut model. Subsequently, the dog joint position-based dataset is prepared based on a camera calibration and triangulation using the Anipose algorithm by predicting the 3D pose from the 2D pose of the dog in a multi-view RGB video extracted using the DeepLabCut model.

The 3D poses in the single-view RGB video are estimated by training the VideoPose3D model based on the prepared dataset and the intrinsic and extrinsic parameters of each multi-view camera. Then, the video is divided into frames based on the dog’s behavior, and the DD-Net model is trained by labeling the dogs’ behaviors in continuous 3D poses. Thus, the dogs’ behaviors can be recognized based on their 3D poses.

3.1. Design Concept

This study defined circling as a problematic behavior in dogs. It has been established that dogs generally circle when they are happy or before they lie down or defecate. Bodnariu [18] explained that circling, classified as a stereotypical activity in dogs, is also caused by stress. However, Kimberly [19] posited that owners tend to encourage circling in their dogs, as they find it interesting. We regarded repetitive circling as a problematic behavior caused by stress and measured the number of circles performed, which served as an index for the level of stress.

Cafazzo et al. [9] studied the relationship between environmental factors and the stress levels of 97 dogs living in animal shelters. Despite the difficulty in identifying a significant relationship, Cafazzo et al. [9] eventually found that dog walks serve as a critical environmental factor that affects stress levels. The study results indicate that dogs who took regular walks showed lower levels of anxiety and a lower frequency of problematic behaviors.

The People’s Dispensary for Sick Animals (PDSA) reported that 19% of companion dogs were left alone for five or more hours on a typical day [20]. RSPCA found that more than 20% of companion dogs were left alone for a longer time than recommended and exhibited behaviors related to SAD [10]. This study determined a trend consisting of a change in the number of problematic behaviors in dogs compared to their history of such behaviors, time spent on walks, and time spent alone, which serves as stress-related environmental information that should be provided for analysis. The analytic processes were as follows.

When information regarding the walk and isolation times of the companion dogs was input into our proposed mobile application, it calculated a stress index and identified stress-related environmental information based on problematic behaviors provided via cameras.

3.2. Creation of the Dataset

This study proposed a mobile app that estimates the three-dimensional poses of companion dogs based on single-view RGB images and uses them to identify the dogs’ behaviors to support their stress care. To this end, the mobile app requires datasets of single-view RGB images and 3D poses. However, the motion capture of an object needs to be performed to measure the 3D poses. Motion capture is the process of recording, in a digital form, the motion of an object through sensors attached to the object’s body. Sung et al. [21] reported that certain motions of animals cannot be tracked via motion capture owing to the difficulty in controlling these animals. Furthermore, the motion capture of animals tends to be expensive because of the need for trainers and well-trained animals. Humans do not show obvious physical differences by race or gender, whereas dogs exhibit significant differences in hair colors, frame, and appearance depending on their breed. For this reason, the acquisition of 3D pose data for dogs of all breeds is extremely costly.

All the videos for the datasets for training the artificial intelligence models were captured firsthand. Four mobile devices were used for the multi-view video shooting: an Apple iPhone 6, an Apple iPhone 8, a Samsung Galaxy Note 8, and a Samsung Galaxy A7. The canine behaviors were recorded as 30-min videos, and the number of frames that could be used as a dataset was 7189 for each view. The dog in the video performed five movements: walk, sit, jump, stand, and track.

3.2.1. 2D Dog-Pose Estimation

This study used DeepLabCut [2], an AI algorithm that provides efficient solutions for estimating the 2D poses of animals without markers, to estimate the poses of companion dogs based on multi-view RGB images. The proposed algorithm facilitates transfer learning based on deep neural networks and can track the location of joints from a training dataset of only 50 to 200 frames. As the algorithm’s Graphic User Interface (GUI) of this algorithm provides functions for the AI model’s formation, verification, and result analysis, users can conveniently estimate the 2D poses of animals using this algorithm. The GUI partially extracts frames to form a training dataset from RGB videos of dogs, saves the joint locations, and generates a DeepLabCut model to estimate the 2D poses of the dogs based on entire frames of the RGB videos. Figure 2 shows the results of predicting the 2D poses of dogs.

3.2.2. Multi-View 3D Dog-Pose Estimation

Anipose [22] is a utility derived from DeepLabCut that estimates the 3D poses of an object from the 2D poses estimated by DeepLabCut. The 3D dog pose-estimation method requires multi-view RGB images, 2D poses of the dogs, and multi-view camera calibration. Triangulation can then be applied to the 2D poses, and multi-view camera calibration then estimates the poses in three dimensions.

Kaustubh et al. [23] state that camera calibration is the process of estimating a parameter that defines a conversion relationship between the world coordinate and the image coordinate. In our study, ChArUco Board, which combines ArUco Marker and Chessboard, was used for camera calibration. A ChArUco Board printed at an A4 size was held carefully without wrinkling or repositioning so that it could be observed clearly from all the cameras. The video was captured over approximately 5 min. Feature points were extracted from the chessboard video sequence to obtain the camera parameter.

The 2D poses of the dogs and the multi-view camera calibration results were analyzed based on triangulation to estimate the 3D poses. The triangulation technique uses properties of triangles to identify the coordinates and distance of a certain point. The coordinates between the multi-view RGB images were identified and used, along with the camera parameter, to calculate information pertaining to the target object’s 3D motion. Figure 3 shows the result of a 2D dog-pose estimation from four perspectives. Figure 4 displays the estimation result calculated by triangulation.

3.3. Identification of Problematic Behaviors in Dogs

3.3.1. Single-View 3D Dog-Pose Estimation

VideoPose3D [24] is an AI algorithm developed by Pavllo et al. that facilitates 3D human pose estimation based on single-view RGB images of people. This algorithm trains an AI model that derives 3D poses from 2D poses estimated from single-view RGB images. VideoPose3D can estimate the 3D poses of dogs only when a dataset of human joints is adjusted to match a dataset of dog joints. To satisfy this requirement, this study adjusted the values of joints_left, joints_right, and parent to be appropriate for the dog joints in the category of Class Skeleton, which represents the joints of an object.

In this study, 5760 3D coordinate triplets out of an overall 7198-coordinate triplet set were used as the training dataset. The remaining 1438 coordinate triplets were used as the evaluation dataset. The first training dataset was analyzed based on the 3D poses of dogs and multi-view camera calibration to form a second dataset that includes 2D coordinate pairs. Specifically, the original training dataset was analyzed based on the results of the multi-view camera calibration to generate 5760 corresponding 2D coordinate pairs. In this process, a rotation vector obtained by multi-view camera calibration was converted to a quaternion rotation vector. The second training dataset was created using metadata, which included 2D coordinates, the number of joints, and keypoint symmetry.

Detectron2 [25] is a platform for object detection and semantic segmentation based on PyTorch, which was developed by Meta’s research team. This platform performs object detection based on a pre-trained model loaded to estimate the 2D poses of the target object. Figure 5 shows the detection process of a dog based on a single-view RGB image by using the pre-trained model and performing 2D pose estimation using the DeepLabCut model. Figure 6a shows a 2D pose of the target dog estimated by the DeepLabCut model from a single-view RGB video, while Figure 6b shows the 3D pose of the dog estimated from the VideoPose3D model.

While training the Videopose3D model, the number of epochs that showed the most optimal performance was 60, the batch-size was 1024, and the learning rate was 0.001. The model arguments with the optimized values are summarized in

Table 1. Model arguments.

Parameter	Value
Stride	1
Epochs	60
batch-size	1024
Dropout	0.25
lr (learning rate)	0.001
lr-decay	0.95
Architecture	3, 3, 3, 3, 3
Channels	1024

. GeForce RTX 3090 with 64GB RAM was used as the graphical processing unit in all the experiments.

3.3.2. Recognition of Problematic Behaviors

DD-NET [26], developed by Yang et al., is an AI algorithm used to recognize the behaviors of dogs through correlations with their 3D poses. AI models that recognize behaviors of an object based on 3D pose estimation are generally time- and resource-intensive. To overcome this challenge, the DD-Net algorithm employs one-dimensional CNNs and a simplified network structure. We found that the lightweight property of DD-Net is appropriate for use within a mobile app. Therefore, this study adopted DD-Net to recognize dogs’ behaviors.

We labeled the behaviors of dogs by dividing single-view RGB videos, which contain 3D motion, into several frames, and generated clips that include the labelled behaviors as a dataset for DD-Net. However, each clip contained a different number of frames. The labels used to classify dog behaviors include five common behaviors (walk, sit, jump, stand, and track) and one problematic behavior (circling). When a single-view RGB video was input, the 3D poses of the dog were estimated by frame. Subsequently, each frame was divided into 30 sub-frames to enable DD-Net to recognize specific behaviors of the dog. Figure 7 shows the image captured from a result derived by DD-Net. When the dog in the video performed circling, the number of circles was recorded.

3.4. Prototype of the Mobile App

The mobile app was implemented using Android Studio and can be run on Android-based mobile devices. The actual appearance of the mobile app is shown in Figure 8.

Figure 9 shows the entire wireframe of the mobile app developed in this study. When the user launches the mobile app, a Login page is displayed, as shown in Figure 9. This page allows the user to sign in with their Google account. Upon successful login, the mobile app displays the User Profile page, where the user can input their member information, including their dog’s name, breed, and birth date.

As shown in Figure 10a, when the user selects the Live Stream menu shown in Figure 9, the mobile app stores a video recorded on a video-recording page in a File-Transfer Protocol storage server based on a 15-min unit. The storage server measures the number of problematic behaviors performed by the dog shown in the video and calculates the dog’s stress index. The stress index is then transmitted to the user’s smartphone.

The stress index can be formulated as:

Dog’s stress index = ((Number of problem behaviors today – Number of problem behaviors on the previous day)/Number of problem behaviors on the previous day) × 100

By touching the stress index button on the navigation bar, the user can enter the Daily Check menu shown in Figure 9. The user can also access their dog’s stress index and stress-related environmental information on the page shown in Figure 10b. They can record the walking and isolation times of their dog on a nightly basis, and the recorded data will accumulate. By accessing the Analyze Stress menu shown in Figure 9, the user can examine the accumulated data over the most recent seven days, the most recent 30 days, and the annual average, as shown in Figure 10c.

For the stress index, the change in the number of problematic behaviors compared to that on the previous day is represented by a percentage. When the user touches the stress index, the mobile app displays the number of problematic behaviors recorded in the form of a bar graph, as shown in Figure 10d.

4. Evaluation

We evaluated our mobile app using the following methods. First, the performance of the developed system was evaluated. Specifically, we assessed the performance of the VideoPose3D model, which estimates the 3D poses of a dog based on its single-view RGB image, and that of the DD-Net model, which recognizes the dog’s behaviors from the 3D poses. Next, we evaluated the usability of the mobile app by considering the clinical experience of experts who specialize in canine behavioral disorders.

4.1. AI Model Evaluation Result

To evaluate the VideoPose3D model, the mean per joint position error (MPJPE), Procrustes-aligned Mean Per Joint Position Error (P-MPJPE), Normalized mean Per Joint Position Error (N-MPJPE), and Mean Per Joint Velocity Error (MPJVE) were measured. To calculate the MPJPE, the average Euclidean distance between the estimated joint location and the location of the Ground Truth (GT) joint was measured.

Kearney et al. [1] used the MPJPE as a measure to evaluate the algorithm for estimating the 3D pose of a dog. Marshall et al. [27] adopted the MPJPE as a measure to evaluate their proposed multi-animal 3D pose dataset. Accordingly, the MPJPE was selected for the evaluation of the VideoPose3D model used in this study.

Pavllo et al. [24] stated that 3D poses estimated by Procrustes transformation should be compared with the GT to calculate the P-MPJPE. The Procrustes transformation eliminates translation, rotation, and scaling. To calculate the N-MPJPE, the estimated 3D pose is compared with the GT in consideration of only size. Finally, the single-dimensional derivative obtained from the estimated 3D pose sequences is applied as the MPJVE, which indicates the dynamic quality of the smoothness of the 3D pose sequences.

Table 2. Reconstruction Errors.

Category	Value [mm]
MPJPE	75.9
P-MPJPE	45.6
N-MPJPE	70.9
MPJVE	4.11

presents reconstruction errors calculated by applying the VideoPose3D model to the test dataset. The VideoPose3D model used in this study had an MPJPE of 75.9 mm.

Table 3. Loss and Accuracy after 2500 epochs.

Category	Value
Training Loss	0.0344
Training Accuracy	1.0000
Validation Loss	0.7443
Validation Accuracy	0.8182

presents the results of calculating the loss and accuracy of the DD-Net model. Training Loss and Training Accuracy refer to the loss and accuracy values, respectively, calculated from the training dataset. Validation Loss and Validation Accuracy refer to the loss and accuracy values, respectively, calculated based on the validation dataset. Google Developers reported that the loss value indicates a poor prediction performance of an AI model. If the AI model performs an accurate prediction, the loss value is 0 [28]. When the DD-Net model was applied to the validation dataset, the loss and accuracy values were calculated to be 0.7443 and 81.82%, respectively. The change in accuracy according to epochs can be seen in Figure 11, and the change in loss according to epochs can be seen in Figure 12. It is evident that the accuracy gradually increases as the epoch increases, while the loss gradually decreases as the epoch increases.

4.2. Qualitative Evaluation by Experts

To evaluate our app’s usability, we conducted in-depth interviews with companion dog trainers and behavioral treatment experts who have considerable clinical experience in understanding stress-inducing situations for dogs. Specifically, this study conducted in-depth interviews with five experts with certificates related to companion dog training and management in South Korea, over approximately a week between 30 November to 6 December 2021. The experts’ credentials are listed in

Table 4. Information on experts who participated in in-depth interviews.

Sample	Gender	Age	Experience
A	Male	In his 20s	Level 2 dog-training certificate Animal-assisted psychotherapist Pet sitter and handler certificates
B	Female	In her 20s	Level 3 dog-training certificate
C	Male	In his 20s	Level 3 companion dog-care certificate Level 3 companion dog-training certificate
D	Male	In his 40s	Six years of experience in dog training A companion dog trainer
E	Female	In her 30s	Level 3 companion dog-training certificate

. The experimental procedures were as follows. The five interviewees were provided with the developed mobile app in advance and asked to use the app by inputting monitoring information on their dogs for one day. Subsequently, we conducted a video conference on Zoom to intensively interview the experts for approximately 40 min.

Prior to the in-depth interview, the experts were asked to perform pre-defined user tasks, in the order listed in

Table 5. Questions of Task Performance.

Order	Task
1	Run the mobile app, check the live video
2	Input the walking time and isolation time
3	Check the stress index

. This was intended to increase the experts’ understanding of the mobile apps prior to the in-depth interviews.

Table 6. Results of Task Performance.

Sample	1	2	3
A	00:15	00:23	01:02
B	00:18	00:24	00:33
C	00:21	00:21	00:43
D	00:15	00:17	00:50
E	00:17	00:38	00:48

summarizes the results of measuring the time required to complete the user task for each expert.

As detailed in

Table 7. In-depth interview questions.

	Categories	Questions
Q1	Validity	Was the walk time of the dog and the number of circles it performed related to its stress?
Q2		Was the isolation time of the dog and the number of circles it performed related to its stress?
Q3		Was the number of circles taken by the dog measured accurately?
Q4		What types of stress-related environmental information for dogs should be additionally included?
Q5	Usability	Do you think the proposed mobile app can analyze the stress of the dog?
Q6		What is the difference between the developed mobile app and other mobile apps?
Q7		What do you think about the usability of the developed mobile app?
Q8		What types of functions should be additionally included?

, the interview questions were related to the validity and usability of the developed mobile app for calculating the stress index of dogs. The factors for evaluating the mobile app’s validity and usability were defined, and based on the defined factors, a questionnaire was created and evaluated against the experts.

4.3. Results of Qualitative Evaluation by Experts

We conducted in-depth interviews with some test subjects to obtain qualitative evaluations of this application. We then used an affinity diagram to compare and analyze the opinions obtained through the in-depth interviews. An affinity diagram is a tool that gathers large amounts of language data (ideas, opinions, and issues) and organizes them into groupings based on their natural relationships [29]. We used the affinity diagram to categorize the opinions from the in-depth interviews into three categories, i.e., experience, ideas, and relevance, listed in

Table 8. Categorizing the opinions from in-depth interviews.

Opinions	Category
It appears that the mobile app performs measurements well.	Experience
It seems that the number of times the dog circles is being measured correctly
I have never seen any mobile apps that can analyze canine behavior. This mobile app is advantageous in that users can be easily informed about the meaning of their dogs’ behaviors.
I find it positive that the developed mobile app provides users with the stress index of their dogs. The bar graph (referring to the times that companion dogs spend walking and being left alone) is helpful because users can understand how much affection they devote to their dogs at a glance.
It is my first time experiencing a mobile app that can analyze the behaviors of companion dogs. Other mobile apps provide only descriptions related to their behaviors. By contrast, this mobile app analyzes their behaviors based on their recorded videos. For this reason, I find this mobile app better than other mobile apps.
By simply installing the developed mobile app on their smartphones, users can automatically ascertain the problematic behaviors of their companion dogs. I believe that this function will help users easily identify problematic behaviors of their dogs.
The accessibility of the mobile app is high enough to allow me to use this app without inconvenience. I can also search detailed information easily.
In general, the developed mobile app operates smoothly. However, when it was installed on my smartphone, at a certain point, this device detected my dog from only a certain angle. For this reason, the problematic behaviors of my dog were not analyzed when my dog was not detected. I wish the app could improve this functionality.
It is also crucial to analyze whether companion dogs naturally like people or only their owners.	Idea
I would like to recommend including additional information on the main owner for the companion dog among family members.
It is recommended to include information on the amount of time that people spend playing with their dogs at home.
I believe that the developed mobile app can analyze the stress of companion dogs more accurately if it detects the behaviors and facial expressions of these dogs simultaneously.
It would be useful if the mobile app sent friendly messages on the analytic results derived by this app to users when the stress index of their dogs reaches a certain level.
It will be appreciated if the developed mobile app provides a function for enabling users to solve quizzes about the causes of problematic behaviors of their dogs displayed on this app to increase the awareness of users.
I hope that the developed mobile app can provide additional information on basic solutions for users to cope with situations where their dogs exhibit problematic behaviors.
It would be useful if the developed mobile app could provide a function for enabling users to record the amount of time that they spent playing with their dogs. I suggest that this app should allow users to identify not only the amount of time that they spent walking their dogs but also the amount of time that they spent playing with their dogs.
It is recommended to analyze a relationship between the stress of the dog with not only the dog walk time but also the behaviors of the dog shown during the dog walk time.	Relevance
Abandoned dogs might perform circling because of past trauma, and certain companion dogs might perform circling because of SAD.
Companion dogs show different tendencies. Some companion dogs might be satisfied with a short walk time, whereas other companion dogs might not be satisfied with even a long walk time.
Companion dogs suffering from SAD can exhibit several problematic behaviors, including circling, when they are separated from their owners.
When companion dogs are isolated from their owners, they suffer from considerable stress.
If puppies did not learn how to live independently during the period of independently leaving their mother dogs, they will suffer from SAD. I think that highly independent dogs might not be stressed even when they are left alone without their owners.
The relationship between companion dogs and their owners and the tendencies of these dogs can be regarded as important stress-related environmental information.
The amount of time people spend playing with their dogs at home and caring about their dogs is as important as the time they spend walking their dogs.
The same behaviors of dogs might have different meanings. The meaning of the same circling behavior can vary according to a minor difference in the ears, tails, pupils, and hair.
It would be difficult to analyze the stress of companion dogs without considering their tendencies.
It is required to inspect situations observed when problematic behaviors of companion dogs occur as well as behaviors that they perform instantly before they exhibit such problematic behaviors.

.

After using the mobile application, among the opinions that fall under the Experience category in the affinity diagram, experts responded using the terms “well measurement”, “easily”, “positive”, “helpful”, and “better”, among other responses.

“It appears that the mobile app performs measurement well.”

“It seems that the number of times the dog circles is being measured correctly.”

“I have never seen any mobile apps that can analyze canine behavior. This mobile app is advantageous in that users can be easily informed about the meaning of their dogs’ behaviors.”

“I find it positive that the developed mobile app provides users with the stress index of their dogs. The bar graph (referring to the times that companion dogs spend walking and being left alone) is helpful because users can understand how much affection they devote to their dogs at a glance.”

“It is my first time experiencing a mobile app that can analyze behaviors of companion dogs. Other mobile apps provide only descriptions related to their behaviors. By contrast, this mobile app analyzes their behaviors based on their recorded videos. For this reason, I find this mobile app better than other mobile apps.”

“By simply installing the developed mobile app on their smartphones, users can automatically ascertain the problematic behaviors of their companion dogs. I believe that this function will help users easily identify problematic behaviors of their dogs.”

“The accessibility of the mobile app is high enough to allow me to use this app without inconvenience. I can also search detailed information easily.”

“In general, the developed mobile app operates smoothly. However, when my smartphone was installed at a certain point, this device detected my dog from only a certain angle. For this reason, problematic behaviors of my dog were not analyzed when my dog was not detected. I wish the app could improve this functionality.”

Among opinions that fall under the Idea category in the affinity diagram, the experts replied with opinions related to ideas such as “I would like to recommend including additional information on the main owner for the companion dog among family members”, “It is recommended to include information on the amount of time that people spend playing with their dogs at home”, and “I believe that the developed mobile app can analyze the stress of companion dogs properly if it detects the behaviors and facial expressions of these dogs simultaneously”.

“It is also crucial to analyze whether companion dogs naturally like people or only their owners.”

“I would like to recommend including additional information on the main owner for the companion dog among family members.”

“It is recommended to include information on the amount of time that people spend playing with their dogs at home.”

“I believe that the developed mobile app can analyze the stress of companion dogs more accurately if it detects behaviors and facial expressions of these dogs simultaneously.”

“It will be useful if the mobile app sends friendly messages on the analytic result derived by this app to users when the stress index of their dogs reaches a certain level.”

“It will be appreciated if the developed mobile app provides a function for enabling users to solve quizzes about causes of problematic behaviors of their dogs displayed on this app to increase the awareness of users.”

“I hope that the developed mobile app can provide additional information on basic solutions for users to cope with situations where their dogs exhibit problematic behaviors.”

“It will be useful if the developed mobile app could provide a function for enabling users to record the amount of time that they spent playing with their dogs. I suggest that this app should allow users to identify not only the amount of time that they spend walking their dogs but also the amount of time that they spend playing with their dogs.”

Among the opinions that fall under the Relevance category in the affinity diagram, the experts stated that “It is recommended to analyze a relationship between the stress of the dog with not only the dog walk time but also behaviors of the dog shown during the dog walk time”, “The relationship between companion dogs and their owners and tendencies of these dogs can be regarded as important stress-related environmental information”, and “It is required to inspect situations observed when problematic behaviors of companion dogs occur as well as behaviors that they perform instantly before they exhibit such problematic behaviors”.

“It is recommended to analyze a relationship between the stress of the dog with not only the dog walk time but also behaviors of the dog shown during the dog walk time.”

“Abandoned dogs might perform circling because of past trauma, and certain companion dogs might perform circling because of SAD.”

“Companion dogs show different tendencies. Some companion dogs might be satisfied with a short walk time, whereas other companion dogs might not be satisfied with even a long walk time.”

“Companion dogs suffering from SAD can exhibit several problematic behaviors, including circling, when they are separated from their owners.”

“When companion dogs are separated alone from their owners, they suffer from considerable stress.”

“If puppies did not learn how to live independently during the period of independently leaving their mother dogs, they will suffer from SAD. I think that highly independent dogs might not be stressed even when they are left alone without their owners.”

“The relationship between companion dogs and their owners and tendencies of these dogs can be regarded as important stress-related environmental information.”

“The amount of time people spend playing with their dogs at home and caring about their dogs is as important as the time they spend walking their dogs.”

“The same behaviors of dogs might have different meanings. The meaning of the same circling behavior can vary according to a minor difference of ears, tails, pupils, and hair.”

“It would be difficult to analyze the stress of companion dogs without considering their tendencies.”

“It is required to inspect situations observed when problematic behaviors of companion dogs occur as well as behaviors that they perform instantly before they exhibit such problematic behaviors.”

5. Conclusions and Future Work

Thus far, the analysis of behaviors in companion dogs has required the animals to wear devices equipped with sensors. However, it might be difficult to equip dogs with these devices, and the devices themselves might cause the dogs to feel uncomfortable. The proposed mobile app in this study recognizes dog behaviors using the VideoPose3D model to estimate the dogs’ 3D poses based on their 2D poses estimated using the DeepLabCut model. Thus, the technology enables users to analyze their dogs’ behaviors without requiring sensors.

As few studies on animal-pose estimation have been conducted [1,2,3,22,27], it is difficult to obtain relevant datasets. The motion capture of an object needs to be performed to measure 3D poses. Certain motions of animals cannot be tracked by motion capture because of the difficulty in controlling these animals. Furthermore, the motion capture of animals is quite expensive, given the need for trainers and well-trained animals. However, the technology proposed in this study successfully generated a dataset of joint locations in dogs by estimating their 3D poses based on multi-view RGB images.

In addition to the two aforementioned technical contributions, this study also proposes a mobile application that can recognize stress-related behaviors in companion dogs based on the 3D poses obtained through deep learning. When their stress becomes unbearable, dogs express their physical and mental conditions through their behavior. However, people often cannot easily recognize dog behavior because the physical appearance—rather than the actions—of dogs draws their attention at first glance. We developed a mobile application that identifies stress levels in dogs by recording their walking time and time spent alone, followed by analyzing problematic stress-related behaviors based on circling, as well as environmental stress factors.

In-depth interviews were conducted with five experts with regard to the proposed app’s performance for the practical management of stress in companion dogs. The interview results verified a relationship between the dog’s walking and isolation times, and the amount of circling that they perform as a result of their stress. However, the relationship between the walking time and stress level was evaluated to be uncertain. All five experts emphasized the importance of the relationship between dogs and their owners as additional stress-related environmental information. In addition, it was confirmed that the app is capable of successfully informing users about the meaning of their dogs’ behaviors and stress level.

The VideoPose3D model estimates 2D poses of dogs from single-view RGB videos, and subsequently estimates 3D poses from the 2D poses. If an unstable 2D pose estimation is derived from a single-view RGB video, this result might also lead to an unstable 3D pose estimation. A high-quality 2D dog-pose estimation result can be obtained only when the camera angle and position of the dog in the single-view RGB video are similar to those in the training dataset of the DeepLabCut model. Therefore, follow-up research should consider camera-based imaging techniques in the process of generating datasets.

The future work directions with respect to good stress management for companion dogs are two-fold. Firstly, we need a comprehensive analysis that includes not only subtle differences in a dog’s body parts but also the tendencies of dogs, as experts stated that stress can also be expressed through the facial expressions of dogs. They also mentioned that different breeds might express stress differently. Secondly, we set the walking and isolation times of companion dogs as stress-related environmental information. However, in addition to external environmental information, such as the walking and isolation time of companion dogs, internal environmental information such as the dog–owner relationship can also serve as significant stress-related environmental information. In the future, the dog–owner relationship must also be considered as stress-related environmental information.

Kang [30] reported that the same calming signals delivered by companion dogs can reflect diverse conditions depending on an encountered situation. In this regard, the researchers emphasized the importance of analyzing the relationship between companion dogs and their surroundings. In this study, the problematic behaviors of dogs were analyzed only depending on the dog’s pose without considering the surroundings. Therefore, if our mobile app is combined with situation-recognition technology based on object detection, it will be able to analyze the problematic behaviors of companion dogs more accurately in specific situations.

Diverse types of companion dog care-services have been recently developed in conjunction with scientific technology. These services require the development of technology for the processing and accumulation of data for dog behavior recognition. In this context, our developed mobile app is expected to contribute toward advancing the technology to analyze the behavior of companion dogs.

Ethology is the study of analyzing animal behaviors through observation. Quantitative and automated technology should be developed to systematically investigate the behaviors of animals and their variability. As the proposed technology produces training data from users who directly store joint locations and train the artificial intelligence model, it can be applied not only to dogs, but also to other animals. In this regard, our mobile app is expected to be widely applicable in industries related to companion dogs, which are deeply associated with ethology, as well as in the fields of animal experiments and wild animal protection.