A Soft Body Physics Simulator with Computational Offloading to the Cloud

Abstract

We describe the gamification of a soft physics simulator. We developed a game, called Jelly Dude, that allows the player to change and modify the game engine by tinkering with various physics parameters, creating custom game levels and installing scripts. The game engine is capable of simulating soft-body physics and can display the simulation results visually in real-time. In order to ensure high quality graphics in real time, we have implemented intelligent computational offloading to the cloud using Jordan Neural Network (JNN) with a fuzzy logic scheme for short time prediction of network traffic between a client and a cloud server. The experimental results show that computation offloading allowed us to increase the speed of graphics rendering in terms of frames per second, and to improve the precision of soft body modeling in terms of the number of particles used to represent a soft body.

1. Introduction

Soft body refers to an object that can deform its shape significantly in reaction to its context [1] or due to an external force, such as push, pull, tear, or bend forces [2]. Soft body simulation has been used in many fields such as medical applications [3], robotics [4], education [5], and games [6]. The topic is especially relevant for preschool education as young children often have incomplete and inaccurate understanding of “how things work” in the physical world, with only vague expectations about the behavior of real-world objects and materials. Therefore, various kinds of simulator games, which allow for mental interaction with world objects and simulation of their future state, are both popular and educationally valid among the young generation [7]. The issue is especially relevant for STEM (Science, Technology, Engineering, and Math) education, which aims to increase the attractiveness of STEM subjects and motivation of high school students to learn by employing attractive and entertaining tools such as games [8,9,10].

Implementing a realistic physics simulation is also important for many simulator games as it contributes to achieving higher Quality of Experience (QoE) and playability. There are different ways to evaluate and quantify the perceived QoE of the end-users, for example, by observing the players’ styles of behavior and patterns of visiting game objects and levels similarly to people visiting exhibits in museums [11,12]. By providing a mathematical model of QoE such as presented in previous studies [13,14], one can solve this model for optimal values of its parameters, maximizing the desired QoE. However, here we adopt a more technical approach in cloud gaming that is linking QoE to the key influencing factors of video graphics quality (frame rate) [15] and network quality (bandwidth, latency) [16].

While many game engines properly tackle various issues of physics simulation, some specific physics of materials are often omitted from game mechanics (e.g., game objects composed by liquids or amorphous jelly-like bodies), and as a result, are unsatisfactorily rendered by game engines. This limitation may hinder game developers when creating highly realistic video games and game mechanics. To improve immersion into a game, the physics laws simulated by the game engine should realistically simulate the laws of physics that the game needs. In order to ensure computing of the physics simulation as fast as possible, simplifying the fundamental mathematical model of the modeled physics behavior is often the solution. Nevertheless, the simulated bodies should have simulated behavior that closely matches expected behavior [17]. The development of realistic physics engines is one of the key technologies named as prerequisite requirements for mobile games [18,19], especially Virtual Reality (VR) and Augmented Reality (AR) games, which require high resolution and low latency of graphics rendering [20,21].

Soft bodies are usually represented as systems consisting of a large number of minute particles connected by flexible spring joints adhering to the laws of kinematics. One of the solutions is using pre-computed data-driven models for rendering physics-based deformable scenes to achieve fast real-time implementation [22]. An example is using the Navier-Stokes equations for modeling soft bodies as compressible fluid encaged in a mesh [1]. Wyvill et al. [23] combine particle systems with implicit functions to model soft bodies. Terzopoulos and Fleischer [24] modeled the behaviors of a deformable body by configurations of plastic, viscous, and elastic components. James and Pai [25] employ linear elasticity methods and use a boundary element method (BEM) solver to achieve desired behavior in real-time. Miller [26] animates the deformable bodies of worms and snakes using a lattice of mass elements linked with springs. Matyka and Ollila [27] adopt the Clausius-Clapeyron state equation for modeling pressure forces affecting a 3D mesh object and apply Newton’s second law to achieve realistic simulation of the behavior of a soft object. Kenwright [28] uses strain models for simulating deformations of rigid objects. McAdams et al. [29] demonstrate surface deformation on hexahedral lattices for a geometric skeleton-based rigging, i.e., inserting a skeleton in a surface mesh and defining transformations on the mesh vertices. Martin et al. [30] unified the modeling of 3D objects by using the elaston (which are the specific rules of integration for volumetric objects) representation. Wu et al. [31] solve subspace deformations and motions of rigid and soft objects using linear solve and cubature optimization schemes to compute the elastic forces and their Jacobian matrices.

Soft physics is rarely used in computer gaming due to high computational requirements, especially on mobile platforms. As a result, this feature of the game engine is quite unique and can attract a lot of attention. Bullet [32], an open-source physics simulation library, has added support to soft bodies. CryEngine 3, a popular game engine, has a soft-body physics simulator implemented by Rigs of Rods (BeamNG). Digital Molecular Matter (DMM) is based on a finite element method (FEM), as described in [33], which is very efficient and achieves realistic results while providing support for many platforms. The Sulfur physics engine [17] uses the Verlet integrator with particles systems to model soft and rigid objects, and also implements collision management. Chrono [34], a multi-physics dynamics engine, also has support for modeling of fluid–solid interaction (FSI) as well as for linear and nonlinear finite element analysis (FEA). SpriteBuilder [35], a 2D game development environment for iOS, which among other features also provides support for soft-body physics.

This paper focuses on cloud-based gamification of an aesthetically pleasing soft body deformation model for real-time interactive environments. We describe the implementation of soft body physics in the game as well as describe the computation offloading management of computation-intensive functions for achieving higher performance.

2. Methods and Materials

2.1. Soft Body Physics

Soft bodies are usually modeled as a collection of elastically linked small particles that are characterized by mass, position, and velocity. The constraints linking two particles connected by an elastic joint can be described mathematically by Hooke’s law with an inclusion of the damping force as follows:

$$\mathrm{F}\left(\mathrm{t},\mathrm{x}\left(\mathrm{t}\right),\dot{\mathrm{x}}\left(\mathrm{t}\right)\right)=-\mathrm{k}\left(\left|\mathrm{d}\right|-{\mathrm{l}}_0\right)\overrightarrow{\mathrm{d}}+\mathrm{b}\left(\dot{\mathrm{x}}\left(\mathrm{t}\right)-{\dot{\mathrm{x}}}_{\mathrm{p}}\left(\mathrm{t}\right)\right)$$

(1)

where $\mathrm{t}$ is time; ${\mathrm{l}}_0$ is the length of the link; $\mathrm{k}$ is the elasticity constant; $\mathrm{b}$ is the damping constant, $\mathrm{d}$ is the variation in the length of the link; ${\dot{\mathrm{x}}}_{\mathrm{p}}\left(\mathrm{t}\right)$ is the velocity of the counterpart particle linked by the joint.

Such a model is easy to calculate and represent in time, however, it provides an approximation of the true physics only to a certain level of precision.

2.2. Game Implementation

The game physics is based on the Java’s JBox2D library. This library has a lot of functions for physics simulations with hard bodies, but it does not directly support the simulation of soft bodies. The Jelly Dude soft body modeling is based on JBox2D constant-length springs. See an example of the spring-mass model of a soft body presented in Figure 1.

The game uses two different strategies to simulate soft bodies: a constant area strategy and a spring-loaded strategy. A constant area strategy simulates fairly realistic soft physics. It is realized by combining a large number of small particles with flexible springs, and then according to the total area and perimeter of the body, each particle of the body is updated to maintain a constant overall area. The operation of this strategy is presented in Algorithm 1 and is visualized in Figure 2.

Algorithm 1

Create small solid particles and link each particle to its neighbor particles using a soft spring. Calculate the initial body area S0. For each subsequent step, perform the following: Calculate the physical perimeter P. Calculate the total area of the body. Find the sum of the extensions E by subtracting the current area S of the initial area S0. This difference E is divided in half and is also divided by the perimeter Pi. Calculate the expansion vector Di for each pair of vertices, by multiplying the sum of the normal vectors of summands of Ni and Ni + 1 of the vertex pairs from the expansion factor E. The tip of the body Vi + 1 is the vector of the expansion of the vector Di, which is added to its vector.

Using this strategy, after the first iterations of the algorithm the shape of the soft body changes, as is shown in Figure 2. After completing several steps of Algorithm 1, the shape of the composite soft body begins to change while maintaining its area (Figure 3).

If this strategy is applied to the ultra-small bodies, it is evident that they are not in place (slightly moving), since in each iteration it is modified by the position of their constituent particles and the movement is visible due to the accumulation of computation error. Also, such calculations are quite expensive in terms of computing resources required, due to the large amount of calculations for each algorithm step.

The spring jointing strategy is based on a similar principle to that of a constant area strategy (combining the body with soft springs). However, this strategy does not add any additional calculations to maintain the body shape. It is realized using only standard JBox2D functions. Its implementation is presented in Algorithm 2.

Algorithm 2

Create an initial body shape from small particles. Fill the inside of the body evenly with small particles. Connect each body component with its neighbors using flexible springs. Perform physics simulation of the internal particles of the body.

This method makes it easy to create polygons and circles. Because the simulation of these types of bodies is relatively cheap in terms of computing resources, it is convenient to use these bodies as objects of the game level, as they can be created in large quantities. Also, this type of body is easy to combine with other soft and hard bodies, thus creating more complex shapes. Examples of sketches for objects that use this strategy are shown in Figure 3.

2.3. Game Architecture

The game is implemented in Java 8. The libGDX library was selected for game development. This library has particularly accelerated the tasks of graphing and creating user interfaces. Using this library, it was easy to create a simple object drawing system and handle player input. However, developing a fluid drawing system required lower-level drawing interfaces that this library does not have. In order to realize the fluid drawing system, a lower-level Lightweight Java Game Library LWJGL provided a software interface. The JBox2D Library was chosen to realize simulation of soft bodies. This library supports the simulation of various fluids but does not have the ability to simulate the physics of soft bodies.

The gdx-holo user interface resource library was used, which is based on the user interface style used by the Android platform. This library has textures for all user interface elements. Game levels were created without focusing on a particular style. We wanted to have a great variety of levels with different styles. For some game levels, our own set of textures was used. However, for the majority of levels, a free set of game textures was used. The game engine consists of classes that are essential for the program’s operation. The activity diagram of the game engine is given in Figure 4.

2.4. Game Interface and Main Elements

When starting the game, the player will initially see the main menu (Figure 5a). From here, they can change the settings, start playing the game, or quit the game. Selecting the game level starts the game right away. An example of a game level is shown in Figure 5b. It features both solid immovable bodies as well as demonstrates the behavior of soft bodies influenced by interaction with the player.

2.5. Computational Offloading Model and Offloading Decision Scheme

Cloud offloading is the employment of physically remote computing resources to perform the computation and energy-demanding functions of the system with data and results sent over the network [36]. The principle challenges of implementing efficient computation offloading concern the high volatility of internet networks, especially mobile networks. An important parameter influencing the effectiveness of cloud-offloading is network delay. If delay is large, the offloading strategy may not succeed. Therefore, constant monitoring of network connection to the cloud server and cloud response time is important. In fact, one needs to predict the cloud server response time in short-time in order to decide whether to offload computations or not. Such short-time forecasting (from ms to seconds) is useful for dynamic task scheduling and can be employed to boost the Quality of Service (QoS) and Quality of Experience (QoE) characteristics and playability of the game by adapting computation offloading mechanisms to mitigate network traffic volatility.

To improve the performance of the developed game we followed the computational model introduced in [37] and the offloading decision scheme suggested in [38]. We considered that a local machine has a computing task $\mathrm{J}\left(\mathrm{w},{\mathrm{S}}_{\mathrm{m}}\right)$ that can be performed locally or remotely via computation offloading in a cloud, where $\mathrm{w}$ denotes the size of input data (e.g., program code or parameters that are needed for a task) used in the task, $\mathrm{J}$ and ${\mathrm{S}}_{\mathrm{l}}$ is the computation speed of the local device. For computing locally, a local device performs its task $\mathrm{J}$ locally, and the execution time of the task $\mathrm{J}$ is ${\mathrm{t}}_{\mathrm{loc}}=\mathrm{w}/{\mathrm{S}}_{\mathrm{l}}$. For computing on the cloud, a device offloads its computation task $\mathrm{J}$ to the cloud. Then the computation time is given as ${\mathrm{t}}_{\mathrm{off}}=\mathrm{w}/{\mathrm{S}}_{\mathrm{s}}$, here ${\mathrm{S}}_{\mathrm{s}}$ is the processing speed of the cloud server, and the sending time is defined as ${\mathrm{t}}_{\mathrm{send}}={\mathrm{d}}_{\mathrm{i}}/\mathrm{B}$, where ${\mathrm{d}}_{\mathrm{i}}$ is the input/output data and $\mathrm{B}$ is the network bandwidth. Then the total time of offloading and executing the remote task is ${\mathrm{d}}_{\mathrm{i}}/\mathrm{B}+\mathrm{w}/{\mathrm{S}}_{\mathrm{s}}$. We can partition all tasks of a simulation program as a part that is running locally and another part that is offloaded to a cloud. We formulate the requirement for increasing computing performance as follows. If ${\mathrm{d}}_{\mathrm{i}}/\mathrm{B}+\mathrm{w}/{\mathrm{S}}_{\mathrm{s}}<\mathrm{w}/{\mathrm{S}}_{\mathrm{l}}$, we can assume that offloading will increase the performance of computing.

2.6. Implementation of Intelligent Cloud Offloading

Here we adopted the opportunistic computation offloading approach suggested in [39]. To manage the cloud offloading efficiently, the game architecture was supplemented with mechanisms for the monitoring offloading process and computation task scheduling to implement adaptive cloud offloading optimizations in a straightforward manner. The Intelligent Offloading Management Module (IOMM) (see Figure 6) manages the CPU-intensive computations by offloading the computation-costly computation for cloud computing. It consists of an Artificial Neural Network (ANN) that analyzes network latency and predicts future network latency. Based on the prediction results, the decision is taken whether to compute locally or offload computations to a cloud.

Here we applied the neuro-fuzzy approach [40] for prediction (see Figure 7). The combination of ANNs and fuzzy reasoning by the neuro-fuzzy approach utilizes the advantages of both approaches simultaneously. The flexibility of fuzzy logic and the learnability of neural networks provides more power and adaptability in modeling the client-to-server connection traffic characteristics.

For short-term network latency prediction, we used the Jordan neural network (JNN) [41] and the Real Time Recurrent Network (RTRN), in which the outputs are delayed and sent back to the network input. Figure 8 illustrates the architecture of the JNN. The network inputs are the delayed signal of the process $\mathrm{u}\left(\mathrm{k}-1\right),$ the output signal of the model, $\mathrm{y}\left(\mathrm{k}\right),$ which are fed back to a delay unit. The JNN has $\mathrm{K}$ hidden neurons with transfer function $\mathsf{\phi }:\mathrm{R}\to \mathrm{R}$ and one linear output neuron, which can be described by the first-order system as follows:

$$\mathrm{y}\left(\mathrm{k}\right)=\mathrm{f}\left(\mathrm{u}\left(\mathrm{k}-1\right),\mathrm{y}\left(\mathrm{k}-1\right)\right)$$

(2)

Let ${\mathrm{w}}_{\mathrm{i},\mathrm{j}}^{\left(1\right)}$ (for $\mathrm{i}=0,\dots ,\mathrm{K},\mathrm{j}=0,\dots ,2$) and ${\mathrm{w}}_{\mathrm{i}}^{\left(2\right)}$ (for $\mathrm{i}=0,\dots ,\mathrm{K}$) be the weights of the first layer and the second layer of the neural network, respectively. The network output can be written as.

$$\mathrm{y}\left(\mathrm{k}\right)={\mathrm{w}}_0^2+{\displaystyle \sum _{\mathrm{i}=0}^{\mathrm{K}}{\mathrm{w}}_{\mathrm{i}}^{\left(2\right)}\mathsf{\phi }\left({\mathrm{z}}_{\mathrm{i}}\left(\mathrm{k}\right)\right)}$$

(3)

where ${\mathrm{z}}_{\mathrm{i}}\left(\mathrm{k}\right)={\mathrm{w}}_{\mathrm{i},0}^{\left(1\right)}+{\mathrm{w}}_{\mathrm{i},1}^{\left(1\right)}\mathrm{u}\left(\mathrm{k}-1\right)+{\mathrm{w}}_{\mathrm{i},2}^{\left(1\right)}\mathrm{y}\left(\mathrm{k}-1\right)$ is the sum of inputs of the $\mathrm{i}$-th hidden node.

The output of the neural network is used to make network delay predictions using the fuzzy logic [42]. A fuzzy subset $\mathrm{A}$ of $\mathrm{X}$ is a set defined by a membership function ${\mathrm{f}}_{\mathrm{A}}\left(\mathrm{x}\right),$ which maps an element $\mathrm{x}$ in $\mathrm{X}$ to a real number in $\left[0,1\right].$ The value of ${\mathrm{f}}_{\mathrm{A}}\left(\mathrm{x}\right)$ is the membership value of $\mathrm{x}$ in $\mathrm{X}.$ As a membership function, we use the probability density function (PDF) of Weibull probability distribution $\mathrm{X}\sim \mathrm{W}\left(\mathrm{k},\mathsf{\eta }\right),\mathrm{x}>0,\mathrm{k}>0,\mathsf{\eta }>0,$ which is often used for survival and failure analysis, and is characterized by the following PDF:

$$\mathrm{p}\left(\mathrm{x}\right)=\frac{\mathrm{k}}{\mathsf{\eta }}{\left(\frac{\mathrm{x}}{\mathsf{\eta }}\right)}^{\mathrm{k}-1}{\mathrm{e}}^{-{\left(\frac{\mathrm{x}}{\mathsf{\eta }}\right)}^{\mathrm{k}}}$$

(4)

The result of defuzzification is the value of $\mathrm{x}$ that corresponds to the maximum value of $\mathrm{p}\left(\mathrm{x}\right).$

3. Results

The simulations were implemented on a desktop computer with 4.3 GHz Intel i9-7900x CPU and NVIDIA GeForce 1080 GPU. Experiments were performed by setting the screen resolution to 1024 × 768 and measuring the frame rate per second (FPS). The game speed was measured in FPS (how many times the game logic was played per second) and the game’s charge rate. If FPS is less than 60, the game loses accuracy and gradually becomes difficult to control. To get results, one of the game levels was fetched and the minimum, maximum, and average counts per second were calculated. Also, before loading the game, the game loading rate was counted. By loading the game level, the virtual camera moved randomly in all directions. This process took ~5 min. The test results for frames per second (FPS) are presented in

Table 1. Summarized results of local computation vs. offloading to cloud.

Computation Scheme	Load Time (s)	FPS (min)	FPS (max)	FPS (mean)
Local computation	4	14	138	40
Offloading (cloud)	2.9	192	656	277

.

A low level testing game was prepared. By activating the test level, the amount of soft body particles used for simulation was gradually increased. During the test, the average number of FPS were registered depending upon the number of particles. The results are presented in Figure 9. It is evident that both faster and slower devices worked, with a relatively good scale of up to 3000 particles. When testing the game on a slow device, it started to cling to ~4000 particles and later the game became difficult to control. From these results we can conclude that the level of the game should be limited to 3000 particles used for soft body simulation.

The performance of the IOMM was evaluated using the real-world network traffic data collected during the game sessions on 24 September 2018, from 9.00 a.m. to 15.30 p.m. The client of the game was physically located in Kaunas (Lithuania), while the cloud server was setup in a physically remote location at Gliwice (Poland). The logarithmic transformation was applied to network round-trip time (RTT) data at $\mathrm{t}$ before the training of the neural network. For learning, the network RTT at $\mathrm{t}+1$ was binned and the accuracy calculated as the number of correctly predicted bins to which the network RTT value had been assigned. Using such a procedure, we achieved a true recognition rate of 84.1%. The example of true vs. predicted values is shown in Figure 10.

4. Discussion and Conclusions

We have developed a two-dimensional graphics game for real-time simulation of simple soft objects. The game demonstrates the simulation of soft physics laws and allows users to control a game character that consists of soft bodies. The game realistically demonstrates the behavior of objects, obeying the soft physics laws applied on simple soft objects, such as circle- and polygon-shaped bodies made of amorphous jelly-like materials, and could be used for educational purposes or as an example of physics law gamification. The limitations of our soft body modeling approach include the lack of collision modeling, which has not been implemented so far. To improve the performance characteristics of the game, we have demonstrated the computational offloading of a game to a cloud server. The offloading decision is managed by an intelligent offloading management module consisting of a Jordan neural network and fuzzy logic, which analyzes the client-to-cloud connection round-trip time (RTT) and takes the computation offloading decisions in real time. We have achieved 84.1% accuracy (true recognition rate) of RTT prediction, which allowed us to improve game performance in terms of frames per second as well as to improve soft body modeling accuracy in terms of the number of body particles, which can be used for modeling.

To facilitate further testing and improvement of the Jelly Dude game, the source code of the game has been made available on GitHub (https://github.com/Edvinas01/jelly-dude), and a set of promotional videos have been launched on YouTube (https://www.youtube.com/playlist?list=PL5No3sH5hcxuFBLlb_7-5g6G21-uueiQf).