Numerical Analysis for Feature Extraction and Evaluation of 3D Sickness

A R T I C L E I N F O A B S T R A C T Article history: Received: 08 December, 2020 Accepted: 21 March, 2021 Online: 15 April, 2021 Artificial intelligence (AI) systems have been applied not only to numerical simulations of the economical sequences but also to the bio-signal, for instance, the statokinesigrams (SKGs). According to the nonlinear analysis of the bio-signal, we have considered that the motion process of the body sway is more random than that of the other bio-signal. In this study, we proposed a method for the numerical analysis of biological data using AI. The AI numerical solutions can indicate graphs that are very similar to the SKGs in degree of the determinism. In addition, we succeeded in extracting partial figure patterns that the AI regarded as a feature of 3D sickness. Compering with the properties resulting from the mathematical analysis, interpretations can be given for the black box processing in the AI.


Introduction
In recent years, artificial intelligence (AI) has been incorporated into the automatic algorithms for buying and selling stocks and bonds. NASDAQ-100, actual stock prices, Standard & Poor's 500 Stock Index, and historical exchange rates have been analyzed to develop an automated trading algorithm. However, when using past stock price fluctuations, there is a drawback in that the test pattern is limited. In addition, when the time series generated by the stochastic process is used, there is a drawback wherein the actual fluctuation of the stock price is not reflected. Therefore, neither case is sufficient to trust the test results. In this study, we attempted to create a mathematical model that can generate innumerable fluctuation patterns after reflecting the fluctuation characteristics of the time series. In previous studies, the generative adversarial network (GAN) was applied to the timeseries generation of exchange rates. GAN is attracting attention because it has succeeded in generating high-precision images. Stationarity [1], fractalness [2], and determinism [3] were measured to compare the actual exchange rates with the pseudo exchange rates generated by GAN. Using these indices, we also measured the similarity between the actual exchange rates and the time series generated by the Winner process [4]. From the stationary perspective, the pseudo-exchange rates generated by GAN showed higher similarity than those of the Winner process, and from the deterministic point of view, both showed higher similarity.
On February 2, 2018, the Dow Jones Industrial Average had the largest decline to date [5]. The actual reason for this decline has not been clarified, but one of the possible causes cited by market participants is continuous selling, such as a chain of automatic loss cuts by the AI or automatic trading algorithms [6]. As a similar example, from March 9th to 18th, 2020, the decline in Standard & Poor's 500 Stock Index triggered four circuit breakers, temporarily suspending trading on the NYSE and NASDAQ [7][8][9][10]. This may indicate that the introduction of AI or automated trading algorithms is accelerating in financial markets. Some financial institutions, for instance, offer mechanical investment services to consumers. These services have solved the wealthmanagement problem of the general public.
However, mechanical investment services using AI cannot always protect assets. This is because reliability and robustness of the system is insufficient, owing to the limited number of tests. As mentioned above, the automatic trading algorithm uses past fluctuations in stock prices and exchange rates. In addition, even if the results look good at first glance, they may be overfitting to past fluctuations. To solve this problem, we propose an evaluation using a new simulated time series that does not depend only on historical data of past stock prices and exchange rates. It is conceivable to use the stochastic process as an idea to generate a simulated time series, but so far, many researchers have considered the stock price as the stochastic process [11,12]. Another approach is the generation of a simulated time series by GAN. In previous ASTESJ ISSN: 2415-6698 studies, the pseudo-exchange rate generated by GAN showed higher similarity to the actual exchange rate in terms of stationarity than the time series generated by the Winner process. In terms of determinism, both showed high similarity.
The body sway is of particular interest regarding the assessment of changes in the postural control, as it is highly related to the risk of falling [13]. In the stabilometry, displacement in the center of pressure (CoP) is recorded in the medial-lateral (x) and the anterio-posterior (y) directions as a statokinesigram (SKG). In the SKGs, variables x and y are regarded to be independent [14]. The linear stochastic differential equation (Brownian motion process) has been proposed as a mathematical model to describe the body sway [15][16][17].
Therefore, the purpose of this study was to establish a new numerical simulation method for biological signals using the same method as the numerical simulation in the exchange rate. We examined whether it is possible to generate a highly accurate simulation for the SKGs by applying the GAN to the SKGs with an unknown mechanism by which 3D images affect the CoP. In addition, if we were able to generate SKGs, we would also verify whether the characteristics of SKGs could be visualized.

Mathematical Model
Stochastic differential equations (SDEs) have been proposed as mathematical models describing the body sway [18][19][20], where μ and ω represent the noise amplitude and the Gaussian white noise, respectively. In Eqs.
where distributions in the x-and y-directions are expressed as ( ) and ( ), respectively. Each distribution is determined as a histogram for each component in the experiment. In some studies, the SDEs have been applied to conduct the numerical analysis of the body sway, however, there is no research using the AI. Using the AI, we perform a numerical simulation in this research.

GAN
In recent years, neural network models, for instance, convolutional neural networks (CNNs) have been used as imageclassification models [21].  In addition, deep-neural-network, which are deeply stacked CNNs, has been used not only for image classification but also for a wide range of applications such as object detection [22], semantic segmentation [23], and the image generation [24]. Two advantages of the GANs use are that it is not necessary to define properties in the images to be generated beforehand, and that a number of images can be generated by using noise as the input to the network. In this study, the GAN was applied to the generator of time series. We can generate time series for the numerical simulation.
In this study, simulative time series (i.e., fake data) can be obtained from the generator G, which is defined as a network input noise. As the side note, the discriminator D distinguishes whether generated data are real (i.e., true data). "G" learns so that "D" determines the fake data as genuine. Inversely, "D" learns so that it can correctly determine whether the input data is true data or false data generated by "G". By repeatedly learning these networks (Figure 1), it can be generated for a large number of simulated time series if the output of "G" can generate data that is very close to true data.

Learning Statokinesigrams in GAN Model
It has been reported that it is difficult to stabilize in the GAN learning. Several models were, in fact, designed and trained to generate pseudo-exchange rates, however, few models could be found to exhibit stable learning. The way has been already suggested how to stabilize the GAN learning [25]. Based on these proposals, the hyperbolic tangent function (tanh) was herein used as an activation function in the output layer of the generator. LeakyReLU was used as an activation function, except for the output layer/fully connected layer in "D" [26]. The hyperbolic tangent function is defined as where the derivative is given as The advantage of using Eq. (4) is firstly that it can output any input value within the range of -1 to 1, thus reducing numerical divergence. Secondly, the hyperbolic tangent function does not have a large maximum value of the differential coefficient, which makes it stop vanishing gradients.

Parameter Optimization
To design a neural network model that achieves the desired performance, it is necessary to adjust the layer configuration to the optimum configuration. In general models for classification and prediction, the accuracy rate is often used as the objective variable [27]. However, because the aim of this study is to simulate the characteristics of SKGs, it cannot be evaluated using the accuracy rate. Therefore, it is necessary to define a new objective variable. In this study, we defined the following optimization function using the learning error for "G" and "D", which was developed in a previous study [28].
where the training error for "G" and the training error for "D" are expressed as LOSS G and LOSS D  respectively. In this study, parameters in the GAN model (see Appendix) were set to minimize the value of the optimization function [29]. The value was evaluated after the parameters were optimized. It should be noted that the parameters in "D" was fixed because of the high computational expense (Table 1). Figures 2 show the a generator model and a discriminator model developed in this study. The "G" uses the combination input of noise and the label. In the subsequent generation process, a branch structure was added so that the variables x and y could be generated independently. This is based on the idea that the variables x and y are considered to be independent in the body sway. The "D", in contrast to the "G", did not have a branched structure. This is because both series are referred to when determining the state of body sway.

Experiment
In this study, we measured the sway of the CoP of the elderly for the GAN learning. Next, pseudo SKGs were generated by the numerical simulation. Finally, SKGs was colored using a GAN discriminator to visualize the characteristics of the sickness.

Measurement
In order to investigate the effects of stereoscopic video clips on the elderly, the body sway was simultaneously measured with the radial motion while/after viewing 2D/3D video clips for 60 s. In this experiment, 204 SKGs were observed from the elderly subjects that stood with Romberg posture on a gravicorder GS3000 (Anima Corp. Ltd., Tokyo) [30]. The SKGs were recorded at 20 Hz sampling in this experiment, which was approved by the Ethics Committee of the Graduate School of Information Science, Nagoya University.  The results have shown that, in the elderly, the equilibrium function is affected while/after tracking the visual target in 3D video clips without statistical significance (sickeness-induced) [30]. Based on the previous studies on visually induced motion sickness (VIMS) [18][19][20], [30], the motion sickness is expected to be induced after/while viewing 3D video clips rather than 2D ones.

Simulation
In the constructed GAN model, it was confirmed that the machine-learning was not stable due to the small amount of raw data. Therefore, independent two-dimensional noise was generated from the Winner process, and one million time series were obtained for each component. Using the weights of the GAN model obtained from this learning process, the model was reconstructed so that GAN simulation that imitated the SKGs of 204 cases in the measurements could be generated (Tables 1-2). At this time, the information is also input to "D" to determine whether the input data is a sickness-induced SKGs or a sicknessfree SKGs in addition to the condition whether the input data is raw data or fake data and determines. A total of 100,000 steps were learned. Thus, 2D/3D video clips are herein regarded as visual stimulus that do not/do induce the VIMS, respectively. SKGs generated from the GAN every 10,000 steps was evaluated by calculating the translation error using the Wayland algorithm [31][32].

Visualization of 3D Sickness Criteria
Using the trained GAN, the data obtained from the subject with the VIMS and the those without sickness were firstly input into the "D", respectively. Secondly, outputs of the intermediate layer close to the output layer were recorded to calculate the contribution rate for the discrimination of the VIMS. Lastly, the input SKGs were a colored with values of the contribution rate.

Results and Discussion
For the simulation SKGs (Figure 3b) generated by the GAN, the translation error was calculated every 10,000 steps using the Wayland algorithm (Figure 4). As a result, the value was approximately 0.7 after the machine learning up to 20,000 steps, and it decreased sharply to approximately 0.3 after 30,000 steps. After that, values of translation error increased to 70,000 steps again, and then decreased in learning after 80,000 steps. The GAN simulation might be influenced by the Winner process used for the pre-learning. That is why values of the translation error was high up to 20,000 steps. After that, the influence of the Winner process would be weakened around 30,000 steps. It was considered that the influence of the SKGs observed up to 70,000 steps gradually increased. After that, mode collapse might occur during the GAN-learning process. That is why values of the translation error decreased again after 80,000 steps. Mode collapse is a problem in which the generated data become similar because the variation of the data generated by "G" becomes smaller. This problem may occur in learning after 80,000 steps. Finally, the SKGs was colored using a GAN discriminator to visualize the characteristics of the sickness (Figures 6). It could be seen that the sickness-induced SKGs were widespread with respect to the sickness-free SKGs. It has been also discovered that GAN captures the shape of the cusp as a factor to determine the sickness. The contribution rate to judge the sickness-induced SKGs has been enhanced by the trajectories of the cusp (Figure 6a). We succeeded in extracting partial figure patterns that the AI regards as a property of 3D sickness. Consistency can be seen in the figure patterns that have been also extracted by our previous mathematical method [33]. Compering with the properties resulting from the mathematical analysis, interpretations can be given for the black box processing in the AI.

Conclusion
In this study, we proposed a method for the numerical analysis of biological data using AI. So far, there have been many studies that have attempted numerical simulations using SDE, but there is no research that performs simulations using GAN and visualizes/extracts the properties of the time series as shapes. As a result, the possibility of SKGs simulation using GAN was suggested. In the next steps, we would like to accumulate fundamental knowledge so that simulations using GAN can be applied in all fields.

Conflict of Interest
The authors declare no conflict of interest.

Appendix GAN model
The generator could be constructed by the following components. Optimum combination was found as Table 2.