Understanding Human Perception of Vibrotactile Feedback in Walking and Running Tasks

Article history: Received: 19 January, 2020 Accepted: 27 March, 2020 Online: 10 April, 2020


Introduction
This paper is an extension of work originally presented in IEEE International Conference on Ubiquitous Robots (URAI 2019) [1].
A key in quantifying and improving locomotion is identification of useful, tractable metrics and providing useful feedback. The most fundamental metric for our study, and arguably of most interest to many affected populations, is an estimate of injury risk. Despite the health benefits, a high number of lower extremity injuries can be linked to running [2,3]. Estimates suggest that 10-20% of Americans run regularly, with 40-50% of these injured annually [4], although causation is more complex, with a survey of results across 17 published studies, involving a range specific population characteristics (age, experience, gender, etc.) showing annual injury rates can vary from 19% to 79% [5]. Among running injuries, 50% of injuries occur at the knee joint and the most common diagnosis is patellofemoral pain (PFP) [3,5]. PFP can lead to severe pain and disability and is a precursor of knee osteoarthritis [6]. Real-time feedback is an effective method for motion and gait retraining. Augmented feedback has been shown to be a useful part in the process of learning a motor task and has been shown to enhance effects of walking and running gait programs [7,8]. Different feedback modalities such as visual, audio, or vibrotactile feedback have been shown to effectively modify knee and impact loading [9]- [13]. Vibrotactile feedback is increasingly becoming an essential feedback component in several non-medical and medical areas [14]. Among vibrotactile feedback, coin vibration motors are chosen for being non-invasive, safe, low cost, small size, and its ability to be effective when placed at almost anywhere on the body [15]- [18]. Vibrotactile feedback can also be a means of delivering cues to a user learning new motor skills [19] or for patients undergoing rehabilitation therapy [20]. Models that are data-driven have been implemented to train gaits involving a combination of different kinematic modifications [21]. One study compared how combining haptic feedback with visual feedback had a greater effect in the learning performance for lower extremity tasks than the feedbacks alone [22]. A different study pointed out that wearable feedbacks in the form of haptic and auditory are commonly used in clinical applications for human gait [23]. Knee and trunk motion are the gait parameters most associated with retraining using wearable feedback [23]. Both visual-auditory feedback and visual-tactile feedback provide advantages in reducing reaction times and improving performance [24]. It has been observed that combining visual feedback with tactile feedback works best when performing multiple tasks that have high cognitive workload conditions [24]. In a study that evaluated assistive navigation systems for the blind, auditory feedback resulted in a 22 times higher cognitive load than haptic feedback [25]. Previous studies have observed that vision feedback provides a high degree of precision [12]. Vibration provides simple and intuitive feedback, particularly when vision is otherwise occupied [21]. In addition, vibration conveys Cartesian space directional cues well. Another benefit of haptic feedback systems is that they can provide feedback to multiple body segments and thus allow modifications of multiple gait parameters [21]. Motor tasks can be guided by using immediate feedback on complex tasks to provide proprioceptive information [14]. Several studies have shown how effective feedback can be in gait retraining and learning a task that can also slow down the progression of joint diseases [9]- [11]. Research still does not provide insight on how feedback needs to be provided and combined in the system to yield better results. It is important to maintain latency as low as possible since it is highly dependent on the motor task and type of vibrotactile stimuli.
To enhance motor learning, visual feedback can be provided in the process of gait retraining. This type of feedback facilitates external focus which has been shown to be beneficial for motor learning due to its ability of promoting automaticity in movement control [26]. Depending on the timing that the feedback is delivered, it can be divided into concurrent (online, real-time) and terminal (at the end) [27]. Concurrent feedback has been reported to enhance learning of complex motor tasks [28] and is especially helpful in early phase of learning [27]. Knowledge of Performance and Knowledge of Results are the two categories of Terminal feedback. While knowledge of performance provides feedback regarding the execution of the task (i.e. joint kinematics and kinetics of gait), knowledge of results pertains information related to the response outcome rather than about the movement (i.e. VO2 consumption during gait) [29]. Terminal feedback has been found to promote motor learning and facilitates motor retention [27]. Despite studies that focus on multi-modal feedback mechanisms on human kinematics, there is a lack of understanding on how feedback interfaces can be used with cyber-physical systems to improve response and engagement. Several studies demonstrate that there is not enough evidence yet to support implementation of tactile feedback to enhance performance [7,16,30].
In addition to augmented feedback systems, there has been much research in developing smart wear with integrated sensors, with applications in different areas. Some recent examples that involve soft suits which can be used in daily activities include the following: (i) A sensing suit using elastic tights and hyper-elastic strain sensors was developed for lower limb biomechanics measurements [31]; (ii) A knee pad and anklet wearable system that contains soft stretchable sensors was developed to track and analyze the joint position of lower limbs [32]; (iii) An IMU integrated shirt, with conductive yarn for circuit patterning and flexible printed circuit boards for interconnections, was designed for posture monitoring [33]; (iv) Leggings with stretch sensors were designed to detect knee valgus [34], commonly observed in athletes at risk of knee injuries; (v) A soft exo-suit, which consists of: a waist belt, leg strap, calf wrap, insole, load cells, gyroscopes, and Bowden cables, was developed to help improve walking in post-stroke patients [35]; (vi) The commercial product by Athos includes a compression shirt and shorts with integrated IMU and EMG sensors, and provides visual information on muscle efforts using a smartphone app (www.liveathos.com); (vii) a portable gait asymmetry rehabilitation system that delivers vibrotactile cues based on gait phase measurement to improve gait symmetry for individuals with stroke [20,36].
It remains unclear how haptic mode (e.g., simultaneous or staggered) or number of motors affects individuals perception of haptic feedback during locomotion. In this study, we present a portable haptic feedback system that uses actuators [37]- [39] to provide the wearer with real-time haptic feedback during locomotion. We designed the system with the wearers comfort during locomotion in mind by making it portable and lightweight. Using this system, we conducted a pilot study to examine subject perceived haptic feedback during different types of haptic mode and numbers of motors during walking and running. Coin-based vibrotactile actuators were chosen and mounted on custom 3D printed cases to avoid impeding the human bodys natural motion. We had 3 main questions in mind when we designed the experiments: (i) How many vibration motors can a subject perceive at once?; (ii) Where on the subjects body should the haptic feedback be provided?; (iii) How effective are the subjects when interpreting higher orders of haptic feedback occurring simultaneously?
Our study is an extension of work originally presented in IEEE International Conference on Ubiquitous Robots (URAI 2019) [1]. Here, we provide a more comprehensive understanding of human interpretation of vibrotactile feedback during walking and running, including the effect of feedback locations. For future work, we plan to train subjects on how interpret haptic cues to increase their performance using this framework and suit by altering key kinematic variables during locomotion.

Vibrotactile Feedback
For the preliminary experiment, we developed a vibrotactile feedback system that consists of the following three subsystems: (1) the hardware, that is comprised of 10 vibrotactile motors (10 x 2.7 mm Coin Mobile Phone Vibration Motor) and 1 Bluetooth-enabled Arduino-Nano microcontroller, (2) an in-house made GUI that sends www.astesj.com 538 Bluetooth commands to the vibrotactile systems microcontroller, and (3) a custom button panel that is placed on a treadmill to record the subjects haptic response (Figure 1). Figure 1: Subject wearing the wireless sensing and feedback system.
Our frameworks GUI is a Windows Presentation Foundation (WPF) application, that uses the 32feet.NET library to discover and connect to the vibrotactile feedback systems Bluetooth module, and a wired serial communication protocol to connect to the custom button panel. When connected to the haptic suit system, the experimenter can use the GUI to manually actuate the 200Hz (slightly below the peak sensitivity of fast-acting mechanoreceptors [40]) haptic suit motors and motor combinations by clicking on the appropriate radio buttons. Figure 2 shows how the motor actuation type (continuous or staggered) and duration of actuation can be manually changed by the experimenter using the GUI. Note that the GUI also provides visual feedback to the experimenter on what motors are being actuated by coloring in the circles that correspond to activated motors on the GUIs body model. When the Start Auto button is pressed, the GUI performs an automatic procedure in which motor actuation commands are executed. Specifically, the sequence and order of actuation modes are based on the selected Subject Number while the time gap between each actuation is randomly chosen at run time to be the subject-specific sequence. While the automatic procedure is running, a separate thread is instantiated and used to collect the subject response data outputted by the custom button panel. The GUI automatically logs: the haptic command events, the subjects response input, and the timestamps at which these events occur, during the automatic procedure of the main process thread. When the automatic procedure finishes, the haptic events and subject response events are combined with the timestamps and then analyzed based on each events unique keywords. The subject response events are grouped with their respective actuator commands (by determining the closest actuator command event to the response event) and, with the use of specific event keywords, compared with their respective actuator commands to quantify the accuracy of the subjects response. For example, the subjects response will be reported as false (zero) if the subjects response is TORSO when the haptic feedback command was activated as LOWER TORSO + L KNEE (Continuous) since the subject failed to perceive the correct feedback. The software then calculates statistical results with the use of the ClosedXML library and saves both statistical and experimental data as an Excel file.

Simulation Framework
The simulation framework was developed to read existing OpenSim models and build them with the Unity game engine environment. OpenSims musculoskeletal properties and definitions are implemented by the framework. This allows the framework to acquire unique behaviors that are only found in OpenSims software. A few unique behaviors include muscle path joints that move during motion, conditional muscle path joints, and application of cubic spline for joint connectivity. The framework takes advantage of Unitys editor window to allow the user, while in edit mode, to upload and adjust dynamic OpenSim models. The simulation framework is capable of symbolic calculus and matrix operations due to implementations of the ALGIB numerical analysis and data-processing library [41], C++ Mathematical Expression Toolkit Library (Ex-prTK) [42], MITs Math.NET Symbolics [43]. At runtime of the framework, the user can symbolically solve both kinematic and muscular Jacobians of an OpenSim multi-body model using the Math.NET Symbolics library. This allows the user to generate the www.astesj.com 539 symbolic expression that can used by the ExprTk library. Due to the high computation expense of symbolic interpretations, dynamic compilation functionality was implemented. Dynamic compilation allows the framework to compile and save symbolic expressions to be used in future runtimes. The simulation framework has the capability to solve complex and dynamic computations during motion tracking due to the implementation of dynamic compilation with Math.Net Symbolics closed-form symbolic computation. The framework can perform these computations at a framerate of 100 FPS or higher depending on the computers specifications. The framework generates generalized coordinates by implementing algorithms to decompose motion of a body [44]- [46] and can act as an interface for generic motion tracking. To evaluate the motion tracking interfaces performance, a Perception Neuron full-body IMU suit was integrated into the framework. As expected, Noitom already provides the software (Axis Neuron) and the rigged humanoid model required to use the IMU motion capture system within Unity. As designed, the only task that must be completed to use the IMU motion capture system with our framework is to connect the individual limbs of the humanoid model to the appropriate JointTracker elements. The ground reaction forces are estimated by our framework using the motion data from IMU and the subject-specific mass matrix. The simulation framework provides haptic feedback commands to an arduino nano microcontroller that is bluetooth enabled. The simulation framework connects to the microcontroller through a haptic feedback interface script that connects to the Bluetooths communication port at 9600 baud rate (Figure 3). Once connected, the simulation runs an experimental procedure script that tracks different kinematic metrics and sends vibration commands in the form of strings to the microcontroller.
Effective indicators of joint loading that can be used in injury prevention and sports performance are joint moments. Progression of patellofemoral joint (PFJ) osteoarthritis can be related to flexion moment impulse and peak knee flexion [47]. High PFJ reaction force and stress has been associated to increased knee flexion moment that suggests requirements for greater quadriceps force [48,49]. Our framework estimates the 3-dimensional joint moments in real-time using the inverse dynamics algorithm. Both the kinematic and kinetic values estimated by our framework were validated against the data reported in the literature [47]. The novel simulation framework and the motion/force data will be used in our future study to estimate joint contact forces and other performance metrics (i.e. metabolic cost) during locomotion.

Experiments
This section presents the experimental evaluation, including experimental setup, subjects and data collection, and statistical analysis.

Subjects
Eighteen subjects (10 males, 8 females, age: 22.5 ± 3.0 years, BMI: 22.6 ± 3.6 kg/m 2 ) participated in the study. All subjects reported no pain or symptoms while walking or running, nor history of surgery or sensory deficit. To have scientifically correct subject-specific scaling for the modeling purposes in our future work, we did not include children or the elderly. Inclusion criteria for subject recruitment are the following: (1) between ages of 18 and 40 years old; (2) familiar with running on a treadmill; and (3) run at least 8 miles/week for 4 weeks prior to participation. Exclusion criteria for subject recruitment are the following: (1)History of lower extremity or low back surgery that may affect running kinematics, muscle activation, or kinetics; (2) low back or lower extremity discomfort or pain during the data collection or within 3 months prior of the experiment; and (3) any mental or physical condition that prevents www.astesj.com 540 the subject from performing the exercises or tasks safely. Medical clearance and written informed consent forms approved by the California State University, Long Beach Human Subjects Review Board (IRB : 1300259-2) were obtained from the participants.

Experimental Setup
We used the same experimental setup detailed in our previous work [1]. The experiment consisted of 4 trials that got divided into walking and running. Subjects were tasked to perform two walking and two running trials on a treadmill at self-selected comfortable speed (walking: 1.1 ± 0.2 m/s, running: 2.2 ± 0.2 m/s). Each trial lasted between 12 to 14 minutes. During each trial, subjects experienced vibrations by motors placed over 10 locations of his/her body bony landmarks: upper and lower back, bilateral posterior superior iliac spines, bilateral medial and lateral femoral epicondyles, and bilateral lateral malleoli. Two types of vibrations were implemented, continuous and staggered. During continuous vibration, the combinations of motors vibrated at the same time. During staggered vibrations, the motors vibrated in a sequential manner for the combination of motors. A 0.5 second delay was programmed between each motor during the vibration sequence. The vibration commands included up to a combination of four motors in a pre-determined random order. Subjects were tasked to verbally report the number and location of the vibration motors they perceived during the experiment. The subject was also tasked to report the vibration locations using the button panel on the treadmill. Subjects were informed if the trial was continuous or staggered with verbal instructions and an LED light on the button panel.

Data Collection
Feedback perception accuracy (FPA), which is defined as the percent accuracy of detecting correct number and location of vibration motors, was calculated for each combination. The average reaction time for each subject was collected during each trial. Subjects comfort level of wearing the system during locomotion was immediately reported after each experiment using a ten-point bipolar Likert-type scale.

Statistical Analysis
Two-way repeated measures ANOVA was used to evaluate effects of number of motors (i.e., 2, 3, and 4) and vibration modes (i.e., continuous and staggered) on FPA during walking and running. The FPA (%) between the first and second trial for walking and running and for the different vibration modes (staggered or continuous) was compared using paired-sample t-tests. Separate t-tests were performed independently for male (M) and female (F) subjects. An alpha of 0.05 was set for the statistical significance level.

Results
Overall, FPAs were greater than 50% in all motor and activity conditions in staggered mode. On the other hand, FPAs reached above 50% only during two motor condition in continuous mode. Results of ANOVAs showed significant number-of-motor, vibration-mode, and interaction effects (p < 0.001) for both walking and running. FPAs reduced as the number of motors increased. Compared to continuous mode, staggered mode yields higher FPAs, especially when the number of motors increased. When examining staggered or continuous mode alone, FPAs were significantly higher during walking than running (p = 0.04 and 0.01, respectively) with a mean difference of 8%.  Table 1 shows the respective FPA p-values, averages, and standard deviations for trial 1 and trial 2 during walking and running. The FPA means for walking and running, respectively, showed an increase of 2.23% and 3.12% that suggests effective learning for both types of locomotion.  Table 2 shows the respective FPA p-values, averages, and standard deviations for males and females during walking and running. A decrease of 12.60% in the standard deviation from walking to running was observed among the male population. An increase of 22.70% in the standard deviation from walking to running was observed among the female population.  Table 3 shows the respective FPA p-values, averages, and standard deviations for continuous (C) vibrations and staggered (S) vibrations during walking and running. A decrease of 9.95% in the standard deviation from walking to running was observed within the continuous vibration mode. An increase of 7.49% in the standard deviation from walking to running was observed within the staggered vibration mode. Figure 6 shows the FPA averages for 1, 2, 3, and 4 motors. Most notably, 1 motor yielded an FPA of 92.78 for walking.
The overall FPA decreased when the subjects were asked to verbally identify the location of the vibrotactile devices they felt. The average decrease for detecting one motor location was 2.98% during www.astesj.com 541   www.astesj.com 542 walking and 2.46% during running (see Figure 7 (left)). For staggered vibration mode, there was an average of 7.45% and 10.39% decrease in FPA of the correct location of vibrotactile motors during walking and running, respectively. For continuous vibration mode, the decrease was more significant: 9.39% and 11.08% decrease in FPA of the correct location of vibrotactile motors during walking and running, respectively (see Figure 7 (right)). The results from Figure 5 (left and middle) suggest that staggered vibrations are easier to perceive by humans. This can be observed by comparing the loss in FPA as the number of simultaneous stimuli increases. Figure 5 (left and middle) show a greater loss (slope) for continuous vibrations as the number of motors is increased compared to staggered, which has less of a negative slope. This may suggest that cognitive load is less when trying to identify vibrations that are staggered throughout body locations compared to simultaneous continuous vibrations. This shows promise when trying to maintain low cognitive load for motor and gait training. Figure 6 (right) and Figure 7 (right) both reinforce that staggered is much more suitable when you want a person to perceive total number of stimulation and accurately distinguish among different locations. Another possible reason that staggered is a more suitable form of haptic feedback for motor and gait training is because it has been observed that continuous vibration on the same location may lead to numbness of that section of the skin [50]. Meanwhile, staggered vibrations provide the user with rest periods that may reduce the numbness associated to constant stimulation. As mentioned in the experiment section, subjects reported an average comfort level score of 8.4 using the bipolar Likert-type with a ten-point scale. Subjects average reaction times can be observed in Figure 5 (right).

Discussion
For this paper, we explored human perception of vibrotactile feedback using a portable system that provides vibrations during walking and running tasks. Findings from our previous study gave insight of how vibrotactile feedback is perceived by humans during locomotion. This study showed that staggered vibrotactile feedback would yield greater subject perceived accuracy during locomotion especially when more than two motors were used. This study provides valuable insights into the feasibility of vibration mode and number of motors during locomotion. Information obtained from this study can be used to improve the haptic feedback systems design and experimental protocol.
This studys results set a foundation to develop multi-modal feedback systems to improve performance during walking and running. We plan to incorporate the simulation framework into the multi-modal system to provide different feedback modalities for coordinated motions. Previous studies have observed that vision feedback provides a high degree of precision [12]. Vibration provides simple and intuitive feedback, particularly when vision is otherwise occupied [21]. In addition, vibration conveys Cartesian space directional cues well. By integrating visual, audio, and haptic feedback, we will combine their advantages to provide rich and intuitive feedback to improve human performance and decrease the risk of injuries during locomotion.

Conflict of Interest
The authors declare no conflict of interest.