Performance Evaluation of a Gamified Physical Rehabilitation Balance Platform through System Usability and Intrinsic Motivation Metrics

Article history: Received: 09 December, 2020 Accepted: 16 February, 2021 Online: 25 February, 2021 Motivation significantly influences the outcome in the rehabilitation of patients. Several developments have been made to assess and increase patient motivation by addressing factors linked to motivation such as the personality of the patient, professional administering rehabilitation, and the rehabilitation environment. The main objective of the study is to evaluate the reliability of a gamified environment for the rehabilitation of stroke patients by testing its functionalities within standard physical therapy time and intervals. To achieve this, calibration was characterized. Also, user feedback was taken in the form of questionnaires based on the System usability scale (SUS) and Intrinsic Motivation Inventory (IMI). Based on the SUS scale, results show that the game manipulability is good, the game concept and design is satisfactory, and the game comprehensibility is also good based on the qualitative conclusion per SUS score. For the IMI ratings, it was found out that the highest rating was the perceived choice which indicates their voluntary participation in the game. Some improvements can still be added to the game itself to increase the motivation of patients. The balance board manipulability and the recalibration time interval can be further improved for comfort and ease of use by the patients.


Introduction
Balance deficiency is one of the common issues for poststroke and post-injury patients, as well as for individuals who wish to train his/her balance even without deficits (e.g. athletes). Out-patients who had such deficiency will undergo a balance therapy program. In the current setting, a patient will attend therapy sessions about six to ten times. Each session, the patient will perform various balance exercises such as static balance, dynamic balance, and manual perturbation exercises using balance board, wobble board, balance rails, and mirrors extensively assisted by physical therapists. The patient will then perform the exercise, but most of the time, the patient will feel the pain of such exercises as well as boredom due to its repetitive nature. Also, the therapist will record the patient's balance quality according to what is observed, making several corrections to the number of balance sessions as well as the physician's initial observation. This also introduces an inconsistency problem when a different therapist observes the patient and records the score for interpretation. Finally, the patient has to rely on his/her safety on the therapist, since the current tools do not have robust feedback for the patient to observe his/her balance performance. With these, the patient is usually discouraged and unmotivated to attend another therapy session as well as not trying to perform them by himself/herself.
In line with this, one of the determinants that greatly affect the success rate of undergoing physical rehabilitation is the motivation level of the patient. Motivation has been identified to be influenced by a variety of factors such as the personality of the patient, the professional administering rehabilitation, and the rehabilitation environment. Research work integrating the use of the Microsoft Kinector the Wii Fit Board with various games to change the rehabilitation environment has proven to be effective in increasing patient motivation and overall affecting the outcomes on a more positive result [1,2]. However, improvements on these previous researches can still be made, such as creating a more affordable and open-sourced design of the hardware and incorporating more game elements that engage the patient which will aid in the success rate of the physical rehabilitation.
Balance-on-action-Team (BOAT) implements affordable hardware and open-source software to enable a wider reach in the utilization of gamification in physical rehabilitation. It is an automated balance board medical device with a gamified element for conducting balance exercises for patients undergoing balance rehabilitation therapy. The overall goal is to provide rehabilitation centers, sports clinics, and personal trainers with safer, faster, and more engaging means of conducting balance training and therapy. In this way, balance exercises in the Philippines setting will become faster, more consistent from therapist-to-therapist, and more fun to perform.
The main objective of the study is to evaluate the reliability of the equipment (calibration and functionalities) based on the standard physical therapy time and intervals. Furthermore, conduct surveys to evaluate the game's impact on motivation. This paper describes the various studies related to rehabilitation and patient motivation in Section 2 of the Review of Related Literature. Section 3 discusses the Methodology which focuses on the hardware and software implementation in Section 3.1 and the testing procedure in Section 3.2, using different metrics. The results are shown and discussed in Section 4 which includes the calibration in Section 4.1, Game Self Evaluation in Section 4.2, Motivation Analysis in Section 4.3, and Game Walkthrough Evaluation in Section 4.4. Lastly, the Conclusion and Recommendation can be found in Section 5.

Review of Related Literature
Several studies have been conducted on the rehabilitation of stroke patients using varying platforms, approaches, and frameworks. A similar balancing platform was studied by [3] where the focus is on utilizing the Nintendo Wii Balance Board for rehabilitation. The study developed a WeHab system that added visual biofeedback based on the center of pressure location. This system also allows for multiple balanced boards to be used together, customization of activities and difficulty level, and integration with a webcam to capture video footage during the sessions. Similarly, visual biofeedback was also used by [4] using Microsoft Kinect for postural rehabilitation. This allows the system to determine if the patient performed the correct postural exercises through the built-in capabilities of the Kinect in movement and gesture recognition. Another type of balancing platform that can be used is a wobble board which was studied by [5]. Here, the instrumented wobble board is used to create multidirectional perturbations and obtain vibrotactile feedback for training dynamic sitting balance. This research work also uses a microprocessor and an inertial measurement unit but has an added eight vibrating tractors. Aside from these approaches, the rehabilitation of patients can also be monitored using EMG on the lower limbs, such as in the work of [6]. In this case, a balance board was also used but accompanied with obtaining sEMGs signals to analyze how the muscle activity reacts from dynamic leaning caused by balance reactions.
Assessing patient motivation and system usability have been determined to help improve rehabilitation for chronic stroke patients. Instead of balance exercises used in this research, the study by [7] evaluates the feasibility of a new technologysupported task-oriented arm training regime (T-TOAT) for chronic stroke patients. The system is comprised of movement tracking sensors, an exercise board, and a software-based toolkit used for skills training. The patient motivation was assessed on the Health Care Self Determination Questionnaire (HCSDT) based on the self-determination theory. Furthermore, system usability was also assessed and was found to be rated good by the users. After performing various tests, it was discovered that the T-TOAT approach improved arm-hand performance significantly for the duration of the post-training. Similar to this work, system usability and motivation were evaluated through various questionnaires and assessment tools for improving rehabilitation.
Another patient motivation assessment tool applied in this work is the Intrinsic Motivation metrics which was also conducted in a study by [8]. This study aims to characterize motivation based on the information of activities for smart wearable health equipment applications. The participants in the study were tasked to solve a computer-generated puzzle under one of the four random conditions including the long-term feedback (LFB), long term graphical feedback (LGFB), short term feedback (SFB), and non-feedback for the trend of correct answers. Intrinsic motivation was then measured using both a 12-item version of the Intrinsic Motivation Inventory and the time it takes the participant to solve the puzzle. Based on these, it was observed that the most desirable type of feedback is one that improves perceived competence, which can then be adapted to health equipment displaying methods.
The BOAT is a research project that was conceptualized and developed through the collaboration between the Ateneo de Manila University and the Philippine Orthopedic Center. The gamified environment was initially developed by [9] using an inertial measurement unit. The goal was to provide the gamified elements in the form of quests and rewards system for providing an interactive and fun rehabilitation process. The hardware prototype was developed and tested in [10] using intervention techniques, motivation, and game design analysis. This was then continued by [11] through a comparative game design analysis. Several changes have been made for the hardware design, shown in Figure 1, and software game mechanics implementation by [12]. The research work added more customizability, and personalized information from the patients to help the physician perform better assessments. This research focuses on evaluating the gamified rehabilitation platform of [12] through various metrics such as system usability and intrinsic motivation metrics.

Design of the System
For the software, the gamified environment of the balance board lets the user experience a seamlessly integrated, open-world seafaring game integrated with a quest system [12]. The quest system allows the patient to perform physical rehabilitation exercises under the guise of the game. There are three quests or game modes designed to correspond with different physical rehabilitation exercises and specific metrics from the Tinetti Balance Assessment Tool and the Equilibrium Score. Each game mode and its corresponding mechanics are shown in Table 1. Table 1: Game Modes and Mechanics [12] Game Mode

Mechanics
Player Input

Collect the Crates
The player must be able to collect as much trash (in the form of crates) as possible within a certain amount of time Lateral Dynamic Balance Exercise.
The player must be able to challenge himself/herself a postural sway and recover from it as much as possible using lateral movements. Avoid the Bombs The player must be able to avoid all possible bombs within the boat's path in a certain amount of time Static Balance Exercise.
Players must be able to stand still as much as possible to avoid the bombs. Follow the Light The player must be able to follow the light in front of the boat without getting too near or too far from it Posterior-Anterior Dynamic Balance Exercise.
The player must be able to challenge himself/herself a postural sway and recover from it as much as possible using posterior-anterior movements.
The hardware design includes the following components: the frame, balance board, accelerometer, and Gizduino Uno. Most of the hardware components were mounted on the frame which features adjustable mechanisms and can be disassembled. The balance board is commonly used in physical rehabilitation for static and dynamic exercises, especially in the Philippine Orthopedic Center. There are two types of balance boards, the two-directional balance board which was used in this study, and the multi-dimensional balance board or wobble board. Relatively, the bidirectional balance board provides more accurate data from the accelerometer readings since only two directions are measured and the rest are disregarded. Several adjustable components are used to cater to different characteristics and categories of patients. One of these is the stopper which adjusts the maximum tilting angle of the balance board for both sides with increments of 5°. This allows the physician to change the maximum angle depending on the patient's capabilities. Furthermore, this helps in stabilizing the platform at 0° before running the application. Also, the handrails where the patient will hold on to when playing the game can be raised or lowered to compensate for patients with different heights. Each consecutive hole for moving the handrail is about 2 inches apart. Lastly, the friction of the balance board can be altered using the tension control knobs with a range of 1 to 8 for each side. Adjusting the magnitude of the sway of the board will help more balance-challenged patients gradually adapt to the game mechanics.
As a safety precaution, the patient is advised to wear a harness that is attached to an adjustable lanyard, presented in Figure 2. The lanyard is then mounted on the hooks of a metal bar which can be rotated 360° along the vertical axis to allow a smoother transition for the dynamic posterior-anterior to lateral sway and vice versa. Moreover, wearing the harness greatly minimizes the risk of falling from the balance board. For the improvements tracking, the calibration of the equipment is accomplished through linear regression using two data points from the sensor readings. From the generated equation in slope-intercept form, the slope and y-intercept will be saved in a .txt file inside the Results folder on the desktop. This will enable the user to see any variations in calibration.
Similarly, the game metrics are recorded in the form of a .txt format in the Results folder. It publishes the patient's name, the date and time when the exercise is completed, the quest played, the type of balance exercise, their in-game score, the Equilibrium Score (ES), the maximum posterior-anterior sway, maximum lateral sway, and game settings information. This information allows the physician to track and evaluate the progress of the patients after engaging in the physical rehabilitation procedure.
Data from each game mode are recorded in the form of the ES, shown in (1), which indicates their quality-of-balance progress as patients play the game. The ES is the probability of fall in which a score of 100 indicates a 0% chance of fall while a score of 0 indicates a 100% chance of fall.
where is the maximum postural sway relative to the vertical is the minimum postural sway relative to the vertical (natural standing angle) is the limits of stability angle for the clinical setting

Testing the System
To fully test the reliability of the system, several methodologies were adapted based on metrics related to the study, which includes the following: An SUS (System Usability Scale) is a self-rated scoring system that measures the usability of a certain device or software. It is usually a ten-item test that has open-ended statements with user responses from "Strongly Disagree" to "Strongly Agree". Each item is scored on a 5 (or 7) point scale. To prevent purely positive or purely negative responses, this questionnaire is designed to have alternate positive-worded and negative-worded statements. Thus, the following rules for computing the SUS are the following: • For every negative statement (odd-numbered items for this test): take the max score (e.g. 7) and subtract it by the raw score (e.g. raw score = 3, SUS score = 7 -3 = 4) • For every positive statement (even-numbered items for this test): take the raw score then subtract by 1 (e.g. raw score = 6, SUS score = 6 -1 = 5) • Add all of the SUS scores of every item. The maximum score should be: (max points -1) * max items (2) • Depending on the maximum number of items, the overall SUS score should be on a scale of 0 to 100. Therefore, for a 10-item questionnaire with 7 as maximum points, the entire sum should be divided by 0.6 since the maximum score is 60. • The overall SUS score is then tallied and averaged.
Normalization is considered for the average therefore a score of 68 and above is a favorable result, while a score of below 68 is an unfavorable result. Figure 3 shows how the scores will be interpreted qualitatively in the conclusion. The Intrinsic Motivational Inventory (IMI) measures the motivation of the test subjects in playing the game. All 22 questions are modified from the template provided by the Self-Determination Theory Organization but have the same measurement (http://www.selfdeterminationtheory.org). For determining the IMI Score of the test subjects, the following procedures are done.
That is, the item score is equal to 8 minus the response (item score = 8response). • These subscale scores in Table 2 are added together:

Results and Discussion
The reliability of the prototype was evaluated to assess the accuracy and precision of the components based on the calibration values before and after using the balance board. The test was also conducted with twenty (20) healthy test subjects consisting of 10 males and 10 females within an age range of 20-30 years old. Each subject has no history of stroke and no injury 6 months prior.

Calibration
Calibration evaluates and adjusts the precision and accuracy of the balance board. In calibrating, the balance board was referenced at +20 and -20 degrees, and sensor values of the MPU 6050 were then recorded on these angles to calculate the slope.
For the reliability of the balance board based on calibration, the slope was recorded before and after the game based on the following parameters: 10min (Game)-15min (Rest), 15min (Game)-20min (Rest), varying tension control, and varying weight of the test subjects.  Figure 4 shows the percentage error of the balance board slope as time increases. The percentage error was computed using this equation Error (%)= 2 |x-y| x+y *100 (4) where x is the initial slope while y is the value of the slope after some time. With 3.5% as the maximum percentage error (4) at around 105 minutes, it is safe to say the calibration of the prototype is still precise after turning it on for around 1 hour and 30 min.
The slope of the balance board was also recorded as the tension control adjusts from 1(low friction) to 8(high friction). The correlation between the percentage error of the slope and the tension control is shown in Figure 5. Since the maximum percentage error is around 0.8%, it shows that the calibration value is still precise even if the tension control is varied.  Figure 6 shows the percentage error of the slope before and after doing the game testing as the weight (kg) of the test subject varies. Based on the figure, we can say that there is no correlation between the weight of the person doing the test and the slope percentage error. It also shows that the person's weight doesn't affect the calibration of the balance board with a maximum error of around 1.5%.

Game Self Evaluation
Game Self Evaluation is an SUS questionnaire to be filled out by the test subjects for their evaluation of both the hardware and the game. It is divided into four groups: Balance Board Manipulability Measures, Game Manipulability Measures, Game Concept and Design Measures, and Game Comprehensibility Measures.
The Balance Board Manipulability Measures is a self-rated survey on how the balance board is used to them. This is collated and calculated based on SUS. For Balance Board Manipulability Measures, the average SUS for all 20 test subjects is 64.06, which is 5.89% lower than the average SUS score of 68. Overall, the result is unsatisfactory and the balance board manipulability measures can be improved.
The lowest SUS is 35.42 while the highest is 93.75. The highest-rated item is #1 (I thought the board was too high for me to use) with an average SUS of 5 (0 being the lowest and 6 being the highest). This is because the handle on the sides of the balance board can be adjusted depending on the height of the test subject. The lowest rated item is #8 (I felt it did not need any much muscle effort in using this platform) with an average SUS of 2.55. This is because the platform was tested for healthy patients while the level of difficulty of the game was intended for post-stroke and post-injury patients. Improvements can be made by adding a motor on the balance board to assist the test subjects when they feel tired or when they executed too much muscle effort.
The Game Manipulability Measures part of the evaluation is based on how playable the game is for the test subjects. The results are also collated and calculated on an SUS basis. The average SUS for all 20 test subjects on Game Manipulability Measures is around 75.94%, which is 11.67% higher than the average SUS score of 68. Based on the results, we can say that game manipulability is good.
The lowest SUS is 47.92 while the highest is 100. Item #8 (I thought the game was simple and uncomplicated) is the highest rated item with an average SUS rate of 5.65. The lowest rated item is #7 (My legs got tired very easily when playing the game) with an average SUS rate of 3.6.
The Game Concept and Design Measures is a self-rated survey on how enjoyable and how appropriate the game is for balance board rehabilitation. The results are collated and calculated on an SUS basis. The average SUS for all 20 test subjects on Game Concept and Design Measures is around 70.64%, which is 3.88% higher than the average SUS score of 68. Based on the results, we can say that the game's concept and design are satisfactory.
The lowest SUS is 51.28 while the highest is 96.15. Item #1 (I thought the game was too hard for me) is the highest rated item with an average SUS rate of 5.7. This proves that the game is easy to play and understand by the test subjects. The lowest rated item is #3 (I got bored at this game) with an average SUS rate of 2.75. This might be because the gameplay was a bit easy for healthy test subjects but it might be challenging for post-stroke and postinjury patients.
The Game Comprehensibility Measures is a self-rated survey on how easy and understandable the instructions of the game when playing, which is collated and calculated based on SUS. Based on the results of 20 test subjects, the average SUS is around 80.42, which is 18.26% higher than the average SUS of 68. This shows that the game comprehensibility measures are on a good level based on the qualitative conclusion per SUS score.
The lowest SUS is 63.89 while the highest is 100. Item #2 (I thought the amount of information to follow this game was enough) is the highest rated item with an average SUS rate of 5.39. The lowest rated item is #11 (The sound was too loud or too soft, or had an annoying burst of sound) with an average SUS rate of 4.17, which is still above the average possible score of 3.

Motivation Analysis
In this project, all twenty (20) test subjects' responses had been collated and analyzed based on the IMI subscales. Table 3 shows the IMI results while Figure 7 shows a graph of the IMI rate in each IMI subgroup.  (Figure 7). This indicates that overall, the test subjects have good motivation in performing the balance exercises.
On all the IMI subscales as shown in Figure 8, the perceived choice is their highest response rate with 86.57%, which indicates that they rated their voluntary participation and continued participation of playing the game. It is followed by perceived competence with 84.86%. This means that the test subjects are skilled and they felt competent in playing the game. Interest/enjoyment which is another motivation subscale is also rated high with 71.94%. The graph also shows that the test subjects didn't feel any pressure/tense and they feel relaxed while playing the game since the Pressure/Tension subscale only has a rate of 35.71% and is below the average rate of 50%. Figure 8: IMI Bar Graph Game Walkthrough Evaluation is a self-rated survey consisting of short answers on what the test subjects thought of the game. Each item is analyzed and compared with other metrics to find out which part of the game is easy or hard, and what would be the difficulty in using the game. Figure 9 shows the feedback of the test subjects while playing the game. The figure shows that most of the test subjects enjoyed while experimenting. They considered "Collect the crates" as their favorite part of the game while Follow the light was the most disliked one. Also, "Collect the crates" was the easiest part of the game while "Avoid the bombs" and "Follow the lights" tied to be the hardest part of the game. Additionally, the participants considered "Follow the lights" as the most tiring game, and difficult to control. The game is scored 7.58 over 10 (shown in Figure 9 (h)), with stability being the strongest feature (9) and rewards being its weakest feature (6.2). The participants also noticed some bugs/errors while playing like delayed movement on the projector screen, squeaking noise during the game, slow pace on the levels, monotonous gameplay, small text on the screen, etc. These need to be improved to increase overall game satisfaction.

Conclusion
The gamified environment and balancing platform were successfully tested on twenty individuals in compliance with the standard physical therapy time and procedures. Based on tests with varying tensions and weights, the equipment was observed to require recalibration every 1 hr and 30 minutes. The users also gave feedback based on a rating from the System usability scale (SUS) and IMI (Intrinsic Motivation Inventory) with regards to the different aspects of the game and equipment. The overall evaluation of the game got a score of 7.58 with stability as the strongest feature and rewards as the lowest one.
Further improvements can be made on the overall system such as the need for technical adjustments in the current build such as bugs, glitches, and sounds, adding a "Congratulations" instead of "Game Over" to increase motivation, integrating more natural special effects (e.g. fish, seagulls, rain, etc.) and also, make the subject immersed in a relaxing-natural coastal area where the endpoint is a docking area/lighthouse. Lastly, testing the system with post-stroke and post-injury patients of different age groups, height, and body mass index (BMI) instead of healthy individuals would yield a more accurate and better reliability evaluation of the platform.

Conflict of Interest
The authors declare no conflict of interest.