1. Introduction

The wireless displacement estimation system in this article is introduced in order to locate the position of user moving along a path indoor and carrying an wearable device only. Such pre-installed devices as fiducial markers on floor [1], infrared LED landmarks on panel [2], wireless communication devices like radio frequency and wireless local area network [3–5] are not required.

The applications of this system are diverse. High-demand areas of application can be listed including defense applications of tactical planning or strategic coordination for soldiers; social life applications for public safety and elderly care; and firefighter rescue applications when they’re stuck and in need of rescue. The common characteristic of the aforementioned applications is the indoor environment with no pre-installed infractructures or devices due to such infractructures as RFID, wireless network, infrared sensor, landmark for vision are unable to installed or have already been destroyed by disaster. Specifically, rescue and security applications often take place in unstable areas that are unsuitable for infrastructure-based systems.

In general, two types of non-infrastructure displacement estimation systems exist and named as the strap-down and the step-and-heading. The strapdown method is integrating the acceleration in order to get displacement. Accumulated errors in the integration process are compensated with reset algorithm in [6]. The step-and-heading method estimates displacement by combining the step size with the yaw (user heading) which is estimated by quaternion-based filters [7–8].

The infrastructure-free displacement estimation system utilizes sensors such as IMU, force sensor, pressure sensor (barometer), ultra-wide band (UWB), etc. Depending on the specific application needs, the sensors are selected and their positions on user’s body can be mentioned as holding in hand [9–10], mounting on foot [11–12], putting at ankle [13], placing at waist [14–16] and attaching at upper torso [17].

All mentioned approaches estimate displacement in two horizontal dimensions. However, Diaz [18] proposed an estimator with IMU put in pocket in order to estimate displacement in the vertical dimension. The inconvenient IMU location for user and the required training are the limitations of Diaz algorithm. Therefore, this article proposes a novel method that doesn’t require training but provide a displacement estimation algorithm for indoor three dimensions (3D), which is able to estimate both the horizontal displacement and vertical displacement. The proposed algorithm is based on the double integration of acceleration with reset at step events in order to eliminate accumulate error due to integrating process. The proposed method is tested when a person climbing/descending staircases.

In this article, the IMU is placed at pedestrian’s waist as shown in Figure 1 for convenient usage.

Figure 1:  IMU is mounted at pedestrian’s waist and used to estimate 3D displacements.

The rest of the article is organized as follows. The proposed system  and algorithm to estimate 3D displacements are introduced in section 2. In section 3, the experiment setting is described and the experimental results are shown. Section 4 concludes the article.

2. Proposed Wireless Displacement Estimation System

2.1. System Overview

The proposed wireless displacement system employed the integrating process with novel velocity update algorithm in [6] in order to convert measured acceleration into displacement. Moreover, the system ultilized IMU module MPU9250 for sensing, Xbee devices for wireless communication and Arduino Fio Board for programming the proposed algorithm. The system was implemented within space as in Figure 1 and various experimental scenarios were designed for testing and evaulating as well. People took part in experiments were tasked to wear mobile device at waist, perform multi-activities freely with normal speed. Experiment data was recorded wirelessly in distance of several hundred meters.

2.2. Attitude Estimation System

In order to make the theoretical basis for all math operations in this article, all coordinate systems (or frames) must be assigned and two frames are assigned as illustrated in Figure 1. The subscript notations of W and B indicate for World and Body, respectively. The East-North-Up (ENU) coordinate system is used as the world reference frame (X_WY_WZ_W) while the body coordinate system is assumed to rigidly attached to the IMU and the IMU is mounted at the pedestrian’s waist.

Let ψˆ, θˆ and φˆ be the user heading, IMU pitch and roll angles, respectively. An attitude in three dimensions is estimated based on a quaternion-based Kalman Filter in indirect form by fusing the information from the gyroscope, the accelerometer and the magnetometer.

The employed attitude estimation algorithm has two advantages as follows: 1) reducing computation cost due to dealing with orientation errors instead of dealing direct with orientation which results the state dimension being smaller  and its response faster, and 2) two stages update  using acceleration and magnetic strength by exploiting two measurements of the difference between accelerometer reading vs the gravity vector and the difference between magnetometer reading vs the magnetic strength vector.

2.3. Classification of Motion Modes

In order to improve displacement estimation accuracy, the motion modes of user carrying IMU must be recognized precisely [19]. Some motion modes including jogging, running, walking with different step sizes have already been done in our previous works [13, 17, 20]. In this article, motion modes of stationary, going up (climbing), going down (descending) staircases and riding a lift are added to update our previous work and enhance our system capabilities. Therefore, the motion modes are organized and indicated by numbers as follows: ’0’ indicates stationary mode; ’1’ indicates descending staircases mode; ’2’ indicates walking on flat surface mode; ’3’ indicates climbing staircases mode; ’4’ indicates riding a lift mode; ’5’ indicates jogging/running mode.

There are many motion modes introduced in [20] and were classified by machine learning based on four decision trees. The device are able to be put in many places including handheld with both portrait and landscape orientations; backpack; purse; belt holder; chest strapped; arm strapped hand still by side; dangling; pocket (consists  of pant pocket, shirt pocket, and jacket pocket, in both horizontal and vertical orientations); wrist (ex: smartwatches) and headmount (such as smartglasses).

The algorithm in [20] is a very convenient but it is complicated and time consuming for learning method. Therefore, a simpler method for classification of motion modes is proposed in this article. The entire classification process, as shown Figure 2, consists of: 1) the acceleration in world frame is normalized and compared with a threshold in order to categorize motion modes into stationary, non-stationary and riding a lift modes, and 2) the non-stationary mode is classified into jogging/running, walking, climbing and descending staircases.

Figure 2:  Motion Modes recognized by our work.  The notation a_W is the acceleration in world frame and the  is its vertical component. All the Th_acc, Th_{acc, run/jog}, θ_{th, low} and θ_{th, up} are thresholds for motion modes classification.

Figure 3:  The blue solid line shows pitch angle estimates whereas the red one is the filtered pitch and the black solid line shows the Motion Modes, with values ’0’, ’1’, ’2’, ’3’ represents modes of stationary, descending staircases, walking and climbing  up staircases,  respectively.

2.3.1. Classification for Non-Stationary mode group:

The mode group of non-stationary consists of walking, jogging/running, climbing/descending staircases. Among them, the climbing/descending staircases motion modes are mentioned in this articles. Basically, classification of climbing/descending staircases is based on the pitch angle thresholds. Firstly, the original pitch estimates (depicted by the blue solid line in Figure 3) are passed through a Median Filter with a 100 samples window before doing a comparison with the pitch angle thresholds. The filtered pitch estimates are depicted by the red solid line in Figure 3. In order to illustrate the feasibility of  the classification algorithm, an experiment that a pedestrian performs the following motion  modes: stationary, walking, climbing up and descending down staircases with normal step sizes, moving at speed two steps per second, going along with a straight path is conducted and the pitch estimates are shown in Figure 3.

2.3.2. Classification for Riding Lift mode

In order to recognize whether a pedestrian riding a lift or not, the surface acceleration and the absolute of vertical acceleration are compared with a threshold. A pedestrian activity is classified into riding a lift mode if the condition shown in Figure 3 is satisfied. A person is tasked to perform a riding lift activity combined with walking then descending staircases in an experiment. Figure 4 proves the feasibility of classification riding a lift mode.

Figure 4:  The blue solid line shows vertical  acceleration  with gravity removal whereas the black solid line shows the Motion Modes,  with values ’0’, ’1’, ’2’,  and ’4’ represents modes of stationary, descending staircases, walking and riding a lift, respectively.

Figure 5:  Overview of the proposed in-door localization system. Values ’1’, ’2’, ’3’, ’4’, ’5’ represents modes of descending staircases, walking and climbing up staircases, riding a lift and jogging/running, respectively.

2.4. Displacement Estimation Method

Authors’ previous works [13, 17] described the pedestrian’s walking motion and proposed two walking models applied to estimate displacement in two dimensions with the accuracy up to 97%. In order to enhance our displacement estimation system, an estimator applied for displacement estimation in (3D) including vertical and horizontal directions is developed. The details are described in this section.

2.4.1. System Overview

The displacement estimation system in this article consists of different algorithms accordingly to motion modes recognized by above described classification method. Figure 5 shows the block diagram of the proposed system for 3D displacement estimation.

2.4.2. 3D displacement estimation for descending/climbing staircases motion modes

During going up/down, the step events are defined at the extreme peak positions of the vertical acceleration, one component of the global acceleration with gravity removal. The method in [13] is used to detect the peaks.

Figure 6: The black solid line shows the Motion Modes, with values ’0’, ’1’,  ’2’,  ’3’ represents modes of stationary, descending staircases, walking and climbing up staircases, respectively. The red rings are the extreme peak positions when descending (top) and climbing (bottom).

Figure 6. shows the peaks detected based on the data from the same experiment shown in Figure 3. The blue solid line is the vertical acceleration and the black solid line shows the Motion Modes. The step events are 100% detected.

Figure 7:  Block Diagram of the estimation algorithm for climbing/descending staircases.

Based on the fact that climbing/descending staircases are the periodic motions and the vertical acceleration is periodic data recored from the IMU mounted on the pedestrian’s body, the entire displacement estimation process is able to be considered as the accumulation of segments. Each segment starts from a step event and ended at the next adjacent step event. However, the accumulating error due to integration affects the estimation accuracy as described in [20]. Unlike putting the IMU at the ankle, placing the IMU at pedestrian’s waist has no chance of zero velocity interval when the normalized global acceleration is equal to zero. Therefore, the authors propose a method in order to obtain the 3D displacement is that integrating twice from global acceleration with gravity removal and that integrating process is reset to zero for velocity and displacement at the step events. The method is illustrated as in Figure 7. The reseting is called refreshing for segments.

2.4.3. 3D displacement estimation for riding a lift motion mode

The 3D displacement estimation when riding a lift is obtained by double integrating the vertical acceleration with gravity removal in world frame. The method was introduced in our previous work [19]. It is described that the integrated velocity is reset in zero vertical acceleration intervals during riding lift interval. Figure 8 illustrates the integrating process. Firstly, the discrete vertical acceleration with respect to the world frame is in the top plot and is with gravity removal when riding a lift one floor. Secondly, the middle plot is the integrated velocity and the bottom plot is the integrated vertical displacement. The red dot value ’non-zero’ in the top plot indicates the zero vertical acceleration intervals where integrated velocity is reset.

Figure 8:  The vertical integration process when riding a lift through one floor.

3. Experiments and Evaluations

The capability of proposed method is tested through experiments conducted by four persons in our research group. In this section, a brief description of the experiment setup is first given, followed by a description of the experimental scenarios and the results obtained.

3.1. Set up

The hardware used to test the proposed method is illustrated in Figure 9, which consists of two parts: 1) the mobile part (attached to the pedestrian) shown in the top portion and 2) the base part shown in the bottom portion. The mobile part consists of an Xbee wireless transmitter, an MPU-9250 IMU sensor board and an Arduino Fio microcontroller Atmega328P board whereas the base part has an Xbee wireless receiver connected to the laptop by virtual serial port via USB port. The transmitter communicates with Fio board using I²C communication protocol. The serial baud rate is set at 115200bps.

Figure 9: Hardware for experiments, with the mobile part (attached to pedestrian’s waist) shown in the top and the base part shown in the bottom.

The proposed algorithm is implemented in the mobile part with the data acquired in every 25 ms and passed through a complimentary filter with 225 ms time constant.

Four experimental scenarios are set and conducted, namely: 1) which is described in section 3.2 is the experiment of going staircases along a straight path with two action modes climbing and descending for 3D displacement accuracy evaluation; 2) which is described in section 3.3 is the experiment of walking along with a triangular path put together with climbing and descending staircases for combination of heading and displacement accuracy evaluation; 3) which is described in section 3.4 is the experiment of riding a lift through floors; and 4) which is described in section 3.5 is the experiment of 3D displacement estimation that consists of many motion modes.

Four people in our research group aging from 25 to 39 with height ranging from 1.6 meters to 1.86 meters are deployed for these experiments, and all the experiments are conducted. Each test is also repeated for four times.

3.2. Results for climbing/descending staircases

For the first experiments scenario, the straight path includes 2m flat path and 15 stairs as shown in Figure 10. Each stair is 15cm in height and 30cm in length. The straight path has 2.25m ground truth in height (vertical direction) and 7.03m ground truth in horizontal direction. It produces 14.06m traveling ground truth in three dimensions.

Figure 10:  In this figure, the volunteers start from the red mark ‘X’ and performs motion modes of stationary, descending, climbing along a straight path depicted by the black bold dot line.

On stairs, both horizontal and vertical displacements are obtained by double integration with zero reset at step events. The heading is determined by method in [9]. Figure 11 shows the side view of an experimental result and illustrates the both vertical and horizontal estimates.

Figure 11:  In this figure, both horizontal and vertical estimates are represented by the red asterisk ‘*’ when walking, descending/climbing staircases along with a straight path. (XY_W) is the horizontal direction. The black solid line is the ground truth.

Figure 12:  In this figure, the horizontal estimates are represented by the red asterisk ‘*’ when walking, descending/climbing staircases along with a straight path.

Figure 13:  In this figure, the vertical estimates are represented by the red asterisk ‘*’ when walking, descending/climbing staircases along with a straight path.

The vertical estimates are positions of pedestrian in upward direction (Z_W) and the horizontal estimates are computed by taking normalization of pedestrian positions in both north (Y_W) and east (X_W) directions. Let (XY_W) be the representation of horizontal direction.

Next, Figure 12 shows the view from top and illustrates the horizontal estimates only. Finally, Figure 13 illustrates the vertical estimates only.

In the three figures, the red asterisk ‘*’ represents the estimated positions, the blue ‘O’ represents the starting point and the black ‘X’ represents the ending point.

3.3. Results for walking and climbing/descending staircases

For the second experimental scenario, a traveling path includes two triangular paths (Figure 14(a)) in two different floors which are connected by 30 stairs of 15cm in height and 30cm in length (Figure 14(c)). The triangle has 3 edges in 11m length and its vertices are circled with 1/3 ring of 1m radius. The pillar in Figure 16(a) is one of triangle vertices. The walking path from a triangle to staircases is 9m for the top level (Figure 14(b)) and 11m for the bottom level (Figure 14(d)). The walking path at middle of staircases is 3m (Figure 14(c)). In order to obtain the highest accuracy in measurement as ground truth for evaluation, the staircase height is measured by Fluke 411D Laser Distance Meter and the number of staircases between two floors are counted. There are 30 staircases with 14.5cm height between two floors at Singapore University of Technology and Design, Building 1. This experimental set produces totally 114.62m traveling ground truth in three dimensions.

Figure 14:  The second experimental scenario in the building.

When walking, the walking model and method in [18] is applied for displacement estimation while double intergrating process is employed for displacement estimation when climbing/descending staircases. For illustrating, an experimental result in 3D is shown in Figure 15.

3.4. Evaluation for the first two experimental scenarios

In order to evaluate the method performance, all the Euclidean distances in three dimensions between the starting point and the ending point are computed for both described experimental scenarios. In addition, all the differences between estimated height for each experiment and the staircase height ground truth are also computed for the first experimental scenario. The root means square errors of the Euclidean distances and the height differences are then calculated. The evaluation results are summarized in Table 1.

Figure 15:  In this figure, the 3D estimates are shown when walking along a triangular path, climbing/descending 30 stairs.

Table 1: Performance of four people conducted experimental scenarios described in Section 3.3

	RMSE [m]	RMSE [%]
3D distance for descending, climbing along with Straight Path	0.5	3.56
Height Differences for descending, climbing along with Straight Path	0.26	11.56
3D distance for travelling on the Triangular Shape Path	1.64	1.43

3.5. Results and evaluations for riding a lift

In this experimental scenario, four members in our research group are tasked to ride a lift upward four times, then downward four times. Three experimental sets are designed such as riding a lift through one floor only as shown in Figure 8, riding lift through multiple floors with one floor stop as shown in Figure 16, riding lift through 40 floors as shown in Figure 17.

Figure 8 shows one of 16 experimental results in total for the first experimental set including 2 upward direction and 2 downward direction rode by 4 persons at Building 1. The experimental results are evaluated by the root mean square error based on the ground-truth of 4.35 meters, which is 0.0334 meters or 0.77%.

Figure 16 shows one of 8 experimental results in total for the second experimental set including 1 upward direction and 1 downward direction rode by 4 persons at Building 1. The experimental results are evaluated by the root mean square error based on the ground-truth of 4.35 meters, which is 0.0384 meters or 0.88%.

Figure 17 shows one of 4 experimental results in total for the third experimental set including 2 upward direction and 2 downward direction rode by 1 person at Block 90. There are 20 staircases with 13.5cm height between two floors of Block 90. The experimental results are evaluated by the root mean square error based on the ground-truth of 2.7 meters, which is 1.7909 meters or 1.66%.

Figure 16:  In this figure, the vertical estimates are shown when riding a lift through multiple floors.

Figure 17:  In this figure, the vertical estimates are shown when riding a lift through 40 floors.

3.6. Result and evaluation for 3D displacement estimation

A person is tasked to conduct experiments of walking from starting point, entering a lift, riding a lift upward one floor, followed by a walking out of a lift, descending staircases and walking again to starting point. The experimental result of this scenario is illustrated as in Figure 18.

Figure 18:  In this figure, the 3D estimates are shown when riding a lift through one floor then walking and descending straircases.

The experiment is repeated 4 times. The Euclidean distances between starting point and ending point are calculated for evaluation, which distance average of four experimental results is 0.71 meters.

4. Conclusion

A simple motion mode classification for climbing/descending staircases comes with an algorithm to estimate both vertical and horizontal displacements by utilizing information from global acceleration with gravity removal have been proposed in this article. The estimation based on integrating process with zero reset at the extreme peaks of the vertical, rather than horizontal, acceleration. Authors have given new method to estimate indoor 3D displacements as a basic for positioning of pedestrian without knowing of stair measurements and provided the most convenient IMU location at waist. In order to verify displacement estimation system’s feasibility and evaluate its performance, a range of experiments which are grouped into two sets of experiment including straight path, and triangular shape path are conducted by four subjects. In each set, people who tasked to perform experiments are required to move at normal step size and at speed 2 steps per second. The experimental results have demonstrated the capability of the algorithm via a RMS displacement error of 3.56% for straight path and 1.43% for the triangular shape path.

Finally, the algorithm in this article is only focusing on climbing upstairs and descending downstairs. The displacement estimations when pedestrian acts in other motion modes like standing, walking on escalator and standing in evaluator will be researched in the future work.

Conflict of Interest

The authors declare no conflict of interest.

Acknowledgment

This project is supported by Singapore University of Technology and Design (SUTD) Temasek Lab, Pillar of Engineering Product Development (EPD), Changi 487372, Singapore.

In addition, this project is supported by Van Lang University (VLU) Electronics and Electrical Engineering Division, Ho Chi Minh City 700000, Vietnam.

1. T. N. Do, and Y. S. Suh, “Gait Analysis Using Floor Markers and Inertial Sensors” Sensors, 12(2), 1594-1611, 2012. https://dx.doi.org/10.3390%2Fs120201594
2. T. N. Do, and Y. S. Suh, “Gait Analysis System Using Infrared LED Landmarks” Journal of Institute of Control, Robotics and Systems, 17(7), 641-646, 2011. https://doi.org/10.5302/J.ICROS.2011.17.7.641
3. H. Liu, H. Darabi, P. Banerjee, and J. Liu, “Survey of wireless indoor positioning techniques and systems” IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 37(6), 1067-1080, 2007. https://doi.org/10.1109/TSMCC.2007.905750
4. K. El-Kafrawy, M. Youssef, A. El-Keyi, and A. Naguib, “Propagation modeling for accurate indoor wlan rss-based localization” in IEEE 72nd Vehicular Technology Conference Fall, Ottawa, Canada, 2010. https://doi.org/10.1109/VETECF.2010.5594108
5. L.-H. Chen, E. H.-K. Wu, M.-H. Jin, and G.-H. Chen, “Intelligent fusion of Wi-Fi and inertial sensor-based positioning systems for indoor pedestrian navigation,” IEEE Sensors Journal, 40(11), 4034-4042, 2014. https://doi.org/10.1109/JSEN.2014.2330573
6. T. N. Do, U-X. Tan, “Novel Velocity Update Applied for IMU-based Wearable Device to Estimate the Vertical Distance” IEEE 1st International Conference on Electrical, Control and Instrumentation Engineering (ICECIE), Kuala Lumpur, Malaysia, 25-25 Nov. 2019. https://doi.org/10.1109/ICECIE47765.2019.8974671
7. X. Yun, E. R. Bachman, and R. B. McGhee, “A Simplified Quaternion-Based Algorithm for Orientation Estimation from Earth Gravity and Magnetic Field Measurements” IEEE Transactions on Instrumentation and Measurement, 57(3), 638-650, 2008. https://doi.org/10.1109/TIM.2007.911646
8. Y. S. Suh, “Orientation estimation using a quaternion-based indirect Kalman filter with adaptive estimation of external acceleration” IEEE Transactions on Instrumentation and Measurement, 59( 12), 3296-3305, 2010. https://doi.org/10.1109/TIM.2010.2047157
9. V. Renaudin, M. Susi, G. Lachapelle, “Step Length Estimation using Handheld Inertial Sensors” Sensors, 12(7), 8507-8525, 2012. https://doi.org/10.3390/s120708507
10. H. Zhang, W. Yuan, Q. Shen, T. Li, and H. Chang, “A Handheld Inertial Pedestrain Navigation System with Accurate Step Modes and Device Poses Recognition” IEEE Sensors Journal, 15(3), 1421-1429, 2015. https://doi.org/10.1109/JSEN.2014.2363157
11. H. M. Schepers, E. H. van Asseldonk, C. T. Baten, and P. H. Veltink, “Ambulatory estimation of foot placement during walking using inertial sensors” Journal of Biomechanics, 43(16), 3138-3143, 2010. https://doi.org/10.1016/j.jbiomech.2010.07.039
12. E. Bachmann, J. Calusdian, E. Hodgson and X. Yun, “Insitu heading drift correction for human position tracking using foot-mounted inertial/magnetic sensors” IEEE International Conference on Robotics and Automation (ICRA), 5425-5430, Saint Paul, MN, May. 2012. https://doi.org/10.1109/ICRA.2012.6225007
13. T. N. Do, R. Liu, C. Yuen, and U-X. Tan, “Design of an infrastructureless indoor localization device using an IMU” IEEE International Conference on Robotics and Biomimetics, 2115-2120, China, Dec. 2015. https://doi.org/10.1109/ROBIO.2015.7419086
14. A. Kose, A. Cereatti, and U. D. Croce, “Estimation of traversed distance in level walking using a single Inertial Measurement Unit attached to the waist” IEEE International Conference in Medicine and Biology Society, 1125-1128, 2011. https://doi.org/10.1109/IEMBS.2011.6090263
15. J. C. Alvarez, D. Alvarez, A. Lopez, and R. C. Gonzalez, “Pedestrian Navigation Based on a Waist-Worn Inertial Sensor” Sensors, 12(8), 10536-10549, 2012. https://doi.org/10.1109/I2MTC.2012.6229714
16. F. Inderst, F. Pascucci, M. Santoni, “3D pedestrian dead reckoning and activity classification using waist-mounted inertial measurement unit” IEEE International Conference on Indoor Positioning and Indoor Navigation (IPIN), 1-9, 2015. https://doi.org/10.1109/IPIN.2015.7346953
17. T. N. Do, R. Liu, C. Yuen, M. Zhang and U-X. Tan, “Personal Dead Reckoning using IMU mounted on upper Torso and inverted Pendulum Model” IEEE Sensors Journal, 2016. https://doi.org/10.1109/JSEN.2016.2601937
18. E. M. Diaz, “Inertial Pocket Navigation System: Unaided 3D Positioning” Sensors, 15(4), 9156-9178, 2015. https://doi.org/10.3390/s150409156
19. M. Elhoushi, J. Georgy, A. Noureldin, M. J. Korenberg, “Motion Mode Recognition for Indoor Pedestrian Navigation Using Portable Devices” IEEE Transactions on Instrumentation and Measurement, 65(1), 208-221, 2016. https://doi.org/10.1109/TIM.2015.2477159
20. T. N. Do, R. Liu, C. Yuen, and U-X. Tan, “Personal Dead Reckoning using IMU device at upper torso for walking to running” IEEE International Conference on Sensors, Orlando, FL, USA, Nov. 2016. https://doi.org/10.1109/ICSENS.2016.7808521

(Click Here)

(Click Here)