Feedback device of temperature sensation for a myoelectric prosthetic hand

A R T I C L E I N F O A B S T R A C T Article history: Received: 21 February, 2017 Accepted: 28 March, 2017 Online: 05 April, 2017 In this study, a control system of a feedback device of temperature sensation for myoelectric prosthetic hand users was improved. The device is mounted on a user's upper arm in contact with the skin, and transfers the temperature sensation in the upper arm corresponding to the temperature detected by the temperature sensor at the fingertip to the user. However, since the feedback device developed in the previous study was controlled in an open loop system, it was difficult to maintain the temperature for a long period. To solve this problem, a closed loop control system was constructed for the feedback device by using a PID controller. In addition, in order to verify the performance of the feedback device controlled in the closed loop control system, performance evaluation experiments were performed. As a result, the sufficient capability of the feedback device controlled in the closed loop control system was indicated. Finally, temperature identification experiments were performed based on psychophysical method to verify an effectiveness of the feedback device. As a result, the difference threshold of the feedback device of temperature sensation, 0.62C was obtained.


Introduction
A myoelectric prosthetic hand is an electrically driven artificial hand in which the intended motion is identified and the movement is controlled based on biosignals generated when muscles are moved. However, myoelectric prosthetic hand users are unable to feel sensations when they touch an object using a prosthetic hand. Therefore, investigation of the functions that provide sensation feedback to the user is required. It is shown that sensory feedback is important for improvement in proficiency level of a myoelectric prosthetic hand [1].
Recent researches focused on developing a feedback device of sensations for myoelectric prosthetic hand users. These included our previous study [2] in which a feedback device of force sensation was developed. In this mechanism, when the myoelectric prosthetic hand held an object, the belt of the device was wound by a motor and tightened the operator's upper arm. This enabled the user to obtain a force sensation. Additionally, the user could feel a difference in the object hardness by changing the winding speed of the belt.
A study [3] also reported a feedback device using vibrations. The device changed the vibration intensity based on the size of the detected force. Another study [4] reported a device that perceives the temperature when touched by a myoelectric prosthetic hand using a hot and cold pad. This device obtains the temperature difference between an object and the user's skin temperature by the changes in the temperature of the hot and cold pads. A study [5] examined transfer of temperature sensation using the Peltier element.
In our previous study [6], a feedback device of temperature sensation was developed by using a Peltier element to perceive the object temperature when the user touches objects by a prosthetic hand. The object temperature is measured by a temperature sensor attached at fingertip of the prosthetic hand. However, the temperature measurement required several tens of seconds. Therefore, a temperature prediction algorithm was proposed to shorten the measurement time. The study [4] examined a device that transmits a temperature difference between skin temperature of cut section and object temperature. However, the temperature sensation differs at each body site. Therefore, the feedback device developed in our study transfers temperature sensation in the upper arm corresponding to the temperature detected at the fingertip of the prosthetic hand. To this end, an experiment to examine the difference in the temperature sensation felt by the fingertip and that felt by the upper arm was performed, and the feedback device reproduces the temperature sensation felt by the fingertip to the ASTESJ ISSN: 2415-6698 upper arm based on the result of the experiment. Furthermore, an identification experiment for temperature was performed to verify the effectiveness of the feedback device. However, the feedback device developed in previous study was controlled in an open loop system. Therefore, it was difficult to maintain the temperature using this control system. This paper is an extension of work originally presented in 2016 International Conference on Advanced Mechatronic Systems [6].
The key changes from the conference paper are summarized as follows.
In this paper, a closed loop control system was constructed to enhance the performance of the feedback device. In addition, to improve the operability of the myoelectric prosthetic hand, a new myoelectric prosthetic hand was built. Then, performance evaluation experiments were performed to verify the performance of the feedback device controlled in the closed loop control system using the new myoelectric prosthetic hand. Furthermore, psychophysics tests were conducted to measure the difference threshold of the feedback device in terms of the temperature difference.

Peltier Effect and Seebeck Effect
Exothermic and endothermic reactions occur when an applied current flows through a junction between two conductors. This phenomenon is called the Peltier effect. The Peltier element is an electronic device that changes surface temperature on the basis of the Peltier effect. It enables both cooling and heating by applying a voltage, and temperature can be adjusted by regulating the voltage. The feedback device has a Peltier element, and transfers the temperature sensation to the user. The Peltier element used in this study is "TEC1-12708" Peltier element made by HB Electronic Components.
In contrast to the Peltier effect, the Seebeck effect is a phenomenon that generates a voltage by providing a temperature difference to a conductor. When the temperature difference is provided to the conductor, voltage that is proportional to the temperature difference is generated due to the Seebeck effect. This principle is used to measure temperature. A thermocouple is a temperature sensor that uses the Seebeck effect. In this study, a K type "AD-1214" thermocouple manufactured by T & D Corporation was used as the temperature sensor. Figure 1 shows the feedback device of temperature sensation (Hereafter, FB device) developed in our study. When the FB device is used for a long period, there is a possibility of high-temperature due to the accumulated heat. Hence, the continuous operating time of the FB device controlled in an open loop control system was limited to 5 s. Figure 2 shows relationship between the applied voltage and the temperature produced by the FB device.

Outline of Feedback Device
The FB device enables refrigeration of surface temperature to a minimum of 15C for 5 s. In addition, when the FB device increases the temperature, it enables heating exceeding 50C. However, the temperature of the FB device was limited to 40C for safety considerations.

Temperature Prediction and Temperature Sensation Investigation
After contact with the object, it takes several tens of seconds until the measured value settles. Thus, the temperature sensor needs a long time for measuring the temperature. Consequently, a temperature prediction was performed to shorten the measurement time.

Temperature Prediction Algorithm
Let y(t) denote a sensor output and u(t) denote an amount of temperature variation, and let relationship between y(t) and u(t) be given by the following equation in transfer function expression: In order to determine the transfer function G(s), the parameters identification was performed by using the gradient method. As a result, G(s) was given by the following equation:

 
Furthermore, let ∆T denote the predicted temperature variation and ̂( ) denote the estimated output, then the estimated output is calculated as follows using the identified transfer function.
The predicted temperature variation ∆ is updated by using ( ) as follows in each sampling period.
where ∆ is the predicted temperature variation one step before and ∆ denotes an updated predicted temperature variation. In addition, K is an arbitrary constant, which was chosen as K=70 in this study, since noise was comparatively low and the reaction time was fast when the temperature prediction was performed.
Let T denote the predicted temperature and 0 denote the room temperature, then the predicted temperature is given by the following equation.
The predicted temperature is updated to reduce the error when the sensor detected a change of the temperature, thus the temperature when reached to the equilibrium state is given by Equation (6). As a result, the sensor can detect the temperature of the object in a short time.

Verification of Temperature Prediction Algorithm
In order to verify the effectiveness of the proposed temperature prediction algorithm, an experiment to investigate the behavior of the temperature prediction when the touched object was changed to an object with a different temperature during the temperature prediction was performed. When the sensor touched an object with a temperature of 20C after touching an object with a temperature of 30C under the room temperature of 25C, the temperature was predicted by using the temperature prediction method. Figure 3 shows the result of the temperature prediction. The results showed that it is possible to perform the temperature prediction even when an object with a different temperature was touched during the temperature prediction.

Purpose of Temperature Sensation Investigation
It is difficult for the FB device to reproduce the low temperature. As shown in Figure 2, the FB device cannot reproduce a temperature of lower than 15C. However, the temperature sensations of an individual differ at each body site. For example, there is difference between the temperature sensations when an object with the same temperature is touched with a finger or with the upper arm. Therefore, an investigation of temperature sensation was performed, and a temperature that corresponds to the temperature at which an object was touched with a finger was determined as feedback temperature to the upper arm.

Method and Result
In order to investigate the difference in the temperature sensation between the fingertip and the upper arm, an experiment was performed in the following procedure using water from temperatures ranging from 5C to 40C for each 5C. The details of the procedure of this experiment are addressed in the literature [6]. Table 1 shows the result of the temperature sensation investigation performed for 7 healthy subjects in their 20s. Based on these results, the FB device is controlled to reproduce the temperature experienced by the upper arm which is equivalent to the temperature experienced by the fingertip. Thus, the temperature sensation which should be reproduced to the FB device is determined by the temperature prediction at the fingertip. However, the temperature of FB device is not changed when the prosthetic hand touched an object at 30C. This is because the temperature of FB device is approximately 30C at normal temperature. The temperature which should be presented at the upper arm and the voltage which should be applied to the FB device were determined as follows. The relationship between the temperatures experienced by the fingertip and by the upper arm was interpolated as shown in Figure 4. Then, the relationship between the predicted  temperature at the fingertip and the target temperature in the upper arm , and the relationship between the predicted temperature at the fingertip and the input voltage to the FB device are approximated as shown in Equations (7) and (8) respectively, by using the least-squares method.

Purpose and Equipment
The FB device transmits the temperature sensation corresponding to the predicted temperature of the object which was touched by the fingertip of the myoelectric prosthetic hand to the user's upper arm. In order to verify the effectiveness of the developed FB device, the temperature identification experiment was performed using a myoelectric prosthetic hand developed in the literature [7].
The myoelectric prosthetic hand needs three types of operations in the experiment, namely bending the finger to touch the object, maintaining the posture, and releasing the finger. To distinguish these operations, electrodes were stuck on the flexor digitorum superficialis muscle and the extensor digiti minimi muscle to measure surface electromyogram (SEMG). The flexor digitorum superficialis muscle is used to flex the fingers, and the extensor digiti minimi muscle is used to extend the fingers. Figure  4 shows the positions where the electrodes were stuck on. The motion of user's hand is identified based on the measured SEMG and the myoelectric prosthetic hand is controlled by using an operation method proposed in the study [8].

Method of Experiment
The subjects executed the identification of the temperature transmitted by the FB device. In the experiment, another Peltier element which is different from that of the FB device was used as the object touched by the fingertip of the prosthetic hand. The following five variations in temperature were employed: hot (approximately 40C), lukewarm (approximately 35C), not much (25C-30C), a little cold (approximately 20C), and cold (approximately 15C).
The temperature identification experiment was performed in the following way. The temperature of the object (Peltier element) is randomly selected from the five variations, and the temperature of the object is adjusted to become the selected temperature. The subject operates the myoelectric prosthetic hand to touch the object. Then, the temperature prediction is performed and the voltage calculated from Equation (8) on the basis of the predicted temperature is applied to the FB device. Based on the corresponding temperature sensation reproduced by the FB device, the subject identifies the temperature of the object from the five variations. The experiment was executed for 10 subjects. The experimental environment is shown in Figure 5.  Table 2 shows the success rates of temperature identification for each subject, and Table 3 shows the success rate of temperature identification for each presented temperature.  The results show that the average success rate for 10 subjects is 88%. In addition, the success rate of each presented temperature exceeded 84%. The above results revealed that the difference in temperature can be recognized with high accuracy. The causes for the failure identification include the failure of the temperature prediction and the individual differences of the sensation of each subject.

Construction of the Control System
The FB device developed in the previous study is controlled in an open loop system, in which the constant voltage through Equation (8) computed from the predicted temperature is applied to the FB device. Hence, it is difficult to maintain the temperature using this control system, because if the voltage is continuously provided to the FB device the temperature keeps increasing or decreasing. Therefore, the continuous operating time of the FB device was limited to 5 s.
To solve this problem, a closed loop control system was constructed. To this end, a temperature sensor which is the same type as one attached at the fingertip of the prosthetic hand was additionally attached on the surface of the Peltier element of the FB device. The target temperature in the upper arm corresponding to the predicted temperature at the fingertip of the myoelectric prosthetic hand is determined by Equation (7). Then, the input voltage to the FB device is determined by a PID controller. The PID gains were determined by trial and error, and chosen as = 1 , = 0.05 and = 0.01.
In order to verify the validity of the closed loop control system with PID controller, sudden change in temperature when the target temperature is decreased from 40C to 15C was examined. The transition of the temperature of the FB device and the input voltage to the FB device are shown in Figure 6. As shown in Figure 6, the constructed closed loop system enabled the adjustment of the presenting temperature of the FB device according to the temperature change and also enabled a long time continuous operation of the FB device.

Purpose and Equipment
In order to verify the performance of the FB device controlled in the closed loop control system, an evaluation experiment was performed.
To improve the operability of the myoelectric prosthetic hand, a new myoelectric prosthetic hand was designed and built by imitating the commercial prosthetic hand, which is shown in Figure 7. The temperature sensor was attached to the silicone finger installed on the fingertip of the index finger of the myoelectric prosthetic hand, and the pressure sensor was attached to the fingertip of the thumb of the prosthetic hand. Thus, when the prosthetic hand holds an object, temperature value of the object and grasping force of the finger are obtained. The new myoelectric prosthetic hand was used in this experiment.

Method of Experiment
The experiment was performed in the following procedure. In this experiment, the following four variations in temperature were employed: approximately 40C, approximately 35C, approximately 15C, and approximately 5C.
1. The temperature of the object (Peltier element) is adjusted to become the selected temperature.
2. The temperature of the object is recorded and the real value of the target temperature in the upper arm _ is calculated through Equation (7) from . 4. The temperature prediction is performed and the target temperature corresponding to the predicted temperature is calculated through Equation (7). 5. The temperature of the FB device is controlled by using the closed loop control system. 6. The temperature of the FB device and the target temperature corresponding to the predicted temperature are recoded.
7. The error between the target temperature and the real value _ (prediction error _ − ), and the error between the target temperature and the temperature of the FB device (control error of the FB device − ) are recorded for 10s.
The above procedure was repeated 5 times for each temperature. Table 4 shows the result of the performance evaluation experiment.

Result
From the experimental result, a maximum error of 3.43C was observed between the target temperature and the real value _ , and a maximum error of 2.86C was observed between the target temperature and the temperature of the FB device . In addition, the overall average of the prediction error and control error of the FB device for 10s were 1.09C and 0.61C, respectively. From this result, it is considered that the FB device controlled in the closed loop control system showed sufficient capability to present the temperature sensation.

Purpose of Difference Threshold Measurement Experiment
The usefulness of the FB device controlled by the closed loop system is objectively verified with a psychophysics experiment method. Namely, a temperature difference threshold of the FB Device is measured by this experiment. The difference threshold is the minimum difference between two stimulations that can be distinguished by the user. Therefore, in this study, the difference threshold means a change in the temperature that the user can recognize using the FB device.

Method of Experiment
In this experiment, the myoelectric prosthetic hand is not used, but each temperature is input directly to a computer, and then the difference threshold measurement of a single body of the FB device is conducted.
The temperature of 30C is used as standard stimulation, and five kinds of temperature, 28C, 29C, 30C, 31C and 32C, are used as comparative stimulation. The following describes the experimental procedure: 2. An experimenter inputs the standard stimulation (30C)to the computer and standard stimulation is presented to the subject by the FB device.
3. An experimenter inputs the comparative stimulation that is randomly selected from the five kinds of temperatures to the computer and comparative stimulation is presented to the subject by the FB device.
4. The subject then answers which temperature is higher or whether the two are the same.
6. The works 2 and 3 was replaced and 25 sets of the work 2 ~ 4 are performed.
In the operations, all the comparative stimulations were used ten times in random orders. The experiment was executed for 5 healthy subjects in their 20s. Table 5 shows the result of difference threshold measurement experiment. When the method of three categories (large, almost equal, small) is used and when the stimulation value at which the occurrence ratio of "large" and the occurrence ratio of the others are both 50% is written as , the stimulation value at which the occurrence ratio of "small" is 50% is written as , the difference threshold ∆ can be obtained by the following equation:

Result
The stimulation value and the stimulation value were calculated through a linear interpolation method. Table 6 shows the results. From the above calculation procedures, the difference threshold of the FB device was found to be ∆ = 0.62C. Therefore, the experimental results suggest that the temperature change can be distinguished if the temperature is changed 0.62C or more under the standard stimulation, namely temperature of 30C.

Conclusion
In this paper, a control system of the FB device of the temperature sensation developed for myoelectric prosthetic hand users was improved. In the previous FB device controlled in the open loop control system, it was difficult to operate the FB device continuously. Also, the FB device could not adjust the temperature for the temperature change. To solve these problems, a closed loop control system was constructed for the FB device and was tested for sudden change of the temperature.
In addition, a new myoelectric prosthetic hand was built to improve the operability of the myoelectric prosthetic hand. Then, a performance evaluation experiment was carried out using the new myoelectric prosthetic hand to verify the performance of the FB device controlled in the closed loop control system. The overall average of the prediction error and control error of the FB device were 1.09C and 0.61C, respectively. Thus, the sufficient capability of the FB device controlled in the closed loop control system was shown.
Furthermore, psychophysics tests were conducted to measure the difference threshold of the feedback device. As a result, the difference threshold of the feedback device of temperature sensation, 0.62C was obtained.