A systematic review on the usability of robotic and virtual reality devices in neuromotor rehabilitation: patients’ and healthcare professionals’ perspective
BMC Health Services Research volume 22, Article number: 523 (2022)
The application of virtual reality (VR) and robotic devices in neuromotor rehabilitation has provided promising evidence in terms of efficacy, so far. Usability evaluations of these technologies have been conducted extensively, but no overviews on this topic have been reported yet.
A systematic review of the studies on patients' and healthcare professionals' perspective through searching of PubMed, Medline, Scopus, Web of Science, CINAHL, and PsychINFO (2000 to 2021) was conducted. Descriptive data regarding the study design, participants, technological devices, interventions, and quantitative and qualitative usability evaluations were extracted and meta-synthetized.
Sixty-eight studies were included. VR devices were perceived as having good usability and as a tool promoting patients' engagement and motivation during the treatment, as well as providing strong potential for customized rehabilitation sessions. By contrast, they suffered from the effect of learnability and were judged as potentially requiring more mental effort. Robotics implementation received positive feedback along with high satisfaction and perceived safety throughout the treatment. Robot-assisted rehabilitation was considered useful as it supported increased treatment intensity and contributed to improved patients' physical independence and psychosocial well-being. Technical and design-related issues may limit the applicability making the treatment difficult and physically straining. Moreover, cognitive and communication deficits were remarked as potential barriers.
Overall, VR and robotic devices have been perceived usable so far, reflecting good acceptance in neuromotor rehabilitation programs. The limitations raised by the participants should be considered to further improve devices applicability and maximise technological rehabilitation effectiveness.
PROSPERO registration ref. CRD42021224141.
In the last two decades, robotic and virtual reality (VR) devices have gained increased interest in the rehabilitation community for their multipurpose application in patient’s physical recovery process [1, 2]. Robot-assisted rehabilitation has showed promising results so far, thanks to its peculiarity to provide intensive, repetitive and task-oriented activities for the treatment of motor impairment resulting from various neurological and musculoskeletal diseases . Compared to conventional therapies, it benefits from smaller workforce, optimized exercise, and quantitative assessment and monitoring. Moreover, among its advantages are also the possibility to better tailor the interventions, by increasing the amount and quality of the therapy that can be administered, and by managing the parameters to make the rehabilitation personalized to the patient . To date, mainly two different types of robotic devices have been implemented, for both lower and upper limbs rehabilitation. The first is based on the use of exoskeletons, which are systems constituted by mechanical and electronic components that completely cover the limb, following its anthropometric characteristics, and assist the kinematic or dynamic activity that patient performs [5, 6]. The second is of the end-effector type. Differently from the exoskeletons, robotic end-effector devices interconnect to the distal part of the limb, allowing the natural kinematic activation of the movement without specific constraints and, thus, with more degrees of freedom [7, 8]. Additionally, another typology is represented by soft-robotics. Soft-robots are wearable devices characterized by a lightweight and flexible structure, and although they are primarily intended to be worn and used for the support and assistance of the activities of daily living (ADLs), they have also been shown promising tools for rehabilitation purposes .
Similarly to robotics, VR is considered an emerging tool in the field of rehabilitation, representing a trending and widely accessible technology for the treatment of different medical conditions [10,11,12]. It can be defined as a system based on computer-simulated 3D environments allowing the user to navigate through and interact with by the integration of auditory, visual, and haptic feedback . Accordingly, VR has three key characteristics: immersion, presence, and interactivity . Immersion refers to the degree to which VR can provide multisensory stimuli, originating from the virtual environment (VE), and a high degree of matching between user’s actions and the cues generated by the system. Consequently, the immersion in and the interactivity with the VEs affect patient’s experience and perception and, thus, his/her sense of presence . Based on the level of immersion, VR devices and systems can be categorized in fully-immersive, semi-immersive, and non-immersive . Fully-immersive systems are characterized by the use of tools, as an head-mounted display (HMD) or a cave automatic virtual environment (CAVE), that enable a high degree of immersion and interaction with the VE, blocking out patient’s perception of the real-world. Differently, semi-immersive systems provide a moderate level of immersion and interaction and usually consist of large monitors or projectors that let the patient perceive the real-world and a part of the VE, simultaneously. Lastly, non-immersive systems allow for a low immersion and interaction and include simpler devices such as a PC or a tablet. Overall, the efficacy and the utility of VR is well recognized, especially when applied to neuromotor rehabilitation, as it allows to provide fully controllable and personalized simulated real-life environments that gives to the patient the opportunity to exercise safely and to increase the motivation and compliance to the treatment .
Despite the several advantages that both robotics and VR offer in rehabilitation programs, considering the perspective of the patient when using the device is essential to guarantee adequate engagement and adherence to treatment. For this reason, the introduction of technological devices in rehabilitation programs has raised the issue of usability. Differently from the concept of feasibility, which is defined as the extent to which a new treatment or innovation can be successfully used or carried out within a given population or setting, usability refers to the patient’s perception and ability to use a system to achieve goals effectively, efficiently, and satisfactorily . According to Nielsen , usability may be explained by five attributes, namely learnability, efficiency, memorability, error rate and recovery, and satisfaction. Accordingly, a patient that considers a device as usable presumably also reports a positive perception as concerns: the ease of learning the functionality and behaviour of the system, the effort made to reach the goal, the ease of remembering the system functionality for any further use, the system capability to support and to let easily recover in case of errors during the use, and the pleasantness of the system design. Therefore, evaluating such aspects provides crucial insights into the perceived acceptability and usefulness of the devices, allowing consequently to understand how to improve patient’s motivation during the therapy. As underscored by prior works, motivation plays a pivotal role during rehabilitation program, as it contributes to make the patient feel competent and satisfied . Moreover, patient’s satisfaction with the treatment was found to be associated with stronger therapy compliance . As a result, both factors may be considered key aspects for therapy efficacy. Furthermore, when we aim at patient’s perception of the usability of the devices, it is important to consider the context in which the devices are used. Since, most of the time, the use of the technology during the rehabilitation is supported or mediated by physiotherapists, it is crucial to also elicit their perception, as their success expectations and views on the technological devices may be transferred to the patient .
Although the term usability is frequently used, it is defined by both the research community and standard organizations inconsistently . Beside the absence of a clear consensus, usability is also recognized as a construct focused more on the task and less on the experience . For these reasons, research on the usability of robotic and VR devices applied to rehabilitation has explored the role of user experience too, providing so far informative, but contradictory, findings on deeper facets like emotions and affective reactions toward the use of the technology . Therefore, exploring usability and user experience in technology-assisted rehabilitation programs should be of paramount concern, as it would help, on the one hand, to strengthen a recovery methodology that has already shown its efficacy  and, on the other hand, it would provide further and informative insight into the perceived evaluation of the specific device implemented. Accordingly, it must be acknowledged that, despite robotic and VR devices have so far shared high technological impact, they basically differ from a technical point of view and for how the user interact with them, ultimately affecting usability and user experience evaluation. For this reason, both technological device typologies were included in the present work specifically with the aim to highlight their impact whether they are implemented in combination or independently. Particularly, this choice may elicit a deeper understanding of the strengths and limitations of the devices described, including their differences. Moreover, in a broader perspective, alongside the well-known increase of life expectancy and, thus, of morbidity, multi-morbidity, and disability , providing clear device-specific guidelines along with optimal and customized recovery is increasingly needed .
Following this line, the present study aimed at systematically reviewing the literature concerning the evaluation of the usability of technological devices, namely robotics and VR, implemented in combination or independently in the neuromotor rehabilitation context, considering both patients’ and healthcare professionals’ perspectives. To the best of our knowledge, a systematic overview on this topic has not been provided yet.
A priori search on registered or ongoing similar contributions was conducted through the International Prospective Register of Systematic Reviews (PROSPERO). The register provided no results and, thus, the systematic review protocol was registered (ref. CRD42021224141). The current work is part of a broader project called PHTinRehab Study (Perception of High Technology in Rehabilitation: a prospective real-life Study on usability, effectiveness, and health-related quality of life) approved by the Ethics Committee of ICS Maugeri—Institute of Montescano (February 2021, protocol n. 2517CE).
Search strategy and selection of the studies
The review was conducted and reported following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines . Electronic searches of PubMed, Medline, Scopus, Web of Science, CINAHL, and PsychINFO were performed on 1 March 2021. The following search query was applied for all databases: ((rehabilitation) AND (robot* OR "virtual reality" OR tech*)) AND (usability). Although the current study aimed at identifying published researches specifically on neuromotor rehabilitation, the general term “rehabilitation” was preferred to ensure a full retrieval despite its applicability to different fields (e.g., neuropsychology, psychiatry). Moreover, in the absence of a clear consensus on the terminology of the technological devices, the word “tech*” was specified besides “robot*” and “virtual reality”. This choice was also driven by the intention to retrieve studies that also included wearable tools (e.g., body-mounted sensors) and/or m/eHealth technologies (e.g., smartphone) in combination to robotics and/or VR during the rehabilitation program. Furthermore, to better optimize and refine the identification of the studies according to the eligibility criteria (Table 1), mutual filters (i.e., year of publication timespan, article language, and document typology) were applied for each electronic database consistently. A reference management and bibliography-creating software (EndNote Web) was implemented during the review process.
To assess the methodological quality of the included studies, the McMaster Critical Review Forms for quantitative and qualitative research [29, 30], which include guidelines for interpreting the criteria [31, 32] to facilitate inter-rater reliability, were used. Since these tools provide a narrative assessment only, the scoring criteria for the guidelines developed by Imms  were applied (Table 2). Accordingly, quantitative researches were scored on a checklist of three criteria: sample, measure, and analysis. Qualitative studies were rated on four criteria based on trustworthiness: credibility, transferability, dependability, and confirmability. For each criterion, a score of one (no evidence of study meeting criterion), two (some evidence or unclear reporting) or three (evidence of study meeting criterion) was assigned.
The included studies were assessed by two researchers working independently and any discrepancies were resolved by discussion until a consensus was reached. Recognising that studies rated as lower methodological quality can still provide useful insights based on the data , all studies were included regardless of assessment results. Though, study quality was considered in the interpretation of the results.
Data extraction and synthesis
Eligibility criteria were discussed and accepted after authors’ full consensus. Thereafter, a progression exclusion of the non-eligible records was performed starting from the title, then the abstract, and finally the full-text. The reviewers conducted the entire process by working independently. To solve the disagreements, periodically planned discussions including all authors were carried out. Each identified article was screened multiple times to increase familiarity and obtain a thorough understanding of the study aim, methods, intervention, and outcomes. A wide range of data was extracted and collected in a structured table that, due to its extent, is provided as supplementary material. This includes: author(s), year of publication, authors’ nation, the rank of the nation according to the Human Development Index (HDI), which is a composite index measuring the average achievement in three basic dimensions of human development, namely ‘long and healthy life’ (life expectance at birth), ‘knowledge’ (expected years of schooling and mean years of schooling), and ‘a decent standard of living’ (Gross National Income per capita) , the research group’s profession specialty field, study design (follow-up presence and duration, if pilot study, if multicentre, if real-life, any fundings), characteristics of the patients (i.e., inpatients or outpatients, disease, sample size, age, and ethnicity) and of the healthcare professionals involved (i.e., specialty, sample size, age, and ethnicity), the study purpose, the name and a brief description of the technological devices, the level of immersion (in case of VR devices), robot typology (i.e., exoskeleton, end-effector, soft-robot), intervention characteristics (i.e., overall duration, number of sessions, session duration), usability factors investigated, measures (quantitative and qualitative), and main results (i.e., devices strengths and limitations) divided by patients and healthcare professionals. Due to the varying design characteristics of the included studies, both quantitative and qualitative results were analyzed using narrative [36,37,38] rather than statistical methods. The main findings were discussed and synthetized descriptively.
Flow of studies through the review
By initial electronic search 3025 records were retrieved. After duplicates removal, 1525 studies were identified and screened by title and abstract. A total of 1418 studies were excluded and the remaining 107 were assessed for eligibility. Of these, 68 met all inclusion criteria. No additional records were identified by hand searching. Details on study selection and reasons for exclusion are outlined in the flow diagram (Fig. 1). Most of the excluded records were labelled as off-topic (n = 1033), as they were not strictly focused on technological neuromotor rehabilitation (e.g., conventional rehabilitation exclusively, cognitive rehabilitation, technological-based interventions aimed at self-care and healthy behaviours promotion/monitoring), others were excluded because considered m/eHealth and/or wearable tools exclusively (n = 122).
Characteristics of the included studies
The full report of the information of the included studies is presented in the synoptic table (Additional File 1: Appendix 1) [39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106].
Forty-seven articles used quantitative methods, and 18 studies adopted a mixed approach. Four [66, 69, 76, 105] of these 65 studies provided evidence to satisfy all three criteria for quantitative studies. Thirty-two articles [47, 52,53,54,55,56, 58, 62,63,64,65, 67, 68, 71, 72, 74, 75, 77,78,79, 83, 88, 90, 91, 96,97,98, 100,101,102,103, 106] satisfied two criteria with some evidence of meeting the third. Of the 18 mixed-methods studies, one article  satisfied three criteria for qualitative studies with some evidence toward the fourth and five studies [73, 77, 83, 87, 95] satisfied at least one criterion.
Three studies [86, 92, 99] used qualitative methods exclusively. All demonstrated at least some evidence of trustworthiness. In particular, one article  satisfied two criteria, and one study  met one criterion.
Tables 3 and 4 summarize the main characteristics of the final studies, including the information on the study design. Most (69.1%) were published in the last 5 years. The most frequent contribution was from research groups from the USA (31.0%) and belonging to the field of physiatry (57.4%) and biomedical engineering (35.3%). Despite this, a heterogeneity of professional contributions was observed denoting a multidisciplinary approach in the study of usability. As for the study design, the majority were feasibility/usability studies (41.2%), which provided a detailed description of the devices and reported their clinical applicability and perceived usability. Only the 11.8% included follow-ups, which ranged from one to three months. Moreover, the majority were supported by fundings (61.7%).
Patients and healthcare professionals involved
As for the participants (Table 5), the total number of patients of the included studies was 1464 and the sample sizes varied widely with a minimum of two patients and a maximum of 157. Notably, most of the studies (86.2%) presented limited sample sizes including less than 30 participants. The age ranged from 18 to 91 years and only one study  reported the ethnicity of the patients. As regards to healthcare professionals enrolled, the total number was 72, with the physiotherapists as the most recruited category (88.9%). The sample sizes varied from two to 20 participants.
Devices adopted and rehabilitation sessions’ characteristics
Table 6 shows the main characteristics of the technology used and of the rehabilitation program. Overall, the 58.8% of the studies implemented VR systems and the 20.6% used robotic devices, while the remaining 20.6% used both. VR systems were mostly non-immersive (77.7%) and provided a wide range of activities such as VR-based treadmill training for lower extremities functionality or exercises in reaching and grasping virtual objects for upper limbs mobility and manual dexterity. The studies describing robot-assisted interventions mainly used exoskeleton devices (60.7%) and proposed both active and passive robot-assisted gait training as well as re-learning activities for the arms and upper extremities through kinematics exercises from complete movement guidance to the absence of support and with the integration of game-based VR environments to interact with during the therapy session. The mean duration of the technological rehabilitation along with the mean of the sessions and duration were extracted. VR-based therapies reported wider ranges than those assisted by robotics. Specifically, the duration of the interventions based on the combination of both technologies lasted from one to 9 weeks with a total number of sessions ranging from one to 36 (session duration range: 10–90 min).
To evaluate usability, 47 studies (69.1%) used quantitative measures only. Three studies (4.4%) conducted a qualitative evaluation exclusively, while 18 (26.5%) adopted a mixed-methods approach. The System Usability Scale (SUS)  was the most frequently administered scale (47.7%). Also, ad-hoc questions (49.2%), and Visual Analogue Scales (VAS) and Numerical Rating Scales (NRS) (7.7%) were implemented (Fig. 2). Of the qualitative and mixed-methods studies, open-ended questions (42.9%) were used and interviews with different structure (38.1%) and focus groups (19.0%) were conducted. Multiple usability-related factors were assessed (Table 7). Overall, the most frequent were ease-of-use (82.4%), learnability (52.9%), and satisfaction (29.4%). Others were more strictly related to the patients’ experience of use of the technological devices, namely motivation (36.7%), enjoyment (22.1%), adverse effects (13.2%), and engagement (8.8%).
Overall, VR devices were rated as having good usability. Patients referred good acceptability regardless the different levels of immersion. Most participated in non-immersive VR-based therapy [40, 44,45,46, 48, 50, 52, 53, 57,58,59,60,61,62, 64,65,66,67,68,69,70,71,72,73, 75,76,77], whereas fully-immersive devices (i.e., HMD) were used in six studies [39, 42, 43, 54, 63, 78]. Semi-immersive VR (i.e., Kinect with large TV or projector screens) was tested in five studies [47, 49, 51, 55, 57, 74]. Moreover, some studies combined the use of VR with devices for conventional rehabilitation as treadmill , force plate , springboard  and bicycle . Satisfactory scores on the devices ease-of-use were observed mostly [39, 40, 43, 45,46,47,48, 50,51,52,53,54,55,56,57,58,59, 62, 63, 65, 67,68,69,70,71,72, 74,75,76,77,78] along with the levels of satisfaction [45, 48, 49, 54,55,56, 58, 59, 61, 63, 70] and learnability [39,40,41,42,43, 46,47,48, 51, 55,56,57, 62, 67, 69, 71, 72, 74,75,76,77,78]. Concerning the user experience, the majority found the rehabilitation as motivating and engaging [40, 42,43,44, 47, 49, 50, 53, 55, 58, 60, 63, 67, 69, 70, 73,74,75,76], reported high levels of enjoyment [40, 45, 48, 55, 59, 64, 66, 68, 69, 71,72,73, 76], and referred none to few adverse effects (e.g., nausea, disorientation, dizziness) [43, 50, 51, 54, 55, 57, 63,64,65,66, 68, 71,72,73, 76, 78]. Some patients agreed with the efficacy of the devices perceiving physical health benefits [39, 58, 61, 72, 75] and expressed the intention of future use in both clinical and home settings [53, 57,58,59, 64, 69, 75]. However, although the devices were on average considered easy to learn and to understand, learnability was mentioned as a factor affecting patients’ performance during the initial phases of the therapy. Accordingly, brief tutorials and more trainings before rehabilitation sessions were suggested [39, 50, 60, 69]. Further, some patients felt that more mental effort was required than usual while performing VR therapies, leading in some cases to experience high cognitive load and consequent loss of concentration [40, 60, 64, 75, 77]. Finally, some technical issues (e.g., size of the screen, device comfort, game design, feedback quality) were raised [42, 49, 50, 52, 53, 56, 57, 64, 70]. These led patients to have difficulties in terms of interactivity with the VE [59, 61, 68] and consequently contributed to the need for more technical support from the therapist [52, 53, 77].
Healthcare professionals provided overall positive feedback on devices ease-of-use, comfort, learnability, and usefulness along with perceived efficacy [39, 40, 48, 49, 53, 60, 62]. Notably, they appreciated the potential for customized therapy sessions and the benefits deriving from the patient’s performance monitoring system [40,41,42,43, 64, 73]. Despite the perceived high applicability to clinical settings, healthcare professionals raised some limitations, too. The role of learnability in affecting initial patient’s performance was confirmed. Consistently, increasing the number of training sessions or the introduction of tutorials before starting the therapy were recommended [39, 41, 60]. Furthermore, some remarked that the complexity of the VE graphic design may represent a distracting factor leading to weaken patient’s compliance . Finally, the importance of system adaptability to patient’s abilities to ensure therapy effectiveness was underscored, too [41, 60].
Generally, robotic devices were perceived as usable. Patients well accepted their implementation to the rehabilitation programs. Most underwent exoskeleton-assisted therapies [80,81,82, 84, 85, 87, 89,90,91,92], whereas others used end-effector devices [79, 83]. Two studies evaluated the use of soft-robots [86, 88]. Moreover, of the studies including exoskeletons, two added the use of devices for conventional rehabilitation, namely a walker and bilateral crutches  and a treadmill . Overall, patients rated positively devices ease-of-use [79,80,81, 83, 85,86,87,88, 90,91,92] and learnability [79, 81, 82, 88, 91]. They considered themselves satisfied with the training program [79, 82, 84, 85, 87, 89,90,91] and found the treatment motivating and enjoyable [80, 82, 83, 85, 89, 90]. Furthermore, health benefits concerning the lasting effects on mobility, balance, gait, independence, and psychosocial well-being were reported [82, 83, 86, 87, 89, 92]. Accordingly, some patients suggested devices implementation in the future [82, 87, 88]. Nevertheless, some technical limitations were raised. Mainly, issues concerning devices design were observed, as low comfort and manageability, and difficulties in donning and controlling the device autonomously [81, 83,84,85,86,87, 92]. Moreover, technical issues like the mechanical resistance of the device were mentioned as difficult to overcome and, in some cases, physically straining . In conclusion, suggestions to improve the adaptability (e.g., transportability, ability to walk up and down stairs, surface adaptability) were reported to ensure the implementation also outside the therapy [87, 92].
Concerning healthcare professionals’ perspective, the devices were judged as having wide applicability and potential for increased treatment intensity and safety with the possibility to quantitatively monitor patients’ parameters throughout the entire recovery process [79, 85, 90, 92]. In contrast, some technical and design issues were reported. Some expressed the need for an effortless, time-saving, and flexible system to ensure optimal clinical applicability [85, 90, 92]. Accordingly, stronger collaborations with developers and final users were considered essential . Finally, patient’s cognitive and communication deficits were evidenced as potential barriers in systems use affecting, in turn, therapy efficacy .
Virtual Reality and Robotics
Some of the studies presented the usability evaluation of combined VR and robotics. The majority implemented exoskeleton devices [93, 94, 97,98,99, 103, 104], whereas others used end-effector robots [94, 95, 104, 105]. Moreover, three studies described the use of soft-robots [100,101,102]. The combination with the VR exclusively consisted of non-immersive systems (e.g., monitor, PC), except for one study, which interestingly included both non-immersive and fully-immersive (i.e., CAVE) technologies . Combining the two different device typologies resulted in satisfactory usability rates. Most of the studies reported the rehabilitation program to be motivating, pleasant, and meaningful [94, 98, 99, 101,102,103] along with positive patients’ feedback on the devices ease-of-use [93, 95,96,97,98, 100, 102, 104,105,106], learnability [93, 96,97,98, 100,101,102, 104, 105] and satisfaction [95, 103]. One study highlighted patients’ perceived effectiveness of the device, which contributed to improve their quality of life . In general, no specific technical support was expressed, except for one study , which also pointed out patients’ desire to use the device again in the future, though . Some limitations were remarked too. These included the robotic active assistance of some devices, which was considered in some cases discordant with patient’s intended movement . In some cases, the mechanical complexity of the devices did not allow a fluent control, which in turn affected the interaction with the virtual objects displayed in the VE and led patients to experience some frustration .
As for the healthcare professionals, only one study describing the combination of robotics with VR was included . The possibility to associate the robot-assisted therapy with the implementation of software interfaces, which provided visual feedback to the patient and consequently improved treatment motivation and compliance was appreciated. Lastly, among the suggestions are the preference of a wider range of VR games to avoid patient’s interest loss during the session and the integration of more therapy techniques-specific game exercises to maximise rehabilitation efficacy.
The present study aimed at systematically reviewing the current literature on the usability of VR and robotic devices applied to neuromotor rehabilitation. Both patients and healthcare professionals’ perspective on this topic was considered.
From the review process, 68 studies were included and synthetized. Promisingly, the number of studies describing the perceived acceptability of the devices has been increasing sensibly through time. As shown in the current review, in the last two decades most of the studies date back within the last five years, meaning that the growing interest in evaluating the usability issues of VR and robotics in neuromotor rehabilitation is rapid and recent. Most of the included researches reported the application of VR, whereas those describing the use of robotics or their combination represented the minority. The absence of an equal distribution in terms of studies numerousness may be attributable to the economic considerations toward the use of robot technology. Indeed, robot-assisted rehabilitation requires higher levels of investments, and its maintenance and routine operation are recognized to be relatively costly. This consequently leads to a growing need for a cost-effectiveness analysis , which for the implementation of VR systems has by contrast received a clearer consensus, as it was evidenced a wider accessible strategy that provides high recovery benefits along with lower costs. Purposely, off-the-shelf and commercially available video gaming systems have been already proposed and adapted for use in VR rehabilitation showing satisfactory results .
Despite these differences, the implementation of robotics and VR in rehabilitation programs entails higher costs when compared to conventional treatment, making the research on this topic more expensive. Accordingly, the majority of the studies included in the present review received a financial support whether they were pilot feasibility researches or clinical trials. Moreover, these studies were essentially carried out by research groups from high-ranked countries according to the HDI  with evidently more economic resources to be invested in this type of studies.
Despite the resources, it should be pointed out that one study out of three was pilot and lacked further evidence, though. Notably, most explored the benefits and the limitations of the technological devices and gave insight into feasibility and formal testing, but almost none moved from intervention efficacy trials to scale-up evaluations in real-life settings. The transition from efficacy to effectiveness represents a still open challenge that future research on this topic should consider net of the issues related to the complexity and the cost of the devices, as well as the characteristics of the population involved . Following this line, addressing these issues and including questions related to the needs and the perception of the end-user from the earlier intervention phases should be of paramount concern and would have the potential to facilitate the scaling-up process . In support of this, longitudinal study designs may also contribute to provide clearer evidence on the effectiveness of technological rehabilitation, as follow-up evaluations would shed light on the effects on patients over time. Most of the synthetized studies did not include follow-ups, and those reporting were not over three months. Future research should take into account this aspect and include larger follow-ups to better clarify the perceived effectiveness and the resonance of VR and robotics in real-life.
A heterogeneity of authors contributions emerged within research groups including different professional categories. Accordingly, not only healthcare professionals contributed, but also experts from different fields of engineering. This reflects the current trend of a constantly evolving healthcare system that encompasses differentiated resources designed to innovation and progress in recovery processes. However, although usability and the experience of use of the devices are mainly recognized to be psychological-related constructs, a paucity of studies including a contribution from the psychological field was observed. In this vein, future studies interested in such aspects should involve more specialised professional figures to discuss the findings more appropriately.
Further heterogeneity was observed concerning the participants characteristics, VR systems and robot typologies used. The included studies involved patients with different neurological and musculoskeletal pathologies and healthcare professionals of different occupations. Although most of the patients suffered from stroke, this main prevalence is not surprising if we consider that nowadays the estimation of stroke cases is 200 per 100.000 inhabitants (70% over 64 years) with an increasing global burden in both sexes [112, 113]. Additionally, a wide range in terms of sample size was observed for both groups of participants. Despite this, most of the articles involved less than 30 patients and less than 10 healthcare professionals. In future studies, larger sample sizes are recommended. Lastly, of the socio-demographic data extracted, only one study reported the ethnicity of the samples. As suggested in a prior work , future clinical research should be encouraged to report participants’ ethnicity diversity for multiple reasons, including increased results generalisability. Concerning the technological devices used, different VR systems with different levels of immersion and various robot typologies were implemented. Regarding specifically the VR, the level of immersion provided by the systems implemented was not always clearly defined within the studies included. This reflects, at least in the field of neuromotor rehabilitation, the absence of a clear consensus on VR immersion classification, making it difficult to navigate the literature and generalize the effectiveness and feasibility of specific rehabilitation systems . Following this line, future studies in this field should adopt a shared framework in order to explain and better clarify the key characteristics of VR and their clinical implications when implementing a specific device. Accordingly, immersion, sense of presence, and interactivity should be considered as key aspects existing on a continuum and, consequently, should be targeted adopting a framework able to address their complexity and the extent of the underlying interacting elements [116, 117]. Overall, the heterogeneity observed in terms of device typology implementation may be ascribable to an absence of clear guidelines orienting the choice of a certain device in relation to specific therapeutic goals and, also, to a widespread difficulty in dealing with the devices upgrade because of the costs and design complexity. Moreover, the ever-growing number of companies providing new devices that require initial testing may represents a possible further explanation.
Usability evaluation was mixed-methods and focused on multiple factors that varied depending on the device typology. Mutual factors included the level of ease-of-use, the degree of satisfaction, and learnability. Others like the level of enjoyment, motivation, and sense of presence were more related to the interaction with VEs, while parameters such as the perceived safety, the level of comfort, and the perceived effectiveness were mostly observed in robot-assisted rehabilitation. Generally, the use of VR [39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78] and robotics [79,80,81,82,83,84,85,86,87,88,89,90,91,92], as well as their combination [93,94,95,96,97,98,99,100,101,102,103,104,105,106], provided informative insights into strengths and limitations of their deployment (Fig. 3). It must be noted that the number of studies describing the usability of combined robotic and VR devices was not sufficient to provide robust inferences. Accordingly, a deeper analysis of the independent implementation of the two devices was preferred. This choice was also based on the fact that, as previously explained, although the combination of robotic and VR devices has so far provided promising evidence regarding the clinical impact, they remain different technologies as for technical and interactivity issues, ultimately needing a differentiated examination on the related experience of use.
As to strengths, patients stated that VR improved their engagement and motivation during the treatment, denoting an overall acceptance of the devices. Besides, positive effects of the devices on their physical health were perceived right after the treatment. Consistently, healthcare professionals provided positive feedback, suggesting the potential of VR for more tailored interventions through the high clinical applicability to different diseases. As for robotics, patients generally felt satisfied with the interventions showing high levels of compliance to the treatment. The assistance of robotics provided lasting effects on their physical health, and a positive impact on the level of independence and on the psychosocial well-being was perceived. Healthcare professionals agreed with the wide applicability of the devices underscoring their potential for increased treatment safety and intensity and the opportunity to quantitatively monitor patient’s recovery process.
Concerning the limits, learnability was cited by both patients and healthcare professionals as a factor affecting initial sessions performance. Some declared that tutorials or training sessions before the treatment would have benefited the use of the devices. Moreover, VR sessions length and the multiple tasks were judged by patients as requiring more mental effort than usual leading, in some cases, to cognitive workload. Also, technical complexity of the system was perceived as affecting the level of interactivity with the VE raising a need for technical support throughout the intervention. Besides, healthcare professionals expressed the need for adaptable parameters to patient’s level and for lower complexity of the VE graphic design to avoid patient’s distraction. Regarding robotics, patients complained about the mechanical resistance or the low comfort of some systems, which made the treatment, in some cases, physically straining. Moreover, a need for a wider applicability across the continuum of care was expressed along with the desire to use the devices outside the clinical setting. Lastly, patients’ cognitive and communication deficits were considered as potential barriers for device use, potentially affecting therapy efficacy. Overall, devices technical complexity and learnability seemed to represent a relevant limit.
According to the above reported comments, to ensure optimal clinical applicability, healthcare professionals suggested a stronger collaboration with developers and end-users. Besides the well-known utility and efficacy, the introduction and the deployment of technologies to rehabilitation programs may not always result easy, especially for the healthcare professionals whose perspective on their work modalities may change requiring time to adapt. Future studies should address this aspect in real-life settings to better respond to the usability issues of such technologies. From the earlier phases of the intervention, making the devices easier to learn and easier to use would help the end-user to optimally benefit from technological rehabilitation, so as to maximise its effectiveness.
Availability of data and materials
Activities of daily living
Cave automatic virtual environment
Preferred reporting items for systematic reviews and meta-analyses
Human development index
Yakub F, Md Khudzari AZ, Mori Y. Recent trends for practical rehabilitation robotics, current challenges and the future. Int J Rehabil Res. 2014;37:9–21. https://doi.org/10.1097/MRR.0000000000000035.
Tieri G, Morone G, Paolucci S, Iosa M. Virtual reality in cognitive and motor rehabilitation: facts, fiction and fallacies. Expert Rev Med Devices. 2018;15:107–17. https://doi.org/10.1080/17434440.2018.1425613.
Iandolo R, Marini F, Semprini M, Laffranchi M, Mugnosso M, Cherif A, et al. Perspectives and Challenges in Robotic Neurorehabilitation. Appl Sci. 2019;9:3183. https://doi.org/10.3390/app9153183.
Giansanti D. The Rehabilitation and the Robotics: Are They Going Together Well? Healthcare (Basel). 2020;9:26. https://doi.org/10.3390/healthcare9010026.
Sawicki GS, Beck ON, Kang I, Young AJ. The exoskeleton expansion: improving walking and running economy. J Neuroeng Rehabil. 2020;17:25. https://doi.org/10.1186/s12984-020-00663-9.
Mehrholz J, Pollock A, Pohl M, Kugler J, Elsner B. Systematic review with network meta-analysis of randomized controlled trials of robotic-assisted arm training for improving activities of daily living and upper limb function after stroke. J Neuroeng Rehabil. 2020;17:83. https://doi.org/10.1186/s12984-020-00715-0.
Maranesi E, Riccardi GR, Di Donna V, Di Rosa M, Fabbietti P, Luzi R, et al. Effectiveness of Intervention Based on End-effector Gait Trainer in Older Patients With Stroke: A Systematic Review. J Am Med Dir Assoc. 2020;21:1036–44. https://doi.org/10.1016/j.jamda.2019.10.010.
Molteni F, Gasperini G, Cannaviello G, Guanziroli E. Exoskeleton and End-Effector Robots for Upper and Lower Limbs Rehabilitation: Narrative Review. PM R. 2018;10(Suppl 2):S174–88. https://doi.org/10.1016/j.pmrj.2018.06.005.
Chu CY, Patterson RM. Soft robotic devices for hand rehabilitation and assistance: a narrative review. J Neuroeng Rehabil. 2018;15:9. https://doi.org/10.1186/s12984-018-0350-6.
Dockx K, Bekkers EM, Van den Bergh V, Ginis P, Rochester L, Hausdorff JM, et al. Virtual reality for rehabilitation in Parkinson’s disease. Cochrane Database Syst Rev. 2016;12:CD010760. https://doi.org/10.1002/14651858.CD010760.pub2.
Lange BS, Requejo P, Flynn SM, Rizzo A. The potential of virtual reality and gaming to assist successful aging with disability. Phys Med Rehabil Clin N Am. 2010;21:339–56. https://doi.org/10.1016/j.pmr.2009.12.007.
Massetti T, da Silva TD, Crocetta TB, Guarnieri R, De Freitas BL, Bianchi Lopes P, et al. The Clinical Utility of Virtual Reality in Neurorehabilitation: A Systematic Review. J Cent Nerv Syst Dis. 2018;10:1179573518813541. https://doi.org/10.1177/1179573518813541.
Baus O, Bouchard S. Moving from virtual reality exposure-based therapy to augmented reality exposure-based therapy: a review. Front Hum Neurosci. 2014;8:112. https://doi.org/10.3389/fnhum.2014.00112.
Ryan ML. Narrative as Virtual Reality 2: Revisiting Immersion and Interactivity in Literature and Electronic Media. Baltimore, MD: JHU Press; 2015.
Rose T, Nam CS, Chen KB. Immersion of virtual reality for rehabilitation - Review. Appl Ergon. 2018;69:153–61. https://doi.org/10.1016/j.apergo.2018.01.009.
Mujber TS, Szecsi T, Hashmi MSJ. Virtual reality applications in manufacturing process simulation. J Mat Process Technol. 2004;156:1834–8. https://doi.org/10.1016/j.jmatprotec.2004.04.401.
Howard MC. A meta-analysis and systematic literature review of virtual reality rehabilitation programs. Comput Hum Behav. 2017;70:317–27. https://doi.org/10.1016/j.chb.2017.01.013.
International Organization for Standardization/International Electrotechnical Commission ISO/IEC 9241–14 Ergonomic requirements for office work with visual display terminals (VDT)s - Part 14 Menu dialogues: ISO/IEC 9241–14. Switzerland; 1998. https://www.sis.se/api/document/preview/611299/.
Nielsen J. Usability 101: Introduction to Usability. Fremont, CA: Nielsen Norman Group; 2012
Monardo G, Pavese C, Giorgi I, Godi M, Colombo R. Evaluation of Patient Motivation and Satisfaction During Technology-Assisted Rehabilitation: An Experiential Review. Games Health J. 2021;10:13–27. https://doi.org/10.1089/g4h.2020.0024.
Tousignant M, Boissy P, Moffet H, Corriveau H, Cabana F, Marquis F, et al. Patients’ satisfaction of healthcare services and perception with in-home telerehabilitation and physiotherapists’ satisfaction toward technology for post-knee arthroplasty: an embedded study in a randomized trial. Telemed J E Health. 2011;17:376–82. https://doi.org/10.1089/tmj.2010.0198.
Luker J, Lynch E, Bernhardsson S, Bennett L, Bernhardt J. Stroke Survivors’ Experiences of Physical Rehabilitation: A Systematic Review of Qualitative Studies. Arch Phys Med Rehabil. 2015;96:1698-708.e10. https://doi.org/10.1016/j.apmr.2015.03.017.
Abran A, Khelifi A, Suryn W, Seffah A. Usability meanings and interpretations in ISO standards. Software Qual J. 2003;11:325–38. https://doi.org/10.1023/A:1025869312943.
Vermeeren, APOS, Law ELC, Roto V, Obrist M, Hoonhout J, Väänänen-Vainio-Mattila K. User experience evaluation methods:current state and development needs. In: Proceedings of the 6th Nordic Conference on Human-Computer Interaction. 2010;521–530. doi:https://doi.org/10.1145/1868914.1868973
Hass C, Berlin D. Usability testing medical devices: A practical guide to minimizing risk and maximizing success. In: Proceedings of the International Conference of Design, User Experience, and Usability. 2013;407–416. doi:https://doi.org/10.1007/978-3-642-39241-2_45
World Health Organization. World Report on Ageing and Health. Luxembourg: World Health Organization; 2015.
Cieza A, Causey K, Kamenov K, Hanson SW, Chatterji S, Vos T. Global estimates of the need for rehabilitation based on the Global Burden of Disease study 2019: a systematic analysis for the Global Burden of Disease Study 2019. Lancet. 2021;396:2006–17. https://doi.org/10.1016/S0140-6736(20)32340-0.
Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6:e1000097. https://doi.org/10.1371/journal.pmed.1000097.
Law M, Stewart D, Pollock N, Letts L, Bosch J, Westmorland M. Critical review form - Quantitative studies. 1998 https://www.unisa.edu.au/siteassets/episerver-6-files/global/health/sansom/documents/icahe/cats/mcmasters_quantitative-review.pdf Accessed 15 Apr 2021.
Letts L, Wilkins S, Law M, Stewart D, Bosch J, Westmorland M. Critical Review Form - Qualitative Studies (Version 2.0). 2007. https://www.unisa.edu.au/siteassets/episerver-6-files/global/health/sansom/documents/icahe/cats/mcmasters_qualreview_version2-0.pdf Accessed 15 Apr 2021.
Law M, Stewart D, Pollock N, Letts L, Bosch J, Westmorland M. Guidelines for Critical Review Form - Quantitative Studies. 1998. https://canchild.ca/system/tenon/assets/attachments/000/000/366/original/quantguide.pdf Accessed 15 Apr 2021.
Letts L, Wilkins S, Law M, Stewart D, Bosch J, Westmorland M. Guidelines for Critical Review Form: Qualitative Studies (Version 2.0). 2007. https://www.canchild.ca/system/tenon/assets/attachments/000/000/360/original/qualguide.pdf Accessed 15 Apr 2021.
Imms C. Children with cerebral palsy participate: a review of the literature. Disabil Rehabil. 2008;30:1867–84. https://doi.org/10.1080/09638280701673542.
Noyes J, Hannes K, Booth A, Harris J, Harden A, Popay J, et al. The Cochrane Qualitative and Implementation Methods Group. Chapter QQ: Qualitative and implementation evidence and Cochrane Reviews. Version 1.02a. 2013. QQ1–QQ25. https://methods.cochrane.org/sites/methods.cochrane.org.qi/files/public/uploads/Handbook52_QQ_Qualitative_web%20update%20Oct%202015.pdf Accessed 15 Apr 2021.
Human Development Report. Human Development Index and its components. http://hdr.undp.org/en Accessed 01 Apr 2021
Bowling A. Research methods in health: investigating health and health services. 4th ed. UK: McGraw-hill education; 2014. https://books.google.it/books?hl=it&lr=&lid=6lOLBgAAQBAJ&loi=fnd&lpg=PR3&ldq=Bowling+A.+Research+methods+in+health:+investigating+health+and+health+services.+4th+ed.+UK:+McGrawhill+education%3B+2014&lots=YfHf9zfOhG&lsig=bOsKFVqcTd8eNxXB0A2C_rBBVt0&lredir_esc=y#v=onepage&lq=Bowling%20A.%20Research%20methods%20in%20health%3A%20investigating%20health%20education%3B%202014&lf=false.
Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol. 2008;8:45. https://doi.org/10.1186/1471-2288-8-45.
Sandelowski M, Voils CI, Barroso J. Defining and Designing Mixed Research Synthesis Studies. Res Sch. 2006;13:29.
Afyouni I, Murad A, Einea A. Adaptive Rehabilitation Bots in Serious Games. Sensors (Basel). 2020;20:7037. https://doi.org/10.3390/s20247037.
Afyouni I, Rehman FU, Qamar AM, Ghani S, Hussain SO, Sadiq B, et al. A therapy-driven gamification framework for hand rehabilitation. User Model User-Adap Inter. 2017;27:215–65. https://doi.org/10.1007/s11257-017-9191-4.
Alves T, Carvalho H, Simões LD. Winning compensations: Adaptable gaming approach for upper limb rehabilitation sessions based on compensatory movements. J Biomed Inform. 2020;108: 103501. https://doi.org/10.1016/j.jbi.2020.103501.
Avola D, Cinque L, Foresti GL, Marini MR. An interactive and low-cost full body rehabilitation framework based on 3D immersive serious games. J Biomed Inform. 2019;89:81–100. https://doi.org/10.1016/j.jbi.2018.11.012.
Avola D, Cinque L, Foresti GL, Marini MR, Pannone D. VRheab: a fully immersive motor rehabilitation system based on recurrent neural network. Multimed Tools Appl. 2018;77:24955–82. https://doi.org/10.1007/s11042-018-5730-1.
Brokaw EB, Eckel E, Brewer BR. Usability evaluation of a kinematics focused Kinect therapy program for individuals with stroke. Technol Health Care. 2015;23:143–51. https://doi.org/10.3233/THC-140880.
Cameirão MS, Badia SB, Oller ED, Verschure PF. Neurorehabilitation using the virtual reality based Rehabilitation Gaming System: methodology, design, psychometrics, usability and validation. J Neuroeng Rehabil. 2010;7:48. https://doi.org/10.1186/1743-0003-7-48.
Chughtai M, Kelly JJ, Newman JM, Sultan AA, Khlopas A, Sodhi N, et al. The Role of Virtual Rehabilitation in Total and Unicompartmental Knee Arthroplasty. J Knee Surg. 2019;32:105–10. https://doi.org/10.1055/s-0038-1637018.
Colomer C, Llorens R, Noé E, Alcañiz M. Effect of a mixed reality-based intervention on arm, hand, and finger function on chronic stroke. J Neuroeng Rehabil. 2016;13:45. https://doi.org/10.1186/s12984-016-0153-6.
Deutsch JA, Lewis JA, Whitworth E, Boian R, Burdea G, Tremaine M. Formative Evaluation and Preliminary Findings of a Virtual Reality Telerehabilitation System for the Lower Extremity. Presence-Teleop Virt. 2005;14:198–213. https://doi.org/10.1162/1054746053967030.
Epelde G, Carrasco E, Rajasekharan S, Jimenez JM, Vivanco K, Gomez-Fraga I, et al. Universal Remote Delivery of Rehabilitation: Validation with Seniors’ Joint Rehabilitation Therapy. Cybern Syst. 2014;45:109–22. https://doi.org/10.1080/01969722.2014.874807.
Foreman MH, Engsberg JR. A virtual reality tool for measuring and shaping trunk compensation for persons with stroke: Design and initial feasibility testing. J Rehabil Assist Technol Eng. 2019;6:2055668318823673. https://doi.org/10.1177/2055668318823673.
Fuertes Muñoz G, Mollineda RA, Gallardo Casero J, Pla F. A RGBD-Based Interactive System for Gaming-Driven Rehabilitation of Upper Limbs. Sensors (Basel). 2019;19:3478. https://doi.org/10.3390/s19163478.
Held JP, Ferrer B, Mainetti R, Steblin A, Hertler B, Moreno-Conde A, et al. Autonomous rehabilitation at stroke patients home for balance and gait: safety, usability and compliance of a virtual reality system. Eur J Phys Rehabil Med. 2018;54:545–53. https://doi.org/10.23736/S1973-9087.17.04802-X.
Hoermann S, Ferreira Dos Santos L, Morkisch N, Jettkowski K, Sillis M, Devan H, et al. Computerised mirror therapy with Augmented Reflection Technology for early stroke rehabilitation: clinical feasibility and integration as an adjunct therapy. Disabil Rehabil. 2017;39:1503–14. https://doi.org/10.1080/09638288.2017.1291765.
Iosa M, Aydin M, Candelise C, Coda N, Morone G, Antonucci G, et al. The Michelangelo Effect: Art Improves the Performance in a Virtual Reality Task Developed for Upper Limb Neurorehabilitation. Front Psychol. 2021;11: 611956. https://doi.org/10.3389/fpsyg.2020.611956.
Jansen-Kosterink SM, Huis In 't Veld RM, Schönauer C, Kaufmann H, Hermens HJ, Vollenbroek-Hutten MM. A Serious Exergame for Patients Suffering from Chronic Musculoskeletal Back and Neck Pain: A Pilot Study. Games Health J. 2013;2:299–307. doi:https://doi.org/10.1089/g4h.2013.0043
Janssen S, Bolte B, Nonnekes J, Bittner M, Bloem BR, Heida T, et al. Usability of Three-dimensional Augmented Visual Cues Delivered by Smart Glasses on (Freezing of) Gait in Parkinson’s Disease. Front Neurol. 2017;8:279. https://doi.org/10.3389/fneur.2017.00279.
Kizony R, Weiss PL, Shahar M. TheraGame: A home based virtual reality rehabilitation system. Int J Disabil Hum Dev. 2006;5:265–70. https://doi.org/10.1515/IJDHD.2006.5.3.265.
Kumar D, González A, Das A, Dutta A, Fraisse P, Hayashibe, et al. Virtual Reality-Based Center of Mass-Assisted Personalized Balance Training System. Front Bioeng Biotechnol. 2018;5:85. doi:https://doi.org/10.3389/fbioe.2017.00085
Lange B, Flynn S, Rizzo A. Initial usability assessment of off-the-shelf video game consoles for clinical game-based motor rehabilitation. Phys Ther Rev. 2009;14:355–63. https://doi.org/10.1179/108331909X12488667117258.
Lange B, Flynn S, Proffitt R, Chang CY, Rizzo AS. Development of an interactive game-based rehabilitation tool for dynamic balance training. Top Stroke Rehabil. 2010;17:345–52. https://doi.org/10.1310/tsr1705-345.
Laver K, Lim F, Reynolds K, George S, Ratcliffe J, Sim S, et al. Virtual Reality Grocery Shopping Simulator: Development and Usability in Neurological Rehabilitation. Presence: Presence-Teleop Virt. 2012;21:183–191. doi:https://doi.org/10.1162/PRES_a_00098
Lee MM, Shin DC, Song CH. Canoe game-based virtual reality training to improve trunk postural stability, balance, and upper limb motor function in subacute stroke patients: a randomized controlled pilot study. J Phys Ther Sci. 2016;28:2019–24. https://doi.org/10.1589/jpts.28.2019.
Lee SH, Jung HY, Yun SJ, Oh BM, Seo HG. Upper Extremity Rehabilitation Using Fully Immersive Virtual Reality Games With a Head Mount Display: A Feasibility Study. PM R. 2020;12:257–62. https://doi.org/10.1002/pmrj.12206.
Ling Y, Ter Meer LP, Yumak Z, Veltkamp RC. Usability Test of Exercise Games Designed for Rehabilitation of Elderly Patients After Hip Replacement Surgery: Pilot Study. JMIR Serious Games. 2017;5: e19. https://doi.org/10.2196/games.7969.
Lloréns R, Colomer-Font C, Alcañiz M, Noé-Sebastián E. BioTrak virtual reality system: effectiveness and satisfaction analysis for balance rehabilitation in patients with brain injury. Neurologia. 2013;28:268–75. https://doi.org/10.1016/j.nrl.2012.04.016.
Lloréns R, Gil-Gómez JA, Alcañiz M, Colomer C, Noé E. Improvement in balance using a virtual reality-based stepping exercise: a randomized controlled trial involving individuals with chronic stroke. Clin Rehabil. 2015;29:261–8. https://doi.org/10.1177/0269215514543333.
Lloréns R, Noé E, Colomer C, Alcañiz M. Effectiveness, usability, and cost-benefit of a virtual reality-based telerehabilitation program for balance recovery after stroke: a randomized controlled trial. Arch Phys Med Rehabil. 2015;96:418-425.e2. https://doi.org/10.1016/j.apmr.2014.10.019.
Lozano-Quilis JA, Gil-Gómez H, Gil-Gómez JA, Albiol-Perez S, Palacios-Navarro G, Fardoun HM. et al. Virtual rehabilitation for multiple sclerosis using a kinect-based system: randomized controlled trial. JMIR Serious Games. 2014;2:e12. doi:https://doi.org/10.2196/games.2933
Meldrum D, Glennon A, Herdman S, Murray D, McConn-Walsh R. Virtual reality rehabilitation of balance: assessment of the usability of the Nintendo Wii(®) Fit Plus. Disabil Rehabil Assist Technol. 2012;7:205–10. https://doi.org/10.3109/17483107.2011.616922.
Seo NJ, Arun Kumar J, Hur P, Crocher V, Motawar B, Lakshminarayanan K. Usability evaluation of low-cost virtual reality hand and arm rehabilitation games. J Rehabil Res Dev. 2016;53:321–34. https://doi.org/10.1682/JRRD.2015.03.0045.
Neil A, Ens S, Pelletier R, Jarus T, Rand D. Sony PlayStation EyeToy elicits higher levels of movement than the Nintendo Wii: implications for stroke rehabilitation. Eur J Phys Rehabil Med. 2013;49:13–21.
Putrino D, Zanders H, Hamilton T, Rykman A, Lee P, Edwards DJ. Patient Engagement Is Related to Impairment Reduction During Digital Game-Based Therapy in Stroke. Games Health J. 2017;6:295–302. https://doi.org/10.1089/g4h.2016.0108.
Shin JH, Ryu H, Jang SH. A task-specific interactive game-based virtual reality rehabilitation system for patients with stroke: a usability test and two clinical experiments. J Neuroeng Rehabil. 2014;11:32. https://doi.org/10.1186/1743-0003-11-32.
Su CH. Developing and evaluating effectiveness of 3D game-based rehabilitation system for Total Knee Replacement Rehabilitation patients. Multimed Tools Appl. 2016;75:10037–57. https://doi.org/10.1007/s11042-015-2820-1.
van Beek JJW, van Wegen EEH, Bohlhalter S, Vanbellingen T. Exergaming-Based Dexterity Training in Persons With Parkinson Disease: A Pilot Feasibility Study. J Neurol Phys Ther. 2019;43:168–74. https://doi.org/10.1097/NPT.0000000000000278.
van den Berg M, Sherrington C, Killington M, Smith S, Bongers B, Hassett L, et al. Video and computer-based interactive exercises are safe and improve task-specific balance in geriatric and neurological rehabilitation: a randomised trial. J Physiother. 2016;62:20–8. https://doi.org/10.1016/j.jphys.2015.11.005.
Vanbellingen T, Filius SJ, Nyffeler T, van Wegen EEH. Usability of Videogame-Based Dexterity Training in the Early Rehabilitation Phase of Stroke Patients: A Pilot Study. Front Neurol. 2017;8:654. https://doi.org/10.3389/fneur.2017.00654.
Weber LM, Nilsen DM, Gillen G, Yoon J, Stein J. Immersive Virtual Reality Mirror Therapy for Upper Limb Recovery After Stroke: A Pilot Study. Am J Phys Med Rehabil. 2019;98:783–8. https://doi.org/10.1097/PHM.0000000000001190.
Aprile I, Cruciani A, Germanotta M, Gower V, Pecchioli C, Cattaneo D, et al. Upper Limb Robotics in Rehabilitation: An Approach to Select the Devices, Based on Rehabilitation Aims, and Their Evaluation in a Feasibility Study. Appl Sci. 2019;9:3920. https://doi.org/10.3390/app9183920.
Bortole M, Venkatakrishnan A, Zhu F, Moreno JC, Francisco GE, Pons JL, et al. The H2 robotic exoskeleton for gait rehabilitation after stroke: early findings from a clinical study. J Neuroeng Rehabil. 2015;12:54. https://doi.org/10.1186/s12984-015-0048-y.
Eicher C, Haesner M, Spranger M, Kuzmicheva O, Gräser A, Steinhagen-Thiessen E. Usability and acceptability by a younger and older user group regarding a mobile robot-supported gait rehabilitation system. Assist Technol. 2019;31:25–33. https://doi.org/10.1080/10400435.2017.1352051.
Gagnon DH, Vermette M, Duclos C, Aubertin-Leheudre M, Ahmed S, Kairy D. Satisfaction and perceptions of long-term manual wheelchair users with a spinal cord injury upon completion of a locomotor training program with an overground robotic exoskeleton. Disabil Rehabil Assist Technol. 2019;14:138–45. https://doi.org/10.1080/17483107.2017.1413145.
Hughes AM, Burridge J, Freeman CT, Donnovan-Hall M, Chappell PH, Lewin PL, et al. Stroke participants’ perceptions of robotic and electrical stimulation therapy: a new approach. Disabil Rehabil Assist Technol. 2011;6:130–8. https://doi.org/10.3109/17483107.2010.509882.
Kwon SH, Lee BS, Lee HJ, Kim EJ, Lee JA, Yang SP, et al. Energy Efficiency and Patient Satisfaction of Gait With Knee-Ankle-Foot Orthosis and Robot (ReWalk)-Assisted Gait in Patients With Spinal Cord Injury. Ann Rehabil Med. 2020;44:131–41. https://doi.org/10.5535/arm.2020.44.2.131.
Nam HS, Hong N, Cho M, Lee C, Seo HG, Kim S. Vision-Assisted Interactive Human-in-the-Loop Distal Upper Limb Rehabilitation Robot and its Clinical Usability Test. Appl Sci. 2019;9:3106. https://doi.org/10.3390/app9153106.
Palmcrantz S, Plantin J, Borg J. Factors affecting the usability of an assistive soft robotic glove after stroke or multiple sclerosis. J Rehabil Med. 2020;52:jrm00027. doi:https://doi.org/10.2340/16501977-2650
Poritz JMP, Taylor HB, Francisco G, Chang SH. User satisfaction with lower limb wearable robotic exoskeletons. Disabil Rehabil Assist Technol. 2020;15:322–7. https://doi.org/10.1080/17483107.2019.1574917.
Radder B, Prange-Lasonder GB, Kottink AIR, Holmberg J, Sletta K, Van Dijk M, et al. The effect of a wearable soft-robotic glove on motor function and functional performance of older adults. Assist Technol. 2020;32:9–15. https://doi.org/10.1080/10400435.2018.1453888.
Stellin G, Sale P, Masiero S, Becchi F, Sieklicki W. Development and test of fex, a fingers extending exoskeleton for rehabilitation and regaining mobility. Int J Mech Control. 2018;19:3–14.
Swinnen E, Lefeber N, Willaert W, De Neef F, Bruyndonckx L, Spooren A, et al. Motivation, expectations, and usability of a driven gait orthosis in stroke patients and their therapists. Top Stroke Rehabil. 2017;24:299–308. https://doi.org/10.1080/10749357.2016.1266750.
Tsai YL, Huang JJ, Pu SW, Chen HP, Hsu SC, Chang JY, et al. Usability Assessment of a Cable-Driven Exoskeletal Robot for Hand Rehabilitation. Front Neurorobot. 2019;13:3. https://doi.org/10.3389/fnbot.2019.00003.
Vaughan-Graham J, Brooks D, Rose L, Nejat G, Pons J, Patterson K. Exoskeleton use in post-stroke gait rehabilitation: a qualitative study of the perspectives of persons post-stroke and physiotherapists. J Neuroeng Rehabil. 2020;17:123. https://doi.org/10.1186/s12984-020-00750-x.
Amirabdollahian F, Ates S, Basteris A, Cesario A, Buurke J, Hermens H, et al. Design, development and deployment of a hand/wrist exoskeleton for home-based rehabilitation after stroke - SCRIPT project. Robotica. 2014;32:1331–46. https://doi.org/10.1017/S0263574714002288.
Guidali M, Duschau-Wicke A, Broggi S, Klamroth-Marganska V, Nef T, Riener R. A robotic system to train activities of daily living in a virtual environment. Med Biol Eng Comput. 2011;49:1213–23. https://doi.org/10.1007/s11517-011-0809-0.
Lam P, Hebert D, Boger J, Lacheray H, Gardner D, Apkarian J, et al. A haptic-robotic platform for upper-limb reaching stroke therapy: preliminary design and evaluation results. J Neuroeng Rehabil. 2008;5:15. https://doi.org/10.1186/1743-0003-5-15.
Lledó LD, Díez JA, Bertomeu-Motos A, Ezquerro S, Badesa FJ, Sabater-Navarro JM, et al. A Comparative Analysis of 2D and 3D Tasks for Virtual Reality Therapies Based on Robotic-Assisted Neurorehabilitation for Post-stroke Patients. Front Aging Neurosci. 2016;8:205. https://doi.org/10.3389/fnagi.2016.00205.
Manuli A, Maggio MG, Tripoli D, Gullì M, Cannavò A, La Rosa G, et al. Patients’ perspective and usability of innovation technology in a new rehabilitation pathway: An exploratory study in patients with multiple sclerosis. Mult Scler Relat Disord. 2020;44: 102312. https://doi.org/10.1016/j.msard.2020.102312.
Nijenhuis SM, Prange GB, Amirabdollahian F, Sale P, Infarinato F, Nasr N, et al. Feasibility study into self-administered training at home using an arm and hand device with motivational gaming environment in chronic stroke. J Neuroeng Rehabil. 2015;12:89. https://doi.org/10.1186/s12984-015-0080-y.
Park JH, Park G, Kim HY, Lee JY, Ham Y, Hwang D, et al. A comparison of the effects and usability of two exoskeletal robots with and without robotic actuation for upper extremity rehabilitation among patients with stroke: a single-blinded randomised controlled pilot study. J Neuroeng Rehabil. 2020;17:137. https://doi.org/10.1186/s12984-020-00763-6.
Radder B, Prange-Lasonder GB, Kottink AIR, Holmberg J, Sletta K, van Dijk M, et al. Home rehabilitation supported by a wearable soft-robotic device for improving hand function in older adults: A pilot randomized controlled trial. PLoS ONE. 2019;14: e0220544. https://doi.org/10.1371/journal.pone.0220544.
Radder B, Prange-Lasonder GB, Kottink AI, Gaasbeek L, Holmberg J, Meyer T, et al. A wearable soft-robotic glove enables hand support in ADL and rehabilitation: A feasibility study on the assistive functionality. J Rehabil Assist Technol Eng. 2016;3:2055668316670553. https://doi.org/10.1177/2055668316670553.
Radder B, Prange-Lasonder G, Kottink AIR, Melendez-Calderon A, Buurke JH, Rietman JS. Feasibility of a wearable soft-robotic glove to support impaired hand function in stroke patients. J Rehabil Med. 2018;50:598–606. https://doi.org/10.2340/16501977-2357.
Resquín F, Gonzalez-Vargas J, Ibáñez J, Brunetti F, Dimbwadyo I, Carrasco L, et al. Adaptive hybrid robotic system for rehabilitation of reaching movement after a brain injury: a usability study. J Neuroeng Rehabil. 2017;14:104. https://doi.org/10.1186/s12984-017-0312-4.
Simkins M, Fedulow I, Kim H, Abrams G, Byl N, Rosen J. Robotic Rehabilitation Game Design for Chronic Stroke. Games Health J. 2012;1:422–30. https://doi.org/10.1089/g4h.2012.0044.
Valdés BA, Van der Loos HFM. Biofeedback vs. game scores for reducing trunk compensation after stroke: a randomized crossover. Top Stroke Rehabil. 2018;25(2):96–113. doi:https://doi.org/10.1080/10749357.2017.1394633
Yeh SC, Lee SH, Chan RC, Wu Y, Zheng LR, Flynn S. The Efficacy of a Haptic-Enhanced Virtual Reality System for Precision Grasp Acquisition in Stroke Rehabilitation. J Healthc Eng. 2017;2017:9840273. https://doi.org/10.1155/2017/9840273.
Brooke J. SUS-A quick and dirty usability scale. Usab Eval Ind. 1996;189:4–7.
Calabrò RS, Müller-Eising C, Diliberti ML, Manuli A, Parrinello F, Rao G, et al. Who Will Pay for Robotic Rehabilitation? The Growing Need for a Cost-effectiveness Analysis. Innov Clin Neurosci. 2020;17:14–6.
Levac D, Espy D, Fox E, Pradhan S, Deutsch JE. “Kinect-ing” with clinicians: a knowledge translation resource to support decision making about video game use in rehabilitation. Phys Ther. 2015;95:426–40. https://doi.org/10.2522/ptj.20130618.
Glasgow RE, Lichtenstein E, Marcus AC. Why don’t we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition Am J Public Health. 2003;93:1261–7. https://doi.org/10.2105/ajph.93.8.1261.
Proffitt R, Lange B. Considerations in the efficacy and effectiveness of virtual reality interventions for stroke rehabilitation: moving the field forward. Phys Ther. 2015;95:441–8. https://doi.org/10.2522/ptj.20130571.
Guzik A, Bushnell C. Stroke Epidemiology and Risk Factor Management. Continuum (Minneap Minn). 2017;23:15–39. https://doi.org/10.1212/CON.0000000000000416.
Feigin VL, Norrving B, Mensah GA. Global Burden of Stroke. Circ Res. 2017;120:439–48. https://doi.org/10.1161/CIRCRESAHA.116.308413.
McCambridge AB, Elkins MR. If we can’t see race and ethnicity in research, how will we see racial inequality? J Physiother. 2021;67:82–3. https://doi.org/10.1016/j.jphys.2021.02.016.
Huygelier H, Mattheus E, Abeele VV, van Ee R, Gillebert CR. The Use of the Term Virtual Reality in Post-Stroke Rehabilitation: A Scoping Review and Commentary. Psychol Belg. 2021;61:145–62. https://doi.org/10.5334/pb.1033.
Høeg ER, Povlsen TM, Bruun-Pedersen JR, Lange B, Nilsson NC, Haugaard KB, et al. System immersion in virtual reality-based rehabilitation of motor function in older adults: a systematic review and meta-analysis. Front Virtual Real. 2021;2:30. https://doi.org/10.3389/frvir.2021.647993.
Maier M, Rubio Ballester B, Duff A, Duarte Oller E, Verschure P. Effect of Specific Over Nonspecific VR-Based Rehabilitation on Poststroke Motor Recovery: A Systematic Meta-analysis. Neurorehabil Neural Repair. 2019;33:112–29. https://doi.org/10.1177/1545968318820169.
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Zanatta, F., Giardini, A., Pierobon, A. et al. A systematic review on the usability of robotic and virtual reality devices in neuromotor rehabilitation: patients’ and healthcare professionals’ perspective. BMC Health Serv Res 22, 523 (2022). https://doi.org/10.1186/s12913-022-07821-w