Next Article in Journal
Approach for Optimisation of Tunnel Lining Design
Previous Article in Journal
A Long/Short-Term Memory Based Automated Testing Model to Quantitatively Evaluate Game Design
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Effectiveness of a Serious Game for Cognitive Training in Chronic Stroke Survivors with Mild-to-Moderate Cognitive Impairment: A Pilot Randomized Controlled Trial

1
College of Information and Computer Sciences, University of Massachusetts Amherst, Amherst, MA 01003, USA
2
Department of Rehabilitation and Movement Sciences, Rutgers University, New Brunswick, NJ 07107, USA
3
Woorisoft, Daegu 42781, Korea
4
Heeyeon Rehabilitation Hospital, Changwon 51420, Gyeongsangnam-do, Korea
*
Author to whom correspondence should be addressed.
Appl. Sci. 2020, 10(19), 6703; https://doi.org/10.3390/app10196703
Submission received: 27 August 2020 / Revised: 21 September 2020 / Accepted: 22 September 2020 / Published: 25 September 2020
(This article belongs to the Section Applied Biosciences and Bioengineering)

Abstract

:
Previous cognitive training games for stroke survivors required the close supervision of therapists. We aim to demonstrate the preliminary therapeutic effectiveness of Neuro-World, serious mobile games for cognitive training, in chronic stroke survivors with mild-to-moderate cognitive impairment without therapist supervision. For that, we conducted a randomized, controlled clinical trial at a single long-term care rehabilitation center with 50 stroke survivors in the chronic stage with mild-to-moderate cognitive impairment. Participants were randomized to standard medical care (n = 25) or standard medical care plus administration of Neuro-World (n = 25) over 12 weeks. A two-way mixed model ANOVA and Tukey’s post hoc tests identified significant differences in outcomes between the experimental and the control groups at post-intervention but not at baseline. Within the experimental group, there were statistically significant improvements in all the outcomes except for the language category of the Mini-Mental State Examination and Digit Forward Span. The improvements were clinically significant for the total Mini-Mental State Examination, Digit Forward Span, and Digit Backward Span. Within the control group, there were no improvements in any of the outcomes. The practice of Neuro-World led to significant improvements in cognitive function and marginal mitigation of depressive symptoms in chronic stroke survivors with mild-to-moderate cognitive impairment.

1. Introduction

Stroke is a leading cause of permanent disability, affecting nearly 800,000 individuals every year in the United States alone [1]. Although motor impairments are a hallmark of stroke, cognitive impairments are also prevalent in chronic-stage stroke survivors. These cognitive impairments are marked by, but not limited to, diminished memory, attention, and executive function [2,3]. In addition to affecting patients’ activities of daily living and quality of life, prior studies have shown that cognitive impairments can serve as a significant barrier to engagement in motor rehabilitation [4], thus affecting the recovery of motor function [5,6]. Moreover, it has previously been reported that cognitive impairments are associated with a decrease in the long-term survival rate [2] and increased symptoms of depression [7], which could lead to a deterioration of patients’ perceived well-being [8,9]. Therefore, stroke survivors with cognitive impairments often experience difficulty achieving independent living [2], which results in an increased rate of institutionalization [10] and cost of care [11]. The high financial burden of stroke care often drives the clinical communities to actively consider discharging stroke survivors early and engaging them with rehabilitation therapies in their residential settings [12]. However, patients have thus far shown poor adherence to rehabilitation therapies when close supervision by therapists is absent—for instance, in their home settings—thereby jeopardizing their continued recovery and maintenance of their function [13]. Therefore, it is essential to devise effective and engaging therapeutic interventions that can be self-administered by stroke survivors where access to therapists or in-person therapy is limited (e.g., patients with mobility limitation or in rural/underserved areas).
Computer-based cognitive training tools have received tremendous interests in the clinical and research communities to enable patients to self-administer therapy in their home and community settings [14,15,16]. However, patients who self-administer computer-based remote training often show poor adherence, and therapists’ frequent involvement was necessary to maintain patients’ engagement to the intervention [17]. As a potential means to enhance patients’ adherence to at-home cognitive training, serious games have been considered to be a viable solution for a wide range of populations, including healthy older adults [14,18], persons with mild cognitive impairments and/or neurological conditions [19,20,21], and survivors of acquired brain injury such as stroke and traumatic brain injury [15,16]. Serious games are video games designed for a primary purpose other than entertainment and enjoyment, such as educating, informing, and enhancing patients’ health-related conditions [22,23,24]. Serious games can complement conventional therapeutic programs to provide additional training in remote settings [25,26]. The entertainment components of serious games are an advantage to improve patients’ compliance to seemingly mundane, repetitive therapeutic activities (e.g., non-game-assisted, computer-based cognitive training) [27], which are a major barrier hindering patients’ potential functional recovery [28]. Furthermore, serious games could improve patients’ engagement during therapeutic activities, which may lead to greater neuroplastic changes and functional recovery in brain injury survivors [29].
Serious games for cognitive training in stroke survivors must be carefully designed to provide simple and user-friendly interfaces so that stroke survivors with motor and cognitive impairments can interact with the interfaces (e.g., a touchscreen or joystick) of the game systems without caregivers’ close assistance outside the clinical setting [30]. However, existing game-assisted cognitive training solutions do not provide adequate interfaces for individuals with some degree of motor impairments. For example, CogMed and RehaCom—the two most widely studied cognitive training game platforms—leverage a keyboard and a mouse [31,32,33] and a custom-designed keyboard-like interface [34,35,36,37], respectively. These interfaces require bi-manual and/or fine-hand motor skills [34,35,36,37], which could be especially challenging for stroke survivors who often have hemiparesis and limited distal upper-limb motor function [38]. Furthermore, these games have been primarily developed for their use in clinical settings, where therapists’ supervision is readily available. For instance, individuals with acquired brain injury who used RehaCom had to rely on their therapists to customize and administer game-based therapies [39].
Our research team has recently developed Neuro-World—a set of six serious games for cognitive training on mobile devices (e.g., smartphones and tablet computers)—that provides simple and user-friendly touchscreen interfaces so that stroke survivors, who often exhibit hemiparesis, can self-operate the technology [40]. In the current work, we examine the therapeutic effectiveness of our serious game-based training on improving cognitive function via a randomized controlled trial in chronic-stage stroke survivors with mild-to-moderate cognitive and motor impairments. The study was conducted in a clinical setting (i.e., nursing home associated with a rehabilitation hospital), where participants were reminded of the training, although they were asked to self-operate the games without any assistance from caregivers or research staff. Our underlying hypotheses included that (1) stroke survivors with hemiparesis could effectively self-operate Neuro-World and (2) chronic-stage stroke survivors with mild-to-moderate cognitive impairment would significantly improve their cognitive function after using Neuro-World for a 12-week period.

2. Materials and Methods

2.1. Neuro-World Games

We developed Neuro-World based on the findings that basic cognitive processes of memory and attention underline various aspects of clinically important cognitive function [41], such as orientation [42], registration, recall [43], and language [44]. These findings imply that training on cognitive processes of memory and attention could lead to improvements in cognitive abilities. Three games (Figure 1a–c) were designed based on concepts related to short-term memory, while the other three (Figure 1d–f) were designed based on concepts related to selective attention. Each of the six games had 20 different difficulty levels, where stage one was the easiest, and stage 20 was the most difficult. The difficulty level of the six games was adjusted by changing game-specific variables, such as the number of game items (e.g., animal avatars to memorize), the length and pattern of the temporal sequence of the items, and the complexity of their visual characteristics (see Table A1 for details). Each game started at stage one and proceeded to later stages as users successfully completed the tasks. In each stage, users were given three chances to select the correct answer. If users failed to provide the correct answer, the difficulty level was automatically reduced by moving to the previous, easier stage. Each game has a predefined time limit of five minutes. When the user uses up the predefined time limit for a game at any stage, the user will be automatically proceeded to the next game (starting from the easiest stage). The detailed mechanics of each game is explained in Appendix A. The user interface of Neuro-World was specifically designed to provide a simple and user-friendly interface (e.g., large buttons to press on the touchscreen), such that stroke survivors with hemiparesis could leverage only the unimanual, gross upper-limb movements to operate the system. Furthermore, Neuro-World requires minimal configurations and automates most of its operation (e.g., automatically changing the games or advancing to more difficult levels).

2.2. Study Design and Enrollment

The experimental procedure was approved by the Institutional Review Boards (IRB) of the University of Massachusetts Amherst (IRB# 2018-4728) and Heeyeon Rehabilitation Hospital, and the approved clinical trial was registered (ISRCTN10613029). As it was the first clinical trial on Neuro-World with post-stroke survivors, the sample size was computed conservatively, aiming for the minimum effect size of f = 0.2 , significance level of α = 0.05 , and statistical power of β = 0.8 . The computed target sample size was 50. We recruited stroke survivors among those who had been institutionalized in the Nursing Center at Heeyeon Rehabilitation Hospital in South Korea, a long-term care facility for individuals with chronic disorders. Participant recruitment was open for one month until the targeted number of patients was met. To be included in the study, participants had to be at least two years post-stroke since most stroke survivors in South Korea have the opportunity to stay in rehabilitation units for two years due to substantial subsidy by the public insurance. In addition, participants needed to score 18 points or greater on the Korean version of the Mini-Mental State Examination (MMSE) [45]. Exclusion criteria were the presence of any issues that could affect the self-operation of Neuro-World, such as physically being incapable of using a mobile device independently (i.e., both limbs do not have the necessary motor function, indicated by the scores of 20 or less on the Manual Function Test [46]) and vision-related issues (e.g., visual neglect). Eligible participants were randomized to the control group (i.e., standard medical care) or the experimental group (i.e., standard medical care plus Neuro-World) using a random number generator. Baseline tests were completed one week before the first Neuro-World session and post-intervention tests were completed one week after the last session for all the subjects by an experienced language pathologist (hereinafter referred to as assessor). The assessor was only involved in the pre- and post-intervention assessments and blinded to the participant group assignment.
Participants in the experimental group used Neuro-World for 30 min per day, twice a week (i.e., Tuesdays and Thursdays), for 12 weeks. In each 30-min session, participants spent five minutes on each of the six games. When the five minutes had elapsed for a given game (at any stage), users were automatically directed to another randomly chosen Neuro-World game. When users reached the highest stage (i.e., stage 20) of a given game within the five-minute duration, stage 20 was repeated with randomized game-specific variables. On the days and times that participants agreed to play Neuro-World, research staff visited the experimental group participants to bring a 12.2-inch tablet computer (Galaxy Note Pro, Samsung) that was pre-installed with the Korean version of Neuro-World. However, the research staff members did not provide any assistance when the participants were using the system. Participants were allowed to freely choose where to play Neuro-World (e.g., on their beds, couches, chairs in their room, or a lounge). The research staff collected the tablet computer when participants finished the suggested 30 min of Neuro-World. Participants in both groups received no additional cognitive or motor rehabilitation therapies other than standard medical care, which included routine checkups (e.g., blood pressure tests, diabetes screening) and/or any medically necessary treatments (e.g., decubitus ulcer).

2.3. Outcome Measures

2.3.1. Primary Outcome

The therapeutic effect of Neuro-World on the overall cognitive impairment level was measured using the total score of the MMSE [45] due to its prevalent use in research and clinical communities [47]. The assessments were performed at baseline and again at post-intervention (i.e., after 12 weeks). We believe that participants’ learning effects for the MMSE between the baseline and post-intervention assessments were minimal, considering the study duration [48].

2.3.2. Secondary Outcomes

The therapeutic effect of Neuro-World on various aspects of cognitive function was assessed using the sub-scores of the MMSE (i.e., orientation, register, attention and calculation, recall, language). In addition, attention and working memory were assessed using the Digit Forward Span (DFS) and Digit Backward Span (DBS) of the Wechsler Adult Intelligence Scale-IV [49], respectively. The Geriatric Depression Scale (GDS) was used to assess the severity of depressive symptoms [50], and the System Usability Scale (SUS) was conducted to measure the perceived-usability of Neuro-World [51]. For all the outcomes of cognitive capability and SUS, a positive net score indicated an improvement, whereas, for GDS, a negative net score indicated an improvement. Among the outcomes, the results of the SUS have been reported in our previous work [40], demonstrating overall “good” usability perceived by the study participants [52]. All participants in this study were able to successfully self-operate Neuro-World without any assistance from caregivers or therapists throughout the study period.

2.4. Statistical Analyses

Unpaired t-tests were used to compare the mean age, chronicity, and the MFT scores on both the affected and unaffected limbs between the experimental and control groups. Chi-square tests compared the ratio of females/males, left/right affected sides, and handedness between the groups. A two-way mixed model ANOVA was used to identify the main effects and interactions between the intervention (experimental vs. control) and time-course (baseline vs. post-intervention) of Neuro-World on each outcome. Tukey’s post-hoc tests were used to assess statistical significance between and within-group changes for each outcome. The threshold for significance was set to p < 0.05 a priori. All analyses were performed using R software (version 3.5.3).

3. Results

3.1. Study Population

Figure 2 illustrates the participant onboarding procedure. Table 1 summarizes the participant demographic information, further details of which are provided in Appendix C. All the participants were recruited and screened for eligibility from 5 June 2018 through 12 June 2018, when the targeted number of participants was met. At baseline, the experimental (EXP) and control (CON) groups did not display statistically significant differences in the mean age (EXP: 72.71 ± 9.86 vs. CON: 72.67 ± 12.64 years, t ( 27 ) = 0.03 , p = 0.98 ), the proportion of males and females (EXP: 0.14 vs. CON: 0.13 , χ 2 ( 1 ) = 0.00 , p = 1 ), the proportion of subjects with left and right hemiparesis (EXP: 0.50 vs. CON: 0.36 , χ 2 ( 1 ) = 0.15 , p = 0.70 ), the mean MFT scores for both the affected (EXP: 14 ± 9.65 vs. CON: 15.80 ± 6.60 , t ( 27 ) = 0.59 , p = 0.56 ) and the unaffected limbs (EXP: 27.07 ± 2.37 vs. CON: 28.33 ± 1.91 , t ( 27 ) = 1.58 , p = 0.13 ), and the proportion of left and right-handed subjects (EXP: 0.07 vs. CON: 0.00 , χ 2 ( 1 ) = 0.00 , p = 0.97 ). However, the mean length of time since the stroke (i.e., chronicity) of the experimental group was significantly greater than that of the control group (EXP: 74.07 ± 40.84 vs. CON: 45.47 ± 19.82 months, t ( 27 ) = 2.43 , p = 0.02 ) albeit both being well within the chronic phase. None of the participants were diagnosed with any language disorders (e.g., aphasia), which may introduce challenges in following text and audio instructions and self-administering Neuro-world games.

3.2. Outcomes

Figure 3 shows the means and standard deviations of the study outcomes for both the experimental and control groups at baseline and post-intervention. The experimental group showed statistically significant improvements in the overall cognitive level (MMSE), working memory (DBS), and depressive symptoms (GDS) after adhering to the Neuro-World training for 12 weeks, while the control group did not. No adverse effects were observed during the study period. More specifically, a two-way mixed model ANOVA showed that there was a significant interaction between the effects of intervention (experimental vs. control) and time (baseline vs. post-intervention) in the total MMSE scores ( p < 0.01 ). Tukey’s post-hoc tests further revealed that there was a statistically significant increase in the mean MMSE scores between baseline (BASE) and post-intervention (POST) in the experimental group (BASE: 21.64 ± 1.08 vs. POST: 27.57 ± 1.45 , t . r a t i o (27) = 6.48, p < 0.01), while there was a statistically significant decrease in the control group (BASE: 20.87 ± 1.30 vs. POST: 16.40 ± 4.22 , t . r a t i o (27) = −5.06, p < 0.01). We performed a comparative analysis between the observed improvements in the total MMSE score vs. clinical information, including the brain lesions, the types of stroke (ischemic vs. hemorrhagic), and chronicity, but did not find any significant correlations. Furthermore, there was a statistically significant difference in the mean MMSE between the experimental and the control groups at post-intervention (EXP: 27.57 ± 1.45 vs. CON: 16.40 ± 4.22 , t . r a t i o ( 27 ) = 12.43 , p < 0.1 ) but with no differences at baseline (EXP: 21.64 ± 1.08 vs. CON: 20.87 ± 1.30 , t . r a t i o ( 27 ) = 0.86 , p = 0.82 ).

4. Discussion

Towards our goal of supporting the independent administration of cognitive training in home and community settings of stroke survivors, we developed a serious game solution, namely Neuro-World, and evaluated its therapeutic effectiveness on cognitive function in chronic stroke survivors through a randomized controlled study. The results herein demonstrate that routine use of Neuro-World (i.e., 30 min per day, twice a week) leads to improvements in several aspects of cognitive function, as well as depressive symptoms, in chronic-stage stroke survivors. Specifically, we observed statistically and clinically significant improvements in the total MMSE score [53] and the DBS [54]. We also observed statistically significant improvements in the GDS and all the MMSE sub-categories other than language. However, the improvement in the GDS was not clinically significant [55]. Furthermore, we observed clinically significant improvements, although not statistically significant, in the DFS [54]. In the rest of this section, we discuss in detail the results pertaining specifically to cognitive function (i.e., changes in the total MMSE scores, DFS, and DBS). The discussions related to the results of the sub-categories of MMSE and the GDS could be found in Appendixes Appendix D and Appendix E.
Previous randomized controlled studies that investigated the therapeutic effectiveness of cognitive rehabilitation solutions suggest that Neuro-World has yielded comparable or greater improvements in cognitive function. For instance, Zucchell et al. reported that acute-stage stroke survivors showed a mean improvement of 2.6 points from 22.8 to 25.5 in the MMSE after 16 sessions of one-hour conventional cognitive therapy (i.e., a total of up to 16 h per person) [56]. Prokopenko et al. also showed a mean improvement of 3 points from 24 to 27 in the MMSE after 14 sessions (i.e., a total of up to 15 h of training) of computer-based cognitive therapy [57]. Similarly, Kim et al. reported that 20 sessions (i.e., a total of 10 h) of a combined program of virtual reality-based upper-limb motor rehabilitation and computerized cognitive rehabilitation resulted in a mean improvement of 3.6 MMSE points from 17.4 to 21.0 in acute-phase stroke survivors [58]. Although promising, the improvements reported in the above-mentioned studies—because they involved acute-stage stroke survivors—might have been affected by spontaneous biological recovery during the initial months following the stroke [59]. Furthermore, the reported improvements in MMSE are not particularly greater than the minimal clinically important changes (MCID) that are necessary to conclude clinical significance, which is from 2 to 4 points in MMSE [53]. On the other hand, our study involved stroke survivors who were in the chronic stage with no spontaneous recovery, and the participants in the experimental group showed a mean improvement of 5.93 MMSE points from 21.64 to 27.57 after 24 sessions (e.g., a total of 12 h) of Neuro-World. Furthermore, the training dosage used in our study was smaller than previous studies except for the study by Kim et al., which collectively emphasizes the clinical effectiveness and lower burden of Neuro-World in cognitive training.
We also demonstrated the therapeutic effectiveness of Neuro-World in improving working memory as assessed by the DBS. Previously, Ȧkerlund et al. reported that chronic-stage brain injury survivors who completed 25 sessions of 30–45 min (e.g., a total of up to 19 h and 45 min) of CogMed—a set of serious games that are specifically designed to train working memory—showed an average increase of 0.61 points in the DBS [60], although the improvement was not greater than the MCID of the DBS (i.e., 1 point) [54]. Westerberg et al. showed a mean improvement of 1.5 points from 5.8 to 7.3 in the MMSE after 25 sessions (e.g., a total of over 16 h) of CogMed [31], which is comparable to the results reported herein: an average increase of 1.79 points in the DBS from 4.14 to 5.93 after 12 h of Neuro-World play. It is noteworthy that CogMed includes a game that specifically targets the DBS by displaying a sequence of numbers and instructing users to remember and reproduce the sequence in reverse order [31]. On the other hand, Neuro-World includes games that ask users to remember and reproduce a sequence of animals, leaving a farm to train working memory, which does not replicate the DBS. There exist studies that reported statistically significant improvements in the DFS after conventional or game-based interventions [56,60]. However, in our study, stroke survivors showed only non-statistically significant improvement in the DFS. This is likely because the participants in our study scored 7.36 on average at baseline, a relatively high score for the DFS [61], which yields limited room for improvement (i.e., ceiling effect).
We envision a clinical scenario in which stroke survivors will first be introduced to Neuro-World while they receive treatments in inpatient facilities. Then, stroke survivors could be prescribed to use Neuro-World in outpatient facilities, in addition to the routine motor and cognitive rehabilitation programs, to enable frequent, low-cost, high-dosage cognitive rehabilitation in naturalistic settings. Stroke survivors’ compliance level to Neuro-World (e.g., frequency and duration of gameplays) can be summarized and made available to clinicians, such that clinicians can remotely follow up (e.g., a phone call or SMS) to improve patients’ adherence to the intervention regimen. Although this remote management of stroke survivors may impose additional burden for clinicians, we believe mobile rehabilitation systems, in turn, will decrease the overall healthcare cost for chronic stroke survivors in the long-term [62]. We may also consider leveraging patient-centered mobile-health solutions (e.g., smartwatch or smartphone-based reminder systems) that use data-driven, personalized health information as part of reminders to promote positive health-related behaviors (e.g., adhering to at-home training) [63,64]. For example, our prior study has shown that quantitative measurements of stroke survivors’ game performance in Neuro-World (e.g., average duration to complete a stage, the most difficult stages reached for each game, etc.) can be analyzed via machine learning algorithms to accurately estimate their cognitive level in terms of MMSE [40]. In other words, Neuro-World could not only be used as a training tool but also as an assessment tool for cognitive level, such that patients can self-monitor the recovery process. Furthermore, clinicians could also use the information to closely monitor patients’ longitudinal recovery trajectories and devise individually tailored therapy programs to maximize the chances of independent living, which is the ultimate goal of rehabilitation.
The current study has several limitations. First, the study had a relatively small sample size. However, the preliminary results reported herein warrant a larger scale study, which would examine the effect of Neuro-World with respect to the level of patients’ education, different domains of cognitive function, stroke subtypes (e.g., size and location), or other medical factors. Second, the passive control group showed unexpected decreases in all outcome measures, where the decrease in the total MMSE score was statistically and clinically significant. At the completion of the data analysis, we subsequently followed up with the assessor to understand the potential factors that could have resulted in this unexpected decrease. After reviewing the data, the assessor suggested that most subjects (i.e., 13 out of 15 subjects) in the control group were less cooperative and poorly engaged during the post-intervention assessment, which may have led to the decrease in the scores rather than from actual deterioration of their cognitive function. We believe this is due partially to the fact that control subjects did not receive any interventions, follow-ups, or therapeutic/monetary compensations. Third, there exists a statistically significant difference in the mean chronicity between the experiment and control groups, which may have influenced the observed changes in the study participants’ cognitive function. However, all participants were more than 12 months beyond their stroke which, by definition, puts them in the chronic stage with minimal spontaneous recovery. Furthermore, the chronicity of participants in the experiment group (75.04 ± 40.84 months) was significantly greater than the control group (45.47 ± 19.82 months) which greatly diminishes the possibility of any spontaneous recovery. Hence, we believe it is unlikely that this statistically significant cognitive improvement in the experimental group could stem from spontaneous recovery. Fourth, the rehabilitation dosage (i.e., 30 min of gameplay, twice a week, for 12 weeks) was chosen arbitrarily by referring to the dosages used in prior studies involving serious games. Although our results suggest that chronic-stage stroke survivors can improve their cognitive function when prescribed with this particular dosage, it remains as an important future study to investigate the optimal dosage of the Neuro-World therapy. Prior studies have shown that older adults with cognitive impairments are amenable to receiving game-based training up to four times a week [18,20,65]. Thus, we may explore the impact of higher intervention dosage on the clinical outcomes. Last, the goal of this study was to evaluate the effectiveness of Neuro-World in improving cognitive level assuming perfect adherence. Hence, other than the entertainment components of the Neuro-World games, we did not employ the components that support other motivating factors, such as self-monitoring of functional improvements [66,67] and providing feedback that is personalized to a specific patient’s condition [68,69]. It remains an important future study to investigate and develop a patient-centered, interactive system that could support motivational factors to further improve patients’ adherence and engagement when they self-administer Neuro-World in uncontrolled settings.

5. Conclusions

The current randomized controlled trial demonstrates that Neuro-World, a mobile serious game platform that was developed for remote cognitive training, could be an effective means to improve cognitive abilities in chronic-stage stroke survivors with mild-to-moderate cognitive impairment. The preliminary results presented herein warrant a large-scale study to investigate the ultimate goal of enabling stroke survivors’ self-administration of the game-assisted cognitive training in their home settings.

Author Contributions

Conceptualization, H.-T.J., J.-F.D., H.L., B.K., Y.K. and S.I.L.; methodology, H.-T.J., J.-F.D., H.L., B.K., Y.K. and S.I.L.; investigation, H.-T.J., H.L., B.K. and Y.K.; formal analysis, H.-T.J., J.-F.D. and T.N.; writing—original draft preparation, H.-T.J., J.-F.D., T.N. and S.I.L.; writing—review and editing, H.-T.J., J.-F.D., T.N., H.L., B.K., Y.K. and S.I.L.; supervision, J.-F.D. and S.I.L. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Acknowledgments

The authors would like to thank all of the staff members and the post-stroke patients from Heeyeon Rehabilitation Hospital, South Korea, and the engineers from Woorisoft, South Korea, for their support throughout the study.

Conflicts of Interest

H.L. and B.K. are the founders and co-chief executive officers of Woorisoft, the company that has the ownership of Neuro-World. H.T., J.D., T.N., Y.K., S.I.L. declare no competing interests.

Appendix A. Neuro-World Games

The first three games (Figure 1a–c) were designed to stimulate memory using visual information, the concept of which is related to short-term memory. In the first game (Figure 1a), when the game is started, a flock of animal avatars enters the farm simultaneously, and a subset of the flock leaves in one of the four directions (i.e., up, down, left, right). Then, the game presents the image of one animal at the bottom center of the screen and instructs the user to select the direction that the animal has left the farm. The user can select an answer by touching one of the four arrows (i.e., directions) on the screen. As the user proceeds to higher stages, the number and the moving speed of animals entering and exiting the farm increase (more detailed mechanics of difficulty level adjustment for all the Neuro-World games is provided in Table A1). In the second game (Figure 1b), a flock of animals enters the farm simultaneously, and a subset of the flock leaves the farm as in the first game. The images of animals are presented at the bottom center of the screen. The user is expected to remember the animals that have left the farm and provide an answer by touching the corresponding animal images on the screen. In higher stages, the number of animals that enter and exit the farm increase. In the third game (Figure 1c), a flock of animals enters the farm one after another in a specific order, and a subset of the flock leaves the farm simultaneously. The user is expected to remember the order that the remaining animals have entered the farm and select them in the correct order by touching them on the screen. In higher stages, the total number of animals, the number of remaining animals, and their moving speed increase.
Table A1. The mechanics of how difficulty levels are adjusted for Neuro-World games.
Table A1. The mechanics of how difficulty levels are adjusted for Neuro-World games.
GameMechanics of Difficulty Adjustment
1One animal enters and leaves the farm in stage one whereas, in stage 20, ten animals enter and five of them leave. The speed of all the animals in stage 20 move 2.1 times faster than the speed in stage one when entering and leaving the farm.
2One animal enters and leaves the farm in stage one whereas, in stage 20, ten animals enter and four of them leave the farm.
3Three animals enter and all three animals remain on the farm in stage one. In stage 20, 16 animals enter and 12 of them remain on the farm. The speed of all the animals in stage 20 move 3.2 times faster than the speed in stage one when entering and leaving the farm.
4A total of four animals are presented while 40 animals are presented in stage 20. In stage one, each type of animal has only one color whereas, in stage 20, each type of animal has multiple skin colors.
5Two animals are presented in stage one whereas nine animals are presented in stage 20.
6Only one shape is used to make a sequence in stage one whereas 15 different combinations of primitive shapes are used in stage 20. The sequence presented at the bottom of the screen consists of seven (identical) shapes in stage one and 14 shapes (using different combinations of primitive shapes) in stage 20.
The next three games (Figure 1d–f) were designed based on concepts related to selective attention. In the fourth game ((Figure 1d), a flock of animals with different visual characteristics is displayed on the screen, in which a subset of the flock shares some common features, such as the type (e.g., dog vs. sheep) and color. A recorded verbal instruction (e.g., ‘Count the number of brown chickens!’) is played to describe the common visual features of the animals that the user needs to count. Users can replay the recorded instructions by touching a button at the top left corner of the screen (i.e., ‘Repeat Speech Instruction’). When the user finishes the counting, he/she can touch the button (i.e., ‘Enter Answer’), which will provide a visual interface that allows the user to input the numbers. In higher stages, the total number of animals increases, and the features of the presented animals become more diverse. In the fifth game (Figure 1e), a flock of animals is presented on the screen. The image of the animal that the user needs to find among the flock is visually presented at the bottom center of the screen. The game instructs the user to identify the matching animal among the presented animals. In higher stages, the difficulty level is increased by presenting a larger number of animals with more complex visual features. In the sixth game (Figure 1f), a sequence of complex shapes (items), each of which combines multiple primitive 3D shapes and colors, is sequentially looped on the screen. For instance, a sequence of an orange-colored hexagonal pyramid on top of a circular support followed by a yellow cylinder on a triangular support and then by an orange cylinder on a circular support is repeated on the screen. The game ends the sequence at a randomly selected item and instructs the user to identify the color and shapes of the item that should follow the sequence. In higher stages, the type of primitive shapes and the sequential pattern become more complex.

Appendix B. Neuro-World Games

Table A2 provides a detailed analysis of participants’ gameplay data for each of the six Neuro-World games. The analysis suggests that the participants found game one (Figure 1a) and four (Figure 1d) particularly easy to play. Participants were able to reach stage 20 for both games and the correct answer rate was 70% and 68% respectively. The participants found game three (Figure 1c) particularly difficult to play as indicated by every aspect of the analysis data. The participants were able to reach stage six on average. During the five minutes given to the participants, on average, they were able to attempt only four different gameplays and spent about 100 s for each. The rate of correct answers was also the lowest among all six games.
Table A2. Analysis of participants’ gameplay data.
Table A2. Analysis of participants’ gameplay data.
Gameplay DataGame 1Game 2Game 3Game 4Game 5Game 6
Highest mean (std) stage20 (0)18 (3)6 (2)20 (0)13 (3)20 (1)
Gameplay mean (std) count18 (4)9 (1)4 (1)11 (2)47 (14)5 (1)
Time for each gameplay, mean (std) seconds17 (4)37 (3)100 (46)30 (7)7 (2)50 (67)
Rate of correct answer, mean (std) %70 (19)45 (16)42 (19)68 (18)60 (21)60 (19)

Appendix C. Study Population

Table A3 and Table A4 describe the demographic characteristics of the participants in the experimental and the control groups, respectively. The information about brain legion for some participants was missing in their medical records, which is marked as N/A in the tables.
Table A3. Patient demographic information for the experimental group.
Table A3. Patient demographic information for the experimental group.
IDAgeSexDiagnosisChronicityAffected SideMFT (Lt/Rt)Location of StrokeHandedness
166FIsc156Lt5/23N/ARt
281FIsc121Rt27/22N/ARt
362FIsc108Lt2/28MCA InfRt
471FIsc & Alz78Lt0/28N/ARt
585FIsc38Lt22/27N/ARt
672FIsc54Rt29/25N/ARt
780MHem64Lt23/24T SDHRt
875FIsc76Rt29/22N/ARt
977FIsc41Rt28/10Lt MCA InfLt
1064FHem132Lt7/31N/ARt
1185FIsc24Rt26/21N/ARt
1250MHem72Rt29/10Lt FPT SDHRt
1370FIsc35Rt23/2Rt BG, IC InfRt
1480FIsc38Lt25/27Lt BG InfRt
Age is in years and chronicity is in months; MFT: Manual Function Test, F: Female, M: Male, Isc: Ischemic, Hem: Hemorrhagic, Alz: Alzheimer, Lt: Left, Rt: Right, Inf: Infarction, BG: Basal Ganglia, IC: Internal Capsule, MCA: Middle Cerebral Artery, SDH: SubDural Hemorrhage, F: Frontal lobe, P: Parietal lobe, T: Temporal lobe.
Table A4. Patient demographic information for the control group.
Table A4. Patient demographic information for the control group.
IDAgeSexDiagnosisChronicityAffected SideMFT (Lt/Rt)Location of StrokeHandedness
181FIsc58Lt22/26Rt ACA InfRt
287FIsc36Lt9/25Rt MCA InfRt
376FIsc53Rt29/11N/ARt
487FIsc70Lt19/25N/ARt
577FIsc40Rt29/4BG Inf w/ HTRt
687FIsc64Rt28/17N/ARt
749FIsc24Rt30/13Lt THRt
866FHem68Rt31/26N/ARt
956FIsc48Lt8/29N/ARt
1080FIsc37Rt29/24MCA, PCA InfRt
1178FIsc84GD30/31BL PVWM InfRt
1273FIsc28Rt30/23Lt TH InfRt
1367MIsc24Rt28/12N/ARt
1475FHem24Lt18/27O ICHRt
1550MIsc24Rt28/11Rt BG InfRt
Age is in years and chronicity is in months; MFT: Manual Function Test, F: Female, M: Male, Isc: Ischemic, Hem: Hemorrhagic, Lt: Left, Rt: Right, GD: Gait Disturbance, BL: Bilateral, ACA: Anterior Cerebral Artery, MCA: Middle Cerebral Artery, PCA: Posterior Cerebral Artery, PVWM: PeriVentricular White Matter, Inf: Infarction, HT: Hemorrhagic Transformation, TH: Thalamic Hemorrhage, O: Occipital lobe.

Appendix D. Secondary Cognitive Outcomes

Table A5. The results of a two-way mixed model ANOVA for intervention (i.e., between-subjects effect) and time (i.e., within-subjects effect).
Table A5. The results of a two-way mixed model ANOVA for intervention (i.e., between-subjects effect) and time (i.e., within-subjects effect).
OutcomesIntervention
Experimental vs. Control
Time
Baseline vs. Post-Intervention
Interaction
Intervention×Time
F ( 1 , 27 ) p F ( 1 , 27 ) p F ( 1 , 27 ) p
Total MMSE 88.33 < 0.001 0.75 0.39 66.87 < 0.001
Orientation 4.86 0.04 0.03 0.85 15.42 < 0.001
Register 23.10 < 0.001 3.66 0.07 41.49 < 0.001
Attention & Calculation 18.36 < 0.001 5.44 0.03 20.15 < 0.001
Recall 69.68 < 0.001 27.57 < 0.001 67.25 < 0.001
Language 8.23 0.008 3.22 0.08 12.45 0.002
DFS 16.71 < 0.001 0.18 0.67 9.16 0.005
DBS 33.81 < 0.001 1.22 0.28 19.87 < 0.001
GDS 46.16 < 0.001 4.91 0.04 23.31 < 0.001
Table A6. Tuckey’s post hoc test results between baseline and post-intervention tests in the experimental and the control groups.
Table A6. Tuckey’s post hoc test results between baseline and post-intervention tests in the experimental and the control groups.
OutcomesExperimental
Baseline vs. Post-Intervention
Control
Baseline vs. Post-Intervention
t . ratio ( 27 ) p t . ratio ( 27 ) p
Total MMSE 6.48 < 0.001 5.06 < 0.001
Orientation 2.96 0.03 2.60 0.07
Register 3.30 0.01 5.85 < 0.001
Attention & Calculation 4.85 < 0.001 1.44 0.49
Recall 9.55 < 0.001 1.92 0.24
Language 1.29 0.58 3.74 0.005
DFS 2.48 0.09 1.79 0.30
DBS 3.97 0.003 2.30 0.12
GDS 5.01 < 0.001 1.76 0.31
Table A7. Tuckey’s post hoc test results between the experimental and the control groups in baseline and post-intervention tests.
Table A7. Tuckey’s post hoc test results between the experimental and the control groups in baseline and post-intervention tests.
OutcomesBaseline
Experimental vs. Control
Post-Intervention
Experimental vs. Control
t . ratio ( 27 ) p t . ratio (27)p
Total MMSE 0.86 0.82 12.43 < 0.001
Orientation 0.20 > 0.99 3.95 0.003
Register 1.90 0.25 6.94 < 0.001
Attention & Calculation 1.02 0.74 6.07 < 0.001
Recall 2.42 0.10 11.49 < 0.001
Language 0.47 0.97 4.52 < 0.001
DFS 1.50 0.45 5.09 < 0.001
DBS 2.22 0.14 7.32 < 0.001
GDS 1.39 0.52 8.22 < 0.001
Further analyses on the sub-scores of the MMSE and the DFS support the therapeutic benefits of a serious game-based cognitive training method (i.e., Neuro-World) for improving the cognitive function and attention in chronic-stage stroke survivors over no therapy. From the two-way mixed model ANOVA, statistically significant interactions for intervention and time in all the assessed outcomes, including the MMSE sub-scores and the DFS scores, were observed (Table A5). The Tukey’s post-hoc test showed that among the MMSE categories, the experimental group achieved statistically significant increases in the orientation (BASE: 7.93 ± 1.82 vs. POST: 8.07 ± 1.49 , t . r a t i o ( 27 ) = 2.96 , p < 0.05 ), register (BASE: 2.57 ± 0.51 vs. POST: 3.00 ± 0.00 , t . r a t i o ( 27 ) = 3.30 , p < 0.05 ), attention & calculation (BASE: 2.57 ± 0.51 vs. POST: 3.00 ± 0.00 , t . r a t i o ( 27 ) = 4.85 , p < 0.01 ), and recall (BASE: 1.21 ± 0.80 vs. POST: 2.93 ± 0.27 , t . r a t i o ( 27 ) = 9.55 , p < 0.01 ) categories and non-statistically significant increase in the language (BASE: 7.71 ± 0.83 vs. POST: 8.14 ± 0.53 , t . r a t i o ( 27 ) = 1.29 , p = 0.58 ) category (Table A6 and Table A7). On the other hand, the control group exhibited non-statistically significant decreases in the orientation (BASE: 8.07 ± 1.49 vs. POST: 6.73 ± 2.79 , t . r a t i o ( 27 ) = 2.60 , p = 0.07 ), the attention & calculation (BASE: 2.13 ± 0.83 vs. POST: 1.40 ± 0.74 , t . r a t i o ( 27 ) = 1.44 , p = 0.49 ), the recall (BASE: 0.67 ± 0.62 vs. POST: 0.33 ± 0.62 , t . r a t i o ( 27 ) = 1.92 , p = 0.24 ) categories and statistically significant decreases in the register (BASE: 2.13 ± 0.83 vs. POST: 1.40 ± 0.74 , t . r a t i o ( 27 ) = 5.85 , p = < 0.01 ) and the language (BASE: 7.87 ± 0.64 vs. POST: 6.7 ± 1.29 , t . r a t i o ( 27 ) = 3.74 , p = 0.01 ) categories. This trend of improvements in cognitive function is further supported by the analysis results of the DFS. The experimental group exhibited non-statistically significant increase in the DFS (BASE: 7.36 ± 2.17 vs. POST: 9.07 ± 2.87 , t . r a t i o ( 27 ) = 2.48 , p = 0.09 ), while the control group showed marginal decrease in the DFS (BASE: 6.13 ± 1.25 vs. POST: 4.93 ± 2.22 , t . r a t i o ( 27 ) = 1.79 , p = 0.30 ). Despite the non-statistical significance, these changes are still clinically meaningful, considering the minimal clinically important changes of the DFS (i.e., 1 DFS point) [54]. Tukey’s post-hoc tests further revealed that while the experimental and the control groups showed no statistical difference in all the MMSE categories and the DFS at baseline, there was a trend toward greater improvements for the experimental group when compared to the control group. These results collectively suggest that Neuro-World training for a three-month period can improve the overall cognitive level and attention in chronic-stage stroke survivors.

Appendix E. GDS Results and Implications

Another factor that can lead to long-term disability in stroke survivors is post-stroke depression, which is reported in 5 to 64% [70,71] of stroke survivors. Post-stroke depression is associated with increased mortality risk, impairments in motor function, and deterioration in the quality of life [72,73]. After the three-month intervention, the experimental group showed a statistically significant decrease in the symptoms of depression, while the control group showed a non-statistically significant increase. Although all the study participants in both the experimental and control groups showed clinically insignificant depressive symptoms ( GDS 4 ) at baseline, Neuro-World gameplay appears to have ameliorated depressive symptoms in the experimental group, which was statistically significant (BASE: 3.79 ± 0.43 vs. POST: 3.00 ± 0.55 , t . r a t i o ( 27 ) = 5.01 , p < 0.01 ). On the other hand, although statistically not significant, the control group showed a deterioration in their depressive symptoms between baseline and post-intervention (BASE: 4.00 ± 0.00 vs. POST: 4.27 ± 0.46 , t . r a t i o ( 27 ) = 1.27 , p = 0.31 ). More importantly, four study participants in the control group became marginally depressed at the post-intervention visit ( GDS 5 ).
However, despite the statistically significant improvement in the GDS score in the experimental group participants, the baseline GDS scores represented non-depressive symptoms (i.e., normal condition), and the observed change ( 0.79 ± 0.70 ) was smaller than the minimal clinically important changes (i.e., 5.4 GDS point) [55]. Hence, although our results show that Neuro-World has the potential to reduce or control depressive symptoms, future work should focus on using Neuro-World in chronic stroke survivors exhibiting more depressive symptoms ( GDS 5 ) to more rigorously examine whether the use of Neuro-World can lead to clinically significant changes in depressive symptoms.

References

  1. Benjamin, E.J.; Muntner, P.; Bittencourt, M.S. Heart disease and stroke statistics-2019 update: A report from the American Heart Association. Circulation 2019, 139, e56–e528. [Google Scholar] [CrossRef] [PubMed]
  2. Tatemichi, T.K.; Paik, M.; Bagiella, E.; Desmond, D.W.; Pirro, M.; Hanzawa, L.K. Dementia after stroke is a predictor of long-term survival. Stroke 1994, 25, 1915–1919. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. Middleton, L.E.; Lam, B.; Fahmi, H.; Black, S.E.; McIlroy, W.E.; Stuss, D.T.; Danells, C.; Ween, J.; Turner, G.R. Frequency of domain-specific cognitive impairment in sub-acute and chronic stroke. NeuroRehabilitation 2014, 34, 305–312. [Google Scholar] [CrossRef] [PubMed]
  4. Cumming, T.B.; Marshall, R.S.; Lazar, R.M. Stroke, cognitive deficits, and rehabilitation: Still an incomplete picture. Stroke 2013, 8, 38–45. [Google Scholar] [CrossRef] [PubMed]
  5. Chen, C.; Leys, D.; Esquenazi, A. The interaction between neuropsychological and motor deficits in patients after stroke. J. Neurol. Neurosurg. Psychiatry 2013, 80, S27–S34. [Google Scholar] [CrossRef]
  6. Mullick, A.A.; Subramanian, S.K.; Levin, M.F. Emerging evidence of the association between cognitive deficits and arm motor recovery after stroke: A meta-analysis. Restor. Neurol. Neurosci. 2015, 33, 389–403. [Google Scholar] [CrossRef] [Green Version]
  7. Nakling, A.; Aarsland, D.; Næss, H.; Wollschlaeger, D.; Fladby, T.; Hofstad, H.; Wehling, E. Cognitive Deficits in Chronic Stroke Patients: Neuropsychological Assessment, Depression, and Self-Reports. Dement. Geriatr. Cogn. Disord. Extra 2017, 7, 283–296. [Google Scholar] [CrossRef]
  8. Montana, J.I.; Matamala-Gomez, M.; Maisto, M.; Mavrodiev, P.A.; Cavalera, C.M.; Diana, B.; Mantovani, F.; Realdon, O. The Benefits of emotion Regulation Interventions in Virtual Reality for the Improvement of Wellbeing in Adults and Older Adults: A Systematic Review. J. Clin. Med. 2020, 9, 500. [Google Scholar] [CrossRef] [Green Version]
  9. Pham, Q.; Khatib, Y.; Stansfeld, S.; Fox, S.; Green, T. Feasibility and efficacy of an mHealth game for managing anxiety:“flowy” randomized controlled pilot trial and design evaluation. Games Health J. 2016, 5, 50–67. [Google Scholar] [CrossRef]
  10. Pasquini, M.; Leys, D.; Rousseaux, M.; Pasquier, F.; Henon, H. Influence of cognitive impairment on the institutionalisation rate 3 years after a stroke. J. Neurol. Neurosurg. Psychiatry 2007, 78, 56–59. [Google Scholar] [CrossRef] [Green Version]
  11. Claesson, L.; Linden, T.; Skoog, I.; Blomstrand, C. Cognitive Impairment after Stroke–Impact on Activities of Daily Living and Costs of Care for Elderly People. The Goteborg 70+ Stroke Study. Cerebrovasc. Dis. 2005, 19, 102–109. [Google Scholar]
  12. Langhorne, P.; Baylan, S. Early supported discharge services for people with acute stroke. Cochrane Datab. Syst. Rev. 2007, 7, 1–25. [Google Scholar]
  13. Bassett, S.F. The assessment of patient adherence to physiotherapy rehabilitation. N. Z. J. Physiother. 2003, 31, 60–66. [Google Scholar]
  14. Kueider, A.M.; Parisi, J.M.; Rebok, A.L.G.G.W. Computerized Cognitive Training with Older Adults: A Systematic Review. PLoS ONE 2012, 7, e40588. [Google Scholar] [CrossRef] [Green Version]
  15. Bogdanova, Y.; Yee, M.K.; Ho, V.T.; Cicerone, K.D. Computerized Cognitive Rehabilitation of Attention and Executive Function in Acquired Brain Injury: A Systematic Review. J. Head Trauma Rehabil. 2016, 31, 419–433. [Google Scholar] [PubMed] [Green Version]
  16. Van de Ven, R.M.; Murre, J.M.; Veltan, D.J.; Schmand, B.A. Computer-Based Cognitive Training for Executive Functions after Stroke: A Systematic Review. Front. Hum. Neurosci. 2016, 10, 150. [Google Scholar]
  17. Wentink, M.M.; Meesters, J.; Berger, M.A.M.; de Kloet, A.J.; Stevens, E.; Band, G.P.H.; Kromme, C.H.; Wolterbeek, R.; Goossens, P.H.; Vliet Vlieland, T.P.M. Adherence of stroke patients with an online brain training program: The role of health professionals’ support. Top. Stroke Rehabil. 2018, 25, 359–365. [Google Scholar] [PubMed]
  18. Anguera, J.A.; Boccanfuso, J.; Rintoul, J.L.; Al-Hashimi, O.; Faraji, F.; Janowich, J.; Kong, E.; Larraburo, Y.; Rolle, C.; Johnston, E.; et al. Video game training enhances cognitive control in older adults. Nature 2013, 501, 97–101. [Google Scholar]
  19. Thapa, N.; Park, H.J.; Yang, J.G.; Son, H.; Jang, M.; Lee, J.; Kang, S.W.; Park, K.W.; Park, H. The Effect of a Virtual Reality-Based Intervention Program on Cognition in Older Adults with Mild Cognitive Impairment: A Randomized Control Trial. J. Clin. Med. 2020, 9, 1283. [Google Scholar]
  20. Manera, V.; Ben-Sadoun, G.; Aalbers, T.; Agopyan, H.; Askenazy, F.; Benoit, M.; Bensamoun, D.; Bourgeois, J.; Bredin, J.; Bremond, C.J.C.; et al. Recommendations for the use of serious games in neurodegenerative disorders: 2016 Delphi Panel. Front. Psychol. 2017, 8, 1243. [Google Scholar]
  21. Mosimann, U.; Tarnanas, I.; Dimitriadis, S.; Laskaris, N.; Bamidis, P.; Tsolaki, M.; Nef, T.; Mueri, R.; Mosimann, U. P4-365: Serious Gaming Enhances Cognitive Function in Mci Due To Alzheimer’s Disease. Alzheimer Dement. 2014, 10, P922. [Google Scholar] [CrossRef]
  22. Michael, D.R.; Chen, S.L. Serious Games: Games That Educate, Train, and Inform; Course Technology PTR: Boston, MA, USA, 2005. [Google Scholar]
  23. Zyda, M. From visual simulation to virtual reality to games. Computer 2005, 38, 25–32. [Google Scholar] [CrossRef]
  24. Manera, V.; Petit, P.D.; Derreumaux, A.; Orvieto, I.; Romagnoli, M.; Lyttle, G.; David, R.; Robert, P.H. ‘Kitchen and cooking’ a serious game for mild cognitive impairment and Alzheimer’s disease: A pilot study. Front. Aging Neurosci. 2015, 7, 24. [Google Scholar] [CrossRef]
  25. Robert, P.; König, A.; Amieva, H.; Andrieu, S.; Bremond, F.; Bullock, R.; Ceccaldi, M.; Dubois, B.; Gauthier, S.; Kenigsberg, P.A.; et al. Recommendations for the use of Serious Games in people with Alzheimer’s Disease, related disorders and frailty. Front. Aging Neurosci. 2014, 6, 54. [Google Scholar] [CrossRef] [PubMed]
  26. Muscio, C.; Tiraboschi, P.; Guerra, U.P.; Defanti, C.A.; Frisoni, G.B. Clinical trial design of serious gaming in mild cognitive impairment. Front. Aging Neurosci. 2015, 7, 26. [Google Scholar] [CrossRef] [Green Version]
  27. Zickefoose, S.; Hux, K.; Brown, J.; Wulf, K. Let the games begin: A preliminary study using Attention Process Training-3 and LumosityTM brain games to remediate attention deficits following traumatic brain injury. Brain Inj. 2013, 27, 707–716. [Google Scholar] [CrossRef] [PubMed]
  28. Radomski, M.V. More than good intentions: Advancing adherence to therapy recommendations. Am. J. Occup. Ther. 2011, 65, 471–477. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  29. Danzl, M.M.; Etter, N.M.; Andreatta, R.D.; Kitzman, P.H. Facilitating neurorehabilitation through principles of engagement. J. Allied Health 2012, 41, 35–41. [Google Scholar]
  30. Benyon, D.; Turner, P.; Turner, S. Designing Interactive Systems: People, Activities, Contexts, Technologies; Pearson Education: Upper Saddle River, NJ, USA, 2005. [Google Scholar]
  31. Westerberg, H.; Jacobaeus, H.; Hirvikoski, T.; Clevberger, P.; Östensson, M.; Bartfai, A.; Klingberg, T. Computerized working memory training after stroke—A pilot study. Brain Inj. 2007, 21, 21–29. [Google Scholar] [CrossRef]
  32. Lundqvist, A.; Grundström, K.; Samuelsson, K.; Rönnberg, J. Computerized training of working memory in a group of patients suffering from acquired brain injury. Brain Inj. 2010, 24, 1173–1183. [Google Scholar] [CrossRef]
  33. Hellgren, L.; Samuelsson, K.; Lundqvist, A.; Börsbo, B. Computerized Training of Working Memory for Patients with Acquired Brain Injury. Open J. Ther. Rehabil. 2015, 3, 46–55. [Google Scholar] [CrossRef] [Green Version]
  34. Mödden, C.; Behrens, M.; Damke, I.; Eilers, N.; Kastrup, A.; Hildebrandt, H. A Randomized Controlled Trial Comparing 2 Interventions for Visual Field Loss With Standard Occupational Therapy During Inpatient Stroke Rehabilitation. Neurorehabilit. Neural Repair 2012, 26, 463–469. [Google Scholar] [CrossRef] [PubMed]
  35. Yoo, C.; Yong, M.; Chung, J.; Yang, Y. Effect of computerized cognitive rehabilitation program on cognitive function and activities of living in stroke patients. J. Phys. Ther. Sci. 2015, 27, 2487–2489. [Google Scholar] [CrossRef] [Green Version]
  36. Cho, H.; Kim, K.; Jung, J. Effects of neurofeedback and computer-assisted cognitive rehabilitation on relative brain wave ratios and activities of daily living of stroke patients: A randomized control trial. J. Phys. Ther. Sci. 2016, 28, 2154–2158. [Google Scholar] [CrossRef] [Green Version]
  37. Jiang, C.; Yang, S.; Tao, J.; Huang, J.; Li, Y.; Ye, H.; Chen, S.; Hong, W.; Chen, L. Clinical Efficacy of Acupuncture Treatment in Combination with RehaCom Cognitive Training for Improving Cognitive Function in Stroke: A 2 × 2 Factorial Design Randomized Controlled Trial. J. Post Acute Long Term Care Med. 2016, 17, 1114–1122. [Google Scholar] [CrossRef] [Green Version]
  38. Donnan, G.A.; Fisher, M.; Macleod, M.; Davis, S.M. Stroke. Lancet 2008, 371, 1612–1623. [Google Scholar] [CrossRef]
  39. Eriksson, M.; Dahlin-Ivanoff, S. How Adults with Acquired Brain Damage Perceive Computer Training as a Rehabilitation Tool: A Focus-group Study. Scand. J. Occup. Ther. 2002, 9, 119–129. [Google Scholar] [CrossRef]
  40. Jung, H.; Daneault, J.; Lee, H.; Kim, K.; Kim, B.; Park, S.; Ryu, T.; Kim, Y.; Lee, S.I. Remote Assessment of Cognitive Impairment Level based on Serious Mobile Game Performance: An Initial Proof of Concept. IEEE J. Biomed. Health Inform. 2019, 23, 1269–1277. [Google Scholar] [CrossRef]
  41. Chun, M.M.; Turk-Browne, N.B. Interactions between Attention and Memory. Curr. Opin. Neurobiol. 2007, 17, 177–184. [Google Scholar] [CrossRef]
  42. Strub, R.L.; Black, F.W. The Mental Status Examination in Neurology; F. A. Davis Company: Philadelphia, PA, USA, 2000. [Google Scholar]
  43. Squire, L.R. Memory and Brain; Oxford University Press: New York, NY, USA, 1987. [Google Scholar]
  44. Downing, P.E. Interactions Between Visual Working Memory and Selective Attention. Curr. Opin. Neurobiol. 2000, 11, 467–473. [Google Scholar] [CrossRef]
  45. Kang, Y.; Na, D.L.; Hahn, S. A validity study on the Korean Mini-Mental State Examination (K-MMSE) in dementia patients. J. Korean Neurol. Assoc. 1997, 15, 300–308. [Google Scholar]
  46. Miyamoto, S.; Kondo, T.; Suzukamo, Y.; Michimata, A.; Izumi, S. Reliability and Validity of the Manual Function Test in Patients with Stroke. Am. J. Phys. Med. Rehabil. 2009, 88, 247–255. [Google Scholar] [CrossRef] [PubMed]
  47. Tombaugh, T.N.; McIntyre, N.J. The mini-mental state examination: A comprehensive review. J. Am. Geriatr. Soc. 1992, 40, 922–935. [Google Scholar] [CrossRef] [PubMed]
  48. Tombaugh, T.N. Test-retest reliable coefficients and 5-year change scores for the MMSE and 3MS. Arch. Clin. Neuropsychol. 2005, 20, 485–503. [Google Scholar] [CrossRef] [Green Version]
  49. Weiss, L.G.; Saklofske, D.H.; Coalson, D.; Raiford, S.E. WAIS-IV Clinical Use and Interpretation: Scientist-Practitioner Perspectives; Academic Press: New York, NY, USA, 2010. [Google Scholar]
  50. Yesavage, J.A.; Brink, T.L.; Rose, T.L.; Lum, O.; Huang, V.; Adey, M.; Leirer, V.O. Development and validation of a geriatric depression screening scale: A preliminary report. J. Psychiatr. Res. 1982, 17, 37–49. [Google Scholar] [CrossRef]
  51. Brooke, J. SUS: A “quick and dirty” usability scale. In Usability Evaluation in Industry; Jordan, P.W., Thomas, B., McClelland, I.L., Eds.; Taylor & Francis: London, UK, 1996; pp. 189–194. [Google Scholar]
  52. Bangor, A.; Kortum, P.; Miller, J. Determining what individual SUS scores mean: Adding an adjective rating scale. J. Usability Stud. 2009, 4, 114–123. [Google Scholar]
  53. Hensel, A.; Angermeyer, M.C.; Riedel-Heller, S.G. Measuring cognitive change in older adults: Reliable change indices for the Mini-Mental State Examination. J. Neurol. Neurosurg. Psychiatry 2007, 78, 1298–1303. [Google Scholar] [CrossRef]
  54. De Paula, J.J.; Malloy-Diniz, L.F.; Romano-Silva, M.A. Reliability of working memory assessment in neurocognitive disorders: A study of the Digit Span and Corsi Block-Tapping tasks. Braz. J. Psychiatry 2016, 38, 262–263. [Google Scholar] [CrossRef] [Green Version]
  55. Huang, S.L.; Hsieh, C.L.; Wu, R.M.; Lu, W.S. Test-retest reliability and minimal detectable change of the Beck Depression Inventory and the Taiwan Geriatric Depression Scale in patients with Parkinson’s disease. PLoS ONE 2017, 12, e0184823. [Google Scholar] [CrossRef] [Green Version]
  56. Zucchell, C.; Capone, A.; Codella, V.; Vecchione, C.; Buccino, G.; Sandrini, G.; Pierelli, F.; Bartolo, M. Assessing and restoring cognitive functions early after stroke. Neuropsychol. Rev. 2014, 29, 255–263. [Google Scholar] [CrossRef]
  57. Prokopenko, S.V.; Mozheyko, E.Y.; Petrova, M.M.; Koryagina, T.D.; Kaskaeva, D.S.; Chernykh, T.V.; Shvetzova, I.N.; Bezdenezhnih, A.F. Correction of poststroke cognitive impairments using computer programs. J. Neurol. Sci. 2013, 325, 148–153. [Google Scholar] [PubMed]
  58. Kim, B.R.; Chun, M.H.; Kim, L.S.; Park, J.Y. Effect of Virtual Reality on Cognition in Stroke Patients. Ann. Rehabil. Med. 2011, 35, 450–459. [Google Scholar] [PubMed] [Green Version]
  59. Sturm, W.; Willmes, K. Efficacy of a reaction training on various attentional and cognitive functions in stroke patients. Neuropsychol. Rehabil. 1991, 1, 259–280. [Google Scholar]
  60. Ȧkerlund, E.; Esbjörnsson, E.; Sunnerhagen, K.S.; Björkdahl, A. Can computerized working memory training improve impaired working memory, cognition and psychological health? Brain Inj. 2013, 27, 1649–1657. [Google Scholar]
  61. Choi, D.J.; Lee, D.Y.; Seo, E.H.; Jo, M.K.; Sohn, B.K.; Choe, Y.M.; Byun, M.S.; Kim, J.W.; Kim, S.G.; Yoon, J.C.; et al. A Normative Study of the Digit Span in an Educationally Diverse Elderly Population. Psychiatry Investig. 2014, 11, 39–43. [Google Scholar]
  62. Patel, S.; Park, H.; Bonato, P.; Chan, L.; Rodgers, M. A review of wearable sensors and systems with application in rehabilitation. J. Neuroeng. Rehabil. 2012, 9, 21. [Google Scholar]
  63. Beursgens, L.; Boesten, F.; Timmermans, A.; Seelen, H.; Markopoulos, P. Us’Em: Motivating Stroke Survivors to Use Their Impaired Arm and Hand in Daily Life. In CHI ’11 Extended Abstracts on Human Factors in Computing Systems; ACM: New York, NY, USA, 2011; pp. 1279–1284. [Google Scholar]
  64. Jamieson, M.; O’Neill, B.; Cullen, B.; Lennon, M.; Brewster, S.; Evans, J. ForgetMeNot: Active Reminder Entry Support for Adults with Acquired Brain Injury. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA, 6–11 May 2017; ACM: New York, NY, USA, 2017; pp. 6012–6023. [Google Scholar] [CrossRef] [Green Version]
  65. Ben-Sadoun, G.; Sacco, G.; Manera, V.; Bourgeois, J.; König, A.; Foulon, P.; Fosty, B.; Bremond, F.; d’Arripe Longueville, F.; Robert, P. Physical and cognitive stimulation using an exergame in subjects with normal aging, mild and moderate cognitive impairment. J. Alzheimer Dis. 2016, 53, 1299–1314. [Google Scholar]
  66. Mawson, S.; Nasr, N.; Parker, J.; Davies, R.; Zheng, H.; Mountain, G. A personalized self-management rehabilitation system with an intelligent shoe for stroke survivors: A realist evaluation. JMIR Rehabil. Assist. Technol. 2016, 3, e1. [Google Scholar]
  67. Choe, E.K.; Lee, B.; Andersen, T.O.; Wilcox, L.; Fitzpatrick, G. Harnessing the power of patient-generated data. IEEE Pervasive Comput. 2018, 17, 50–56. [Google Scholar]
  68. Parker, J.; Mawson, S.; Mountain, G.; Nasr, N.; Davies, R.; Zheng, H. The provision of feedback through computer-based technology to promote self-managed post-stroke rehabilitation in the home. Disabil. Rehabil. Assist. Technol. 2014, 9, 529–538. [Google Scholar]
  69. Gulotta, R.; Forlizzi, J.; Yang, R.; Newman, M.W. Fostering engagement with personal informatics systems. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems, Brisbane, Australia, 4–8 June 2016; pp. 286–300. [Google Scholar]
  70. Hosking, S.; Marsh, N.; Friedman, P. Depression at 3-months poststroke in the elderly: Predictors and indicators. Int. J. Rehabil. Res. 2000, 7, 205–216. [Google Scholar]
  71. Ayerbe, L.; Ayis, S.; Crichton, S.; Wolfe, C.; Rudd, A. The natural history of depression up to 15 years after stroke: The South London Stroke Register. Stroke 2013, 44, 1105–1110. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  72. Pan, A.; Sun, Q.; Okereke, O.; Rexrode, K.; Hu, F. Depression and risk of stroke morbidity and mortality: A meta-analysis and systematic review. J. Am. Med Assoc. (JAMA) 2011, 306, 1241–1249. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  73. Sturm, J.; Donnan, G.; Dewey, H.; Macdonnell, R.; Gilligan, A.; Srikanth, V.; Thrift, A. Quality of life after stroke: The North East Melbourne Stroke Incidence Study (NEMESIS). Stroke 2004, 35, 2340–2345. [Google Scholar] [CrossRef] [Green Version]
Figure 1. The screenshots of the six games of Neuro-World. All the games were administered in Korean, and English translations were added post clinical study for readers. A detailed explanation for each game is provided in Appendix A.
Figure 1. The screenshots of the six games of Neuro-World. All the games were administered in Korean, and English translations were added post clinical study for readers. A detailed explanation for each game is provided in Appendix A.
Applsci 10 06703 g001
Figure 2. Participant Onboarding Procedure. Fifty-six volunteered chronic stroke survivors were screened for eligibility. Six were determined to be ineligible for the study because they scored less than 18 points in the initial MMSE assessment. Fifty participants were randomly assigned to the experimental vs. control groups. During the study, three participants expressed cognitive fatigue during gameplay and withdrew their consent. Ten participants were either discharged or transferred to other medical institutions. Eight participants withdrew their consents due to their health issues (e.g., headache or pain from arthritis), which were not related to the administration of Neuro-World. As a result, 14 participants from the experimental group and 15 participants from the control group completed the study. All 14 participants from the experimental group who stayed in the study completed the entire 24 sessions of Neuro-World gameplay (i.e., twice a week for 12 weeks).
Figure 2. Participant Onboarding Procedure. Fifty-six volunteered chronic stroke survivors were screened for eligibility. Six were determined to be ineligible for the study because they scored less than 18 points in the initial MMSE assessment. Fifty participants were randomly assigned to the experimental vs. control groups. During the study, three participants expressed cognitive fatigue during gameplay and withdrew their consent. Ten participants were either discharged or transferred to other medical institutions. Eight participants withdrew their consents due to their health issues (e.g., headache or pain from arthritis), which were not related to the administration of Neuro-World. As a result, 14 participants from the experimental group and 15 participants from the control group completed the study. All 14 participants from the experimental group who stayed in the study completed the entire 24 sessions of Neuro-World gameplay (i.e., twice a week for 12 weeks).
Applsci 10 06703 g002
Figure 3. The means and the standard deviations of the total MMSE score and its category scores at baseline and post-intervention (af). The means and the standard deviations of the DFS, DBS, and GDS scores by groups at baseline and post-intervention (gi).
Figure 3. The means and the standard deviations of the total MMSE score and its category scores at baseline and post-intervention (af). The means and the standard deviations of the DFS, DBS, and GDS scores by groups at baseline and post-intervention (gi).
Applsci 10 06703 g003
Table 1. Baseline patient demographic information.
Table 1. Baseline patient demographic information.
Baseline Demographic Data for 29 Stroke SurvivorsExperimentalControl
Patients, No.1415
Age, mean (standard deviation), y.72.71 (9.86)72.67 (12.64)
Sex, No. (%)
Men2 (14.29)2 (13.33)
Women12 (85.71)13 (86.67)
Diagnosis, No. (%)
Ischemic stroke11 (78.57)13 (86.67)
Hemorrhagic stroke3 (21.43)2 (13.33)
Chronicity, mean (standard deviation), m.74.07 (40.84)45.47 (19.82) *
Affected Side, No. (%)
Left7 (50.00)5 (35.71)
Right7 (50.00)9 (64.29)
Manual Function test, mean (standard deviation) score
Affected Limb14.00 (9.65)15.80 (6.60)
Unaffected Limb27.07 (2.37)28.33 (1.91)
MMSE, mean (standard deviation) score
Orientation7.93 (1.82)8.07 (1.49)
Register2.57 (0.51)2.13 (0.83)
Attention and calculation2.57 (0.51)2.13 (0.83)
Recall1.21 (0.80)0.67 (0.62)
Language7.71 (0.83)7.87 (0.63)
Total score21.64 (1.08)20.87 (1.30)
Digit spanning, mean (standard deviation) score
Forward7.36 (2.17)6.13 (1.25)
Backward4.14 (1.29)2.93 (0.88)
GDS, mean (standard deviation), score3.79 (0.43)4.00 (0.00)
* indicates that the difference is statistically significant ( p < 0.05 ).

Share and Cite

MDPI and ACS Style

Jung, H.-T.; Daneault, J.-F.; Nanglo, T.; Lee, H.; Kim, B.; Kim, Y.; Lee, S.I. Effectiveness of a Serious Game for Cognitive Training in Chronic Stroke Survivors with Mild-to-Moderate Cognitive Impairment: A Pilot Randomized Controlled Trial. Appl. Sci. 2020, 10, 6703. https://doi.org/10.3390/app10196703

AMA Style

Jung H-T, Daneault J-F, Nanglo T, Lee H, Kim B, Kim Y, Lee SI. Effectiveness of a Serious Game for Cognitive Training in Chronic Stroke Survivors with Mild-to-Moderate Cognitive Impairment: A Pilot Randomized Controlled Trial. Applied Sciences. 2020; 10(19):6703. https://doi.org/10.3390/app10196703

Chicago/Turabian Style

Jung, Hee-Tae, Jean-Francois Daneault, Tenzin Nanglo, Hyunsuk Lee, Byeongil Kim, Yangsoo Kim, and Sunghoon Ivan Lee. 2020. "Effectiveness of a Serious Game for Cognitive Training in Chronic Stroke Survivors with Mild-to-Moderate Cognitive Impairment: A Pilot Randomized Controlled Trial" Applied Sciences 10, no. 19: 6703. https://doi.org/10.3390/app10196703

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop