Serious games for improving knowledge and self-management in young people with chronic conditions: a systematic review and meta-analysis

Abstract

Objective To conduct a systematic review and meta-analysis of randomized controlled trials assessing the effectiveness of serious games in improving knowledge and/or self-management behaviors in young people with chronic conditions.

Materials and Methods The authors searched the databases PubMed, Cochrane Library, Web of Sciences, and PsychINFO for articles published between January 1990 and January 2014. Reference lists were hand-searched to retrieve additional studies. Randomized controlled trials that compared a digital game with either standard education or no specific education in a population of children and/or adolescents with chronic conditions were included.

Results The authors identified 9 studies in which the effectiveness of serious games in young people with chronic conditions was evaluated using a randomized controlled trials design. Six studies found a significant improvement of knowledge in the game group from pretest to posttest; 4 studies showed significantly better knowledge in the game group than in the control group after the intervention. Two studies reported significantly better self-management in the game group than in the control group after the intervention. Seven studies were included in the meta-analysis. For knowledge, pooled estimate of Hedges’ gu was 0.361 (95% confidence intervals, 0.098-0.624), demonstrating that serious games improve knowledge in patients. For self-management, pooled estimate of Hedges’ gu was 0.310 (95% confidence intervals, 0.122-0.497), showing that gaming improves self-management behaviors.

Conclusions The authors’ meta-analysis shows that educational video games can be effective in improving knowledge and self-management in young people with chronic conditions.

INTRODUCTION

Epidemiological studies have indicated that about one-quarter of children in the Western world have one or more chronic conditions. ¹ By the time they reach adolescence, 10–15% of children live with a chronic condition. ² Most of these conditions are long-lasting and continue into adulthood. Hence, it is paramount that these individuals acquire, at the earliest possible age, adequate knowledge about their medical condition and develop appropriate self-management skills as they transition from being a dependent child to an independent adult. ³ Self-management can be defined as the strategies that individuals undertake to promote health (e.g., healthy living, exercising), manage an illness (e.g., manage symptoms, medication, and lifestyle changes), and manage life with a medical condition (e.g., adapt leisure activities or deal with losses caused by illness). ⁴ Patient education is frequently provided in order to improve their understanding of the condition, but also to enhance the self-management skills, which in turn can improve the overall health status, reduce healthcare utilization, and minimize the overall burden of the condition. ^5–8

Although individually tailored educational programs are most effective, ^9–11 these are very resource consuming. ¹² By contrast, more traditional and passive methods of patient education, such as oral lecturing or offering printed reading material, fail to substantially improve clinical outcomes. ^5,13 Especially in the case of adolescents, methods that motivate individuals to learn may be more effective. In response, innovative systems of supportive, evidence-based educational interventions have been created in order to provide education and to improve self-management in a financially sustainable way, while still being effective.

A more recent alternative approach relies on video games as a medium for improving medical skills and knowledge and as a tool in medical treatments, therapy, and disease management. ¹⁴ Since the rise in popularity of video games over the past 30 years, researchers have started to explore the potential of video games for “serious purposes.” ^15–17 Serious games are defined as “a mental contest, played with a computer in accordance with specific rules, that uses entertainment to further government or corporate training, education, public policy, and strategic communication objectives.” ¹⁸ Digital game-based learning has the power to evoke intense interest among gamers, motivating them to engage in a task at a regular basis for a long period of time. These are qualities that are often hard to obtain via traditional learning materials and approaches, and hence may be responsible for the difference in educational effectiveness. ^16,19–23

Serious games in healthcare (also called “health games”) as well as commercial games related or unrelated to healthcare serve several goals ranging from training healthcare providers and supporting patients in their therapy and disease management to promoting healthy wellness and lifestyles to the broader public. ¹⁴ Games appear to be especially eligible for young persons, because several characteristics of games match the learning styles of these “digital natives” who grew up around computers, video games, and the Internet. ²⁴ Young learners are typically more visually oriented than older age groups, can easily manage several flows of information simultaneously, and have a preference for inductive reasoning and fast interactions. ^15,16

Health games can provide young persons with flexible learning environments in which they can learn about their medical condition in a dynamic and personalized setting that allows for accessible and appealing exploration, information seeking, and practice. ^9,14 Games can adapt content and challenges to the developmental stage, educational level, personal interest, and specific diseases of the gamers, thereby allowing them to design a self-management plan with their own personal educational goals, which is likely to result in a more effective education approach. ^25–27 In contrast with other electronic media, contemporary games typically combine both intrinsic and extrinsic motivational elements, ²⁸ active learning processes, ^20,21 provision of immediate feedback, ²⁹ and opportunities for socialization with others. ^30–32 Based on these powerful and persuasive game mechanisms, ^27,30 it is hypothesized that playing health games increases the gamers’ learning, which results in increased knowledge and a better adoption of healthier lifestyles and self-management behaviors. To test this hypothesis, we sought to conduct a systematic literature review and meta-analysis on the effectiveness of digital games in improving knowledge and self-management behaviors in young persons with chronic conditions.

METHODS

Literature sources and searches

Relevant studies were identified using 2 strategies. First, we performed a comprehensive literature search of the databases PubMed, the Cochrane Library, Web of Sciences, and PsychINFO for studies published between January 1990 and January 2014. We searched for articles published from 1990 onward because the first studies on the effects of video games in the area of health education were published in the 1990s. In PubMed and Cochrane Library, the following Mesh terms or keywords were used: “Video Games,” “Experimental,” “Play and Playthings,” “Self Care,” “Chronic Disease,” “Patient Education,” “Health Education,” “Adolescent,” and “Teaching.” In Web of Sciences, the keywords were “game” and “education”, each in combination with “health,” “child” or “adolescent,” and “patient,” and with “chronic disease,” “asthma,” “diabetes,” “cancer,” “cystic fibrosis,” “anorexia,” “malnutrition,” “cerebral palsy,” and “autism.” In PsychINFO, we searched using the keywords “Education,” “Games,” and “Health.” All these searches were limited to “outcome studies,” “randomized controlled trials (RCTs),” and “efficacy studies,” and were restricted to studies in children and adolescents. English, French, German, and Dutch were used as language limits. Second, we hand-searched the reference lists of all relevant articles to find additional studies (snowball technique). This systematic review and meta-analysis was conducted in line with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines ( Supplementary Data ).

Eligibility criteria

Studies that met the following criteria were considered eligible for inclusion: 1) RCTs that compared a digital game (serious game or commercial) with either standard education or no specific education, 2) a study population of children or adolescents with chronic conditions at any stage of disease, and 3) a quantitative assessment of patients’ knowledge and/or self-management as one of the outcomes variables. Articles referring to computer game interventions in relation to health promotion programs in preventive healthcare (physical activity, mental health, nutrition); articles focusing on symptom management (e.g., burn pain relief) or distraction (for surgery, chemotherapy, or radiation treatment) without measuring behaviors; articles focusing on measurement (e.g., spirometry) and diagnostic methods (e.g., biofeedback games to diagnose Attention Deficit Hyperactivity Disorder); and articles on game theory (learning processes), game development, and evaluation (e.g., playability and usability research) were excluded.

Study selection and data extraction process

A flow diagram of the search and selection procedure is shown in Figure 1 . Database searches resulted in 1119 records. On the basis of title review, we identified 122 potentially relevant studies matching health games for children and adolescents. After exclusion of duplicates, 2 of the authors (N.C., N.Z.) completed an abstract review of 107 articles. Articles referring to healthy lifestyle games for prevention ( n = 29), articles focusing on symptom management or distraction without measuring behaviors ( n = 20); articles focusing on measurement and diagnostic methods ( n = 12), game development, and evaluation or conceptual frameworks ( n = 27) were excluded. The full-text of the 19 remaining articles were reviewed and the references were hand searched, identifying two additional articles. In this phase of the selection process, 3 investigators (N.C., N.Z., P.M.) reviewed the full-text articles independently, in order to evaluate whether the studies met the proposed criteria for eligibility. When necessary, authors were contacted to obtain more information. Twelve articles were removed from the selection of 21 articles for different reasons ( Figure 1 ). Hence, 9 articles were retained for the systematic review. ^33–41 Two of the nine studies were not included in the meta-analysis, ^35,39 because the effect sizes could neither be derived from the article nor obtained from the authors. These 2 studies both investigated knowledge and self-management.

Using a structured form, data were extracted by 1 reviewer (N.Z.) and subsequently checked by 2 other reviewers (N.C., P.M.). Extracted data were characteristics and key outcomes of the studies: study type, year of study, target population (age, diagnostic group), description of the group conditions (the intervention and usual care), and measurement and outcome variables. In addition, narrative descriptions of the games as well as a set of game elements were detailed: setting, game content, play frequency and intensity, and game platform.

Assessment of quality of individual studies

Three investigators (N.C., N.Z., P.M.) assessed independently the methodological quality of the individual studies using 8 criteria. ⁴² These criteria took several methodological aspects into account—for example, clinical heterogeneity and attempts to reduce other potential sources of bias. Two criteria that are typically used in critical appraisals were not applied in the present review: blinded patients and blinded outcome assessors. Games as a health intervention cannot be implemented without patients being aware of it. Furthermore, the outcomes under study were knowledge and self-management.

All criteria were scored on a scale ranging from 0 to 2, indicating the criteria were not met or vaguely described (0 points), partially met (1 point), or completely met (2 points). A total score was calculated and ranged from 0 (low quality, high risk of bias) to 16 (high quality, low risk of bias).

Disagreements between reviewers during the selection, quality assessment, and data extraction process were resolved by consensus meetings. If needed, the authors of the original article were contacted for more information to (i) establish eligibility according to the inclusion criteria, (ii) appraise the methodological aspects, or (iii) obtain data to determine the effect sizes.

Statistical analysis

Data analysis was done in SAS software (version 9.2, SAS Institute Inc, Cary, NC, United States). For each study an effect size was calculated as the standardized mean difference (SMD) of the posttest values between the control and the intervention group—that is, the difference between both means divided by the pooled standard deviation. To handle the upward bias present in small samples, a correction is applied to the SMD yielding the so-called Hedges g.⁴³ A positive effect size refers to a better result in the gaming group. Observed differences in effect size between the studies reflect true variability (between-study variability or heterogeneity) and sampling variability (within-study variability). Heterogeneity was quantified by the I² statistic, ⁴⁴ which is the percentage of total variation in study estimates that is due to heterogeneity, and tested by Cochran’s χ ² -test. The random-effects approach of DerSimonian and Laird was used to obtain a pooled estimate of the SMD as a weighted average of the study-specific estimates. ⁴⁵ For 2 studies that did not report a standard deviation for the posttest result, ^33,40 the pretest information on variability was used.

RESULTS

Characteristics of selected studies

The 9 RCTs included in this systematic review enrolled a total of 1168 patients, 966 (83%) of which completed the respective studies. Of this latter group, 514 patients were assigned to the intervention group and 452 to the control group. The studies included patients with asthma, ^{33,35–38,40} diabetes, ^34,39 or cancer. ⁴¹ Seven out of nine studies were conducted in the United States ( Table 1 ).

Within the game intervention group, patients played a video game with educational content aiming at knowledge improvement or promotion of self-management behaviors. Games were software packages that run on a personal computer, ^{33,35–38,40,41} console, ³⁴ or mobile phone. ³⁹ They usually contained some sort of competition (e.g., adventure game, jump n’ run game, quiz), mental challenge, chance factors/luck, and motivational aspects. These games did not contain virtual reality programs, software that exclusively provided health information, or systems that were only meant for storage and management of health-related data. In 6 studies, the intervention group received the game only. ^33–37,41 In the 3 other studies, the game was combined with written materials, ^38,40 a nondisease-related computer program, ³⁸ and/or monitoring software. ³⁹ A large variability in gaming frequency, duration, length of exposure, and setting was observed ( Table 1 ).

The control groups received either a noneducational, nondisease-related computer game only, ^34,41 a disease monitoring system without a playing component, ³⁹ standard education, ^33,35,38,40 or no education at all. ^36,37 Standard education included any form of education, ranging from verbal instructions ³³ to printed material, ^35,38,40 and with ^33,35,38 or without ⁴⁰ a noneducational, nondisease-related computer game.

Knowledge was measured as an outcome variable in 8 studies. ^{33–38,40,41} Knowledge was assessed using standard disease-specific knowledge tests. Self-management was evaluated in 6 studies ^{33–36,39,41} and was operationalized in terms of self-monitoring, ^34,36,39 medication adherence, ^33,34,41 symptom trigger avoidance, ³⁶ response to acute episodes of the disease, ^33,36 or general disease-related behaviors. ³⁵ It was measured by means of objective measures, ^39,41 auto- and hetero-anamnesis with standardized measurement scales, ^33,34 or interview protocol. ^35,36

Quality and publication bias assessment

All studies clearly described the game characteristics, such as the theoretical basis, game purpose, scenario, content, and patients’ information ( Supplementary Data ). All studies used an identical assessment of the outcome variables in both the experimental and control group. Most of the studies explained the randomization procedure ^{33–35,38,40,41} and clearly defined inclusion and exclusion criteria. ^{35,36,38,40,41} However, 3 studies reported the included subjects, without providing a clear definition of exclusion. ^33,34,39 In 1 study, the criteria were vague. ³⁷ In 5 studies, intervention and control groups were comparable in terms of socio-demographic variables and baseline knowledge. ^{33,35,38,39,41} Four studies partially met this criterion: differences between groups were observed in terms of baseline knowledge, ^34,37 mean age, ⁴⁰ and parental employment status. ³⁶ Comparison of socio-demographic characteristics was not mentioned in 1 study. ³⁴ For the “standard program” criterion, 2 points were allocated if all subjects were exposed to the exact same condition, except for an additional educational game in the intervention group. Six studies met this criterion. ^36–41 In 5 of these studies, controls received standard care without other educational interventions. ^36–39,41 The criterion was not met if young persons in the control group received an intervention that the young persons in the game group did not. ^33–35 Only 2 studies mentioned intention-to-treat analysis and had a complete follow-up. ^40,41 The total quality scores ranged from 8 to 16.

Effectiveness of serious games

Table 2 summarizes the results reported in the individual studies. In terms of knowledge, 6 studies found a significant improvement in the game group from pretest to posttest. ^{33,35–37,40,41} In 1 study, a significant improvement of knowledge was observed in the control group, as well. ³⁶ In the posttest, four studies showed significantly better knowledge in the game group than in the control group. ^33,35,37,40 For self-management, no significant pretest–posttest differences were found in the game group. In one study, a significant deterioration from pretest to posttest was found in the control group. ³⁴ Two studies reported significantly better self-management in the game group after the intervention compared to the control group. ^33,34

Based on these data, a meta-analysis was performed, using Hedges’ gu as measure of effect size. All seven studies that investigated the effect of games on the level of knowledge had a Hedges’ gu higher than zero, which favors the games ( Figure 2 ). In 3 studies, this effect size was significantly different from zero. ^33,37,40 The combined estimate of Hedges’ gu was 0.361 (95% confidence interval (95% CI), 0.098-0.624), demonstrating that serious games improve knowledge compared to controls. However, there is a high level of heterogeneity between the effect sizes from the various included studies ( I² = 62.3%, χ ² = 18.9, df = 6, P = .004), questioning the appropriateness of combining the study-specific estimates into a combined one. A sensitivity analysis (repeating the meta-analysis, each time excluding a single study) reveals that the heterogeneity is mainly due to the study of McPherson et al. ⁴⁰ Exclusion of this study decreases the percentage of variability explained by heterogeneity to 23% ( I² = 22.7%, χ ² = 6.5, df = 5, P = .26). Since the excluded study is the one with the strongest effect size, the combined effect size decreases to 0.222 (95% CI, 0.046-0.399). Separate analyses were undertaken for studies in which video games were compared to conventional education ^33,38,40 (Hedges’ gu 0.596; 95% CI, 0.018-1.174) and studies that compared gaming with no education ^34,36,37,41 (Hedges’ gu 0.015; 95% CI, 0.015-0.346).

Of the 6 studies that assessed the effect of games on self-management, ^{33–36,39,41} 4 studies could be included in the meta-analysis, all of which had a Hedges’ gu higher than zero ^33,34,36,41 ( Figure 3 ). In only 1 study, the effect size of the difference between the intervention and control group was statistically significant. ³³ The combined estimates of Hedges’ gu was 0.310 (95% CI, 0.122-0.497), showing that gaming improves self-management behaviors. For self-management, the differences between the effect sizes do not exceed sampling variability ( I² = 0%, χ ² = 2.6, df = 3, P = .46). Three out of the 4 studies compared self-management in the gaming group with a control group that did not receive any education. ^34,36,41 The effect size on self-management for these 3 studies was 0.263 (95% CI, 0.064-0.463).

DISCUSSION

We conducted a systematic review and meta-analysis on the effects of serious health games in improving knowledge and self-management in young persons with chronic conditions. Nine studies were identified. Six studies found a significant improvement of knowledge in the game group from pretest to posttest; 4 studies showed significantly better knowledge in the game group than in the control group after the intervention. Two studies reported significantly better self-management in the game group than in the control group after the intervention. Our meta-analysis showed that educational video games are effective in improving knowledge and self-management of young persons with chronic conditions.

To date, several reviews on gaming as a healthcare intervention have been published. ^8,15,46–49 These reviews addressed the use of digital games in health education, ^15,47,48 physical education, ¹⁵ patient treatment, ⁴⁸ prevention and health promotion, ^48,49 specific health outcomes, ^8,46,49 or the use of games for training health professionals. ^47,48 In general, the reviews suggested that digital games have the potential of improving people’s knowledge, skills, attitudes, and behaviors in relation to health, ^15,47,48 and can results in improvements of health outcomes. ^8,46 However, firm conclusions on the effectiveness of serious health games could not be drawn from these reviews, because they did not limit their review to RCTs—thus studies with weak designs were also included—and they did not use a standard method of systematic reviews described by PRISMA guidelines.

Only 1 systematic review, that merely included RCTs and that used the PRISMA guidelines, has been published. ⁴⁹ These authors included 38 studies that used video games to provide physical therapy, psychological therapy, improved disease self-management, health education, distraction to discomfort, increased physical activity, and skills training for clinicians. ⁴⁹ In that review, study inclusion was not limited by age of the patient; the studies were not restricted to patients with chronic conditions; and only studies with positive outcomes were included. In this respect, the present systematic review and meta-analysis is substantially different from Primack’s one, because we specifically focused on (i) young people (ii) with chronic conditions, and (iii) we conducted a meta-analysis.

However, the findings of our review and meta-analysis should be interpreted with caution due to some methodological limitations. First, the methodological quality of the studies included varied substantially. Although we identified some studies with a rather low methodological quality, we did not exclude these from our review and meta-analysis. We observed that more recent studies generally have better methodological rigor. Second, the studies that were included in our review were not homogeneous. Indeed, different games or game platforms for different patient populations were assessed. We tried to tackle this issue by performing a random-effects meta-analysis. This technique accounts for sampling variability and heterogeneity of the study populations. Also, we tested heterogeneity and performed sensitivity analysis in case a high level of heterogeneity was observed. Third, we were not able to investigate a potential dose-effect relationship. Indeed, the intensity of the intervention and the adherence to the implementation protocol is deemed to be important for gaming to be effective. Hence, it would have been valuable to assess the gaming intervention quality in addition to the RCT quality. However, the articles reviewed lacked the necessary information to do so. Fourth, we only investigated the effect of games on knowledge and self-management. Some studies also included other variables as outcomes of the game, such as limitations in activity and symptoms of the disease. Such outcome measures should be considered in future research and can be included in future meta-analyses. Fifth, we included only games that were described in the scientific literature and were tested using an RCT. However, other games for health do exist. Sixth, we could not take the developmental stage of the patients or the game into account. As evidence is mounting, this issue should be addressed in future trials. Seventh, we did not investigate the interplay between knowledge and self-management. Although games have shown to be capable in improving young people’s knowledge, a direct impact in health behaviors is not necessarily warranted.

We call upon researchers to investigate the impact of games from a broad and systematic perspective. First, existing studies have evaluated the effectiveness of games as an alternative for traditional patient education. However, future studies should investigate the relative contribution of games above and beyond that of traditional patient education. An intriguing finding of our study, for which we do not have an explanation, is that subanalyses showed a larger effect size when gaming was compared to standard education, then when gaming was compared to no education. This finding should be scrutinized in future studies, or in meta-analyses in other populations. Second, future effectiveness studies should not only assess the cognitive and behavioral benefits in terms of increased knowledge and improved self-management, but also should scrutinize the emotional and attitudinal aspects relating to personal well-being, identity development, sense of peer-belonging, social support, enjoyment, and entertainment. Third, now that we have gained evidence for the effectiveness of serious games, careful attention should be paid to determine how and why the games involved in our meta-analysis were effective in achieving their goals. It is only by gaining a deeper understanding of the mechanisms underlying the success of serious gaming that we can generalize our results beyond the current studies and provide constructive insights for the design of new serious games. The use of qualitative research methods is appropriate in this context. Hence, a meta-synthesis of qualitative studies in this respect can be advocated.

CONCLUSION

Serious games have been considered as potential healthcare interventions, but empirical data on their effectiveness has been scarce and inconsistent. We conducted a systematic review and meta-analysis on the effectiveness of serious games in improving knowledge and self-management behaviors in young people with chronic conditions. Previous investigations suggested that games affect the outcomes under study. Our meta-analysis allows us to firmly conclude that serious games improve the level of knowledge and self-management in young people with chronic conditions.

FUNDING

This research received no specific grant from any funding agency in the public, commercial, and not-for-profit sectors.

COMPETING INTERESTS

The authors have no competing interest to declare.

CONTRIBUTORS

N.C., N.Z., and P.M. extracted the data from the original studies, conducted the initial analyses, and designed the data collection instruments. They all contributed (from first draft to final version) to the writing of the manuscript. S.F. and K.D. carried out the statistical analyses and contributed to the writing of the methods and results section. B.Z. provided guidance and expertise in the overall conceptualization of the review and critically reviewed the manuscript. All authors approved the final manuscript as submitted and agree for accountability for all aspects of the work.

SUPPLEMENTARY MATERIAL

Supplementary material is available online at http://jamia.oxfordjournals.org/ .

REFERENCES