Digital health interventions (DHIs) show vast potential in supporting patients and health care systems with the globally increasing prevalence and economic costs of non-communicable diseases (NCDs) – the leading causes of death and disability worldwide [1, 2]. More specifically, Mobile Health Applications (mHealth apps) are now considered an accessible and scalable solution to promote behavior change among patients, improve health outcomes, and reduce healthcare costs [3-5]. Correspondingly, the number of available mHealth apps has continuously grown to more than 300.000, with approximately 200 new mHealth apps released each day [2, 6].
However, despite increasing evidence and availability, mHealth apps are subject to significant dropout rates, with a substantial proportion of users not adhering to them as intended [7, 8]. Recent research has shown that up to 80 percent of all participants in mHealth interventions only engage at a minimum level, not logging into the mHealth app more than once and not consistently using the app long term . Studies examining mHealth app usage in more extensive real-world settings further show that less than five percent of participants are daily active users, with the large majority only downloading the corresponding app but not using it regularly [10, 11].
Since non-adherence relative to intended use jeopardizes treatment success and thus might lead to an increased number of hospitalizations, it is considered a fundamental concern in developing mHealth apps [8, 12-15]. Still, the scientific body of literature lacks concise conceptualizations and measures for intended use of mHealth apps, while intervention components and other factors influencing adherence remain to be explored [13, 16].
Therefore, understanding factors that act as barriers or facilitators to adherence is crucial to prevent intervention dropouts and increase the effectiveness of digital health interventions. Furthermore, identifying users at risk and preventing them from dropping out indicates the potential to increase the efficiency and quality of DHIs as many interventions are only beneficial if people use them regularly and for a prolonged period [14, 15, 17-19].
Predicting and preventing intervention dropouts and thus increasing the effectiveness of mHealth apps requires multiple steps: First, understanding the factors that act as barriers or facilitators to user adherence. Second, being able to predict dropouts based on individual real-time data. Third, incorporating effective preventive intervention components and actions that induce behavior change at the right time before the dropout occurs. To this end, we will use our technical expertise in the state of receptivity to fill this research gap and apply it to the prediction and prevention of non-adherence [20, 21].
In this regard, our project has the following objectives:
- Identifying factors that influence adherence and predict dropouts based on existing literature and real-world data from longitudinal studies of different health domains.
- Identifying intervention components that effectively intervene in critical situations to prevent dropout
- Developing and evaluating an early warning system that can predict and prevent dropouts.
From a practical perspective, these objectives aim to help stakeholders of DHIs in developing more compelling and effective mHealth apps.
Planned Studies and Research Questions
We are planning three distinct studies which combine evidence from the existing body of research, retrospective analysis of longitudinal data from real-world mHealth apps, and prospective data analysis to validate our findings.
Study 1: Systematic Literature Review
Conducted in cooperation with the Swiss Institute for Addiction and Health Research (ISGF) and funded by the Swiss Federal Office of Public Health, Study 1 aims to identify intervention- and patient-related factors influencing adherence to the intended use of mHealth apps. We further derive quantified adherence scores for different health domains, which may serve as adherence benchmarks.
A comprehensive systematic literature search (January 2007- December 2020) was conducted in MEDLINE, Embase, Web of Science, Scopus, and ACM Digital Library. Data on intended use, actual use, and factors influencing adherence were extracted. Intervention-related and patient-related factors with a positive or negative influence on app use are presented separately for the health domains NCD-Self-Management, Mental Health, Substance Use, Nutrition, Physical Activity, Weight Loss, Multicomponent Lifestyle Interventions, Mindfulness, and other NCDs. Quantified measures of adherence were derived for each study and compared with qualitative findings.
The following research questions are addressed in Study 1:
- RQ 1.1: Which intervention-related factors influence adherence relative to the intended use of mHealth applications targeting NCDs in adults?
- RQ 1.2: Which patient-related factors influence adherence relative to the intended use of mHealth applications targeting NCDs in adults?
- RQ 1.3: How do adherence rates of mHealth apps for NCDs compare across different health domains?
Study 2: Retrospective Data Analysis
In study 2, we strive to identify variables (e.g., login rates, session length, or interaction frequency with specific features) and methods (e.g., logistic regression, decision trees, or random forest models) that effectively predict dropouts in mHealth apps. We also aim to identify intervention components affecting intervention adherence during the process, thus validating and potentially adding to the findings from Study 1. Furthermore, we are going to visualize cohort retention rates and compare them across different mHealth apps, which may grant further insights into common points of dropouts. For this purpose, we collaborate with the following industry project partners:
Manoa – Pathmate Technologies AG
The digital coach Manoa supports self-managing high blood pressure, type 2 diabetes, and sleep problems in everyday life and motivates users to reduce influenceable risk factors. The app was developed with medical experts and supports the correct documentation of relevant data (e.g., blood pressure, blood glucose, sleep), calculates average values, and provides feedback based on the current state of scientific knowledge and medical guidelines. Manoa is the name of the Pathmate Coach, which is registered as a medical device class I.
Ready4life – Swiss Institute for Addiction and Health Research (ISGF)
The lifestyle intervention ready4life promotes coping with stress, strengthening social skills, and resisting addictive substances. Targeting young people from 15 to 25 years, the app is aimed at learners in vocational schools, polytechnic schools, and training companies in Germany, Liechtenstein, Switzerland, and Austria. Within the app, users can choose a coach (avatar) to guide them through the program.
MySwissFoodPyramid – the Federal Food Safety and Veterinary Office (BLV)
The nutrition app “MySwissFoodPyramid” aims at increasing food literacy in the Swiss population by an interactive food diary and personalized feedback through the MobileCoach-based chat interface and the digital coaches Lukas or Anna. The app is based on Swiss dietary recommendations and aims at people aged 16 and older. The Federal Food Safety and Veterinary Office (BLV) developed the app in collaboration with Pathmate Technologies.
To find effective variables and methods to predict dropouts within these interventions, we will explore a variety of machine learning techniques that have been successfully applied for customer churn prediction in healthcare [14, 17-19], but also other areas such as telecommunications [22-24] or gaming [25-29].
In summary, Study 2 addresses the following research questions:
- RQ 2.1: Which variables predict dropouts in mHealth apps? (e.g., login rates, session length, or interaction frequency with specific features)
- RQ 2.2: Which methods effectively predict dropouts of patients in mHealth apps? (e.g., logistic regression, decision trees, or random forest models
Study 3: Prospective Analysis and Outcome Validation
Applying findings from Studies 1 & 2, we want to develop and evaluate a prototype of an early warning system that predicts and prevents intervention dropouts. As part of Study 3, we plan to assess the efficacy of this tool with one of our project partners.
In a first step, the tool is supposed to predict dropouts processing adequate real-time socio-demographic, medical, and usage data derived from Studies 1 & 2 applying methods derived from Study 2. In a second step, once a dropout is predicted for a specific user, the tool triggers a preventive action based on findings from Studies 1 & 2. The tool’s efficacy will be evaluated by comparing dropout predictions with actual dropout data and comparing dropout rates of the intervention group (receiving preventive action) with dropout rates of a control group.
The outcome validation of Study 3 aims to answer the following research question:
- RQ 3: Are selected actions effective in preventing dropouts?
Results and Possible Real-World Implementations
The literature search for Study 1 yielded 2862 potentially relevant articles, of which 99 were included as part of the inclusion criteria. Four intervention-related factors indicated positive effects on adherence across all health domains: (1) personalization or tailoring the content of the mHealth app to the individual needs of the user, (2) reminders in the form of individualized push notifications, (3) a user-friendly app design, and technical stability, and (4) personal support complementary to the digital intervention. Social and gamification features were also identified as drivers of app adherence across several health domains. A wide variety of patient-related factors like socio-demographic characteristics or type of user acquisition channel further affected adherence. Derived adherence scores of included mHealth apps averaged 56.0% and were highest for Multicomponent Lifestyle Interventions (60.1%) and lowest for apps aimed at reducing substance use (46.1%).
From a practical perspective, the identified intervention-related factors can be used by mHealth app developers as Feature, UI, and UX development guidelines. The identified intervention components positively influencing adherence serve, in combination with findings from Study 2, as an inspiration for potential preventive actions in Study 3. The derived adherence scores across different health apps may also serve as benchmarks to evaluate mHealth apps and draw further inspiration from relatively high-performing apps within their specific health domain. Finally, the identified patient-related factors reveal potential confounding factors when evaluating apps regarding adherence or predicting dropouts for specific populations. These identified factors are now subject to validation in Study 2 and serve to develop preventive actions in Study 3.
Study 1 also confirmed previous findings by Sieverink et al. (2017), that “adherence to eHealth technology is an underdeveloped and often improperly used concept in the existing body of literature”  and that “justifications for intended use are often missing in evaluations of adherence” . Deriving intended use of 97 individual mHealth apps in Study 1 showed that intended use differs across mHealth apps with daily tracking (e.g., daily diary entries, 36/97, 36.1%), activity completion (e.g., completion of a certain number of coaching modules, 19/97, 19.6%), and daily usage (e.g., daily login, 17/97, 17.5%) being the most common categories of intended use
The results of Study 1 suggest that adherence must be defined for each intervention individually, as mHealth apps vary significantly in terms of intended use, intended interaction frequency, and intended interaction duration. Still, we will continue to explore how adherence can be standardized across different mHealth apps, also reflecting on how adherence is measured in non-digital medical treatment and how it developed over time. One extreme measure of adherence with potential universal application for DHIs is the dropout rate, defined as the percentage of users who entirely cease to use the intervention during the intervention period, which closely relates to the term “customer churn” established in other app industries [22-29].
Retrospectively analyzing longitudinal data from mHealth applications in Study 2, we want to identify intervention components affecting intervention adherence, thus validating and potentially adding to the findings from Study 1. We also aim to visualize retention curves and compare them across different mHealth apps to identify common points of dropouts. Most importantly, we strive to identify effective variables and methods predicting intervention dropouts. If identified variables and methods do not predict or explain dropouts with considerable sensitivity and specificity, we can include more features or data sets from additional mHealth apps in the model. Combining the results from Study 2, we expect to predict potential dropouts and identify ideal situations for preventive actions in real-time for the studied interventions.
Based on the results from Studies 1 & 2, we strive to develop a software tool capable of predicting and preventing dropouts in mHealth apps. Evaluating the tool in Study 3, we aim to measure the degree of beneficial effect under real-world clinical settings. This process will grant insights into the extent to which dropouts can be predicted and if selected actions effectively prevent intervention dropouts. In the best case, the model can predict dropouts with good sensitivity and specificity, while the dropout rate is significantly lower in the intervention group than the control group. Regarding the design of the preventive actions, we aim to minimize the potentially harmful effects of type 1 errors (False positive), which will also determine the underlying measures to evaluate the model’s predictive performance.
Potential real-world applications based on these findings could be the development of a SaaS solution that supports mHealth app developers in predicting and preventing intervention dropouts. Such a service might also extend the scope of mHealth interventions with potential applications for other app industries, particularly for subscription-based services.