Finding Order in the Chaos of Behavioral Intervention Technologies
Imagine this: you're trying to improve your health by being more active, so you download a popular behavior change app. It has great reviews and an appealing interface, but does it actually work? Is it safe? Could it even be harmful? This scenario plays out millions of times daily across the globe as people turn to digital tools to help them change behaviors related to physical activity, mental health, and wellness.
Welcome to the unregulated frontier of Behavioral Intervention Technologies (BITs)—the apps, websites, and connected devices designed to support health behavior change. Unlike medications that undergo rigorous testing and approval processes, many of these digital interventions reach consumers with little evidence of their effectiveness or safety.
In 2019, a team of researchers set out to answer this question by conducting a comprehensive review of existing frameworks for validating and monitoring BITs. What they discovered was both concerning and enlightening: a landscape of 46 different frameworks with no consensus on how to evaluate these technologies 2 5 7 . This article explores their fascinating findings and what they mean for the future of digital health.
Behavioral Intervention Technologies (BITs) represent a fascinating intersection of psychology and technology. The term refers to "behavioral and psychological interventions that use information and communication technology features to address behavioral and mental health outcomes" 3 . These technologies include:
For conditions like depression, anxiety, and insomnia
That provide coaching and monitoring
Psychotherapy extensions
That track behaviors and provide feedback
What makes BITs so promising is their potential to overcome traditional barriers to healthcare access. Studies show that approximately 75% of primary care patients with depression identify structural or psychological barriers that prevent them from accessing behavioral treatment—barriers that BITs can potentially overcome 3 .
of primary care patients with depression face barriers to accessing behavioral treatment
Recognizing the critical need for better evaluation standards, a research team embarked on what's known as a narrative review of the existing literature. Their goal was straightforward but ambitious: to identify and categorize all proposed frameworks for validating and monitoring BITs 2 5 7 .
They searched through major scientific databases including MEDLINE, PsycINFO, and Web of Science, looking for any proposed models or frameworks that could guide the evaluation of BITs from development through real-world implementation.
Once they identified relevant frameworks, the researchers analyzed them based on several key characteristics 2 7 :
The research team discovered something remarkable: 46 distinct frameworks coexisting without clear dominance or convergence 2 5 7 . This proliferation has occurred mostly recently, with 57% of these frameworks (26 out of 46) created in the 2010s alone 2 7 .
Distinct Validation Frameworks Identified
| Decade | Number of Frameworks Created | Percentage of Total | Visual |
|---|---|---|---|
| 1970s | 2 | 4% |
|
| 1980s | 5 | 11% |
|
| 1990s | 4 | 9% |
|
| 2000s | 9 | 19% |
|
| 2010s | 26 | 57% |
|
The researchers found considerable diversity in how these frameworks structured the development and evaluation process 2 7 :
Only 4% followed a purely linear sequence
37% combined linear and iterative structures
33% added evolutive structures (circular patterns)
24% incorporated parallel processes
Encouragingly, the majority of frameworks (61%) involved end-users early and systematically in the development process 2 7 .
Perhaps most revealing was the finding that only 12 of the 46 frameworks (26%) covered the complete continuum from initial prototyping to market surveillance 2 7 . Similarly, only 12 frameworks (26%) integrated all three relevant paradigms: biomedical, engineering, and behavioral 2 7 .
Covered complete translational continuum
Integrated all three paradigms
Involved end-users early and systematically
Amidst the framework confusion, some researchers have proposed more comprehensive models to guide BIT development. One particularly promising approach is the BIT Model, which provides a framework for translating treatment aims into implementable technologies 6 .
This model addresses the crucial "why," "what," "how," and "when" of BITs 6 , acknowledging that effective technologies must do more than just incorporate evidence-based behavior change techniques—they must present them to users in the right way at the right time.
Once a BIT has been developed, how do we measure whether it's successfully integrated into real-world practice? This is the domain of implementation science, which focuses on moving evidence-based interventions into routine practice.
A 2019 paper proposed recharacterizing established implementation outcomes specifically for BITs 1 . The researchers identified eight key outcomes for evaluating implementation success:
Note: These outcomes shift attention from whether a BIT can work under ideal conditions (efficacy) to whether it does work in real-world contexts (effectiveness)—a crucial distinction for realizing the public health potential of these technologies 1 .
The discovery of 46 competing frameworks reveals a field at a crossroads. The researchers who conducted the review identified three potentially "dangerous scenarios" that could emerge if this fragmentation continues 2 5 7 :
Companies continue developing BITs with confusing amalgamations of health benefits and usability claims, with limited implementation across countries
A move toward drug-like evaluation frameworks that are heavy, costly, and potentially stifling to innovation
Dependence on post-market surveillance and market self-regulation that fails to address safety risks
Instead, the researchers recommend convergence toward an international validation and surveillance framework that accounts for the specificities of BITs without treating them exactly like medical devices 2 7 .
Such a framework would need to be:
The great framework investigation reveals both the vitality and the growing pains of a field undergoing rapid expansion. The existence of 46 different approaches to validating BITs reflects tremendous creativity and interest but also concerning fragmentation.
The takeaway isn't that BITs are inherently untrustworthy, but that the ecosystem lacks standardized quality controls. This underscores the importance of looking for technologies developed with scientific rigor and transparency about their evidence base.
The challenge is to work toward consensus frameworks that protect users while enabling innovation. As one team of researchers noted, "BITs have the potential to transform health care delivery. Realizing this potential, however, will hinge on high-quality research that consistently and accurately measures how well such technologies have been integrated into health services" 1 .
The path forward will require collaboration across disciplines—behavioral scientists, clinicians, engineers, implementation researchers, and end-users working together to build better tools and better ways to evaluate them. Only through such collaborative efforts can we hope to harness the full potential of BITs to address some of our most pressing health challenges.
This article is based on the narrative review "Identifying Frameworks for Validation and Monitoring of Consensual Behavioral Intervention Technologies" published in the Journal of Medical Internet Research (2019), supplemented by additional research on Behavioral Intervention Technologies.
References to be added manually in this section.