We really like validation: in fact, you might say we take it personally. It’s the foundation of everything we do. We are excited to report that we’re nearing the completion of a study that we believe will greatly expand the possibilities of digital outcome measures in clinical trials.
The research includes steps 1 and 2 of the V3 framework, verification and analytical validation. The outcome will be validated, machine learning algorithms to derive novel, real-world measures of physical function and mobility from wearable kinematic sensors (e.g., accelerometers, gyroscopes). Here’s an overview of our approach to the study and development efforts.
The Most Critical Study Requirement
Good data is required to build trust, prove relevancy, drive adoption, and ultimately see digital health technologies reach their potential in clinical trials. We develop and validate signal processing methods using data labeled with ground truth and tested in challenging real-world environments.
Study Data Collection Process
In our study, sensor verification was performed and research participants underwent two data collection procedures while wearing multiple kinematic sensors at various anatomical locations.
V3 Framework Step 1
Sensor Verification
The purpose of verification is to demonstrate sensor technologies are capturing data:
- • Within a minimum defined accuracy and precision
- • Consistently over time (intra-sensor comparison)
- • Uniformly across multiple sensors (inter-sensor comparison)
Sensors were bench tested using an electronic orbital shaker at frequencies relevant for human movement.
Images 1. Multiple devices positioned on an orbital shakerV3 Framework Step 2
Motion Capture Laboratory
Research participant data collection started in a motion capture laboratory, where a multi-dimensional infrared camera system gathered ground truth gait and mobility data.
Images 2 and 3: Participant suited up and ready to go in the Motion Capture Lab.
Living Science Laboratory
Further real-world development and validation occurred in a Living Science Laboratory. This is a unique home-like setting with a living room, kitchen, bedroom, and bathroom. Researchers investigated human behavior using wearable sensors and a discrete multi-camera system. We collected training and validation datasets under conditions similar to how the participant moves and interacts in their natural environment for prolonged periods.
In clinical research, these tools are used in real-world settings, not laboratories. Therefore, this level of validation is necessary because research has repeatedly shown that the accuracy of sensor derived measures of movement and behavior are significantly reduced when tested under real-world conditions.
Images 4: Participant doing computer work while sitting at a table in the Living Science Lab. The participant is wearing multiple kinematic sensors, some of which you can see on his wrists, and a mobile indirect calorimeter for ground truth energy expenditure.
Novel, real-world measures of physical function and mobility
To study how a given therapeutic impacts disease manifestation and progression, the pharmaceutical industry is increasingly interested in using continuous, real-world measures of physical behavior as outcome measures. There are two distinct types of physical behaviour outcome measures.
1. Participation
A common hypothesis is that when disease progression is slowed or remission is reached, individuals physically feel better, thus increasing participation in active behaviors. However, while the measurement of participation in real-world behaviors might reflect a patient’s lived experience, it can’t be assumed that for all individuals, “feeling better” translates to increased participation in active behaviors.
For many individuals, participation in active and inactive behaviors is unrelated to functional ability or health status but rather is influenced by individual, social and environmental factors such as professional responsibilities, urban vs. rural living, or seasonal variability. For this reason, participation (or lack thereof) in a given behavior is not always an accurate reflection one’s ability to perform that behavior.
2. Function
In our current R&D, we have identified several novel measures of physical function that are less likely to be confounded by other factors. Function variables are measures that qualify how a patient performs a given behavior and reflect physical ability.
For example, we go beyond simply summarizing the amount of time a patient spends walking in their real-world environment (participation). We evaluate the characteristics of walking, such as stride length and velocity (function). These measures are derived from unstructured, real-world data and do not require the patient to change their behavior or perform a prescribed activity.
We have also identified several common in-clinic performance assessments that we believe can be advanced using a wearable accelerometer by objectively assessing outcomes and providing an “out-of-clinic” measurement option. These measures require the participant to complete a defined/structured performance test and are most useful when combined with patient-reported outcomes or context provided from patient diaries.
Advancing the future of digital endpoints is exciting work, and we look forward to sharing our final results and providing a complete list of the developed and validated novel measures from this study soon. We’ve been working with clinical researchers to deliver end-to-end digital biomarkers since 2010 and have an existing pipeline of analytically validated outcome measures.
Give us a call to discuss your clinical trial or healthcare research application today.
1. NCBI: A Refined 2-Regression Model for the ActiGraph Accelerometer
1. NCBI: A Method to Estimate Free-Living Active and Sedentary Behavior from an Accelerometer
1. American College of Sports Medicine: A Framework to Evaluate Devices That Assess Physical Behavior