Suche
This article is part of the 2018 NMJ Oncology Special Issue. Download the full issue.
Relation
Peake JM, Kerr G, Sullivan JP. A critical review of consumer wearables, mobile applications, and devices for providing biofeedback, stress, and sleep monitoring in physically active populations.Front Physiol. 2018;9:743.
Objective
To summarize the characteristics of wearable health technologies and assess their suitability for consumer use by assessing whether the data has been validated, is reliable, and does for consumers what manufacturers claim.
Draft
Review of commercially available wearable health technology devices. Researchers identified devices for inclusion in the review by searching the Internet and databases of scientific literature (e.g., PubMed) using key terms such as “technology,” “hydration,” “sweat analysis,” “heart rate,” “biofeedback,” “respiration,” “muscle oxygenation,” “sleep,” “cognitive function,” and “Concussion.”
Study parameters assessed
Researchers examined commercial technology websites for links to research results and, where appropriate, obtained published research literature. They then divided technologies into the following categories:
- Geräte zur Überwachung des Hydratationsstatus und des Stoffwechsels
- Geräte, Kleidungsstücke und mobile Anwendungen zur Überwachung von physischem und psychischem Stress
- Tragbare Geräte, die physisches Biofeedback bieten (z. B. Muskelstimulation, haptisches Feedback)
- Geräte, die kognitives Feedback und Training bieten
- Geräte und Anwendungen zur Schlafüberwachung und -förderung
- Geräte und Anwendungen zur Auswertung von Gehirnerschütterungen
Primary outcome measures
The investigators assessed the available information using 4 measurements:
- Welchen Anspruch hat die Technologie?
- Wurde die Technologie unabhängig anhand anerkannter Standards validiert?
- Ist die Technologie zuverlässig und ist eine Kalibrierung erforderlich?
- Ist die Technologie kommerziell verfügbar oder noch in der Entwicklung?
Key insights
Researchers identified and evaluated 89 devices; some were commercially available and others were not. They found that the vast majority (82/89) had never been officially validated. Only 10% were used in research settings.
Almost all devices (87/89) had no published reliability tests. Calibration of the devices fell into 1 of 3 categories: 1) not reported; 2) reported as “self-calibrating”; and 3) manufacturers stated that no calibration was required.
When it comes to sleep tracking devices, of the 15 wearables reviewed, only 3 had validation information (UP, FitBit Charge2, OURA), and none of them had undergone reliability testing. The 1 device that had reliability data (FitBit Flex) did not report validation testing.
Practice implications
This study was the first to evaluate different types of wearable devices to determine whether the data generated by wearable devices is valid and reliable.
The wearable device industry is growing at 15% annually and is expected to be worth $51.50 billion worldwide by 2020.1Undoubtedly, clinicians have encountered patients using wearables and will continue to do so. Understanding the technology, particularly as it relates to the demands of a particular device, is essential for thoughtful conversations with patients who are likely to use data from their device as “medical information” during clinic visits.
However, the vast majority of devices the researchers evaluated failed in every way. Manufacturers have not validated the data produced by the devices, most have not calibrated the data to ensure consistency of data readings over time, nor have they conducted reliability testing or disclosed how reference limits were established. Therefore, unless confirmed otherwise, physicians should not assume that data generated by wearable devices is accurate.
For example, the 2 devices with validation data (UP and FitBit Flex) were compared to the gold standard for sleep studies, polysomnography. Each device correlated with total sleep time and time in bed, but they did not correlate with deep sleep, light sleep, or sleep efficiency.2.3
Despite current limitations in accuracy, a potential benefit of wearable technology is that it may provide an opportunity to increase patient awareness of certain health issues. For example, a sleep tracker could provide a starting point for a helpful conversation about sleep and how to improve it. Through this conversation, the doctor can better assess the quality and quantity of sleep. This can lead to conversations about how to improve sleep or help determine whether a more formal sleep study is warranted.
The silver lining, however, is that these devices can provide opportunities to promote healthier patient behavior while verifying (or unmasking) potential health problems through more rigorous and validated testing methods.
A 2018 systematic review and meta-analysis published inAmerican Journal of Health Promotionevaluated the effectiveness of wearable devices to improve physical activity in patients diagnosed with cardiometabolic disease.4Primary endpoints included physical activity, measured in steps per day, and moderate-to-vigorous physical activity [MVPA], which may include activities such as jogging, lap swimming, tennis or racquetball, cycling, aerobics, and dancing.
35 studies involving 4,528 volunteers met the inclusion criteria. The pooled data showed a significant increase in physical activity and MVPA among volunteers who used wearable devices.4This study supports the idea that wearable devices can promote physical activity by making patients more aware of their activity levels.
Similarly, wearable devices that monitor other health parameters such as stress and emotions, heart rate, and blood oxygen levels may provide an opportunity to direct further clinical assessments to patients' concerns and help them either validate or refute the data provided by each device.
While the current study found that manufacturers are releasing unvalidated products, researchers are conducting post-market studies to test the accuracy and reproducibility of specific devices. A 2019 study by Nelson and Allen evaluated the heart rate accuracy of Apple Watch 3 and Fitbit Charge 2 and compared the data they produced to an ambulatory ECG (Vrije Universiteit Ambulatory Monitoring System).5The authors concluded: “Apple Watch 3 and Fitbit Charge 2 delivered acceptable heart rate accuracy (<±10%) over the 24 hours and during every activity, with the exception of Apple Watch 3 during daily activities.”5
The caveat to Nelson and Allen's review is that devices may only perform well under certain conditions. However, it may not be possible to know these limitations from the manufacturer's literature alone. This puts doctors in a difficult situation. Without manufacturers disclosing the limitations of their devices, it is not possible to know under which specific usage conditions devices are more or less accurate.
The silver lining, however, is that these devices can provide opportunities to promote healthier patient behavior while verifying (or unmasking) potential health problems through more rigorous and validated testing methods.