How accurate are fitness trackers?
Look down at your watch. Now, look at me. Now, look at your watch.
Sadly, your watch isn’t me. But, I would be willing to bet it’s a Garmin, Apple Watch, Fitbit, or another popular fitness tracker (unless you’re rocking an old Timex. Kuddos if that’s you!). When you first purchased your fitness tracker, it was probably advertised to have many different health features. And because you bought from the brand you liked, you likely assumed it was an accurate piece of hardware, especially given the cost of some of these things. I am no exception.
Some of you may have purchased to keep track of the calories you burned during exercise or use throughout the day. After all, calories-in calories-out is the fundamental law behind weight gain, loss, and maintenance. But, looking a little deeper behind the glowing numbers on your watch, do you ever wonder how accurate that number is?
Heart rate
Your heart rate is the number of beats your heart contracts throughout the day or a given amount of time. Most fitness trackers will give you a summary of your resting heart rate (RHR), maximum heart rate and even provide you with high & low notifications. Some trackers even have a built-in ECG for measuring your heart rhythm, warning you of possible heart conditions. Cool stuff.
https://www.garmin.com.sg/minisite/garmin-technology/wearable-science/heart-rate/
Data-wise, the coloured charts show how much variance there is in tracking heart rate. If they did a perfect job, we wouldn’t see any dots or lines originating from where it says 0%. All in all, fitness trackers tend to underestimate heart rate.
Variance for heart rate by device brand.
Variance for heart rate by device.
Step count
I have another article about the importance of step count here. I wrote it for a good reason- the number of steps you average daily can profoundly impact your health. So, do these devices accurately track your steps?
Similarly to heart rate, the coloured charts show how much variance there is in tracking steps. If they did a perfect job, we wouldn’t see any dots or lines originating from where it says 0%.
Variance for step count by device brand.
Variance for step count by device.
To sum it up: just under half of the fitness trackers varied in their total step count by 10%. Also, they were likely to overestimate your steps.
I will again reference the Quantified Scientist here, who compares popular fitness trackers with a basic step counter. For example, the Garmin Venue 2 did tend to overestimate step count when compared to 1000 true steps.
“[fitness trackers]…were likely to overestimate your steps.”
Calories
Most fitness trackers will give you an estimate of the number of calories you burn throughout the day. This is often your Basal Metabolic Rate (BMR) combined with your daily activity. In short, the number of calories you need to keep basic functioning running in your body, plus the energy needed for physical activity.
Garmin will combine your active + resting calories and give you a total. This is likely similar to other fitness trackers as well.
But how reliable are these devices in determining your daily calories?
According to a systematic review and meta-analysis by the British Journal of Sports Medicine, there is a significant difference between devices measuring calories. As we have seen already, this shouldn’t surprise us.
Below are graphs and a ‘device legend’ to tell which trackers overestimated and underestimated certain metrics. To follow along, be sure to match the short form tracker on the left-hand side of the graph with the legend below, and look for the big black diamond, which tells you the big-picture result if they overestimated or underestimated calories.
This graph estimates calories used during non-specific exercise protocols, circuits, arm ergometer, rowing and resistance exercises. Some underestimate, and some overestimate calories. Some by a huge amount!
Another systematic review a similar trend in the fitness trackers to underestimate calories burned. The coloured charts below show how much variance there is in tracking calories. Again, if they did a perfect job, we wouldn’t see any dots or lines originating from where it says 0%.
Variance for calories by device brand.
Variance for calories by device.
Next, let's look at some specific measurements.
…AGT3X, Actigraph GT3X; AW, Apple Watch; BA, Beurer AS80; BMC, Bodymedia CORE armband; BP, Basis Peak; EPUL, ePulse Personal Fitness Assistant; FC, Fitbit Charge; FCHR, Fitbit Charge HR; FF, Fitbit Flex; GF225, Garmin Forerunner 225; GF920XT, Garmin Forerunner 920XT; GVA, Garmin Vivoactive; GVF, Garmin Vivofit; GVS, Garmin vivosmart; JU, Jawbone UP; JU24, Jawbone UP24; MB, Microsoft band; NF, Nike FuelBand; PL, Polar Loop; Polar AW200, Polar: AW200; SWA, SenseWear Armband; SWA p2, SenseWear Armband Pro 2; SWA p3, SenseWear Armband Pro 3; SWAM, SenseWear Armband Mini; V, Vivago; WP, Withings Pulse; WPO, Withings Pulse O2.
I couldn’t find an actual definition for Sedentary and Household tasks in the study. Therefore, we could interpret the terms as they are according to their definition.
Sedentary- (a person) tending to spend much time seated; somewhat inactive. (Google)
Household tasks- Performing basic tasks around the home. Although certain tasks may be more exertive than others.
As with the general trend, we see during these tasks that fitness trackers tend to underestimate calories burned.
…AGT3X, Actigraph GT3X; AW, Apple Watch; BA, Beurer AS80; BMC, Bodymedia CORE armband; BP, Basis Peak; EPUL, ePulse Personal Fitness Assistant; FC, Fitbit Charge; FCHR, Fitbit Charge HR; FF, Fitbit Flex; GF225, Garmin Forerunner 225; GF920XT, Garmin Forerunner 920XT; GVA, Garmin Vivoactive; GVF, Garmin Vivofit; GVS, Garmin vivosmart; JU, Jawbone UP; JU24, Jawbone UP24; MB, Microsoft band; NF, Nike FuelBand; PL, Polar Loop; Polar AW200, Polar: AW200; SWA, SenseWear Armband; SWA p2, SenseWear Armband Pro 2; SWA p3, SenseWear Armband Pro 3; SWAM, SenseWear Armband Mini; V, Vivago; WP, Withings Pulse; WPO, Withings Pulse O2.
Similarly, with the first graph, no definitions.
Ambulation- the act, action, or an instance of moving about or walking (Merriam Webster)
Stairs- I’m assuming this is the energy expended while climbing or descending stairs.
Still a ton of variance between devices, with the trend being to underestimate calories. Now, this last graph is the most interesting.
Pooled Hedges’ g and 95% CIs for estimates of energy expenditure relative to criterion measures per device for total energy expenditure (TEE). Total refers to number of effect sizes. A negative Hedges’ g statistic represents an underestimation and a positive Hedges’ g represents an overestimation. DLW, doubly labelled water; EP, Epson Pulsense; FF, Fitbit Flex; GVF, Garmin Vivofit; JU24, Jawbone UP24; MS, Misfit Shine; SWA, SenseWear Armband; SWA p2, SenseWear Armband Pro2; SWA p3, SenseWear Armband Pro3; SWAM, SenseWear Armband Mini; WPO, Withings Pulse O2.
What is DLW? Doubly Labelled Water is the gold standard for measuring energy expenditure and likely the best measurement against these fitness trackers. Again, the fitness trackers tended to underestimate total energy expenditure. I love it when a good review comes together!
Considerations
There are a couple of things to consider about all the data above.
Controlled lab settings yielded better accuracy for step count vs. free-living conditions (lab vs. real-world).
The systematic review and meta-analysis by the BMJ noted that most of the fitness trackers estimated energy expenditure via indirect calorimetry, which may differ by 5.2%. As we noted above, DLW is likely the best control, and only a handful of fitness trackers used DLW as such.
The systematic review and meta-analysis by the BMJ also noted that the population used in the review was only healthy individuals.
The individual methodology behind gathering data differed- some gathered short-term data, some recorded and exported to a database, while others collected data from a brand API (Application Program Interface, the preferred method).
The technology used in this review is up to 2019. The technology found in these fitness trackers changes considerably year over year, yielding better accuracy and more consistent results compared to older devices.
What to take from this?
To conclude, it appears that fitness trackers (up to 2019) are likely to overestimate your steps and underestimate heart rate and energy expenditure. You may think- what’s the point of using them if they aren’t accurate?
First, I would suggest using these trackers as tools to look at trends. For example, you purchased an Apple Watch and started running. After a couple of months, you notice your resting heart rate is declining. As a trend, this is generally a good thing, even though the device may be underestimating your heart rate. Resting heart rate is strongly associated with a decreased risk of all-cause mortality.
Second, fitness trackers can be an introduction to other healthy behaviours. They can be encouraging in implementing physical activity, and with new devices tracking water and demonstrating exercises on the watch, you have opened a new world to fitness and health.
Third, having data like this makes us mindful of the shortcomings of current technology. If someone isn’t losing weight and they are following the calories-burned data from their watch, it may be because the watch is overestimating total calories.
Conclusion
Although they aren’t perfect, fitness trackers can be tools to introduce someone into the world of fitness and health tracking. For the athlete, they can be a way of measuring and increasing performance. If you are fitness & health-oriented, they can be a great tool to develop new habits and become more mindful about specific metrics like resting heart rate and more.
References
Photo- https://www.garmin.com.sg/minisite/garmin-technology/wearable-science/heart-rate/
https://bjsm.bmj.com/content/54/6/332
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7509623/#app1
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5486561/#:~:text=The%20doubly%20labelled%20water%20method%20is%20the%20indicated%20method%20to,the%20behavior%20of%20the%20subjects.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4754196/#:~:text=Interpretation%3A,mortality%20in%20the%20general%20population.
