Google Health has announced a new wellness feature coming to its Google Fit mobile app that uses machine learning to give users heart rate and respiratory rate readings without the need for any hardware other than their smartphone's camera.

The two tools are slated to launch for all Pixel devices in the next month, and will come to additional Android devices running version 6 within the months to follow, according to the tech company. Both are accessed within the app and include prompts guiding users on how to position the camera for accurate readings.

For the respiratory rate feature, a user holds the phone's front-facing camera so that their head and upper torso are in view (similar to taking a portrait selfie). The software tracks the rise and fall of the user's chest as they breathe normally over about 30 seconds. According to the company, it is accurate within a single breath per minute on average.

For the heart rate feature, the user instead focuses the camera on their fingertip. The tool detects heart beats through photoplethysmography (PPG), in which minor changes in the optical color of the blood relate to changes in volume. Google said that this approach has a mean absolute percentage error of less than 2%, and that the company is looking into updating the algorithm to detect color changes in a user's face.

The algorithms' measurements and calculations are all conducted on the user's device, Cristina Stancu, product manager at Google Fit, said during a press briefing. Only the final value is uploaded to the company's servers if the user chooses to save their reading, she said. The data is encrypted during transit. Within the Google Fit app, users will be able to view a summary of their recent saved readings.

Jiening Zhan, technical lead at Google Health, said during the briefing that these features were tested internally across a sample of participants comprising different health statuses and skin tones, as well as in various lightings. The company is planning to release these validation studies as pre-print papers "within the coming weeks," and will also be seeking publication in a peer-review journal down the road.

The Google group described the performance of both features during these studies as comparable to existing clinical-grade devices, but stressed that the tools are strictly intended for use as a wellness resource. As such, heart rate and respiratory rate readings are not displayed with additional context or guidance. They also provide a note telling users not to use the readings as a diagnosis or other medical guidance.

WHY IT MATTERS

In a world where heart rate and respiratory rate sensors are increasingly prevalent on consumer smartwatches, Google Health product manager Dr. Ming Jack Po said that the new Google Fit features are an opportunity to deliver these health-and-wellness tools to those who haven't bought – or are unable to afford – a wearable device.

"Equity is an important part of Google's mission, and it actually turns out that relatively few people in the U.S., let alone the world, have wearables," he said during the press event. "So one of the things we were really focused on was trying to get it ... on as ubiquitous a device as currently available, which is of course a cell phone. We know that a lot of people, especially in disadvantaged economic classes right now, don't have things like wearables, but would still really benefit from the ability to be able to track their breathing rate, heart rate, etc."

For now, that benefit is limited to intermittent readings in a personal-wellness capacity, rather than, say, the more robust health insights one could glean from a continuous wearable monitor. For Po, Google Fit users will see the most benefit by taking readings on a consistent basis before or after specific activities – for instance, right after a regular workout or once a week just before bed.

He also didn't rule out the possibility that a future version of the concepts could provide more actionable data for patients with health conditions

"[Data from] the research community [on] the technology generally does seem to suggest that it might work for sick patients as well," he said. "But frankly we haven't done enough testing and validation to say that we can definitively work in those use cases yet, but it's definitely something we're exploring."

Smartphones already house a wide range of sensors that could be repurposed for health and wellness with the proper algorithms, Shwetak Patel, director of health technologies at Google Health and a professor at the University of Washington, offered. While this initial set of features relies exclusively on the camera, Patel said that his team is also looking into how additional sensors could either improve these initial features' readings or provide new information to users.

"There is a desire to explore sensor fusion. So, using multiple sensors, you can imagine combining the accelerometer and camera to improve the measurement," he said. "It's an area of exploration, but right now the focus was how much can we extract from the camera itself."

THE LARGER TREND

News of the Google Health team's latest consumer health-and-wellness tools come shortly after the company's long stalled acquisition of Fitbit was finally set in stone. Ostensibly, the smartphone camera features would act as a complement to the smartwatch and tracker offerings Fitbit is bringing to the table – although Patel noted that any collaboration between the two teams on a joint project is still in its earliest stages.

Still, Google has been pumping out a fair number of health products and resources designed for consumer use over the past year. In December, the company launched an Android app that streamlines health-study recruitment while showing consumers how their personal data is being employed by researchers.

The company also revealed a pilot project designed to help patients prepare for an upcoming doctor's appointment around that same time, and just last week said that it will be updating its Search and Maps services to help users find nearby COVID-19 vaccination sites.

Google isn't the only tech giant in town that's homing in on consumer health-and-wellness products. Apple has been hard at work on new offerings primarily focused around the Apple Watch, such as updates to its Fitness+ subscription service and upgrades to the smartwatch's blood oxygen and ECG measurements.

At the same time, its personal health record tool has continued to gain steam among provider groups in the U.S. and abroad, while its various digital health research studies continue to enroll iPhone and Apple Watch owners.