Amazon may not have unlocked the secret to happiness. But with the announcement of a new voice monitoring tool called Tone, the company promises that it knows what happiness sounds like. And that—with a new gadget and a little tracking—you, too, can sound happy.
Tone will be a feature on Amazon’s new wearable health tracker, dubbed Halo. Users can opt in to let it sample snippets of their speech throughout the day, or turn it on for up to 30 minutes at a time to get a detailed report on how they sounded in a particular conversation. Powered by AI algorithms designed to detect the “positivity” and “energy” in human voices, the tool purports to offer users feedback on their tone so they can improve their communication skills and relationships.
Of course, it’s hard to define fuzzy traits like positivity—and it’s an even more Herculean task to train an AI model to objectively quantify and measure them. In a blog post, Amazon simply says that “positivity” measures how happy or sad a voice sounds. But humanity (and the field of positive psychology) have been wrestling with how to define happiness for eons.
“It’s hard for me to imagine that there could be a single objective measure,” said Jim Allen, an associate professor of psychology at the State University of New York at Geneseo who writes and teaches about the psychology of happiness. Our perception of what a happy voice sounds like, he notes, varies depending on culture, gender, ethnicity, and other personal factors.
An Amazon spokesperson said that the developers had accounted for these differences by drawing on vocal samples from tens of thousands of voices from across US regions and demographic groups. A team of Amazon employees then listened to the recordings and rated the voices as happy or sad to determine “positivity” and tired or excited to measure “energy.” The model associated those emotional ratings with vocal qualities like pitch, intensity, tempo, and rhythm, which the AI uses to label users’ speech.
Training sets, however, are highly susceptible to bias from the humans who build them, as researchers have extensively documented in fields like facial recognition. That makes vetting the data, and the people who label it, very important. Amazon declined to offer any detail about the demographic breakdown of its vocal samples, or the team whose perceptions of positivity and energy form the basis for the model. “Throughout product development, we’ve focused on ensuring the data we use to train and evaluate our models accounts for all demographic groups,” a spokesperson said in an email.
In particular contexts, Allen said some version of a tool like Tone could work well. “In the hands of a skilled counselor giving feedback to a client about how they come across to other people, it could be really helpful,” he said. But, he noted, constantly monitoring yourself for signs of happiness—or worse, projecting a positivity you do not feel—has been shown to make people less happy.
Pattie Maes, an MIT professor who studies wearable technology designed to enhance people’s lives, pointed out that the AI would be more likely to return meaningful results if it didn’t try to treat happiness as a universal truth. “People have different speaking styles,” she said in an email. “I believe a personalized AI model trained on an individual’s own data would perform better.” (While Tone learns to pick out a user’s voice in a conversation, it does not calibrate its ratings to that user’s emotional baseline.)
But these approaches to boosting the model’s validity are not compatible with mass consumer tech. In its announcement blog post, Amazon medical officer Maulik Majmudar describes a gadget that comes out of a box ready to coax users into better communication. He writes about the ease with which his colleagues can turn on Tone and rehearse for a big presentation at work. Majmudar says he switches the system on before talking to his children, to make sure he’s not taking work stress out on his family.
It’s an intriguing vision for an AI-enabled future.But it might not be the one we live in right now.
Amazon has entered the health and fitness world with Halo, a subscription service and accompanying fitness band that unlocks an array of health metrics, including activity, sleep, body fat and tone of voice analysis, to determine how you sound to others. Amazon’s entry into the fitness space is odd indeed, and ambitious. And we’re just getting our minds wrapped around it.
The band itself looks a lot like a screenless Fitbit tracker, but with a few different elements: It has temperature sensing, much like Fitbit’s newest smartwatch, the Fitbit Sense, and a microphone that continually scans a wearer’s voice to determine emotional tone. Yes, it’s a lot to take in. And the service is immediately available for early access. We haven’t even had a chance to try it out yet.
The membership part will start at $65 for the first six months ($100 once the early access deal is over) and then $3.99 a month after that. (International prices aren’t currently available, but $65 converts to about £50 or AU$90.) The subscription to Halo includes the basic fitness band that has one button, no screen and tracks your heart rate, steps and temperature. The lack of screen means you’ll have to rely on the mobile app to see all your data, but it does a lot more than just count your steps and log your weight.
A tone-analyzing, Amazon health band that also lets you scan your body fat may sound like Black Mirror incarnate, but it’s also opening up some ideas in fitness that we’ve never seen before.
Now playing: Watch this:
Amazon’s Halo takes fitness tracking to new and uncomfortable…
Body fat analysis with a smartphone camera
Amazon thinks the concept of weight loss is flawed, and that body fat is a much better predictor of health.
Most of us have been conditioned to obsess over our weight. The entire diet industry was built on it with programs, apps and devices that revolve around ways to lose pounds.
But weight can fluctuate daily based on factors including humidity, medication, menstrual cycle and illness. Plus muscle is more dense than fat, and a scale can’t tell the difference between the two. You could literally work your ass off building muscle and burning fat, and not see the numbers on the scale go down.
Rather than relying on weight, Halo focuses on body fat percentage, which is less volatile and takes a lot more time and work to change.
The gold standard in the medical world for body composition analysis is a DEXA scan (dual-energy absorptiometry), which can cost up to $100 at a lab. The Halo app does it all using your smartphone camera. Once you take your photos, the app automatically eliminates everything else in the background, calculates body fat percent based on body indicators, and then creates a 3D model of your body, which is both cool and terrifying. The app requires you to wear minimal form-fitting clothing and trust Amazon to take a picture of you wearing it. The entire process takes seconds.
If you’re feeling uncomfortable, that’s not surprising: The idea of body-scanning with a camera is already an awkward proposition. Amazon doing this on a health platform makes it feel more so. The sample body-scan images Amazon showed me look very personal — not necessarily something I’d ever want anyone else to see.
That’s why Amazon promises that the finished body scans stay on your phone and won’t be shared with anybody, including the company, unless you opt into that. According to Amazon, “the images are processed in the cloud, but encrypted in transit and processed within seconds, after which they’re automatically deleted from Amazon’s systems and databases. All scan images are fully deleted within 12 hours. The scan images aren’t viewed by anyone at Amazon and aren’t used for machine learning optimizations.”
Watch that tone!
Halo also offers a Tone analysis, which has nothing to do with body tone, but rather analyzes the nuances of your voice to paint a picture of how you sound to others. It can let you know when you’ve sounded out of line, weirdly enough.
The fitness band has two built-in mics to capture audio and it listens for emotional cues. The company says it’s not intended to analyze the content of your conversation, just the tone of your delivery. It takes periodic samples of your speech throughout the day if you opt in to the feature. You enable the microphones by tapping the side button and you’ll know when the mic is off when a red LED lights up on the band.
The voice scanning pulls out the wearer’s specific voice in conversations and delivers analysis with related emotional-tone words (like “happy,” or “concerned” in the Halo app). The idea, according to Amazon, is to help guide you to deliver better tones of voice and speaking styles, like a vocal form of good posture. It isn’t intended as a form of psychological analysis, but it seems awfully hard to draw the line on a concept like this.
Amazon’s been exploring the idea of emotional tone-sensing since at least 2018, but this is the first time it’s approached the idea in any device. And according to Amazon, the Tone feature is only available on the Halo band for now. It will be limited to the band’s microphone, but Amazon sounds open to exploring the idea on other devices, depending on how the early access response goes from first-wave wearers. It’s a very odd thing to put on a fitness band, and we have no idea what this is like to use yet.
Amazon promises that Tone voice samples are encrypted and stored only on a wearer’s phone (shared from the band via Bluetooth with the encrypted key), are deleted after analysis and won’t be shared to the cloud or used to build machine-learning models.
Sleep analysis with temperature tracking
The app provides a comprehensive sleep analysis with a breakdown of the different stages of sleep and overall sleep score, much like other fitness trackers. It also goes beyond the basics by keeping track of your overall body temperature during sleep and creating a baseline for each person. It then charts your average temperature each night relative to your baseline to help you identify variations that could affect your health and the quality of your sleep.
The Halo band won’t provide a specific body temperature, similar to the way other temperature wearable devices like the Oura Ring already work.
Temperature has become a trending wearable metric in the COVID-19 era: The Oura Ring has one and Fitbit’s newest Sense watch has one too. Amazon’s Halo team is pursuing research for COVID-19 symptom detection on its wearables, much like other health wearable companies, but no specific studies or plans have been laid out yet.
Activity tracking: A week at a glance
Halo also does basic fitness tracking based on the information from the band. It can automatically track walks and runs, but you’ll have to go into the app and tag any other workouts manually.
It rewards you for any type of movement or activity, but gives more points for more intense workouts and subtracts points for sedentary time. And it doesn’t keep a daily tally of your activity, your score is based on the points you accrued during the entire week. The entire picture of exercise, sedentary time and active time is combined into one tally.
Amazon’s sleep and activity scores and other AI tools will require an Amazon Halo subscription; otherwise, the band will default to more basic tracking data. Much like Fitbit and its Premium service, this looks to be continuing a trend of fitness devices that expect a subscription model as part of the package.
A lot of labs and partners, but no Google or Apple integration
A Labs section of Amazon Halo looks similar to what’s on Fitbit’s Premium service, with a lot of multiweek health and fitness goals to opt into, and partners lined up from OrangeTheory to Weight Watchers. Amazon promises these challenges are scientifically vetted, but it also sounds like these challenges will keep being added to over time.
But at least at launch, Halo will not tie in to Apple’s HealthKit or Google’s Fit App which puts it at a disadvantage with people who are already deeply invested in either for health tracking. Amazon is leaning on Weight Watchers, John Hancock Vitality wellness program, and a few others that will be able to hook into Amazon Halo health data.
The looming privacy question
There’s a lot of process in terms of features, and while some seem interesting and innovative, the biggest barrier to entry is privacy. Sharing any kind of health data (let alone unflattering seminudes) requires next-level trust, and you might not be prepared to give Amazon that trust. The company doesn’t exactly have the most pristine track record when it comes to keeping user data private. Alexa-enabled devices have been in the hot seat for storing private conversations “for machine learning purposes.” And Amazon’s Ring doorbell has had a series of privacy dust-ups.
Halo puts privacy in your hands by allowing you to opt out of data sharing with Amazon and third-party apps as well as disable the microphone on the band, but it’s still going to be an uphill battle. That is unless its features prove to be earth-shattering and worth the privacy risk, which remains to be seen.
Amazon is late on arrival
The lack of connection to Apple or Google is telling. Amazon’s making a play in the health and fitness data space, and with Google, Fitbit and Apple already deep in, it’s a big question as to how Amazon will make waves. Or, where Amazon Halo will go next. It’s a platform as much as a wearable, and it sounds like Halo’s early-access experiment may just be the tip of the iceberg.
Our Health & Wellness newsletter puts the best products, updates and advice in your inbox.