Google is investigating the source who leaked information about how it reviews and transcribes recordings from Google Home smart speakers and Google Assistant virtual assistant.
Last week, Belgian public broadcaster VRT revealed a Google contractor had shared recordings the company had made via these devices.
The search giant said the recordings came from one of the human reviewers who helps to refine Home’s linguistic abilities.
VRT said the majority of the recordings it reviewed were short clips logged by the Google Home devices as owners used them.
However, it said, 153 were “conversations that should never have been recorded” because the wake phrase of “OK Google” was not given.
These unintentionally recorded exchanges included:
- blazing rows
- bedroom chatter
- parents talking to their children
- phone calls exposing confidential information
It said it believed the devices logged these conversations because users said a word or phrase that sounded similar to “OK Google” that triggered the device.
VRT journalist Tim Verheyden said he had received thousands of these recordings, many of which had been captured by accident after the assistant had misheard its wake word.
The report has brought further scrutiny on Google. Following the revelations, the company responded in a blog post written by David Monsees, a product manager for search.
“As part of our work to develop speech technology for more languages, we partner with language experts around the world who understand the nuances and accents of a specific language. These language experts review and transcribe a small set of queries to help us better understand those languages. This is a critical part of the process of building speech technology, and is necessary to creating products like the Google Assistant.”
“We just learned that one of these language reviewers has violated our data security policies by leaking confidential Dutch audio data,” said Monsees. “Our Security and Privacy Response teams have been activated on this issue, are investigating, and we will take action. We are conducting a full review of our safeguards in this space to prevent misconduct like this from happening again.”
The Google contractor who leaked the audio data to VRT, using the pseudonym Peter, told the outlet: “Every month we are sent a number of audio recordings. In my case, these are Dutch-language recordings from Belgium and Holland. We log in to a Google Platform, where we can see the audio file. We have to transcribe what is said there as accurately as possible. Sometimes, we indicate whether it is a man’s voice, a woman’s voice or a child’s voice. We do around a thousand such transcriptions per week.”
In letting smart products into our homes and lives we should be made aware of the price we have to actually pay. They are, of course, two-way devices, with essentially a chatbot as a middle man. Terms and conditions may detail in fine print that recordings are made and listened to for service improvements, but it still seems an oddly grey area that tech firms seem less inclined to dwell on.
“Google is happy that it has the right to share recordings but is unhappy that this third party has leaked that audio data,” said Frank Jennings, a Cloud, IP, GDPR & commercial contracts lawyer. “Everyone else was surprised to discover that human beings are listening in to their conversations at all.
“It seems speech recognition has progressed so far that we don’t expect humans to be involved at all when we talk to our smart speakers. Perhaps this shouldn’t have been such a surprise since it was revealed that humans were fulfilling some bookings requested via Google Duplex. While asking humans to assist with language recognition and booking fulfilment is a ‘legitimate purpose’ under GDPR, the real question is whether Google is doing so in a ‘transparent manner’ and for ‘specified and explicit purposes’.”
Responding in its blog to the VRT expose, Google said it shared recordings with experts who “understand the nuances and accents” of specific languages to make its speaker more accurate.
“This is a critical part of the process of building speech technology,” it said, adding that the storing of recordings is turned off by default when people start using its Home devices.
It added that it was probing how one of its contractors had been able to give VRT access to the database of recorded chatter.
“Our security and privacy response teams have been activated on this issue, are investigating, and we will take action,” it wrote.
Google admitted language experts are employed to analyse “snippets” of recordings made by users, which Google claims helps improve its voice recognition technology.
This is then used to develop the Google Assistant artificial intelligence system, which is used in its Google Home smart speakers and Android smartphones.
The assistant understands and responds to voice commands given to it, answering queries about the news and weather as well as being able to control other internet-connected devices around the home.