Tracking call logs, Facebook posts and other electronic communications are child's play compared to a new wave of TV room monitoring proposed by Verizon. A supersmart, sensor-laden cable box could anticipate when you need anything from a cold beer to the services of a therapist.
Verizon filed a patent on just such a technology designed to serve up ads on your TV based on what you and others are doing, saying — and yes, feeling. Verizon said its technology would detect "ambient user actions," including "eating, exercising, laughing, reading, sleeping, talking, singing, humming, cleaning and playing a musical instrument." It could also determine what's going on between two people, such as "cuddling, fighting, wrestling and talking."
The three inventors listed on the application are Brian F. Roberts (director of user experience design at Verizon and Redbox Digital Entertainment Services), Anthony M. Lemus (manager of convergence platforms at Verizon) and Michael D'Argenio (director of product design at Verizon Wireless). (Titles are those listed on each man's LinkedIn account.)
The patent was initially rejected by the U.S. Patent and Trademark Office (USPTO) in November 2012 because of two existing patents for similar technology. Patent examiner Christine Kurien wrote, "At the time of the invention [Verizon's] it would have been obvious to one of ordinary skill in the art to modify the teachings to create a television audience-feedback collection system for providing content to the users." One of the existing patents involved newspapers rather than TVs.
Verizon denies plans
Verizon spokesman Ed McFadden told TechNewsDaily that the company "has never had plans to offer this type of technology." He dismissed the contents of the patent, saying that "every year, a number of our innovators file a number of patents. This is a common practice."
Furthermore, McFadden said that Verizon is not the leader in this technology. "We're not even close, unlike Comcast and Microsoft," he said.
Despite denying the plans, Verizon later filed a Request for Continued Examination, which was received by the USPTO on June 12, according to its records. Updated today (June 19), the status of Verizon's patent application reads, "Docketed New Case - Ready for Examination."
Verizon's application integrates all sorts of recognition technology — including motion, facial expressions and voice — that could identify each viewer in the room, as well as determine their emotional states and their actions. Fighting over the remote with your spouse? Sensors connected to a camera, microphone and thermal-imaging devices built into a DVR, cable box or other video-related equipment would immediately assess the situation. During the next commercial break, you might "see an ad for marriage counseling or an aromatherapy candle for stress relief," the description in the patent application reads.
Verizon's plans notwithstanding, other companies have explored similar technology. In fact, most of the pieces for Verizon's system already exist.
Way beyond Siri
It is indeed possible to reliably identify emotions and intent just from listening to someone talk. Apple's Siri can't do it, but Israeli startup Beyond Verbal Communication can. After 18 years of tests and trials, the company recently launched Moodies, a voice-recognition app that can reliably detect underlying emotions in real time. (You can try a beta version of Moodies now.) For instance, Moodies could tell a user if his date agreed to come up just for coffee, or if she was really agreeing to a whole lot more — just "by looking at a mobile app," the company said.
Facial recognition and gesture software is widely available. Look no further than Microsoft's Kinect for Xbox, which was introduced in 2010. The new Kinect, due to ship this fall with Xbox One, is even more astute than its predecessor. And, the new Kinect is always on, even when the Xbox One is turned off. It includes a 1080p camera and sensors that can read emotions and measure heart rate, a biometric that's handy for people using an exercise game.
It's not much of a jump to imagine a set-top box with the capability to know that you're thirsty when you return to the "detection zone" with only a bowl of popcorn and nothing to drink. Yep, here comes an ad for your favorite beverage — make that a Bud, not a Coors, because the system has been watching you all along and presumably knows your brand preferences.
Mobile device monitoring
But Verizon's patent aims to extend beyond the TV. The patent calls for its devices to be able to not only detect when you're using a phone or tablet while you're watching TV, but also to determine what you're doing on them — for example, to "browse the Web, draft an email, review a document, read an e-book , etc." It may also see "one or more Web pages browsed by the user, an email drafted by the user" and more.
While this all seems a little creepy, actual humans aren't watching your TV-room antics. But what happens when that sort of data are shared with others, either unintentionally — like with a security breach — or on purpose, as in a company giving the government access, based on a court order, like what happened with Verizon's business phone logs? [See also: What's Going On With These Tapped Verizon Calls? ]
Lawmakers step in
Lawmakers want to put privacy protections in place before these types of products are available for consumers to purchase. Rep. Mike Capuano, D-Mass., and Rep. Walter Jones, R-N.C., have introduced the “We Are Watching You Act of 2013,” which would compel manufacturers to not only get permission from users to employ the technology, but emblazon a warning sign across televisions connected to such devices.
"Think about what you do in the privacy of your own home, and then think about how you would feel sharing that information with your cable company, their advertisers and your government,” Capuano said in a statement.
If passed, the legislation would require cable companies and any others using the monitoring technology to display the message "We are watching you now" on TV screens in type large enough to be read at "a reasonable distance from the television ."
Further, companies that offer such supersmart devices would also have to offer a similar device that "does not collect this information but is otherwise identical in all respects," the draft bill reads.
"At a minimum, consumers should have the option of saying no to being watched," Capuano said in a statement.