• by John Capone , April 13, 2011

It's been said that it's the fate of every medium to be devoured by its offspring. Screens have been devouring their parents at increasingly faster rates and we've sat in front of them watching, as they multiplied from the one we all gathered around in theaters, to the one we all gathered around in living rooms, to ones we all carry around in our pockets. Through it all we've watched them with rapt attention. It's about time the screens returned the favor.

Mark Weiser, the late chief technologist at Xerox's PARC, who coined the term ubiquitous computing, wrote in a 1991 essay, "The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it." Weiser posited an age of "machines that fit the human environment, instead of forcing humans to enter theirs."

Moore's Law, named for Intel cofounder Gordon Moore, states that the number of transistors on a chip will double every two years (and has held true for more than four decades). With the rise of increasingly powerful and inexpensive hardware, machines have been getting faster and smarter by order of magnitude. The eventual and inevitable flatlining of Moore's Law, though, once doubted by engineers, now is a crisis for the tech industry on par with what Peak Oil is for big energy.

Bell's Law, Moore's complement in many regards, states that machines are getting smaller at the very same time they are getting faster. It holds that every decade or so we'll be introduced to a whole new class of computers; so we've gone from mainframes to desktops to smartphones and other attendant variations. Hence most of us hold more computing power in the palm of our hands than NASA had in 1969 (no matter that it used that power to put two men on the moon and we use it to follow @charliesheen).

This is not to be confused with the "historical exponential view" extolled by futurist Ray Kurzweil, which states that the rate of technological progress cannot be calculated linearly; that because technological progress begets faster and faster technological progress exponentially, it must be charted on a steep curve. That progress now seems to dictate that computers and by default, screens (which may be even more bizarre to contemplate) will be basically absorbed into the real world.

"The destiny of computers - like other mass technologies like electricity, paper and running water - is to become invisible, that is, to disappear into the fabric of our lives, to be everywhere and nowhere, silently and seamlessly carrying out our wishes," theoretical physicist Michio Kaku writes in Physics of the Future (Doubleday, 2011). We are on a precipice - a generation or so (of hardware, not people) away from screens not only being inescapable but aware.

Screens have become like Pringles: You can't stop at just one. We are getting over - or perhaps we are still in the midst - of an insatiable gadget-hungry phase. It's one in which we expect screens at every turn, going from single-screen households to multiple-screen households. Maybe it started to go off the rails with Super Bowl parties where people put TVs in their bathrooms.

Then we began to snack voraciously on screen content. It was always there on the desk at work. TVs in every room. Screens at the train station giving us the news and suggesting places around town to visit. Tinny-speakered monitors next to the pump at the gas station selling us deodorant or showing videos of babies roller-skating and drinking Evian. The map display on the dashboard blinking at us. At a stoplight? Check your phone for updates. Waiting on a line? Post a status to Facebook. By the time we brought another screen into our homes so we could play "Angry Birds" on tablets held on our laps, some had to be thinking, "Did I really just eat all of that?" Personal hardware is developing apace with the technology that's driving outdoor screens, so could our gadget-happy ways curtail the need and applications for public screens? Those concerned with UI and form factor are, of course, asking, "Does there need to be a screen everywhere?"

"If you have customers who are walking around with really sophisticated hardware on them, whether that's an iPad or an iPhone or something else, what is the benefit of putting in interactive touchscreens?" asks Jason Brush, executive vice president for user experience at Possible Worldwide.

"We've already had interactions with clients where they said they wanted to put in in-store kiosks and we said, 'Well, everybody's carrying [interactive devices] around in their pockets - do you need to make the investment?' " relates Matt Lindley, director of brand experience and innovation at SapientNitro.

This is not to say that we'll soon see anything like a decline in public screens. In fact, we are probably just getting started, though by the time we're done we may not recognize the result as screens. "The best applications are ones where it's social [in the old-fashioned sense] and where it makes sense for it to be public," says Brush. "And we've only dipped our toe into what those applications are."

"When a company sees what kind of investment they are going to make in a vending machine with a 42-inch plasma, they have to figure out what they want to compare it to," says Lindley. "And there are a lot of companies that have approached us and said, 'If we spend $10,000 on a kiosk, we better sell $10,000 worth of whatever comes out of it.' And that's the kind of pressure we never put on a billboard." While the cost of elaborate screen installations has come down, it still isn't cheap, though exactly what you'd be comparing the infrastructure and set-up cost to isn't quite clear. "There's no standard. The interaction levels are obviously higher, but what does that mean? What sort of metric do we apply to it? If I spent x to install all this stuff what kind of return am I looking for in having this out there? That's why the connectedness is so important," says Lindley. "If someone touches an IP-connected vending machine, 10 seconds later I know they touched it, and I know what they did with it, and that's data we've collected. Compare that to an ad that used to be static or lit up or even a video: I didn't have that kind of interaction, so I didn't have that kind of data. As soon as we record a touch on a screen for a product, we are eons ahead of a static ad or a video."

The current default is enabling interactive screens with touch- and multitouch capabilities; however, gesture controls in public screens of the sort used by Microsoft Kinect are certainly on the way. "As soon as screens start seeing people, there's a whole host of questions that companies have to consider. It's one thing for somebody to install a gaming system in their home and to think, "I'm in control of this, and this is just another way for me to control it,'" says Brush. "But as soon as you take some of that technology and put it into a public space, there's a question of how much are you veering into a CCTV world?"

At this stage, the data collected from facial recognition technologies is aggregate and of a basic sort: Are you laughing, smiling, confused, saddened, shocked, bored?

Early uses such as SapientNitro's "Share Happy" smile-activated ice cream vending machine for client Unilever (which continues to rack up industry recognitions, recently adding an award at the SXSW Interactive Festival to a long list capped by a Cannes Cyber Lion) have been small- scale and viewed as promotional. That is undoubtedly changing.

Out-of-home screens (including, and right now perhaps especially, those at the movies) will become increasingly aware. Movie screens have had cameras watching the audience for many years, though only for the purpose of spotting the infrared signal of a video camera pirating the film. However, UK-based tech security firm Aralia Systems received a grant late last year to develop a 3-D facial recognition technology it was working on into a system (of the sort employed by Sapient in the smile-activated vending machine) that can collect sentiment (and to some extent demographic) data from audiences during ads and films. Once deployed, this would be the largest-scale such system.

Late last year Microsoft's chief operating officer and chief financial officer Dennis Durkin raised a privacy shitstorm when he made a somewhat off-the-cuff comment during an earnings call about the potential to use the Kinect gesture-control system for ad targeting purposes. "We can cater which content we present to you based on who you are," he said. "How many people are in the room when an ad is shown?" Microsoft quickly made statements emphasizing that it was not spying on people in their living rooms or collecting any of the data from Kinect cameras for advertising purposes. It left out that Durkin was just saying, in effect, "Hey, we could." The system could certainly help create a very robust profile of its users. "If you're using a system at home that knows who you are, has a lot of data about what you are doing - the content you're consuming, your behaviors online, your behaviors in games - and it can cross-index that with knowing who's logged on automatically, it's going to create a pretty sophisticated profile," says Brush. "Brands have to work really hard to explain to people what information is public and what's not, and how they can use this stuff." When you are on your phone looking for a McDonald's, in a way McDonald's is looking for you. Technology is becoming capable of tracking people through a cycle, where someone saw an ad on his phone and perhaps made an online purchase. It's a short leap to add in there when he saw an ad on the display in the airport and at the train station and went to the store and bought a hamburger and a shake.

"We've always had this question about advertising: 'Half the money's wasted, but we don't know which half.' If I advertise in a theater, am I 30 percent more likely to get half the people to go left into my fast-food joint?" Lindley says. "It's really about people patterns supporting the media spend. An affirmation: Am I spending my media dollars correctly and are they having the effect I hope they'd have - and can I confirm that by watching the patterns people follow?" People are becoming accustomed to giving up some data for convenience and relevant information. "There is absolutely that tug-of-war going on," says Peter Rose, senior vice president at market research firm The Futures Company. "That's where the question around transparency really comes to the fore. If this is going to help them at the end of the day, you've got very practical consumers who are saying, 'Sure, I get it.' " He points out the data on those willing to make this trade-off clearly shows that the youngest are most willing, which points to a trend toward a more open and less private consumer (though their kids may really freak out). There is the expectation to be connected anytime and anywhere, and they seem, as a cohort, willing to make that trade. Though, says Rose, they are becoming more conscious of the risks and deliberate in decisions they make. Just as people now online may be more likely to click away on a banner ad from Coke than a sketchy offer to win a free iPad2 or trust Amazon or Apple with reams of personal data for the convenience of one-click checkout, they are also more likely to interact with a Coke machine than a flashing screen displaying something they never heard of.

Will there always be the freaked-out people? Surely. Cookies can be turned off fairly easily and they are currently at the heart of the debate over online tracking. If screens are ubiquitous and commonly equipped with, say, facial-recognition capabilities so you essentially would take your cookie with you everywhere you went, the tinfoil-hat brigade may have a tougher time of it.

Especially when the screens, like Weiser's computers, become ubiquitous. "Eventually we're gonna over-crank this thing and there'll be screens everywhere and we need to begin to control some of this," says Lindley.

"We have to reach a point, and we're slowly getting there, where screens and the objects in which they are embedded or placed on are becoming more and more indistinguishable," says Brush. "And the technology is getting there." As an example, he offers the work being shown in some concept cars now, where the dashboard and the display are seamless. We are still a ways off from moving beyond the sorts of applications that lend themselves to this integration: flat, hard and small surfaces. Though as screens become more malleable, the options for public spaces are myriad. Screens won't just be in boxes or panels. They'll be walls and benches and columns and tables (not to mention your refrigerator, oven and toaster).

While there's a whole spectrum of new consumer experiences opening up - and some people will do some amazing things with that - the ad industry, as a whole, has never been known for its great restraint. It took 40-odd years and legislation for it to lower the volume on TV commercials. Brands that are not careful about where and when they engage consumers may wind up with a big problem.

"Companies have a real responsibility when they are affecting people in public spaces to do things which are respectful of people's lives and their needs," says Brush. "And that doesn't mean not engaging. That doesn't mean not making the screens, or not putting this stuff out there. But it means doing it in a smart and responsible way." "The technology should be aware enough to know that most of the time I don't need it," offers Lindley. "The broadcast model will change to a narrowcast model and I get just the information I need ... The concept has to be really relevant. It has to be that this is a really good application for a screen that happens to be outdoors. If you're at the airport, it's great to have a screen up there telling you when the plane leaves ... but do you need nine screens in bar? Where does it make the most sense and where is it noise?"

"Think about architecture," says Brush. "We all know the spaces which have been thoughtfully designed and the spaces which have not been thoughtfully designed." As much as we may hope that media and advertising will play a part in respectfully designed spaces, we can't hold our breath. Brush, though, is hopeful. "One of the ways to think about it is to make a shift from focusing on communications to focusing on interactions and experiences. This is something that's happening in the marketing world right now, on a larger level." Though Lindley mulls a moment and says, "We may wind up regretting this when we see a shot of the earth from space and it's lit up with old episodes of 'Seinfeld'"