Prediction and Reactions
PREDICTION: In 2020, the most commonly used communications appliances prominently feature built-in voice-recognition. People have adjusted to hearing individuals dictating information in public to their computing devices. In addition “haptic” technologies based on touch feedback have been fully developed, so, for instance, a small handheld Internet appliance allows you to display and use a full-size virtual keyboard on any flat surface for those moments when you would prefer not to talk aloud to your networked computer. It is common to see people “air-typing” as they interface with the projection of a networked keyboard visible only to them.
Expert Respondents’ Reactions (N=578)
Mostly Agree 64%
Mostly Disagree 21%
Did Not Respond 15%
All Respondents’ Reactions (N=1,196)
Mostly Agree 67%
Mostly Disagree 19%
Did Not Respond 14%
Note: Since results are based on a nonrandom sample, a margin of error cannot be computed. The “prediction” was composed to elicit responses and is not a formal forecast.
Overview of Respondents’ Reactions
A clear majority of respondents favored the idea that by 2020 user interfaces will offer advanced talk, touch, and typing options, and some repondents added a fourth “T” —think. Those who chose to elaborate in extended responses disagreed on which of the four will make the most progress by 2020, with a fairly even yes-no split on the success of voice-recognition or significant wireless keyboard advances and mostly positive support of the advance of interfaces involving touch and gestures. A number of respondents projected the possibility of a thought-based interface—neural networks, mind-controlled human-computer interaction. Many expressed concerns over overt public displays of ICT use and emphasized the desire for people to keep private communications private.
Nearly two-thirds of survey participants mostly agreed with this scenario about advances in network interfaces, with just one-fifth mostly disagreeing. “It is these technologies that will enable the mobile device to become powerful enough for use in serious applications,” responded Brad Templeton, chairman of the Electronic Frontier Foundation and an Internet pioneer who has been active online since 1979. Cliff Figallo, social innovator and original member of the first online community, The WELL, agreed. “More time on the move, less time sitting at orthodox computer interfaces,” he wrote. “The need to communicate and think through handhelds will stimulate growth in use of such features.”
“In addition to this,” predicted David Brin, futurist and author of “The Transparent Society,” “there will be ‘subvocal’ inputs that detect ‘almost speech’ that you will, but do not actually voice. Small sensors on teeth will also let you tap commands. Your eyeballs will track desires, sensed by your eyeglasses. And so on.”
Respondents noted that intuitive, human-centric interfaces allow technology to eliminate some social, economic, and physical divides. “Ease of access + usability will entice more people to interact with technology—in other words, it will not only be limited to computer-literate people,” commented Sam Ozay, an e-learning and e-communication specialist and solutions architect at Postmodern-Asia/Pacific. Jan Schaffer, executive director of J-Lab, the Institute for Interactive Journalism, wrote, “I see great benefits for education in the form of alternative learning, and assessments of learning for dyslexics and LD children.”
Security is always an issue, as noted by Alejandro Pisanty, director of computer services at the Universidad Nacional Autónoma de México, an active leader in the Internet Society and ICANN. “They will all be hacked big-time!” he predicted. “Think of using a cell phone for video-recording a person who types on her lap while riding a subway.”
Many are concerned about social ramifications of new interfaces. “By 2020 I would hope that there is some other way to get information without a public display of any kind through ubiquitous technology,” suggested Teresa Hartman, associate professor and head of education at the University of Nebraska Medical Center. “Interactions with personal communication interfaces should be less intrusive to others than taking out a notepad today and writing a note. Communication users have allowed the public display of their interactions to continue and even increase due to what they perceive as a ‘wow’ factor—‘look at me, I have a cell phone and know how to use it.’ I see the prediction of us air-typing to be in the same category. In the future, using technology (hopefully) won’t be a status item, and can be conducted discreetly and with panache. Somehow, interactions with communication/information have to be put back in the individual’s world, instead of bleeding over into everyone’s world, and not causing any more interruption or notice than a quick cough into a handkerchief.”
All respondents expect evolution of some kind. “Yes, yes, and yes,” noted Leonard Witt, associate professor at Kennesaw State University in Georgia and author of the Webog PJNet.org. “It’s all disruptive technology, which means as Clayton Christensen says, cheaper, smaller, faster, and easier to use. It can’t be stopped.”
“Solitude will soon become a thing of the past, as no one is ever disconnected,” commented Lisa Carr, director of strategy for Targetbase Interactive.
Yes, Talk Will Be Effectively Developed, or, No, Talk Can’t Be Effectively Developed, at Least by Then
Respondents debated the idea of voice as a user-interface, with some in support of its development to perfection, some saying the technical issues to develop it correctly have been and will continue to be too difficult to overcome by 2020, and some expressing concern over social acceptability.
“By 2020 the voice ‘interface’ will be more sophisticated,” predicted Maz Hardey, a social analyst and blogger completing a doctorate funded by the Economic Social Research Council at the University of York. “When not touch typing, voice commands will allow the user to talk to those in the immediate and physical vicinity, as well as to update and ‘chat’ across SNS.”
“We are already used to the way dictating to devices would sound, since Bluetooth headsets and cells create a similar hearing experience,” commented Paul Greenberg, president of the 56 Group LLC. “This is not a difficult one to see, given the rates of technological advance, especially in computers and electronic gear that we are seeing today.”
“WiFi- and WiMax-enabled badges with voice recognition will act as personal assistants—allowing you to talk with someone by saying their name, to post a voice blog, or access directions from the Internet for the task at hand,” predicted Jim Kohlenberger, director of Voice on the Net Coalition, a senior fellow at the Benton Foundation.
Those who disputed the likely use of talk as a UI by 2020 generally noted how difficult it has been up to this point to overcome the technical barriers in designing a usable talk interface. “Speech-recognition and even natural-language understanding are evolving, but it’s been a very gradual process over several decades and it is likely to take several additional decades before we approach Hal-like performance,” commented NMS Communications CTO Brough Turner, referring to the AI computer Hal in the film “2001: A Space Odyssey.”
“Voice will continue to be the most over-sold, over-hyped, but un-used interface,” noted Walt Dickie, executive vice president and CTO for C&R Research. “Voice recognition has been a holy grail of computing since ‘Star Trek’ in the 1960s,” wrote Charles Ess, a researcher on online culture and ethics based at Drury University in Springfield, Missouri, a leader of the Association of Internet Researchers. “Like the artificial intelligence that was supposed to make it happen…it has faltered for a host of reasons, beginning with technical ones. Perhaps there will be some sort of technological breakthrough in the next few years that will make voice-recognition workable and affordable—but I’m not optimistic.”
“Although voice control will progress to where it can be mainstream, it will not surpass other input mechanisms—mostly touch screen and accelerometers,” commented Todd Spraggins, chairman of the board of directors of the Communications Platforms Trade Association and a strategic architect with Nortel Carrier Networks.
Clay Shirky, consultant and professor in the Interactive Telecommunications Program at New York University, author of “Here Comes Everyone,” wrote, “Ben Shneiderman’s work on the limits of voice recognition and the weakness of the human brain’s ability to co-process other information alongside the spoken word are, in my view, dispositive critiques.”
“I worked on voice-activated technologies and AI in the 1980s, and I am familiar with the overblown predictions that were made then,” responded Micheál Ó Foghlú, research director for the Telecommunications Software & Systems Group, Waterford Institute of Technology, Ireland, arguing in support of the positive future of the talk interface. “Steady progress has been made, and the need to use innovative interfaces on small mobile devices is a good spur for these developments in the next 12 years.”
Some noted that those with special needs are most likely to use speech-recognition UIs first. “With increased attention being given to the need of ‘specially-abled’ people, ‘talk and touch’ will become more popular (and profitable) as devices that employ such will help empower more people who never had the chance,” noted Gbenga Sesan, a consultant for Internet development with Paradigm Initiative in Nigeria, adding that 2020 may be too soon for it to be practicable.
Concerns over the appropriate use of talk interfaces were expressed by some respondents. “I mostly agree with the scenario, although a rise in voice-driven interactions might lead to social reactions against the use of these devices in public spaces,” commented Paul Miller, technology evangelist on the senior management team at Talis, a company delivering human-centric Web applications, based in the UK. “See, for example the differing attitudes to speaking on phones in restaurants, etc., today. In some places this is acceptable, in others most definitely not.”
“The sound rules out using [voice] in many environments,” commented Christine Boese, researcher and analyst for AvenueA-Razorfish and Microsoft, “(and I even avoid listening to podcasts on the subway because my hearing is so bad already, and the train noise is too loud). Privacy concerns arise with too much spoken technology, or should, when we see people walking up and down aisles at the grocery store, talking out loud on their mobile phones with the ear bud hanging out of their ears.”
“People in airports and grocery stores who talk to themselves using those stupid looking knobs in their ears are already annoying. Imagine an office where people in cubicles are all talking to themselves—composing proposals, sending e-mails, making notes on their next presentation to the boss. Yipes!” noted Mike Samson, an interactive media writer and producer.
“I expect to see some use of these things, but my use of them so far (Amtrak’s ‘Julie’ for example) suggests that they only work when conflicting sound can be stopped and when talking to a computer is not disruptive to others,” wrote Fred Baker, fellow at Cisco Systems and a longtime leader of the Internet Society and IETF. “That imposes quite a limit.”
Touch Is Natural and Intuitive and It Will Succeed
While talk drew heated debate from the respondents who wrote elaborations to their answers on this scenario, positive support for the future of the touch interface was nearly unanimous. “Touch is there already, with the Microsoft Surface computer, the iPhone, the Wii,” noted Christian Huitema, distinguished engineer with Microsoft and an Internet pioneer and active leader of the IAB and Internet Society.
Jerry Michalski, founder and president of Sociate, a technology consulting firm, responded, “Touch is the first major step away from the windows/mouse interface, which is very long in the tooth. We’re due for some more advances in the next 13 years.”
Jonathan Dube, president of the Online News Association, director of digital media at CBC News and publisher of Cyberjournalist.net, wrote, “Touch feedback will be the primary mode, with voice recognition an increasingly common tool (but not on airplanes!)”
An anonymous respondent predicted, “touching machines in ways that we have not imagined will become possible.”
Many Say Typing Has Advantages and It Will Advance
Many respondents see the survival of keyboards as input devices as highly likely. “Most people form a tactile bond with their keyboards and a comfort with their workplace/desktop environments that will be difficult to replace with haptic appliances and voice recognition,” noted Michael Edson, director for Web and new-media strategy for the Smithsonian Institution.
Jeff Jarvis, blogger at Buzzmachine.com and a professor at City University of New York, predicted, “We will have control environments that don’t require us to read buttons. We will also have some means of typing specific wording quickly and accurately without two-handed (or two-thumbed) keyboards. I await their invention.”
“Air keyboards or projected keyboards will be a great advancement, as they will allow small devices to become fully-functional computers that finally will allow people to work the way they want and with a maximum of convenience,” wrote John Jordan, associate professor of communications at the University of Wisconsin-Milwaukee.
Survey participants had mixed reactions to the idea of air-typing. Many thought it unlikely. “This one sounds too much like The Kitchen of the Future at some 1930s World Fair; I think we’ll have better, more adaptable devices, but I doubt we’ll be air-typing,” commented Susan Crawford, founder of OneWebDay (celebrated each September 22) and an ICANN board member and law professor at Yale. “Roll-out, flexible keyboards might be the more likely development,” wrote Seth Finkelstein, author of the Infothought blog and an EFF Pioneer Award winner. An anonymous respondent commented, “Laser-based keyboards are available today but are often inaccurate and inconvenient. It’s hard to imagine this situation will change much by 2020.” Another anonymous respondent wrote, “Keyboards will remain. So will street signs and the alphabet.”
Many who disagreed with the idea of air-typing noted the lack of physical feedback one gets when typing in an empty space. “Tactile interaction requires feedback,” noted Richard Osborne, a Web manager at the University of Exeter. “That’s why our hands are designed the way they are.”
But Internet sociologist and author Howard Rheingold responded, “The point-and-click user interface is 40 years ago. It’s time for more human-machine bandwidth. You are a typist, try ‘air-typing’ and see if it doesn’t feel natural very quickly.” And Havi Hoffman, of the Yahoo developer network noted, “I can imagine air typing of a kind, and a flat and more fluid electronic paper than we’ve seen yet.”
Some respondents disputed the idea that keyboarding will still be a dominant UI. “I still envisage a replacement for the keyboard, virtual or otherwise,” commented Adrian Schofield, manager of the applied research unit at the Johannesburg Centre for Software Engineering. “My vision is of a virtual pen that can interpret any type of script.”
“Typing will be a thing of the past; it seems reasonable that some forms of subvocalization, not to say ‘mind reading’ will eliminate the need for a manual interface—kind of like reading ‘almost aloud,” suggested Oscar Gandy, author, activist, and emeritus professor of communication at the University of Pennsylvania.
Look, Ma, No Hands or Voice; Communicating by Thought Alone
Many respondents predicted that brain science will advance to the point at which there will be at least some human-machine interaction conducted through the reading of brain activity. Most who offered this view did not note that they expected this to be true as soon as 2020.
“Future technologies (although perhaps not by 2020) will involve physically connecting our bodies ‘wirelessly’ to computer/digital networks through true ‘neural nets,’” responded Benjamin Ben-Baruch, senior market intelligence consultant and applied sociologist for Aquent. “It will literally become possible to interface with these networks via neural nets that connect our nervous systems to the networks. The common technology interfaces will be ‘talk-touch-think.’”
“I totally expect even mind-controlled interaction by thought using a simple range of commands which in combination allow ‘joystick’-style interaction,” commented Robert Eller of Concept Omega, a marketing and communication company. “This is already today virtually possible. Some research even shows that we can grow additional synapses into minute glass vials that will connect to wires allowing fighter pilots to steer a jet. Nano and bio technologies should yield some significant advances here making such interaction, if not mind-controlled, at least be part of the body.”
“I suspect we will eventually move beyond voice and touch interfaces for computing in the future,” predicted Gary Kreps, chair of the department of communication at George Mason University. “Instead we will direct computing directly through our cognitions, through thought.”
Bruce Turner, director of planning services for a US regional transportation commission, agreed, writing, “As brain-mapping technology improves, we may forego the virtual of the real world to direct our consciousness to type inside our brains for transmittal to the surface.”
Respondents Suggest Additional Interface Innovations
Some respondents expressed various additional expectations for UI in 2020. Many noted that gestures and body language (as exemplified in Nintendo’s Wii game system) may be more common than talk, typing, or touch. “Air-typing (difficult without tactile feedback) will be less commonplace than seeing people make gestures into thin air,” noted Ivor Tossell, technology columnist and journalist for the Toronto Globe and Mail.
Ed Steinmueller, a professor whose research expertise is the industrial structure of high-technology industries, commented, “Although I doubt that the keyboard metaphor is entirely apt, the extension of the interface to gesture seems very likely.” Scott Brenner, technologist and consultant, predicted, “The haptic technologies will prevail, although we’ll be getting away from the keyboard method of input. Instead, data-getting and giving will be more intuitive, using icons, structured gestures, and a more semantic information universe.”
“Other types of inputs, such as simple gestural inputs, may prove more popular than full-keyboard inputs,” suggested Scott Smith, a futurist with Changeist LLC, consultant, and writer based in North Carolina. “Additionally, interfaces will be more predictive, taking into account contextual information about a user to determine data we might have to manually enter today.” Tiffany Shlain, founder of the Webby Awards, commented, “I could see a whole physical way of communicating with our technology tools that could be part of our health and exercise. A day answering e-mails could be a full-on physical workout ; ) .”
Smaller movements made by the eyes and face were also noted as possible interface methods by respondents. Jay Neely, social strategist and founder of News Armada, a Boston-based Internet-news community, wrote, “Advancements in eye-tracking technology, combined with the miniaturization of components needed to create devices in the same size and form as eyeglasses, make sight a more likely interface for services that only require information consumption and very limited data entry.”
A number of respondents noted that devices will interface with ubiquitous computing built into human architecture. “It will be common to see people interacting with signs,” responded Fred Hapgood, technology author and consultant.
“We will see the display interface device separated from the input device over the next 12 years,” wrote Ross Rader, a director with Tucows who is active in the ICANN Registrars constituency. “Display devices will be everywhere, and you will be able to use them with your input device. The input device might be virtual, as in the case of the iPhone or a holographic keyboard, or they might resemble the keyboards and touchpads that people are using today. Likely, some combination of these will prevail. These devices will be able to securely interact with any display device that the user selects, using common standards that permit the user to interact with data in a variety of resolutions and formats.”
Chris Miller, senior vice president for digital operations for Element 79 predicted, “Common objects, desks, countertops, etc., will become haptic-sensitive and provide feedback and content and send/receive information based on touch. This will correspond to the everyWeb which allows appliances, objects, etc., to be networked. ‘The Minority Report’s’ haptic gestures and feedback will be a reality.”
Some respondents suggested a cluster of alternative user-interfaces. Sean Steele, CEO and senior security consultant for infoLock Technologies, predicted that by 2020: “While air-typing and haptic gestures are widespread and ubiquitous, the arrival of embedded optical displays, thought-transcription, eye-movement tracking, and predictive-behavior modeling will fundamentally alter the human-computer interaction model. What we think is performed almost in real time, when and how we imagined it to be.”
The Scenario Is Wrong; It Is Not Going to Happen This Way
Those who disputed the scenario expressed a variety of viewpoints in their elaborations. Layered reasoning came into play in a number of the responses. “Neither of these are particularly efficient interfaces, at least as described,” wrote Jamais Cascio, blogger, public speaker, and futurist. “The social response to mobile-phone conversations in public—resigned/resentful acquiescence—is a likely model for voice interfaces, slowing or even halting their widespread adoption. As for haptics, these seem more likely, but not as described; ‘air typing’ and similar non-responsive interfaces have a poor record of usability. More likely is some kind of touch-based interface, possibly even a finger-on-opposite-palm model.”
“Products continue to be driven by short product lives and lowest-possible-cost, and dim displays and flat membrane switches,” answered Tom Jennings, the creator of FidoNet, the first message-and-file networking system online and builder of Wired magazine’s first online presence. “Extreme power management will continue to work apparent miracles in ubiquity, and will have unpredictable side effects. People forget that ‘lack of interface’ also allows for perceptual partitioning and maintaining of separate cognitive spaces. Eg. I can let the phone sit on the table and not answer it and it doesn’t impinge on my conversation. Talk to people over 16 years of age. Oh, I forgot, you’re too busy taking their money.”
“Future communications devices are unlikely to remain tethered to QWERTY or any other similar relic,” commented Buddy Scalera, vice president for interactive content and market research for CommonHealth Qi. “The tools are likely to be icon-based, batched and routine-oriented. That is, it takes too long to type certain concepts, so taking a cue from programming language, we’ll have communication subroutines that we’ll drag and drop in highly streamlined conversations. Physical objects tagged with information will be part of an overall, organic language that’s able to be virtualized over long distances.”
