Spatial Computing and the Potential of Innovation and Inclusion

In the early 1950s, pioneering computer scientist and U.S. Navy Officer Grace Hopper conjured up a different—and democratic—vision for computer programming languages. Until then, programming languages had remained in the hands of a few scientists who felt comfortable using esoteric machine codes. But Hopper saw that computing could serve the broader interests of society, including those of the civilian government and private industry, if its programming language could, ironically, be decoded and made familiar to non-specialists. Which is why, in 1952, Hopper introduced the world’s first compiler, a computer program that translated code written in English into machine code that could be understood by computers. 

Because of scientists like Hopper, who insisted that technology’s potential should be tethered to democratic ideals, computing was put on a trajectory that propelled it toward greater accessibility and usability for ordinary people. This path enabled generations to remake society from the ground up, designing everything from the first personal computers, to the Internet, to mobile computing.

One perennial hitch has beleaguered technologists, though. Despite this participatory evolution of the machine-human interface, access to the new economy that has cropped up around it hasn’t been evenly distributed across society. For instance, millions lack the skills or ability to fill the jobs created by technological disruptions, leaving many to conclude, unfortunately, that unemployment and disempowerment are the inevitable offspring of innovation.

But that’s not really true.

Today, there’s an emerging technology that, like Hopper’s compiler, promises to make computing radically more accessible—even obvious and intuitive—and it’s being applied right now to some of our nation’s biggest challenges. It’s called spatial computing, a term used by some to describe augmented reality (AR) technology, and if we enlist it in service of our national priorities, we can ramp up society’s inclusivity, while also empowering citizens. That’s exactly why public officials, nonprofit leaders, and aspiring social entrepreneurs ought to understand its current applications in order to start envisioning strong civic-use cases.

Imagine this: What if the information on your desktop computer or mobile phone wasn’t bound to the screen, but instead projected into the space in front of you—such as on your kitchen table, inside your child’s classroom, or into the produce aisle at the grocery store? That’s the basis of spatial computing. Its aim, basically, is to create a sort of Iron Man, just without the suit.

Its superpowers won’t be imaginary, however. Research by Boeing suggests that when job training material is integrated into a worker’s physical environment, rather than on paper or on a desktop computer, via spatial computing, they’re more likely to remember it faster and more accurately. Spatial computing could play an important role in the efforts to close our nation’s growing skills gap. In a few months, Apple plans to introduce this technology on its new smartphone. Hundreds of millions of people will be able to program AR applications into the real world around them.

But while that access is important, superpower technology alone isn’t enough. As an activist and immersive computing entrepreneur, I believe that access must be paired with a vision that matches tech’s greatest potential with the deepest needs of society. Put differently, how can we, in the words of Tim Wu, apply technology to “cater to our more complete selves rather than just our narrow interests”? 

Whether or not you’re convinced people will wear smartglasses in the future, spatial computing is already transforming the way we work, learn, and spend our leisure time. From advanced robotics, to self-driving cars, to the viral app Pokémon GO, elements of spatial computing are everywhere. And its strongest-use cases come from applications that promise real benefits for society, including innovations in productivity and job training methods, remote access to reliable health care, enhanced tools for the visually impaired, and new forms of civic engagement.

For example, GE recently equipped its technicians with AR smart glasses and found a 34-percent improvement in productivity on first use compared to their non-AR-enabled counterparts. That’s at least in part because their instructions—which are usually read from a manual or tablet—were seamlessly overlaid onto the technician’s glasses, offering line-of-sight guidance on the job. This video shows a side-by-side time-lapse comparison between the two approaches.

In health care, SugAR Poke, an iPhone app published by the the Eradicate Childhood Obesity Foundation, merges AR and object-recognition technology to show users how much added sugar is hiding in everyday food products. It’s a sterling example of how AR, in combination with other advanced technology, can be used to integrate health information and remote expertise into our physical world, nudging us to make healthier, more informed choices.

In a similar vein, for blind or visually impaired citizens, spatial computing is enhancing their ability to navigate society. Already, academics at Dartmouth University were awarded a research grant to develop a Microsoft HoloLens app that uses pattern and voice recognition technology to help those with low vision identify and read indoor signs so that they can better maneuver text-heavy environments.

Spatial computing is also reimagining community organizing by encouraging young people to better engage with their neighborhoods. This month, ENGAGE Boston, a youth-centered civic nonprofit, announced a partnership with the city of Boston, Emerson College, and Niantic, the developers behind Pokémon GO, to encourage middle-school and high-school students to explore their community’s cultural history and identify important locations to be added to the game as new PokéStops (the real-world locations that players must visit to play the game).

Because spatial computing affords us the opportunity to rethink entire processes in the physical world, the biggest hurdle to disrupting any one of these spaces is our imagination. But to be clear, this isn’t to say that spatial computing will be a panacea. In fact, many of the problems facing today’s technology and policy communities—inequality, privacy, security, and algorithmic bias, to name a few—are likely to become further complicated, at least in some ways, by spatial computing. Still, that’s why we must understand both technology’s promise and its limitations as we continue to think about social applications.

In Hopper’s old navy office, she hung a clock on her wall that ran backwards. This was to remind herself, as well as others, of what she believed was the most dangerous phrase in the English language: “We’ve always done it this way.” The clock exemplified a different but equally valid way of doing something, and it captured the spirit of innovation that moved Hopper and so many others of her generation.

As my generation wrestles with how to think about using spatial computing for good, Hopper’s clock offers us a possible direction: Think counterclockwise. It sounds a bit ironic, sure, but it could be just the type of freewheeling thinking that could move us forward.