AI Marketing as a Trojan Horse

JOSEPH TUROW
ROBERT LEWIS SHAYON PROFESSOR OF COMMUNICATION AT THE ANNENBERG SCHOOL FOR COMMUNICATION AT THE UNIVERSITY OF PENNSYLVANIA. TUROW’S CURRENT BOOK PROJECT CENTERS ON VOICE AND INTELLIGENT AGENTS IN 21ST-CENTURY MARKETING.

One can imagine amazing things when contemporary marketing meets sophisticated AI. While you are at work, your knowledgeable digital helper purchases plane tickets for your upcoming trip using criteria it developed from analyzing dozens of flights you’ve taken. As you travel home, the same virtual assistant orders a week’s worth of groceries with the help of another virtual assistant that conveys your family’s culinary interests. That evening it suggests movies and TV shows for various members of your family based on their past interests, current social situations, and real-time emotional states. These activities can’t happen for most of us in quite these ways right now, but artificial intelligence technologists will tell you they are right around the corner as marketers apply artificial intelligence for collecting, evaluating, and acting on data about us in unprecedented ways. Look a bit deeper into these developments, though, and you might see a less optimistic reality. Marketing technology driven by artificial intelligence can also be a Trojan Horse—a surreptitious channel through which marketers discriminate against many of the people who welcome it and by which governments create new paths to impose their will on citizens who just wanted better entertainment and shopping.

Gartner, a research consultancy, celebrated the rise of artificial intelligence in marketing as early as 2013. It predicted that within five years sellers would be able to identify and follow individuals through apps on mobile phones, home appliances, and wearable devices. Gartner identified four stages in the evolution of these activities: Sync Me, See Me, Know Me, and Be Me. Sync Me and See Me “are well underway,” it said, referring to marketers’ ability to connect with and follow customers as they use different devices [1]. In 2018, the exciting action involves using artificial intelligence for the last two stages. These activities increasingly take place in a form some marketers are calling “conversational commerce”: personalized computer-customer interactions where the computer is working with as close to a 360-degree knowledge of individuals as possible. On a day-to-day basis, the results may not appear all that successful. One of the most visible forms involve chatbots—the text versions when you try to communicate with a bank on Facebook and the voice incarnations that try to help you when you phone an airline. Some general chatbots, such as Mitsuku, delight their users. In the marketing domain, the results are often maddening. Go a bit off script and you will likely have to call for a human agent.

A step up from these sorts of frustrating relationships are virtual digital assistants. In the US, the best known are Apple’s Siri, Amazon’s Alexa, Google Assistant, Microsoft Cortana, and Samsung’s Bixby. The poster products for this future are the Amazon Echo and Google Home “smart speakers.” According to Edison Research and NPR, Americans are buying them at a faster rate than they adopted even smartphones and tablets. Their surveys in May and June 2018 found that 18 percent of Americans (around 43 million people above age 18) owned one or more of those voice-activated devices, mostly Amazon’s Alexa [2]. The gizmos come in different sizes and prices so they can be scattered around the house to recite the weather, adjust compatible lights and thermostats with voice command, play subscription music, read subscription audiobooks, buy products on command, and more. But they too fail the conversational intelligence test. When it comes to basic knowledge and phrasing, they are not nearly as smart as standard search engines. Instead of asking whether it will snow today (which it can tell you), ask Google when it will start snowing. You’ll get, “I’m sorry; I don’t understand,” or some variant. That happens a lot.

AI marketers are convinced these are just gestational difficulties and that hard work by many teams internationally will yield intelligent, talkative, intuitive companions. They emphasize a rosy future where you can get the products and services you want, when you want them, by reaching out to always-on helpers who know everything about you through deep dives into your personal history. The downside, of course, is that the helpers know everything about you. In that sense the AI marketing world is not the beginning of a new order but the next phase in a trajectory that decides the value of individuals based on fervid information processing. Computer-driven scoring and categorization engineer practical and prejudicial discriminations that separate those who don’t count from those who do, and decide the prices specific individuals receive compared to others.

AI will push this approach forward with important new wrinkles emerging from the sometimes-surprising conclusions about customers derived from heavy-duty, predictive analytics. Probably the most intense of these new developments will promote people’s use of biometrics—fingerprints, voice prints, retinal scans, implanted chips, facial recognition—during their shopping. The pitch will center around security (a firm in Michigan already encourages chip implants for this reason) [3], but in reality retailing consultants see the body as a data bonanza. One consultant highlighted the value of installing chips in the human arm for gauging shoppers’ emotions during a real-time shopping episode in order to evaluate when and whether to give discounts [4].

Conversational marketing with biometrics will be today’s discriminatory world on steroids. If the assistant gauges that a shopper’s voice is nervous in the aisle, it may offer that shopper a specific discount because doing so has worked in the past with that same level of anxiety. Another person might get offered less or more, depending on the emotion in their voice as tied to purchase history. All this will take place behind a veil. The customer will have little understanding of the profiles created through the use of deep data using real-time analytics. Marketers, though, will have increasingly asymmetrical knowledge and leverage in their relationship with customers.

AI marketing technologies would push us further toward a society in which people know they are treated differently but can’t quite figure out when or why.

AI marketing technologies would be troubling even if they just stayed in the advertising and selling domains. They would push us further toward a society in which people know they are treated differently but can’t quite figure out when or why. The public spheres that people use most—the online, app, and physical shopping arenas—will become places of personal and social tension as people realize they are getting different ads than others, seeing different products, and paying different prices based on databases they cannot see and analytical profiles they might not want.

But it gets worse. There is virtually no chance that the technology developed for marketing will stay in marketing. Sophisticated AI has the potential to be applied to populations for all purposes, including for governmental leverage and control. When it comes to data flows about populations, the boundaries between business and government are blurry, and the potential of artificial intelligence will make them blurrier. Edward Snowden‘s revelations about US snooping on Americans was clearly the tip of the iceberg. Moreover, the controversy that ensued in the West about government surveillance has parallels elsewhere. China is a poster government for using AI against its population. Pulling together data about individuals’ behaviors, including books they are reading, the regime has implemented soon-to-be mandatory participation in a Social Credit System that will rate the trustworthiness of its citizens to get good loans, rent cars without a deposit, rank high on a popular dating site, and other perks [5]. All but “at least a small group of the country’s ruling elite” have been entered into the system [6]. One China specialist dubbed it “Amazon’s consumer tracking with an Orwellian political twist” [7]. At the recent Las Vegas Consumer Electronic show the chief operating officer of Baidu, China’s Google, extolled the nation’s leadership for encouraging artificial intelligence in business [8]. It’s hard to see how the fruits of business will not also be the fruits of government surveillance.

At the end of this sort of contribution, it may be customary to call for greater public education about what is happening, along with more regulation to stop egregious invasions of privacy. But the rise of marketing AI raises global issues about the simultaneous rise of what might be called authoritarian surveillance capitalism. Tinkering with weak measures around the edges of education, business, and law in the West will not address the fundamental problems. Strident global insistence by citizens meddling in international forums might be a good start. While AI-enabled shopping can bring some joy, the future it portends should worry us all.

References

1. Stephanie Baghdassarian et al., “Predicts 2014: Apps, Personal Cloud and Data Analytics Will Drive New Customer Interactions,” Gartner Inc., November 22, 2013, https://www.gartner.com/doc/2628016/predicts--apps-personal-cloud.
2. “The Smart Audio Report,” Edison Research and NPR, July 18, 2018, https://www.nationalpublicmedia.com/wp-content/uploads/2018/07/Smart-Audio-Report-from-NPR-and-Edison-Research-Spring-2018_Downloadable-PDF.pdf, accessed August 16, 2018.
3. Mary Bowerman, “Wisconsin Company to Install Rice-Sized Microchips in Employees,” USA Today, July 24, 2017, https://www.usatoday.com/story/tech/nation-now/2017/07/24/wisconsin-company-install-rice-sized-microchips-employees/503867001/.
4. Joseph Turow, The Aisles Have Eyes (New Haven, CT: Yale University Press, 2017), 2.
5. Rachel Botsman, “Big Data Meets Big Brother as China Moves to Rate Its Citizens,” Wired, October 21, 2017, https://www.wired.co.uk/article/chinese-government-social-credit-score-privacy-invasion.
6. Xin Dai, “Toward a Reputation State,” paper presented at the Privacy Law Scholars Conference at George Washington University, Washington, D.C., May 30–31, 49, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3193577.
7. Johan Lagerkvist of the Swedish Institute of International Affairs, quoted in Rachel Botsman, “Big Data Meets Big Brother as China Moves to Rate Its Citizens,” Wired, October 21, 2017, https://www.wired.co.uk/article/chinese-government-social-credit-score-privacy-invasion.
8. Blair Hanley Frank, “Baidu COO Says China’s Government Will Help the Country Dominate AI,” Venture Beat, January 9, 2018, https://venturebeat.com/2018/01/09/baidu-coo-says-chinas-government-will-help-the-country-dominate-ai/.