A few weeks ago, my partner and I made a bet. I said that there was no way ChatGPT could believably replicate my writing style for a smartwatch review. I already asked the bot to do that months ago, and the results were hilarious. My partner bet they could ask ChatGPT the exact same thing but get one many better results. My problem, they say, is that I don’t know the right questions to ask to get the answer I want.
To my chagrin, they were right. Written by ChatGPT many better reviews I said when asked by my spouse.
That memory came to mind while blogging at Google I/O. This year’s keynote is a two-hour thesis on AI, how it affects Search, and all the ways it can do it. COURAGEOUS and responsible make our lives better. A lot of it is neat. But I felt shivers down my spine when Google openly acknowledged that asking AI the right questions is difficult.
During its demo of Duet AI, a series of tools that live inside Gmail, Docs, and more, Google showed off a feature called Sidekick that can proactively give you prompt changes. or based on the Workspace document you are working on. In other words, it prompts you how to prompt IT by telling you what it can do.
That showed up again later in the keynote when Google demoed a new AI search result, called Search Generative Experience (SGE). SGE takes any query you type in the search bar and creates a mini report, or a “snapshot,” at the top of the page. Below the snapshot are follow-up questions.
As someone whose job it is to ask questions, both demos are troubling. The questions and prompts Google uses on stage are not the same as the questions I type in my search bar. My search queries always read like a baby talking. (They’re also usually followed by “Reddit” so I get answers from a non-SEO content mill.) Things like “Bald Dennis BlackBerry movie actor name.” When I’m looking for something I wrote about Peloton’s earnings in 2022, I pop on “Site:theverge.com Peloton McCarthy ship metaphors.” I rarely search for things like “What should I do in Paris for a week?” I didn’t even think to ask Google that.
I will admit that when looking at any kind of generative AI, I don’t know what to do. I can watch a zillion demos, and still, the blank window mocks me. It’s like I’m back in second grade and my stupid teacher just called me for a question that I don’t know the answer to. When I ask for something, the results I get are not good – things that take me more time to make presentable than if I were doing it myself.
On the other hand, my partner took to AI like a fish to water. After our bet, I watched them play around on ChatGPT for a solid hour. What surprised me the most was how different our prompts and questions were. Mine is short, open, and wide. My partner left AI very little room for interpretation. “You have to hold it,” they said. “You have to feed it everything you need.” Their commands and questions are hyper-specific, long, and often include links to references or data sets. But even they have to rephrase prompts and questions over and over to get what they’re looking for.
This is just ChatGPT. What Google pitching goes one step further. Duet AI is meant to extract contextual data from your emails and documents and intuit what you need (which is funny because I don’t know what I need half the time). SGE is designed to answer your questions — even those that don’t have a “right” answer — and then anticipate what you might ask next. For this more intuitive AI to work, programmers need to make it so that the AI knows what questions to ask users so that users, in turn, can ask it the right questions. This means that programmers must know what questions users want answered before they ask them. It’s a headache to think about.
Not to sound philosophical, but you could say that all of life is finding the right questions to ask. For me, the most uncomfortable thing about the AI era is that I don’t think any of us know what we are. Granted want from AI. Google says that’s what it showed on the I/O stage. OpenAI thinks it’s chatbots. Microsoft thinks it’s a really horny chatbot. But whenever I talk to the average person about AI these days, the question everyone wants answered is simple. How AI is changing and impacting Am I life?
The problem is that nobody, not even bots, has a good answer for that. And I don’t think we’ll get any satisfactory answers until everyone takes the time to rewire their brains to communicate with AI more fluently.
