Is Your AI Just a Chatbot? The COACH Test

I've spent thirty years in the technology that became generative AI. Not adjacent to it — in it. Enterprise search, semantic enrichment, taxonomy and classification, the whole apparatus that tried for two decades to make machines find the right document, then the right fact, then the right answer. So when people describe today's models as bewildering, I don't share the feeling. What I feel instead is closer to vertigo: not confusion about the technology, but astonishment at the speed with which it's turning into plumbing.

That's a specific kind of surprise, and it's not the one most people are reporting.

The talking dog decade

For almost twenty years we've had conversational interfaces sitting on our phones and countertops. Siri arrived in 2011. Alexa followed in 2014. Both were sold as a glimpse of the future, and both spent a decade proving something closer to the opposite: that a device which could parse a sentence and still couldn't act on it was a parlor trick, not a revolution. Ask it to set a timer, fine. Ask it anything with a second clause, and you got the conversational equivalent of a dog walking on its hind legs — the astonishment was that it happened at all, not that it happened well.

The reason wasn't a lack of ambition. It was architecture. Siri and Alexa were intent classifiers wired to a fixed menu of scripted actions. They didn't understand your request; they matched it against a list of things they already knew how to do, and anything off that list got the same shrug in a friendlier voice. That's not a smaller version of understanding. It's a different mechanism entirely, one that was never going to grow into the thing it was pretending to be, no matter how many launch events promised otherwise.

Jeff Bezos's own choice of wake word is the tell. In 2017 Amazon added "Computer" as an alternate to "Alexa" and "Echo" — a nod from a famously devoted Star Trek fan, the same Bezos who later put William Shatner on an actual rocket, sending his own boyhood Captain Kirk into real suborbital space. The wake word points at a specific scene: Spock ordering a possessed, dangerous computer to compute pi to the last digit, trapping it in a loop it can never finish, a trick that only worked because 1967 imagined a machine that could do exactly one thing at a time. Naming your product after that computer and then shipping something that can set a timer is a confession dressed as a tribute. My own Echoes are still set to "Computer," not "Alexa" — which is either nostalgia or an admission that the name was always a promise nobody had cashed yet.

Which is why an entire generation of engineers grew up assuming the ceiling was real. HAL 9000 could hold a genuine conversation about the mission, argue for his own survival, and lie convincingly while doing it. The Enterprise's computer — voiced by Majel Barrett across three decades of the franchise, the same actress who played Nurse Chapel — could take an ambiguous spoken request and route it correctly through a starship's worth of subsystems. Both were fiction, and both stayed fiction for the entirety of a working career spent inside the actual field. Not because nobody wanted it. Because nothing behind the curtain could do it.

What actually changed

The models running today are a different mechanism than a faster Siri: something that can hold context across a conversation, reason about an ambiguous request instead of matching it to a script, and — critically — reach out and operate other systems instead of just describing them. That last piece is the one the talking-dog decade never had. A voice assistant that can tell you the weather is a lookup. A system that can read your calendar, negotiate a scheduling conflict, and rebook three downstream meetings without being walked through each step is something else, and the gap between those two isn't a matter of degree.

Star Trek's LCARS interface got dismissed for years as production design — mauve panels and confident beeping, nothing more. Looked at as an architecture instead of an aesthetic, it was always a claim about where control should live: the crew stated intent, and the computer handled implementation. That claim wasn't viable with 2011-era assistants, because there was no genuine reasoning underneath the panel to route the intent correctly. It's viable now. The bottleneck wasn't imagination. It was capability, and the capability arrived faster than the industry's expectations did — which is exactly why so many products still bolt a chat window onto software built for a mouse, the same way early automobiles kept a whip socket on the dashboard.

COACH, not chatbot

I use a name for the paradigm that's actually available now, because "conversational AI" has been drained of meaning by a decade of chatbots that weren't: COACH — Conversational Operations and Control Hub. The distinction the name is doing work to preserve is the one the talking-dog decade blurred past: a COACH doesn't explain the console. It operates the console, on your stated intent, and shows you what it did.

That's a design discipline, not a slogan, and it implies a concrete test an engineer can run against anything being built or bought right now. Ask what happens when a user states an outcome instead of a step. A chatbot answers a question about the outcome. A COACH produces the outcome, or tells you precisely why it can't. If the real answer is "it walks you through the menu where you can do that yourself," what's been built is a very well-spoken tour guide, not the thing the name promises. That test costs nothing to run and it is uncomfortable on most of what currently ships under the AI label — which is the point of having it.

Engineers spent a professional lifetime being told the ceiling on this was fixed, high, and fictional. It wasn't fixed. Treating it as though it still is — designing the next system around a chat window bolted onto the old menu, one more polite way to ask a machine to do what it was always going to do anyway — is the one mistake this moment doesn't forgive twice.