Everyone wants AI. Few ask what for.
AI is brilliant at some tasks and unreliable at others that look just as easy. A plan names the task.
Written by NickStrategy & Adoption Lead, Zipline
A sentence I hear in almost every first conversation is some version of "we need to do something with AI." It is an understandable thing to say. Competitors are announcing pilots, the board is asking questions, and the tools get more impressive every month. It is also a sentence without an object. Nobody ever says they need to do something with spreadsheets.
My answer is always the same question: which task? Not which department and not which job. Which task, done by whom, how often, and what happens when it goes wrong.
Jobs are bundles of tasks
The idea comes from labour economics. In 2003 David Autor, Frank Levy and Richard Murnane argued that to understand what computers do to work, you should stop looking at jobs and look at tasks. A job is a bundle of them, and technology takes over some and leaves others. Twenty years of research have held that up, and it is still the first thing I check in any AI plan.
It matters even more for AI than it did for earlier software, because of what researchers call the jagged frontier. The term comes from a 2023 field experiment with 758 consultants at Boston Consulting Group, led by Fabrizio Dell'Acqua. AI turned out to be excellent at some tasks and poor at others that looked no harder. On the first kind, consultants with AI produced more work and better work. On the second kind, they did worse than colleagues who had no AI at all. A confident, plausible answer is hard to argue with.
So "AI for customer service" is not a plan. Customer service is a bundle: answering a question about a delivery date, calming an angry customer, spotting that three complaints this week share one cause. The first sits well inside the frontier. The second is a judgement call with a relationship attached. The third depends on context that may not be written down anywhere. A plan names the task.
When AI costs more than the person
There is a second reason to be specific, and it is money. KPMG's Global AI Pulse reported this year that 49 percent of companies had delayed or scaled back AI agents once the costs began to outweigh the value.
The economists Daron Acemoglu and Pascual Restrepo have a name for what those companies bought: so-so technologies. Automation that is just good enough to replace a person, and not good enough to make the company noticeably more productive. The self-checkout is the standard example. The work did not vanish. It moved to the customer, and someone still has to stand nearby for when it goes wrong.
Erik Brynjolfsson describes the thinking behind it in his essay "The Turing Trap". If you aim AI at imitating what a person already does, the best outcome is the same output with a smaller wage bill. If you aim it at what your people could not do before, you get something new. The second target is harder to see, because it is not sitting in an existing job description.
The human in the loop needs a real job
The usual safeguard is to keep a person in the loop. It is the right instinct, and it is often badly built. In 1983 Lisanne Bainbridge published a short paper called "Ironies of Automation" that I would make required reading for every AI project. Her point: when you automate the routine part of a job, you leave the human with the hardest cases, plus the task of monitoring a system that is usually right. People are bad at that.
Dell'Acqua later measured it. In an experiment with professional recruiters, those given a very good AI became less attentive and ended up making worse decisions than those given a weaker one. He called it falling asleep at the wheel.
This is the part most project plans miss. Improving the AI makes the human check weaker, unless you redesign the check at the same time.
The task test
Before we point AI at anything, we run the task through four questions. I call it the task test, and it fits on an index card.
- How often does it happen? Rare tasks seldom repay the effort of automating them well.
- Can someone tell quickly whether the result is right? A draft reply can be judged in seconds. A subtly wrong customs classification cannot.
- What does a mistake cost? The answer decides how much human judgement stays in.
- Would something simpler do? A rule, a template or a better form is cheaper to run and easier to trust.
The tasks that pass are usually unglamorous: quotes that go out the same day, orders that nobody has to retype, books that close in two days. That is the point. None of those is an AI goal. They are business goals, and AI happens to be a good tool for them.
Sources and further reading.
The research referred to in this note, for anyone who wants to read it in full.
- David Autor, Frank Levy and Richard Murnane, "The Skill Content of Recent Technological Change", Quarterly Journal of Economics (2003). The task view of work starts here.
- Fabrizio Dell'Acqua and co-authors, "Navigating the Jagged Technological Frontier" (2023). The source of the term jagged frontier.
- KPMG, Global AI Pulse (2026).
- Daron Acemoglu and Pascual Restrepo, "Automation and New Tasks", Journal of Economic Perspectives (2019). The source of the term so-so technologies.
- Erik Brynjolfsson, "The Turing Trap: The Promise and Peril of Human-Like Artificial Intelligence", Daedalus (2022).
- Lisanne Bainbridge, "Ironies of Automation", Automatica (1983).
- Fabrizio Dell'Acqua, "Falling Asleep at the Wheel: Human/AI Collaboration in a Field Experiment on HR Recruiters" (2022).
More field notes
Co-existence is a team sport.
Most advice about working with AI is written for one person at a keyboard. A company is not one person.
4 min readAI changes fast. People don't.
Every company has an absorption rate: the speed at which its people can take in a new way of working. Trust sets it, not technology.
4 min readWhen building is nearly free, the wrong thing gets expensive.
Cheap software sends three bills that never appear in the quote. Your team pays all of them.
4 min read