The Automation of a Bad Question
Article by Ayman Fouad Abdelgawad
Imagine a team struggling to answer a question that was poorly chosen. They have been asked to rank applicants by a convenient measure that captures only a small part of what the work requires.
A new system makes the ranking instantaneous.
The improvement is real in one sense. Time has been saved, and the same calculation can be applied consistently. Yet the original difficulty has not been addressed. The team can now produce an inadequate answer with greater confidence and at greater speed.
Automation encourages attention to the process because the process is what can be built. The underlying question may receive less scrutiny precisely when its consequences are about to become easier to repeat.
Before automating a decision, an organisation should be able to explain what the chosen measure represents, what it leaves out, and what would count as evidence that it is misleading. Those questions matter whether the system is a simple formula or something more complicated.
Human judgement is not a guaranteed cure. People also use poor proxies, inherit assumptions, and mistake confidence for understanding. The comparison should be between accountable methods of judgement, rather than an idealised human and an imperfect machine.
The central design question is where reconsideration remains possible. Can somebody challenge the premise, or are they allowed only to dispute the input? A perfectly corrected number can still answer the wrong question.
The fastest route through a process is useful only after we have examined where the process is going. Sometimes the most valuable interruption is a person asking why this was the question in the first place.