Many years ago I wrote a post about a scene in Star Trek II: The Wrath of Khan in which Captain Kirk is confronted with a training simulator designed to test his ability to survive a battle against the Klingons.
He was the only person in the history of Starfleet Academy to beat it. As the film progresses, we discover how. Kirk had not simply outmanoeuvred the simulator. He had reprogrammed it.
Imagine you are Captain Kirk, flying the USS Enterprise in the simulator. The Klingon fleet divides into two groups. One moves around the Enterprise from the left, while the other circles around from the right. Their intention is to attack simultaneously from both sides, leaving you nowhere to turn. Yikes!
Kirk realised that if he could change the rules of the simulator, he could change the battle.
But what happens when the simulator is powered by AI?
A traditional simulator is controlled by a program: a set of rules written in advance. An AI system works differently. It can be trained on thousands of battles and learn patterns from them. When a new battle begins, it uses what it has learned to decide what to do next. There is therefore no simple set of battle rules for Kirk to rewrite.
So how could Captain Kirk beat an AI simulator?
Perhaps he would do what Kirk has always done: look at the problem differently. Instead of changing the rules, he could change the data from which the AI learns. A small conventional program could manipulate the training data, subtly altering the patterns that the AI discovers. The simulator would still be following its instructions. It would simply have learned the wrong lesson.
This points to an interesting shift. With traditional programming, influence comes from changing the rules. With AI, influence can come from changing the data from which the system learns.
A program designed to manipulate the information presented to an AI could cause its decisions to become systematically biased towards a particular conclusion. This is a real area of research in AI security and robustness.
Computer hackers like giving things names. So perhaps we should give this one a name of our own: Silent Whisper. Silent, because the influence may go unnoticed. Whisper, because it does not need to shout or take control. It simply nudges the system towards learning something different.
In Star Trek II, Kirk receives a commendation for “original thinking” after finding a way to beat the simulator. He goes on to become a Starfleet Admiral.
But suppose Kirk faced the simulator today. He could no longer simply reprogram its battle rules, because an AI simulator learns its behaviour from the battles it has been shown. To beat it, he would have to find a new way of interfering with what it learns.
And perhaps that is the real challenge for the age of AI.
The old Kirk changed the rules. The new Kirk would change what the AI learned from.
So, Captain Kirk, the question is no longer whether you can outwit the simulator.
Can you outwit the way it learns?




