Last week a friend sent me a video of him on a bus ride through central Shanghai where the driver was sitting behind the wheel, but the wheel was moving autonomously. It really got me thinking about how many perverse incentives we bake into technological progress and the impact they may have down the line.
In the Chinese bus example, the ‘driver’ is employed as a failsafe, which means his measurable value is only felt when something goes wrong. The rest of the time he is simply waiting for the day when the number of incidents is low enough that his role becomes redundant. Is it therefore in his interest to subtly ensure as many non catastrophic mishaps as possible happen during each shift?
The situation reminded me of a colleague I had in a previous job who had concluded that deliberately breaking and then fixing things was the fasted route to rapid promotion. He had observed in his first year at the company that the culture was one that optimised for maximum chaos – the move fast and break things approach to product development.
With a new baby on the way and wanting to get promoted as fast as possible, he began looking for bits of infrastructure he could break that would cause enough outages and incidents that all eyes would be on him to fix the problem. And the more problems he fixed, the more highly regarded he became, until eventually he’d gone from Mid Level to Senior Principal Engineer in record time.
This month, You Can See Everything was released, delving into the exploits of the now infamous fraudster Elizabeth Holmes. The story has continued to fascinate years beyond her sentencing, yet the ‘fake it ’til you make it’ principles she doggedly employed are right out of the Silicon Valley startup playbook.
If the environment you launch your business into is one where stories, false claims and abstract promises are the norm in securing increasingly absurd amounts of money until the whole market periodically comes crashing down, white collar prisons should be very crowded places indeed.
In each of these cases I’m reminded that we get what we optimise for and sometimes we have to think very carefully about the perverse incentives, and apocalyptic paperclips, we create in the process.
