Faulty algorithms
Apps and social media platforms can purposely or accidentally help shape societies, but such concerns are of a very different nature to mistakes made by programmers and coders now shaping the algorithmic economy.
"When it comes to systems that monitor vital signs, like automated diabetes pumps or drug injection systems, then a flaw could prove disastrous," says Curran, who notes that AI is used in aviation autopilot systems, in Segways, and even during a Google search, natural language and speech processing, and translation.
Mistakes could be costly – a wrongly transcribed word could lead to an accidental libelling of someone, while a mistranslated word could cause a huge legal misunderstanding.
Lesser of two evils
However, we're not just talking about mistakes. "If we think about it, amoral computer programmers are trying to solve moral dilemmas with algorithms," says Curran, but he's quick to jump to the defence of technological progress. "We also have to believe that sophisticated AI is just as likely to be nurturing and ultimately beneficial to mankind in so many ways."
There are millions of road deaths every year and huge problems with unemployment and poverty. If algorithms can alleviate those ills – or try to – shouldn't they be forgiven in advance for the odd mistake, or unintended consequences?
"Pervasive software, algorithms and big data are a key feature of the Fourth Industrial Revolution – the deep digital transformation affecting our world that is just starting," says Kemp. "So you can add 'lesser of two evil' moral choice issues around smart cities, IoT, robotics and even 'designer beings' to driverless car dilemmas."
Where do algorithms go to die?
An algorithm found not to be fit for purpose, blamed for an accident, or considered politically irresponsible can, of course, be made to disappear. "An AI algorithm could be banned or deleted," says Curran. "If a system flaw in an algorithm was brought to light, then the organisation deploying that algorithm should seek to rectify the situation … in reality, the system would be tweaked."
After all, isn't that how the tech world works? Invent something, code it, beta test, sell it to the public, then iron out the bugs in the software. Even the very biggest, long-established tech companies like Apple approach their global businesses in this bizarrely risky way.
"Outside of aviation and medical products, technology experts tend not to follow stringent testing methodologies, but lazily rely on fixing problems as they arise," says Curran, "but a mis-configured service in a fast-moving truck could lead to death."
Blame the programmers
That said, tech companies will have to take responsibility for what they create. After all, in the programmable economy of the near-future, there's an obvious group to blame: the programmers of the algorithms on whose decisions lives will rest.
"The motivation to build rigorous and secure systems should be there because it is quite possible that all involved in its design could be held liable if a defect caused or contributed to a collision," says Curran, who thinks that there needs to be guards put in place to protect the public from algorithms deployed in software that could lead to catastrophic events.
He adds: "If computer programmers eventually play a bigger role in the way driverless cars move than drivers do, it is likely manufacturers will build the cost of litigation and insurance into their trucks."
In the algorithm-driven automated world now being created by programmers there's got to be a massive cultural shift towards more rigorous testing, and more responsibility taken by the tech industry. "There's always someone behind the artificial intelligence, algorithm or program," says Kemp.
Algorithms represent the coders' value – and usually it's only about maximising profits. With autonomous vehicles now chosen as the tech industry's next big project, the media is only going to get more obsessed with driverless cars having accidents – and that means that the algorithms behind the wheel are going to come under intense scrutiny.
