Self-driving cars have captured the imaginations of businesses and consumers alike but fully autonomous vehicles are still a ways off. This is because there are a number of safety, security and logistical issues that still need be solved before human drivers can begin to share the road with self-driving vehicles.
To better understand the challenges that need to be overcome before level 5 autonomy can finally be reached, TechRadar Pro spoke with the director of automotive solutions and platforms at Arm, Robert Day.
- Mapping the world: solving one of the biggest challenges for autonomous cars
- When will self-driving cars earn our trust?
- Securing self-driving cars on the road to level 5 autonomy
What are some of the biggest advancements in autonomous vehicle technology in the last few years?
The auto industry has made significant headway with developing and trialing autonomous prototypes, particularly in the U.S., and is starting to hone the sensor arrays, software stacks and algorithms, and understanding of the level of computing required to process the information for the vehicle to make the right decisions in every scenario. For example, in California, the autonomous vehicle companies must provide a report every year showing the number of vehicles, number of miles driven and the number of disengagements of the autonomous system as it hands back control to the human safety driver.
We are seeing many companies from ride-sharing pioneers, technology giants, Silicon Valley start-ups, and the traditional automotive OEMs and tier 1s, all putting their technology on our roads, fuelled by big investments and big expectations.
However, these are still prototypes and are some way from real deployability as the computing systems are too big, too power hungry, too costly, and thermally inappropriate. The other challenge is that the software stack is still struggling to deal with all corner cases, and making a fully autonomous car functionally safe through the all sensing, perception, decision, and driving phases is very challenging which requires there to be a safety driver in attendance at all times (which is not very deployable).
Over the last few years, we’ve started to see an interesting shift in the industry where OEMs and autonomous technology providers have broken the chain of command by now partnering with each other and discussing these issues directly with silicon tech companies, like Arm, to find out exactly what it is they need to implement in their designs and get from prototype to deployment. Recently announced, the Autonomous Vehicle Computing Consortium (AVCC) is an example of industry collaboration focused on deployable autonomous compute.
What challenges do automotive manufacturers need to overcome before AVs can become the kind of product that consumers can go into a dealership and buy?
We are seeing many hurdles in the industry which we need to collaboratively overcome in order to move from prototype to mass market models. The most obvious being cost. A recent report estimated that in 2020 a Level 4 or Level 5 car would cost an extra $75,000 to $100,000 more than the price of an average premium car, and we believe that estimate is conservative just based on the number of sensors and compute needed to achieve full autonomy. This means that the only business ventures like ride-hailing services will be able to justify the high cost by the appropriate return on a driverless 23/7 robotaxi.
Safety is also a big challenge as AV practitioners are often concerned that the software available is not behaving acceptably in universal situations. When we increase autonomy in a car, we are essentially replacing the safe decision making of a human driver with a complex computer system, comprising many diverse compute elements. The right software is incredibly important for the car to react perfectly in any environment, whilst it also needs to ensure the vehicle systems are safe from cyberattacks.
Even if automotive firms get the technology right, today’s industry certification standards and governmental regulatory frameworks don’t address AVs. Automotive industry executives must accelerate the development of these new standards to ensure that once AVs meet the right criteria.
The initial deployments of autonomous vehicles will have a very specific operational design domain (ODD), where the vehicle will only operate in certain geo-fenced areas, in certain weather conditions, and at certain speeds. This is not appropriate for a consumer-based car, as we don’t want to be limited in where we travel and use our car, and we are already seeing how humans abuse some of the new Advanced Driver Assistance Systems (ADAS) features that are now available in certain premium vehicles, leading to potential safety concerns.
As part of leaving the comfort of a specific ODD, extra information will be required to augment the traditional sensors, such as high precision GNSS signals with centimetre level accuracy and HD maps with highly accurate environmental information, so when there are poor visibility conditions for the AV’s cameras, such as snow or no road markings, the car will still be able to drive in autonomous mode.
How has the sensor technology used in AVs improved and what roadblocks still need to be overcome?
The move from ADAS to autonomous demands a much greater awareness of everything around the car. In order to accomplish this, the number of sensors on the car are dramatically increasing, with multiple LiDAR, camera, ultrasonic, and radar sensors required to essentially replace and enhance human sight and situational awareness. Other sensor inputs such as GNSS and thermal cameras could also help with the positioning of the vehicle and the recognition of objects around the vehicle.
Currently, sensors are showing one error every 5 or 6 minutes. Next generation sensors will likely show an error approximately every 1,000 hours, however, to meet Level 3 safety requirements, it is essential that sensors show one error no more than every 11,000 hours.
The fusion and perception processing of the sensor inputs are still a roadblock, as even if we get the sensors perfect (at the right price point) the industry is a long way off from getting this right for every corner case because the computing horsepower and algorithms don’t exist yet. It will take over a decade before a computer can make all of the judgments in a guaranteed safer way than a human in boundless environmental conditions.
How is the automotive industry working to tackle consumer fears surrounding the idea of self-driving cars?
I saw recently that recent statistics show 73% of American drivers are too afraid to ride in fully self-driving vehicles.
In order to win over the trust of consumers, the industry needs to prove to them that the technology in the vehicle is statistically safer than human drivers. For example, AVs are able to have a full 360 view, which humans cannot achieve alone.
Essentially, the compute needs to be able to make better decisions than basic human instinct. Consumers trust in automated systems to fly our airplanes, as the public are attracted to the vision of a relaxing journey in which they can spend their time usefully. For decades, airliners have been using autopilot and auto-throttle, so why can’t the same be done with cars?
In my opinion, a large portion of this can be addressed by starting to alter the conversations in the media around the benefits AVs will bring - there are positive stories out there right now. ADAS systems today are already saving thousands of lives by the use of lane departure warnings, adaptive cruise control, and automatic emergency breaking, progress we shouldn’t be overlooking.
Companies are already trying to alter people’s perception of self-driving cars, including educating children and offering free robotaxi rides to consumers. Ultimately, the best way to alleviate our fears is for us to live in a world that is shared with autonomous vehicles, and to understand first-hand both the safety and convenience benefits of the self-driving car.
Technology aside, government regulation and insurance are also practical matters that will also address common questions from consumers. A lot of legislation needs to be implemented to prove that autonomy can become a reality in people’s everyday lives.
What kind of security features are being added to cars today as well as to AVs to prevent them from falling victim to cyberattacks and malware?
For an autonomous vehicle, security cannot be an afterthought, as a compromised AV becomes a serious safety hazard for those in and around the vehicle. Security of an AV also needs to be a defense in depth strategy, as the attack surface of an autonomous car is huge; multiple connectivity ingress points, many sensors, hundreds of millions of lines of software, many CPUs and ECUs are all potential attack vectors for malicious agents.
At Arm, we are focusing on protecting the many Arm-based modules that are helping power the AV, offering technologies like TrustZone and cryptography along with our Platform Security Architecture (PSA) framework that is aimed at securing connected devices. These technologies provide a root of trust to enable security functions such as secure boot, key storage, cryptography, attestation, and auditing functions.
As computing systems become more complex with multiple functions being put into domain controllers, there needs to be additional protection of the software that runs on these systems. Arm provides isolation technologies with its built-in hardware virtualization and system MMUs that, when used with a hypervisor, can separate software and I/O devices, and can isolate the critical components from the key attack vectors such as the external connectivity.
Lastly, as part of the defence in depth, there are a large number of companies, many whom are Arm partners, that are focusing on bringing specific security functionality to the car and module manufacturers. For example, technologies are available to encrypt CANbus communication to protect critical driving functions from attack, and others to make sure that any over the air (OTA) updates are secured, an example of the latter being the open source Uptane project that is being utilized in connected car systems.
Arm is also bringing its Platform Security Architecture to work with many companies that can achieve PSA certification and working with standards bodies such as the National Institute of Standards and Technology (NIST) and Society of Automobile Engineerings (SAE) to inform them of the threat models and security analysis for autonomous use cases.
How is AI-capable compute technology being implemented in AVs and how will it help improve their safety and security?
The perception, prediction, and planning functions in an autonomous vehicle require a vast amount of computing horsepower that is best served by a heterogeneous complex of traditional CPUs for general purpose compute plus domain-specific processors, such as machine learning (ML) neural processing units (NPUs), for numerically intensive workloads. As the level of autonomy rises, the more information is required, resulting in more sensor information that needs to be processed in a fast, efficient, safe and secure manner.
Current AV prototypes are using datacentre-like processing to get the performance needed, but as companies look to a deployment strategy it is becoming clear that right-size heterogeneous compute is what is needed in order to tackle the power-dissipation challenges of the compute levels required for AVs. A fundamental part of this is getting the right AI technologies to be able to process the autonomous workloads in an efficient way, applying inference processing in the vehicle to process real-time sensor information using pre-built models for correct object and situational recognition. The correctness of this processing in a real-time situation will dramatically effect both the safety and security of the AV, as it will be able to not just understand the situation, and make a safe decision, but it can also watch out for anomalies caused by potential security attacks.
This AI processing has to be scalable with functional safety characteristics that allow the autonomous drive computer to process the sensor information, make the correct decisions, and then safely plot and implement a course of action. Security also needs to be built into this compute complex, to ensure that both the hardware and software are not compromised by a malicious attack, and that sensor information is not spoofed. This applies to all the compute and software elements including CPUs, OSes, NPUs, and applications through to AI and associated models.
At Arm we are focusing on bringing right-size heterogeneous compute to AVs using high-performance Cortex-A CPUs with built-in functional safety features, and our powerful, efficient NPUs for AI processing, coupled with our security modules to protect the compute complex.
How far off are we from fully autonomous vehicles being commonplace on roads and how are governments helping to make this a reality?
We are still a long way from achieving fully autonomous vehicles and realistically, we can expect that the industry will take approximately another 5-10 years before first actual fully autonomous deployments, in which time automakers will need to adopt a number of changes before we get there.
It is likely that high prices of AVs will mean that the first real deployments will be part of Mobility-as-a-Service (MaaS), including e-hailing, ridesharing or robotaxi fleets. By replacing the cost of a human driver, and by having higher utilisation of the vehicles than consumers, companies could build a business model that can afford the high cost of the initial autonomous vehicles, and potentially get them on the roads within 5 years.
For autonomous vehicles to be commonplace, and not just in initial trial cities, the technology will need to reach Level 5 capabilities, where the autonomous drive system will need to be able to operate in any conditions at any speeds and in any areas. This will require much more sensor technology, and more advanced software and computing capabilities.
Two other factors will also weigh in here: public opinion and government regulations. Initial deployments of autonomous robotaxis will be in autonomous-friendly cities, where that friendliness can mean clement weather, easy traffic conditions, straight roads, and local government that embraces this technology. It could also mean technologically friendly smart cities where roadside infrastructure will help augment the on-vehicle sensors, implementing V2I and making for a safer introduction of autonomy.
For autonomous vehicles to become commonplace and venture out from the confines of their “safe” cities, government regulations and guidelines will need to be put in place, which will also go a long way to easing public opinion. This is where we will start to see which countries/regions are most friendly to autonomy and where it has the potential to become most widespread.
From a technology perspective, there are many tasks that will take a significant amount of time. For example, a Boeing 787 Dreamliner aircraft needs companies to create approximately 14 million lines of code. In comparison, Level 4 AVs are going to need hundreds of millions of lines of code, and Level 5 over one billion. That’s the scale of the challenge that lies ahead.
Once the AVs have even been manufactured, auto makers also need time to certify them. Traditionally, OEMs need to complete 3 million miles of validation before a Level 2 vehicle can be certified. However, Level 4 AVs are going to need around 150 million miles of validation for on the road and simulation certification.
Clearly, no one company can do this alone. In order to speed up this process, collaboration is key, and this is why myself and the team at Arm work directly with OEMs, tier ones and software vendors to show them what is available to deploy in 5 years’ time.
I am positive about the progress the industry is making. Mobility in any part of the phase, whether fully electric or autonomous, has the ability to impact positive change towards the biggest societal issues the world is currently trying to tackle, such as urbanisation, digitisation and sustainability. The quicker the auto industry focuses on working together, the quicker we will start to see a fully autonomous world.
