Logo Assembly Logo Hemicycle

Legal aspects of "autonomous" vehicles

Report | Doc. 15143 | 22 September 2020

Committee
Committee on Legal Affairs and Human Rights
Rapporteur :
Mr Ziya ALTUNYALDIZ, Turkey, NR
Origin
Reference to committee: Doc. 14839, Reference 4442 of 12 April 2019. 2020 - October Standing Committee

Summary

The circulation of semi-autonomous vehicles on European roads is likely to increase significantly in the coming years, with some believing it possible that completely autonomous vehicles may become available within the next decade. These developments pose questions in relation to criminal and civil liability, the obligations of manufacturers and insurers and the future regulation of road transportation. Important ethical and privacy concerns also arise.

Modern automated driving systems are distinguished by their reliance on artificial intelligence systems. The expectation is that automated vehicles will be significantly safer that those driven by humans. Appropriate regulation will be needed to realise this potential. As a starting point, this regulation must ensure full respect for the right to life.

The Committee on Legal Affairs and Human Rights therefore proposes that the criminal law, civil law and human rights implications of the development and introduction of autonomous vehicles are regulated in accordance with Council of Europe standards on human rights and the rule of law. It also proposes that the Committee of Ministers take into account the particularly serious potential impact on human rights of the use of artificial intelligence in automated driving systems when assessing the necessity and feasibility of a legal framework for artificial intelligence.

A Draft resolutionNote

1 The circulation of semi-autonomous vehicles on European roads is likely to increase significantly in the coming years, with some believing it possible that completely autonomous vehicles may become available within the next decade. These developments pose questions in relation to criminal and civil liability, the obligations of manufacturers and insurers and the future regulation of road transportation. Important ethical and privacy concerns also arise.
2 In the case of a semi-autonomous vehicle operating under the proper control of an automated driving system (ADS), or of a fully autonomous vehicle, criminal law is not designed to deal with the conduct of non-human actors. This may create a ‘responsibility gap’, where the human in the vehicle – the ‘user-in-charge’, even if not actually engaged in driving – cannot be held liable for criminal acts and the vehicle itself was operating according to the manufacturer’s design and applicable regulations. This may require new approaches to apportioning criminal liability, or alternatives to criminal liability in cases where no human can reasonably be held responsible.
3 Similar concerns apply to civil liability for damages incurred by a vehicle operating under the proper control of an ADS. Current fault-based liability regimes may leave the user-in-charge absolved of any liability, with responsibility shifted to the ADS. This may require new approaches, such as strict liability, to ensure that injured parties receive compensation for the damage they suffer.
4 In cases where road traffic regulations are violated by a vehicle under the proper control of an ADS, whether establishing the facts of a criminal offence or causing damage to third parties, the responsibility of the manufacturer may raise issues of product liability. The complexity of autonomous vehicles, however, may make it difficult to prove the existence and nature of any technical fault. Again, it is important that future regulations do not leave lacunae in this respect.
5 These concerns are closely related to ethical issues that arise in relation to autonomous vehicle technology. Human drivers are regularly required to make ethical decisions, including forced-choice decisions of life and death. ADS will have to make the same decisions, but according to an ethical framework that was defined by their manufacturer. Given that the purchasers of autonomous vehicles may prefer that priority is given to their own safety, competitive market pressures on manufacturers may not generate outcomes that are optimal from a general utilitarian point of view. There may certainly be a need for government regulation to standardise the ethical choices implicit in ADS design, to ensure compatibility with the general public interest.
6 ADS are data-dependent and data-generating, including sensitive personal data relating, for example, to an individual’s movements. The data from autonomous vehicles is automatically shared with other autonomous vehicles and with a central system and may need to be shared with regulatory and law enforcement bodies in certain circumstances. Particular care will be needed to ensure a correct balance between data processing that is necessary for the safe operation of autonomous vehicles and respect for and protection of the privacy of drivers, passengers and other users.
7 Modern ADS are distinguished by their reliance on artificial intelligence (AI) systems; indeed, modern autonomous vehicles are, in effect, robots. The introduction of autonomous vehicles means putting AI-controlled robots in charge of fast-moving projectiles in a situation of proven, serious potential risk to their passengers and other road users. The expectation is that autonomous vehicles will have the potential to be significantly safer that those driven by humans. Appropriate regulation will be needed to realise this potential. As a starting point, this regulation must ensure full respect for the right to life; including positive obligations to prevent foreseeable and avoidable threats.
8 The Parliamentary Assembly considers that ethical and regulatory standards applicable to AI in general should also be applied to its use in autonomous vehicles. It therefore considers that the work of the Ad hoc Committee on artificial intelligence (CAHAI) on a possible legal framework for AI will be especially significant, and notes the important contributions to work in this area of other international organisations including the Organisation for Economic Co-operation and Development, the European Union and United Nations bodies.
9 Once fully autonomous vehicles, designed to respect road traffic regulations and avoid any collision, become available, the legislator will have to solve the problems resulting from their coexistence with vehicles driven by humans who may not always respect the rules. The democratic legislator will have to decide on the most appropriate balance between minimising the number of accident victims and allowing for an efficient flow of traffic.
10 The Assembly concludes that the above considerations give rise to a variety of novel challenges to regulatory regimes. It takes note of the work underway in specialised regulatory bodies, including the Working Party on Autonomous and Connected Vehicles (GRVA) of the United Nations Economic Commission for Europe, which is addressing a range of essential technical issues, as well as the European Union and different national authorities. It further notes the work within the Council of Europe on “Artificial intelligence and criminal law responsibility in Council of Europe member States – the case of automated vehicles” underway in the European Committee on Crime Problems (CDPC).
11 The Assembly therefore calls on:
11.1 the member States of the Council of Europe to ensure that the criminal law, civil law and human rights implications of the development and introduction of autonomous vehicles are regulated in accordance with Council of Europe standards on human rights and the rule of law, including respect for the right to life, privacy and the principle of legal certainty;
11.2 the GRVA to conduct a human rights impact assessment as part of its preparatory work on future regulation of autonomous vehicles, as part of a general, comprehensive framework for ensuring that safety in all its forms is maximised during future development and production of autonomous vehicles;
11.3 the CDPC to ensure that possible lacunae in the applicability of criminal law to the operation of autonomous vehicles are identified and addressed;
11.4 the CAHAI to pay particular attention to the application of AI in ADS, where there is a particular risk of adverse consequences for the enjoyment of fundamental human rights, in its mapping of the risks and opportunities of AI and its examination of the feasibility of a legal framework.

B Draft recommendationNote

1 The Parliamentary Assembly refers to its Resolution … (2020) on legal aspects of “autonomous” vehicles. It recalls that this resolution was adopted as relevant work was ongoing within the Council of Europe by the Ad hoc Committee on artificial intelligence (CAHAI).
2 The Assembly therefore calls on the Committee of Ministers to take into account the particularly serious potential impact on human rights of the use of artificial intelligence in automated driving systems when assessing the necessity and feasibility of a legal framework for artificial intelligence.

C Explanatory memorandum by Mr Altunyaldiz, rapporteur

1 Introduction

1.1 Background

1 The motion for a resolution (Doc 14839) underlying this report , which I tabled on 7 March 2019, was referred to the committee on 12 April 2019, following which the committee appointed me as rapporteur on 29 May 2019. The committee held a hearing with experts: Ms Theodora Hamsen, German Federal Ministry of Transport and Digital Infrastructure, Professor Sahin Albayrak, executive director of the Distributed Artificial Intelligence Lab (DAI-Lab), Berlin, and Mr Connor Champ, Automated Vehicles/Public Law Team at the Law Commission, London, at its meeting in Berlin, Germany on 14-15 November 2019. I would like to thank all three experts for their contributions to the preparation of this report. I had intended to conduct a fact-finding visit to a research centre working on autonomous vehicle technology but unfortunately the restrictions due to the Covid-19 pandemic made this impossible.
2 Technical progress has already made the circulation of semi-autonomous vehicles a reality. As noted in the motion, both the increased circulation of semi-autonomous vehicles and the expected circulation of fully autonomous vehicles pose questions in relation to criminal and civil liability, the obligations of manufacturers and the future regulation of car transportation that will need to be addressed by legislatures in Council of Europe member States, and beyond. Intertwined with these legal questions are a number of ethical and privacy concerns that will also need to be tackled.
3 The race to develop vehicles with increasing levels of autonomy led to its first fatality in May 2016. A driver who had put his car into Tesla’s autopilot mode crashed into a large white 18-wheel truck and trailer crossing the highway which in the bright conditions it failed to detect. The investigation by the US National Highway Traffic Safety Administration found that Tesla was not at fault.Note The first bystander was killed in March 2018 after a Volvo XC90 that Uber was using to test its self-driving technology hit a pedestrian. The car had detected the pedestrian, but its emergency braking system had been disabled. Prosecutors have determined that Uber is not criminally liable although it is possible that the backup safety driver will face criminal charges.Note
4 The stakes are constantly being raised and assisted driving technology is constantly developing. However, much technological and regulatory progress is still required before fully autonomous vehicles will be commercially available and allowed on the public roads. The European Road Transport Research Advisory Council (ERTRAC) predicts that fully automated shuttles and buses operating within defined urban conditions will not become available until 2030 and fully automated passenger cars will only be available sometime after 2030.Note

1.2 Key concepts – vehicle automation

5 Vehicles equipped with automated driving systems are often colloquially referred to as “autonomous”. However, in order to have a complete understanding of the ethical and legal issues it is important to clarify the distinctions between the different levels of technology. SAE International (formerly the Society of Automotive Engineers) has defined six levels of driving automation in their international standard J3016 for consumers and their classification has been widely accepted.Note
6 The levels start at level 0 which refers to vehicles that are entirely operated by humans and end with level 5 which refers to vehicles that are completely autonomous in all circumstances, requiring no human intervention. In between these vehicle classifications are driver assistance, for example power steering and anti-lock braking systems (level 1); partial automation, for example automatic braking systems to sense and prevent imminent collisions (level 2); conditional automation, with the system additionally monitoring the driving environment and prompting intervention by the driver (level 3); and high automation, with the system able to control the vehicle even in the absence of the driver’s intervention (level 4). The technical expression “semi-autonomous vehicles” refers to level 3 and level 4 automation. My report is principally concerned with semi-autonomous vehicles, but it will also look into the legal challenges raised by fully autonomous vehicles (level 5).
7 The current maximum level of vehicle automation that is publicly available is level 3. These vehicles have environmental detection capabilities but still require human override. The driver must remain alert and take control if the system is unable to perform a task. Level 4 technology does exist in prototype and is currently in testing; some manufacturers claim that their most recent models already include level 4 technology, although this appears to be based on a generous interpretation of the definition and is not accepted by independent experts.Note These vehicles are able to intervene if there is a system failure and do not require human interaction in most circumstances. However, the driver still has the capacity to manually override. This type of technology is being used at present but only along predefined routes and under specific circumstances. For example, shuttle bus services using level 4 technology are being used in some retirement communities, university campuses and airports. Complete level 5 automation is not currently possible.

2 Autonomous vehicles and artificial intelligence

8 Modern autonomous vehicles are entirely dependent on artificial intelligence (AI) systems. Advanced sensory systems, notably LiDAR (Light Detection And Ranging) and radar, provide detailed 360° information on the vehicle’s operating environment. This information, along with information from satellite positioning systems and on-board digital maps, must be processed so that the vehicle can identify its location, plan and follow a route and recognise and respond appropriately to markings such as road signs and lines and hazards such as actual and even potential obstacles in the path of the vehicle, including other road users (motorised vehicles, cyclists, and pedestrians). This processing is done by AI machine learning algorithms, trained on huge historical datasets and constantly refining their own performance through accrued real-world experience (including that of other vehicles operating the same system). For level 3 automation and beyond, the AI system will be in full control of the vehicle for at least some of the time, and the decisions it takes may be literally a matter of (human) life or death. As noted above, even at level 3 automisation and during (theoretically) human-supervised testing of level 4 systems, fatal accidents have already occurred.
9 Professor Albayrak highlighted an important difference in how future autonomous vehicles systems may be developed. The best-known projects – Google’s Waymo or Tesla, for example – involve ‘intelligent’ vehicles driving on ‘dumb’ roads, and thus are entirely dependent on their own sensing and data processing capacities. Professor Albayrak is developing a different approach with technology embedded not only in the vehicle, but also in the road and its infrastructure. Static cameras and other sensors monitor the road itself and the traffic upon it, and both this infrastructure and the automated vehicles themselves form a single integrated system exchanging data with a central computing system. Professor Albayrak’s research team is currently experimenting with elements of such a system on a real-life test-track in central Berlin. The advantages of such an integrated system in terms of overall traffic management appear obvious, especially in the context of a dense urban driving environment – although its fixed infrastructure costs and relative usefulness may make it less relevant on rural roads, for example. That said, as Mr Champ pointed out, there are also legitimate doubts about the suitability of level 4/5 systems for driving on small rural roads, where environmental conditions are quite distinct and challenging in different ways than in cities.
10 Automated vehicles are, in fact, robots – an essentially self-contained, computer-controlled machine designed to perform a particular function autonomously. Giving robots responsibility for transporting human passengers on the public road network has enormous safety implications. In 2018, there were 268 million cars, over 33 million vans, and 6.6 million trucks on the roads of EU member States;Note over 25 000 people were killed on those same roads.Note The introduction of automated vehicles means putting AI in control of fast-moving passenger-carrying projectiles in a situation of serious potential risk to both their passengers and other road users. Obviously, the expectation is that automated vehicles will prove safer than those driven by humans. A great deal of regulation and precautionary action is needed, however, before this can be assured. From a human rights perspective, therefore, a key consideration will be to ensure that automated vehicles and the AI systems that control them are regulated in a way that ensures full respect for the right to life, including positive obligations to prevent foreseeable threats.
11 The Council of Europe has already begun working on criminal law issues concerning the application of AI in the context of autonomous vehicle systems. I will examine this work, which is taking place in the inter-governmental European Committee on Crime Problems (CDPC), in more detail below.
12 Alongside the work of the CDPC, in September 2019, the Committee of Ministers established the inter-governmental Ad Hoc Committee on Artificial Intelligence (CAHAI). The CAHAI has been instructed to examine the feasibility and potential elements of a legal framework for the design, development and application of artificial intelligence. Its work is based on Council of Europe standards of democracy, human rights and the rule of law, as well as other relevant international legal instruments and ongoing work in other international and regional organisations. Along with the usual participants representing Council of Europe member and observer States and other Council of Europe bodies (including the Assembly), the CAHAI has an exceptionally high level of involvement of representatives of private sector bodies, civil society, and research and academic institutions.
13 The CAHAI held its first meeting on 18-20 November 2019. Amongst other things, it decided that a key element of the future feasibility study would be a “mapping of risks and opportunities arising from the development, design and application of artificial intelligence, including the impact of the latter on human rights, rule of law and democracy”. The CAHAI currently expects to adopt the feasibility study at its third meeting, scheduled for December 2020.
14 This is the institutional context within which the Assembly will debate the present and the various other AI-related reports currently under preparation in different committees. The Assembly has chosen to approach the topic on a contextual basis, examining the impact of AI in different areas. Within the Committee on Legal Affairs and Human Rights, for example, there are also reports on the impact of AI on policing and the criminal justice system, on brain-computer interface technology and (in the early stages of preparation) on lethal autonomous weapons systems. The recommendations that the Assembly may adopt on the basis of these reports will thus provide important guidance for the CAHAI when mapping the risks and opportunities of AI and its impact on human rights, rule of law and democracy, and subsequently determining the need for a binding international legal framework.
15 It should also be noted that other international organisations are also working on the ethical and/or legal regulation of AI, with varying focus and approach depending on their institutional perspective. Within Europe, the European Union, for example, has elaborated a European Strategy on Artificial Intelligence, whose implementation is supported by a High-Level Expert Group on Artificial Intelligence; and the OECD has adopted Principles on Artificial Intelligence with a strong ethical component.
16 Further, general information on AI, including a description and an examination of the applicable ethical principles can be found in appendix to the present report.

3 Ethical concerns

17 The introduction and potential proliferation of semi-autonomous and autonomous vehicles raise a number of ethical questions. Driving is not as simple as merely following the rules of the road. Drivers are regularly required to make what are, in effect, ethical decisions, particularly in situations of forced-choice such as certain unavoidable collisions.Note As the level of automation increases, so too does the decision-making capacity and function of the technology. Machines will be programmed with sophisticated forced-choice algorithms to allow them to make ethical decisions, such as whether it is better to hit two pedestrians or two cyclists. The ethical assumptions inherent in such algorithms can be problematic. There is a debate surrounding the basis of the decision-making criteria in these algorithms and whether governments should regulate to standardise these criteria or set a minimum moral standard. Several factors could be relevant in the event of an unavoidable crash, such as the number of people affected, the severity and likelihood of various types of injury, and, potentially, personal characteristics such as age and disability.
18 A key ethical concern in relation to the development of autonomous driving technology is that from a commercial perspective, it may be more beneficial for manufacturers to design vehicles that prioritise the safety of the car and its passengers, since people may prefer to purchase cars that will keep them safe. However, from a general utilitarian point of view, this will not always result in the most ethical course of action being taken. There is a need to balance manufacturers’ freedom to innovate and desire to make commercial gains against general public safety concerns.
19 From a socio-economic perspective, there are concerns about the accessibility of autonomous driving technology. A key benefit of the development of autonomous vehicles would be the reduction of road traffic accidents, since more than 90% of road accidents are caused by human error.Note This should logically in turn result in a reduction in insurance premiums for owners of semi-autonomous and autonomous vehicles. However, this new technology is currently only affordable for the wealthiest members of society, meaning that those from lower socio-economic classes are more likely to be driving cars that are less safe and see their insurance premiums rise, at least relatively.
20 Whilst there is broad common understanding, including at the international level, of the relevance and meaning of the main ethical principles applicable to autonomous vehicles, the challenge will be to ensure that this understanding is translated into regulations that are equally broadly accepted and implemented.

4 Liability issues

21 In the event that damage is caused by a semi-autonomous or autonomous vehicle, there will inevitably be questions as to who can be held criminally or civilly liable and in what circumstances. The answer to these questions will invariably depend on the level of driving automation. In relation to level 3 or 4 vehicles, problems with liability are particularly acute during the transition phases between automated and manual driving modes. In the case of level 5 vehicles there is also the potential dispute about whether the final decision on handover or takeover of control lies with the human driver or the machine.

4.1 Criminal liability

22 The difficulty of dealing with the “criminal behaviour” of non-human beings has been identified by the CDPC as a key concern in modern criminal law.Note Where a machine and not a human being is driving the car, there is the risk of the emergence of a ‘responsibility gap’. The concept of culpability or mens rea is crucial in European criminal law systems. Where an accident involves a semi-autonomous vehicle, apportioning the blame between the driver and the system can be extremely challenging. At present, in most European states, if assisted driving technology is in operation and there is an accident, the driver can still be charged with negligence since they are under an obligation to monitor the system. In the absence of negligence, the manufacturer can be held criminally liable. For level 3 and 4 automation, the exact demarcation between when the human driver is considered responsible (and therefore potentially liable under criminal law), and when not, will have to be precisely defined in law on the basis of principles established at the international level.
23 In the future, if level 5 technology becomes commercially available, new ways of establishing guilt (criminal liability) will have to be developed in criminal procedural law, potentially along with the creation of new offences in substantive criminal law. This may even raise the question of whether a non-human entity, namely the AI responsible for driving the vehicle, should be subjected to criminal law liability, perhaps in a similar way to a corporation may be liable as a ‘legal person’. Clearly this would have novel very complex conceptual and legal implications.
24 Moreover, not all questions concerning criminal law and autonomous driving technology relate to accidents caused by these vehicles. Decreased driver responsibility could also impact on the applicability of other offences such as drink driving or using mobile phones while driving. In this respect, the different levels of driving automation make it difficult to develop a comprehensive system of regulation.
25 The United Kingdom Law Commission is currently undertaking detailed exploratory work on the legal challenges posed by the introduction of autonomous vehicles, including the criminal law implications. In a series of public consultations, the Law Commission has been seeking comments on a wide range of issues and proposals for the future regulation of autonomous vehicles. This work reveals the wide range of novel and sometimes surprising challenges that arise in relation to criminal law regulation of autonomous vehicles, for example:Note
  • For level 3 and 4 automation, where there is still a human ‘user-in-charge’ (to use the Law Commission’s expression), the problem of driver distraction and resulting lack of situational awareness and delayed reactions was a predictable problem, but most comments thought that there should be “no relaxation of the laws against distracted driving”. Manufacturers, however, “supported permitting some activities other than driving” for the user-in-charge.
  • In the event of an accident occurring when the autonomous vehicle system was ostensibly in charge, insurers would need access to data on the event. In the absence of automatic reporting of accidents (by the autonomous vehicle system itself), the question arose whether the user-in-charge or insured owner should be required to notify the accident, and within what timeframe.
  • Where the autonomous vehicle system was in charge at a time when driving rules were breached, the Law Commission proposed that the matter be referred by the police to a regulatory authority, with the power to apply regulatory sanctions on the ‘Automated Driving System Entity’ (a self-selected entity, in practice likely to be the body that had put forward the automated driving system for regulatory approval, for example the manufacturer). The question of whether an AI could be subject to provisions of criminal law that are normally reserved for human beings, and which depend on concepts of personal autonomy and moral responsibility, would thereby be circumvented.
  • Questions about permissible exceptions to normal driving rules divided opinion: whether an automated vehicle should ever be allowed to drive up on to the pavement (for example to allow emergency vehicles to pass); whether it should ever be allowed to exceed the speed limit (drivers’ associations thought this could be allowed, up to a point); and whether it could ever slowly push forward through a crowd of pedestrians (most comments held that this should almost never be permitted, although a few considered that otherwise, autonomous vehicles could too easily be blocked).
26 The CDPC is conducting a multi-year project entitled ‘artificial intelligence and criminal law responsibility in Council of Europe member States – the case of automated vehicles’. The concept paperNote underlying this project opens with the most pertinent questions, namely: “who will be responsible if a completely automated vehicle injures or kills a human? With self-learning algorithms driving a car, the more general question arises: how should criminal law address Artificial Intelligence (AI)?” It later notes that “the legal framework currently applicable to the development and utilisation of automated vehicles (or other AI deployment) is based on normative principles developed during the pre-digital era… It could be therefore valuable to set up rules governing any potential criminal liability in advance to ensure that in cases such as a car collision or a drone crash, no State will have to face an unclear legal situation due to unsuitable or out-of-date rules… As the potential adoption of automated vehicles will affect all Council of Europe member States and beyond, there is a role for the Organisation to play in facilitating the general development of the principles pertaining to AI deployment… several issues should be addressed including the question of how different approaches in testing and using automated vehicles can translate into ‘permissible risks’ not criminalised in domestic law (like the different uses of technology in cars) as well as the question of whether an automated vehicle may eventually have to answer to the law as an e-person (similar to corporations as legal persons) or whether criminal justice is for ‘human persons’ only.”

4.2 Civil liability for damages

27 Where a semi-autonomous or autonomous vehicle causes damage, the victim may want to seek compensation. In the case of accidents involving conventional vehicles, harm or loss is generally evaluated in terms of the responsibility of the road users involved. A system of fault-based liability is in operation whereby the party who negligently or deliberately violated road traffic rules is obliged to compensate the party who suffered a loss. This type of system permits an absence-of-fault defence where the driver could not have prevented the accident. In the context of semi-autonomous vehicles there are more possibilities for the use of this defence when it can be argued that the automation system was responsible for the damage. Moreover, in relation to fully autonomous vehicles, a fault-based system of liability would mean that the vehicle user would never have to compensate a party who suffered loss, since the operation of the vehicle is completely out of their control.
28 In order to ensure injured parties can be compensated for their losses, many are advocating for a system of strict liability. Thus, even where there is no evidence of fault, the owner or user of the vehicle would be automatically liable for any damage. In this case, the existence of automated driving technology would not affect the owner’s liability. Under such a system the manufacturer could be required to contribute a portion of the insurance for each vehicle while limiting their product liability. Another alternative would be a system of first party insurance whereby the victims in each vehicle obtain compensation directly from that vehicle’s insurer, while non-motorised road-users (pedestrians, cyclists etc.) are still protected by third party liability. In this system, the damage caused by assisted driving technology would be automatically compensated by the insurer of the vehicle involved.
29 The suitability of these various regimes will depend on the degree of vehicle automation. This will indicate how much responsibility for the damage the driver and the insurer can fairly be expected to assume given the amount of control the driver had, or should have had, over the behaviour of the vehicle. In the United Kingdom, the Automated and Electric Vehicles Act 2018 is an example of good practice in this area and includes a comprehensive list clarifying the liability in a wide range of circumstances of vehicle owners and insurers if an accident occurs.Note

4.3 Product liability

30 Since the driver has less control over the behaviour of a semi-autonomous vehicle than over a conventional vehicle, often the damage that occurs is the fault of the manufacturer, who can then be pursued on the basis of product liability. However, automated driving technology poses a number of specific concerns in relation to current product liability rules.
31 In terms of the presentation of the semi-autonomous vehicles, there have already been issues with misleading marketing. For example, Volvo’s Pilot Assist system was initially advertised under the “autonomous driving” section of the company’s website, despite the fact drivers were expected to keep their hands on the steering wheel at all times. Such errors can mislead customers who do not understand the complexities of this new technology and manufacturers could be held liable for any damage caused as a result of such drivers’ reliance on claims that vehicles are “autonomous”. Manufacturers are only liable for products that are deemed defective within the bounds of their reasonably expected use. For this reason, there is a need to incorporate human factors into the testing of assisted driving technology, as certain types of careless behaviour are to an extent foreseeable, even with new technology. Manufacturers can be found liable for damage caused by a product where it should have been adapted in line with alternative designs that were available at the time it was marketed. Conversely, the ‘development risk’ defence means that the producer cannot be held liable if the defect could not have been known at the time it was put into circulation, given the state of technical knowledge at the time. Considering the rate at which assisted driving technology is evolving it is thus possible that consumers will have to bear the burden of a number of as yet scientifically unknown risks.
32 In addition, the burden of proof in product liability cases normally lies with the injured party. However, the complexity of assisted driving technology makes it incredibly difficult for an individual or an insurer to prove that a technical fault was the cause of an accident. This could create an unfair burden on the consumer. Equally, the extent to which the manufacturer can be held criminally or civilly liable in relation to victims of accidents could affect their liability to the consumer for selling a defective product in line with the ne bis in idem principle.

4.4 Insurance concerns

33 Insurers are key stakeholders in the development of assisted driving technologies, since car insurance is obligatory. They have the power to decide whether or not to insure semi-autonomous and autonomous vehicles and thereby determine their commercial viability. Currently insurers are in a difficult position in terms of setting premiums for drivers of semi-autonomous vehicles, as although these vehicles are designed to be safer than conventional vehicles, there are inevitably various new risks and dangers at play during the initial stages of launching the technology. In the long-term, however, average premiums are likely to fall due to the reduced risk of accidents.
34 One way in which insurers can address some of the premium-setting and liability concerns is through the use of pay-as-you-drive (PAYD) systems and black boxes. PAYD is an insurance model that uses telematic systems to calculate premiums according to individual driving behaviour, and could be employed to ascertain liability. This technology could be used to monitor the vehicle interior and ensure the driver is still paying attention even when using assisted driving technology. PAYD systems are considered fairer as users are charged according to their own driving behaviour and studies have shown that they can positively impact on driver behaviour.Note However, increased monitoring of consumers in this way also raises privacy and security concerns.

5 Privacy and cybersecurity issues

35 Semi-autonomous and autonomous vehicles operate using vehicle-to-vehicle and vehicle-to-infrastructure communication, sensors and high definition maps. All of this allows them to learn from other vehicles and maximise safety. However, all this information also amounts to a significant collection of personal data, notably data on a vehicle’s (and its driver’s and passengers’) location. The introduction of biometric features, such as fingerprint, facial and iris recognition, into vehicles’ security and other systems also implies the processing and storage of sensitive personal data. There are still several unanswered questions in relation to the information systems used in semi-autonomous vehicles such as, what type of information is being collected and why; who controls and has access to this information; and how long is it stored for? Should data recorded by autonomous vehicles prior to and during accidents be automatically shared not only with the central system, but also with insurance companies, or the police and other regulatory/enforcement bodies? Within the European Union (EU), the General Data Protection Regulation (GDPR) is relevant in this respect.Note These regulations apply to all companies processing data from subjects residing in the EU, regardless of the location of the company. It is necessary to find the correct balance between fostering innovation and protecting the privacy of individuals. Effective data protection regulations, appropriate also to the context of connected, (semi-)autonomous vehicles, will be an important and necessary part of the overall regulation of autonomous vehicles.
36 Putting a computer in charge of a passenger vehicle also gives rise to cybersecurity concerns. Hackers could potentially take control of a vehicle through wireless networks, such as Bluetooth, keyless entry systems, cellular or other connections. The dangers of hacking would be particularly high in relation to a level 5 vehicle where the human driver – or in this case, rather a passenger – is not required to oversee or intervene in any of the driving tasks. Moreover, the data hacked would be financially valuable and could be sold on to third parties. Thus, securing the information systems used in assisted driving technology should be a priority.

6 Current regulation in Europe

37 All Council of Europe member States have their own regimes for regulating autonomous and semi-autonomous vehicles. Many countries, such as Germany and the United Kingdom, have developed specific laws on assisted driving technology, while others apply existing laws for conventional vehicles to semi-autonomous vehicles. Notwithstanding, there are also a number of international and regional instruments that are relevant. In the area of the specification of vehicle safety requirements it can be argued that the relative importance of specific national law is reducing rapidly, particularly within European Union States.
38 As Mr Champ pointed out, there is already a “lot of law” at various levels of jurisdiction surrounding road vehicles. This law covers a host of issues including insurance, product liability, international vehicle standards, criminal liability for driving offences, civil penalties for driving infractions, roadworthiness standards and procedures, consumer information and marketing standards, driving licences, accident investigation procedures, data protection regulations, regulation of the taxi and private hire markets, regulation of public service vehicles, general traffic regulations (rules of the road), etc. This law was developed with non-autonomous vehicles in mind, or at most those with some driver assistance features, under the constant control of a responsible human being. This division – between driver and vehicle – is now breaking down, and new laws and regulations will have to decide whether safety assurance for automated driving should focus on the driver or the vehicle. In some cases, the technology itself will largely resolve this dilemma, notably when it comes to fully autonomous vehicles with no user-in-charge (or at most, a remote supervisor able to take decisions in exceptional circumstances).
39 At an international level the Vienna Convention on Road Traffic of 1968 deals with general traffic law. It has been signed and ratified by 38 Council of Europe member States.Note Article 8.5 and Article 13 state that all drivers must be able to control their vehicle at all times. In 2016 a new paragraph was added to Article 8 of this Convention to make provision for automated vehicles.Note As a result, autonomous and semi-autonomous vehicles will be considered to be in conformity with the Convention provided the system can be overridden by the driver or it fulfils the requirements of the United Nations Economics Commission for Europe (UNECE) Agreement of 1958 and the Global Technical Regulations (GTR) Agreement of 1998.
40 The 1958 UNECE Agreement was established to facilitate the adoption of uniform conditions of approval of motor vehicle equipment and parts across Europe. The GTR Agreement of 1998 was designed to create a global process for this type of approval that was applicable to countries from all regions of the world. A directive was adopted at EU level to ensure that once vehicles or vehicle components are certified in one member State, they cannot be excluded from markets of other States unless there is sufficient evidence that it would be seriously threatening to traffic safety.Note This Directive makes large use of the UNECE Regulations for vehicle safety rules.
41 In 2018 the UNECE set up a dedicated Working Party on Autonomous and Connected Vehicles (GRVA). This group helps to mobilise expertise from key industries together with civil society in order to realise the vision of new sustainable mobility and support the mass introduction of autonomous vehicles on the roads. There are also a number of Informal Working Groups (IWG) on specific vehicle safety issues. The IWG on Automated Commanded Steering Function (ACSF) proposed amendments to UNECE Regulation 79 in order to include provisions on ACSF technology.Note The adopted amendments include that in case of an ACSF intervention the driver must be able to gain control of the vehicle within four seconds; immediate override is required and the vehicle must be able to stay in lane, keep a distance and handle rear impact scenarios on highways.
42 The EU has also adopted instruments regulating the appropriation of risk in relation to the use of motor vehicles. The two main legislative acts governing liability are the Motor Insurance Directive and the Product Liability Directive.Note Substantive rules on liability for damages resulting from motor vehicle accidents are not harmonised at an EU level. The Motor Insurance Directive prescribes only minimum third-party liability insurance. Autonomous vehicles fit the definition of “vehicle” set out in Article 1 of this Directive and so will automatically be covered by its provisions.Note The Product Liability Directive sets out rules relating to the liability of producers and the rights of consumers. It is based on a no-fault liability regime, meaning that the producer of a defective product must provide compensation for personal injuries caused by their product irrespective of the negligence of an individual.Note The presentation of the product, the product’s reasonably expected use and the time when it was put to market are all influential in determining whether it is “defective”.Note There is a limited list of derogations that can waive product liability.Note However, this Directive only covers liability of producers of defective products, which may not be sufficient to deal with producer’s liability for injuries caused by autonomous and semi-autonomous vehicles. In addition, under this regime the cost of scientifically unknown risks would be shouldered by the injured party.
43 Ms Hamsen told us the approach to regulation of autonomous vehicles being taken by the authorities in Germany, the European country with the largest vehicle manufacturing sector. Since 2017, road safety legislation has allowed up to level 3 automation, which in some cases did not require the driver’s intervention. This legislation was beneficial for the car industry, as it provided clarity and security for investments. The legislation makes clear when the human driver may be subject to ‘adapted’ obligations to pay attention to traffic, and on what preconditions. If the conditions are not satisfied, the driver remains responsible in case of an accident. The technical preconditions include a system capable of respecting the normal traffic rules and regulations that a human driver would have to observe. The system must be described in international regulations that are applicable in Germany, or an application made for exemption under EU legislation. There are not yet any applicable international rules on autonomous vehicle systems; the GRVA (see above) is expected to make proposals but, according to Ms Hamsen, will probably not complete its work until next year. If the manufacturer’s technical description of the system states that the vehicle has level 3 automation and is able to comply with traffic regulations, then its driver is permitted to cede control of the car to the autonomous vehicle system – although the driver must still be able to monitor the situation and be able to resume control should the preconditions for reliance on the autonomous vehicle system no longer be fulfilled.

7 Conclusions and recommendations

44 The circulation of semi-autonomous vehicles is likely to increase significantly in the coming years as the technology becomes more advanced and more reliable. Some even believe it possible that within the next decade a completely autonomous vehicle will have been developed. These vehicles have the potential to transform personal mobility and dramatically improve road safety. At this stage, it will be technically possible to create autonomous vehicles that will never be “at fault” in an accident (except in cases of malfunction) and that will even be able to avoid most or all collisions that would be the “fault” of the other vehicle (or its driver), or of a pedestrian or cyclist. In a world where not all cars will be autonomous, autonomous vehicles programmed to avoid any collision will not be able to advance very quickly at all, as they will be “cut off” regularly by other, non-autonomous vehicles under the control of less cautious, human drivers. In order to give guidance and legal certainty to the engineers, legislators will have to “set the cursor” somewhere: should autonomous cars be programmed to assume that other road users will respect traffic rules too? What about children? Last but not least, when autonomous cars will be far safer than even the best and most attentive and unimpaired human driver, should legislators forbid non-autonomous vehicles? Would they even have a positive obligation to do so, in order to protect the right to life under Article 2 of the European Convention on Human Rights (ETS No. 5)?
45 Since vehicles regularly cross borders, the future regulation of autonomous vehicles should be developed with a global vision in mind. This should take into account the perspectives of all regions and the work being undertaken in different contexts around the world. The various international bodies, each working within its area of specialisation, should take account of one another’s activities in order to ensure a harmonious and comprehensive overall result. Those working specifically on regulation of autonomous vehicles should pay particular attention to the general regulatory principles and legal frameworks being developed for AI, and should conduct a specific human rights impact assessment of autonomous vehicle technology in order to anticipate the issues that may arise.
46 The Council of Europe is already attentive to aspects of autonomous driving that fall within its institutional mandate. By concentrating on criminal law aspects of autonomous vehicles in particular, and on the impact of AI on human rights in general, the Organisation plays to its strengths and can make an important contribution to the ongoing debate. One of the purposes of my report is to feed into this process. I therefore propose that the Assembly should underline the development and implementation of autonomous vehicles as one of the areas in which AI systems may have a particular impact on human rights and democratic societies. This should be taken into account by the CAHAI when mapping risks and opportunities of AI and the need for a binding legal framework to regulate its development and operation.
47 Further recommendations appear in the attached draft resolution and recommendation.

Appendix

Artificial Intelligence – description and ethical principles

There have been many attempts to define the term “artificial intelligence” since it was first used in 1955. These efforts are intensifying as standard-setting bodies, including the Council of Europe, respond to the increasing power and ubiquity of artificial intelligence by working towards its legal regulation. Nevertheless, there is still no single, universally accepted ‘technical’ or ‘legal’ definition.Note For the purposes of this report, however, it will be necessary to describe the concept.

The term “artificial intelligence” (AI) is generally used nowadays to describe computer-based systems that can perceive and derive data from their environment, and then use statistical algorithms to process that data in order to produce results intended to achieve pre-determined goals. The algorithms consist of rules that may be established by human input, or set by the computer itself, which “trains” the algorithm by analysing massive datasets and continues to refine the rules as new data is received. The latter approach is known as “machine learning” (or “statistical learning”) and is currently the technique most widely used for complex applications, having only become possible in recent years thanks to increases in computer processing power and the availability of sufficient data. “Deep learning” is a particularly advanced form of machine learning, using multiple layers of “artificial neural networks” to process data. The algorithms developed by these systems may not be entirely susceptible to human analysis or comprehension, which is why they are sometimes described as “black boxes” (a term that is also, but for a different reason, sometimes used to describe proprietary AI systems protected by intellectual property rights).

All current forms of AI are “narrow”, meaning they are dedicated to a single, defined task. “Narrow” AI is also sometimes described as “weak”, even if modern facial recognition, natural language processing, autonomous driving and medical diagnostic systems, for example, are incredibly sophisticated and perform certain complex tasks with astonishing speed and accuracy. “Artificial general intelligence”, sometimes known as “strong” AI, able to perform all functions of the human brain, still lies in the future. “Artificial super-intelligence” refers to a system whose capabilities exceed those of the human brain.

As the number of areas in which artificial intelligence systems are being applied grows, spreading into fields with significant potential impact on individual rights and freedoms and on systems of democracy and the rule of law, increasing and increasingly urgent attention has been paid to the ethical dimension.

Numerous proposals have been made by a wide range of actors for sets of ethical principles that should be applied to AI systems. These proposals are rarely identical, differing both in the principles that they include and the ways in which those principles are defined. Research has shown that there is nevertheless extensive agreement on the core content of ethical principles that should be applied to AI systems, notably the following:Note

  • Transparency. The principle of transparency can be interpreted widely to include accessibility, explainability and explicability of an AI system, in other words the possibilities for an individual to understand how the system works and how it produces its results.
  • Justice and fairness. This principle includes non-discrimination, impartiality, consistency and respect for diversity and plurality. It further implies the possibility for the subject of an AI system’s operation to challenge the results, with the possibility of remedy and redress.
  • Responsibility. This principle encompasses the requirement that a human being should be responsible for any decision affecting individual rights and freedoms, with defined accountability and legal liability for those decisions. This principle is thus closely related to that of justice and fairness.
  • Safety and security. This implies that AI systems should be robust, secure against outside interference and safe against performing unintended actions, in accordance with the precautionary principle.
  • Privacy. Whilst respect for human rights generally might be considered inherent in the principles of justice and fairness and of safety and security, the right to privacy is particularly important wherever an AI system is processing personal or private data. AI systems must therefore respect the binding standards of the EU General Data Protection Regulation (GDPR) and the Convention for the Protection of Individuals with regard to Automatic Processing of Personal Data of the Council of Europe (ETS No. 108 and the ‘modernised’ convention 108+, CETS No. 223), as applicable.

The effective implementation of ethical principles in relation to AI systems requires an ‘ethics by design’ approach, including a human rights impact assessment so as to ensure compliance with established standards. It is not sufficient for systems to be designed on the basis of technical standards only and for elements to be added at later stages in an attempt to evince respect for ethical principles.

The extent to which respect for these principles should be built into particular AI systems depends on the intended and foreseeable uses to which those systems may be put: the greater the potential impact on public interests and individual rights and freedoms, the more stringent the safeguards that are needed. Ethical regulation can thus be implemented in various ways, from voluntary internal charters for the least sensitive areas to binding legal standards for the most sensitive. In all cases, it should include independent oversight mechanisms, as appropriate to the level of regulation.

These core principles focus on the AI system and its immediate context. They are not intended to be exhaustive or to exclude wider ethical concerns, such as democracy (pluralistic public involvement in the preparation of ethical and regulatory standards), solidarity (recognising the differing perspectives of diverse groups) or sustainability (preserving the planetary environment).

;