Logo Assembly Logo Hemicycle

The need for independent and credible expert assessments

Report | Doc. 12873 | 20 January 2012

Committee
(Former) Committee on the Environment, Agriculture and Local and Regional Affairs
Rapporteur :
Mr Cezar Florin PREDA, Romania, EPP/CD
Origin
Reference to committee: Doc. 11892, Reference 3562 of 29 May 2009. 2012 - March Standing Committee
Thesaurus

Summary

In order to be credible and trustworthy, expert assessments in fields such as the environment, health, energy, finance or civil protection must be independent and impartial.

Recent events have shown that economic interests and a lack of information have often led to health scandals (contaminated blood, growth hormones, asbestos).

It is therefore essential, among other measures, to instigate model rules, to impose the traceability of expert assessments and for them to operate in a collegial framework.

A Draft resolutionNote

1 Recent events have highlighted the vital need for independent, impartial expert assessments in fields such as the environment, health, energy, finance or civil protection.
2 Too many expert assessments are based on causal hypotheses, some of whose repercussions are unknown. Experts therefore lack the requisite distance to gauge the medium- and long-term impacts on the environment and human health.
3 In this connection, the Parliamentary Assembly notes that the findings of these assessments vary according to the source of funding, and that this approach can be different when assessments are made after incidents, when the implications are economic, financial and possibly political.
4 The Assembly regrets that economic interests and the lack of full, transparent information has led to many health-care scandals (contaminated blood, growth hormones, asbestos, etc.), which have had serious repercussions on human health.
5 The Assembly considers that a legal framework could help reinforce the credibility of expert assessments by preventing external pressure.
6 The Assembly considers that if expert assessments are to be transparent and independent, they must be the subject of a debate in which all the different points of view can be expressed, and it would strongly advise involving representatives of universities, scientific and technological research circles and non-governmental organisations, either as experts or as observers.
7 The Assembly also stresses the need to ensure traceability of expert assessments as a means of ensuring their independence.
8 The Assembly consequently invites the governments of member and non-member States of the Council of Europe to:
8.1 call on the services of independent experts, particularly in fields requiring in-depth scientific and technical expertise;
8.2 set up a public fund to finance so-called “sensitive” expert assessments;
8.3 draw up a handbook of good practice and set up a high-level multidisciplinary committee responsible for ensuring compliance with ethical rules;
8.4 establish a system of consultation with representatives of civil society;
8.5 prevent conflicts of interests, particularly vis-à-vis so-called “sensitive” expert assessments, by means of a declaration;
8.6 introduce an assessment traceability system in order to enhance transparency and independence;
8.7 ensure proper follow-up of the consequences of expert assessment conclusions;
8.8 systematically disseminate expert assessment conclusions;
8.9 set up joint committees for transfrontier expert assessments;
8.10 adopt procedures for ensuring transparency and public information.

B Explanatory memorandum by Mr Preda, rapporteur

1 Introduction 

1 The independence and impartiality of expert assessments form a condition sine qua non for their reliability and legitimacy. Unfortunately, recent events have shown us that the assessments were not always performed independently. It is fitting to recall that the etymology of the word “im-partiality” implies that the opinion delivered is not that of a single party but is a matter of collective debate by two or more parties.

2 What is a scientific assessment?

2.1 General considerations

2 An assessment is an investigation which one person or a group of persons possessing knowledge and know-how in the field concerned is asked to carry out by another person or group, generally a public institution.
3 An assessment is needed when a decision which is to be taken requires knowledge of a field which is not within the sphere of common knowledge. When the field concerned is known to be of a scientific nature, one or more players within the sector concerned, and possibly in associated sectors, are usually called upon. The assessment thus provides the link between knowledge and a policy decision.
4 It is also usual for an expert assessment to be requested after a natural or other disaster which has serious effects on the environment and on individuals' health.

2.2 Assessment frameworks

5 The first questions which arise are who has requested the assessment and who is to finance it. In practice, it is very often the case that the findings vary according to the intended follow-up action, particularly when those commissioning the assessment have direct interests in the field being assessed. This is also and especially true of scientific assessments. On the other hand, the approach is very different when assessments are made following an incident, and particularly following a disaster. Indeed, the implications are economic, financial and possibly political.

2.3 The particular nature of the scientific approach

6 The scientific approach has certain particular features. It operates in accordance with its own structure and with a specific language and codes.
7 Making a scientific assessment of a situation presupposes building an analytical model, that is to say a set of abstract processes describing how a real situation develops. A model is necessarily based on consideration of a limited number of parameters, the selection of which – often on an unclear basis – is crucial and necessarily open to challenge. The model is based on assumptions and takes account of the parameters used in its production.
8 The results yielded by this model are then compared to known results. The model is then presented to a gathering of scientists (and/or published) and subjected to collective review, during which its author is called upon to reply to questions from his or her peers. Several models are sometimes developed for the same situation.
9 The models serve as tools for predicting and simulating possible future situations.
10 The question whether a given model is well adapted to another situation is rarely raised. However perfect a model may be, it is always confined to a particular, often ideal, situation. The free-fall model, for example, takes no account of friction. At present, models that have to reflect the effect of friction vary in their efficiency – it is very difficult to describe the fall of an object in precise terms owing to the considerable number of parameters to be taken into account. In the case of a climate model, the number of parameters is even greater and it will be even harder to create.
11 Models devised in this way enable the development of a situation to be predicted. This prediction is based on experimental data used by the model, and the accuracy of these data will significantly influence the results produced by the model. Let us take climate forecasting models as an example. In its 2007 report on climate change, the Intergovernmental Panel on Climate Change (IPCC) based its findings on the temperature changes recorded from 1850 to the present day. Temperatures have been recorded precisely for only fifty to seventy years; earlier values are not as exact, and the results provided by the various models will accordingly not be very precise. It will then be necessary to gauge the reliability of the results obtained, putting a figure on the probability, for instance, that the earth is warming up. In this connection, it may come as a surprise to note that the temperatures announced by the IPCC for the 1850s are accurate to one tenth of a degree.
12 Scientists are used to dealing with models. They know their limits and are therefore aware that the credibility of the result provided by the model is limited. This credibility is, incidentally, often calculated: a GPS location is pinpointed to within a few metres for example.
13 The non-scientific public is frequently unaware of this aspect and often expects a perfectly accurate and indisputable result.
14 It is extremely difficult to model, and hence scientifically assess, actual physical situations. Some details of the water cycle remain unclear, and it is scientifically impossible is say exactly why a particular forest is showing signs of weakness or what will become of each molecule of fertiliser spread on a field. By contrast, climate study centres have a considerable amount of data at their disposal, such as information on sulphur and CO2 concentrations in the atmosphere. These data enable assumptions to be made and warnings sounded, but no more than that from a scientific standpoint. The process of scientific identification is necessarily limited by the multiplicity of factors that come into play in natural processes.
15 Finally, it must be pointed out that scientific research work is steered by the scientific group itself and not from outside it. The focus of new work is often dependent on the results of previous work. The development of a new law or model rarely follows a linear course.

3 Position and special role of the scientific expert

16 Frequently, it is a decision-making body which commissions an assessment from a scientist or group of scientists, who usually belong to a public research institution not part of the decision-making body. This is the case, for instance, in France with the National Centre for Scientific Research (CNRS). The scientist is thus positioned at the interface between the relevant knowledge and the policy decision and has to input the scientific knowledge into the decision-making process.
17 The scientist thus faces what for him or her is an unfamiliar situation, namely the requirement to provide, by a specified deadline, the answer to a question that must make it possible to take or at least help to take a decision. In a scientific context, replying “I don’t know” is perfectly acceptable, whereas this reply is more difficult to formulate in the context of an expert assessment and is often unacceptable for the decision-making body.
18 This body needs at least a pseudo-commitment or, at all events, a diagnosis.
19 Quite often, the scientist who has become an expert will then be called upon to go beyond the replies resulting from scientific analyses and to give his or her own opinion, which will normally only commit him or her on the basis of his or her own convictions.
20 By way of illustration, mention might be made of the answers provided by the IPCC in 1990 concerning the future of the climate: “The calculations convince us that …”, “we consider that …”. In this particular case, the models and their results are marred by significant uncertainties and the only correct answer from the scientific point of view is, “We are not in a position today to provide a scientific reply to the question asked”. However, the group goes further and gives its opinion – it oversteps, or rather disregards, its scientific role so as to comply with that of an expert.

4 Credibility of the assessment

21 The question of the credibility of a scientific assessment therefore naturally arises. In order to reply to the question asked, the scientist or his or her group will necessarily be called upon to go beyond the purely technical results. This action is not open to reproach in itself but must be clearly acknowledged. It no longer has to do with the result of an indisputable technical analysis but with the group’s conviction, which is also important. A clear distinction must be drawn between scientific results and personal conviction. The scientist’s “I don’t know” must be rehabilitated and not denigrated.
22 In order to obtain a credible assessment, should it be entrusted to just one expert or should a panel of experts be set up? The choice of experts is crucial, and that is the crux of the problem. In this context, it seems essential to avoid preferring hyper-qualification as a guarantee of quality, as that is often synonymous with considerable specialisation, whereas an assessment requires multidisciplinary and transdisciplinary knowledge.
23 Making this kind of assessment can clearly be seen to constitute a discipline with its own requirements, but it is not currently recognised as such. It appears important to specify the skills appropriate to assessment. Some of the scientists employed in laboratories could thus be involved on a more or less permanent basis in producing assessments. Themes such as “water” or “oil” could be the subject of ongoing work, with publications, specialist debates and public debates.
24 Another pitfall to be avoided is conflict of interests. In order to do their work, most researchers need to find funding in addition to public money (which is often insufficient). This leads them to work for private interest groups, which have recourse to the best specialists in their field. In the context of an expert assessment, it would be a pity to do without these skills just because they also benefit a private group involved in the subject area of the assessment. But of course the potential conflicts of interests must be realised, and their compatibility with the specific assessment determined.
25 One question that arises is whether or not it is necessary to call on so-called independent experts, government experts or dependent specialist experts. These three categories offer both advantages and drawbacks, but the choice of the category primarily depends on the nature of the assessment to be carried out and of the persons or organisations directly involved.
26 It is also important to provide a precise definition of the term “independence”.
27 That leaves the issue of the methodology employed to produce the assessment. It appears necessary to set up groups of scientists whose members are bound to have divergent opinions. The group must consist of at least two sub-groups, one arguing in favour of the proposition and the other having an opposite stance. Scientific disputation is essential for the proper conduct of the analysis.
28 In this connection, and in view of the nature of the questions asked in the context of an assessment, it would seem worthwhile to establish a “public forum” for assessments. A public process constitutes a sort of critical sounding board.
29 It should be noted that this approach is different from that often employed, which is to produce an assessment followed by a counter-assessment. This method excludes the cut and thrust of debate.

5 Expert assessment and the concept of risk

30 A scientific assessment does not eliminate risk, but has to determine whether a risk is present, and what kind of risk it is. It is here that the crux of the problem lies. The question of strategic or environmental evaluation should obviously be raised.
31 Expert assessments following disasters are not only required to establish their causes, but should also have the capacity to lay down and put forward safety and prevention rules for the future.
32 The need has also emerged to consider the issue of the consequences of expert assessments where the findings have cross-border implications, for example an assessment relating to the laying of a gas pipeline or the construction of a nuclear power station.

6 Use of expert assessments

33 What use is made of an expert assessment? In order to answer this question, it can be reiterated that an assessment is generally commissioned by a decision-making body, often of a political nature. The aim is therefore to help policy makers take a decision. That is the primary function of the assessment.
34 Responsibility is then shared between the decision maker and the scientist. It might, incidentally, be asked whether in some specific cases an assessment is not commissioned when the decision has already been taken.
35 An assessment may, unfortunately, also serve the interests of lobby groups and private groups with financial objectives.

7 Comments on a few specific cases

36 At this stage of the discussion, it seems wise to look at certain specific cases, which are not exhaustive.

7.1 Asbestos

37 The problem of asbestos is a most telling example. True, for very many years certain groups met with a rebuff and went unheard. Thanks to stubbornness and more objective assessments, as well as the European Union’s successive programmes of environmental action accentuating the importance of preventing and reducing pollution of the environment, asbestos was finally recognised as a first-category pollutant with serious effects on human health and the environment.Note
38 European Council Directive 87/217/EEC on the prevention and reduction of environmental pollution by asbestos seeks to prevent environmental pollution by asbestos; in addition, the substance was designated in Annex XVII of Regulation (EC) No. 1907/2006 concerning the Registration, Evaluation, Authorisation and Restriction of Chemicals (REACH).
39 The aim of these legislative provisions is to protect public and environmental health, to prevent, reduce and control environmental pollution by asbestos, to restrict the sale and use of asbestos and of products containing asbestos and to ensure that such products are labelled.
40 Inadequate management of the risks inherent in the use of dangerous substances can have irreversible consequences for human and environmental health. Workers' health can be affected in a number of ways, ranging from slight irritation of the eyes or skin, or asthma, to reproductive health problems and congenital malformations, or even cancer. The health impact can result from short-term exposure or multiple periods of exposure with the long-term accumulation of substances in the organism.
41 To prevent or reduce substances' negative environmental effects, above all surface and ground water pollution, soil pollution and air pollution, including the greenhouse effect, and any risk to public health, it is necessary to conduct environmental and human health hazard assessments.
42 Since hazard assessments are made by a number of entities (industry, public authorities, non-governmental organisations (NGOs)), differences may arise with regard to the interpretation of the results, and it is important that the hazard assessment should be underpinned by a code of ethics, which could constitute a starting point for overcoming these difficulties.

7.2 Pesticides

43 Environmental hazard assessment is also encountered in the case of biocidal products, plant protection products and all other chemical substances which, by virtue of the quantity placed on the market, come within the scope of Regulation (EC) No. 1907/2006 mentioned above, which encompasses a registration procedure, an evaluation procedure and a restriction procedure.
44 These procedures involve interpreting all the existing information on certain uses of substances, regarding which a number of hazard management options are developed.
45 In Romania for example, with regard to pesticides/plant protection products, in accordance with national and EU law, applicants for the authorisation of various chemical products/substances destined to be used, sold or placed on the market are required to seek an environmental opinion from the ministry of environment and forestry. To obtain this opinion, the applicant must submit a file containing data on the eco-toxicological aspects and the action of the product in the environment.
46 To prevent or reduce the negative environmental effects of pesticides/plant protection products, above all surface and ground water pollution, soil pollution and air pollution, and the dangers they pose for public health and the environment, it is necessary to conduct environmental and human health hazard assessments. Following these environmental assessments, the following are determined:
  • environmental restrictions (aquatic organisms, birds, bees, etc.);
  • the distances at which pesticides can be used without entailing a risk of polluting surface or ground water;
  • the indications of average risk level, which must obligatorily appear on the labels of the products in question.
47 The process of interpreting the results of hazard assessments gives rise to much discussion among the European Union member States' experts involved in these procedures, as a result of differing interpretations of Community law.
48 Standardisation of the action taken by the experts on the basis of a code of ethics could constitute a starting point for overcoming these malfunctions.

7.3 Natural disasters

49 Credible damage assessment during and after natural disasters (floods, landslides, dangerous weather events such as droughts or forest fires) constituted a key consideration in the drafting of national and EU law in this field. This aspect was, moreover, addressed both in Directive 2007/60/EC on the assessment and management of flood risks and in Romania's national strategy for flood risk management in the medium and long term.
50 The floods that have occurred in Romania in recent years have shown that correct, rapid assessment of damage necessitates high resolution satellite images to supplement the data supplied by the damage assessment committees established at the level of each affected administrative-territorial entity.
51 Apart from flood intervention measures, consideration must be given to related events such as accidental pollution of residential areas or industrial estates as a result of flooding. To manage these risks, owners must make budget provisions for flood protection work and the authorities administering watercourses must budget the equipment and resources needed to deal with pollution.
52 During periods of drought, particular attention must be paid to the risk of forest fires, which, along with floods, cause considerable damage to forest resources, damage which must be properly assessed with a view to rehabilitating the areas affected.

8 Conclusion and recommendations

53 In the light of the foregoing, it is clearly apparent that specimen rules urgently need to be laid down for the purpose of defining the context in which an assessment must be placed in order to make the final results indisputable.
54 Unarguably, experts will not always be able to ascertain the full implications and the human and environmental impact of the problem referred to them for assessment, and some uncertainty will always remain.
55 That is why these assessments must plainly be made in a collegial framework, including groups of scientists with presumably divergent opinions as well as civil society players.
56 Furthermore, the traceability of assessments will make for greater transparency and independence.
57 Lastly, it would be advisable to have specialised experts, especially in fields requiring technical knowledge, and to produce a handbook of good practice for them. These experts might be supervised by a committee of wise persons which would ensure due application of ethical rules.
;