Report of the Expert-Symposium at the Berlin-Brandenburg Academy of Sciences and Humanities, January 2006

What is the proper role of science in relation to policymaking? How should scientific advice to policymakers be institutionalised in government in a way more accountable to academic science and public concerns alike? Science is the major institution for producing knowledge pertaining to political decision making and regulation. However, concerns about the quality of scientific expert advice to policy-makers have been raised for years in the UK and by the EU. Past experience with public debates such as the BSE case or the controversy about genetically engineered food show that the legitimacy of experts and of the policy makers whom they advise essentially depends on the reliability and transparency of scientific advice. This has highlighted the absence of clear rules to follow as well as a legal framework and structures for obtaining institutionalised advice from academics. Thus, the issue of quality control and assurance in scientific expert advising is of vital importance for both, decision makers and the academic community.

With these problems in mind, the interdisciplinary research group “Scientific Advice to Policy in Democracy” of the Berlin-Brandenburg Academy of Sciences and Humanities has organised the expert symposium “Quality Control and Assurance in Scientific Policy Advice”, January 12-14, 2006 (with generous support by the Fritz Thyssen Foundation). The idea of this international high-level symposium was to have scientific consultants, agency heads and distinguished science studies scholars exchange their ideas and perspectives. Presentations were given amongst others by leading scientific advisers like Sir David King, chief scientific advisor to the UK Government, Geoffrey Podger, by that time Executive Director of the European Food Safety Authority, and Peter D. Blair, former Assistant Director of the Office of Technology Assessment (OTA) of the United States Congress, as well as by distinguished science studies scholars like Sheila Jasanoff, Naomi Oreskes and Peter Weingart.

For the first time, this symposium has assembled the perspectives and experiences of advisory bodies that have been functioning as “model organisations” for the institutional implementation of scientific advice across Europe and the United States. These advisory bodies can be regarded as model organisations because they have been functioning as blue prints or as reference points for the institutionalisation of scientific policy advice in their field. The symposium gave a review of the broad spectrum of the different procedures of quality control as well as the different institutional arrangements putting these procedures into practice within these organisations.

On a theoretical level, the role of scientific expertise in the policy process has been a core issue in the social study of science: Quality control, and particularly review by peers, is a central institution of the scientific system itself (cf. Merton 1973). As such it has been of major interest in science studies (cf. e.g. Chubin / Hackett 1990; Jasanoff 1987; Jasanoff 1985). However, regulatory or policy oriented science differs in important respects (such as goals, tasks institutions, time-frames and its accountability) from basic or curiosity driven science. Moreover, even within a scientific community, different experts may weigh evidence differently and adhere to different standards of demonstration – particularly in cases where their results have political or economic ramifications (cf. Oreskes 2004). Therefore, controlling and assuring the quality of scientific advice for public policy making is neither a uniform nor even a well-defined procedure, as more than a decade in science studies has shown (cf. most notably Jasanoff 1990).

In addition, recent developments in the relationship between science and politics are also affecting the system of quality control (cf. e.g. Maasen / Weingart 2005; Hemlin / Rasmussen 2006): ‘Fitness for function’ is increasingly becoming the norm in policy oriented science and scientific advice to policy (cf. e.g. Funtowicz 2001). But results that work may not be easy to accomplish, because the production and provision of science advice takes place in particular institutional arrangements and under particular constraints like uncertainty and emergent time frames. The need for knowledge quality assessment at the science-policy interface was emphasised in the presentation by Arthur Petersen and Silvio Funtowicz on quality management and extended peer review at the Netherlands Environmental Assessment Agency. Moreover, as Willem Halffman pointed out in his discussion note, norms guiding professional conduct in scientific expert advising will have to take into account a broad variety of different tasks performed under the heading of expert advice such as reviewing, reflecting and mediating. This resonates with a reflexive turn and a process (rather than product) orientation in scientific advisory organisations (cf. Hemlin / Rasmussen 2006).

On the political level, evaluation and quality control in regulatory science and scientific policy advice is also in urgent demand of politics and in government settings, as became apparent in the presentation by Manfred Hennecke. In his paper, he discussed the German Federal Institute for Materials and Testing that was founded in 1870/71 as the former Prussian Royal Mechanical and Technical Research Institute. However, when it comes to such tasks as the evaluation of the production and provision of expert knowledge, the uncritical use of success criteria often yields unexpected and advert consequences in science (cf. for the use of scientometric indicators Weingart 2005). The basic dilemma for a scientific advisory body is how to effectively produce, assess and provide scientific expertise to public policy makers in a way accountable to both sides of the science-policy boundary. This dilemma has led to a shift in the organisational forms of policy advice towards so called “boundary organisations” (a nice review of the discussion in science studies and organisational studies is given in: Jacob 2005). One instructive kind of boundary organisations that was discussed at the symposium is the Dutch sector council model presented by Bert de Wit. The sector councils function as a kind of “knowledge broker” by giving meta-level advice from a cross-departmental perspective. As another important example of a boundary organisation, the agency model that gains increasing popularity on the European level was discussed on the examples of the European Food Safety Authority (Geoffrey Podger) and of the European Environment Agency (David Gee; cf. also Waterton / Wynne 2004). From the perspective of scientific advice to risk regulation, Podger emphasised the eminent importance of stakeholder consultation at expert level before a final opinion is reached.

But how can the impact of scientific expert advice be evaluated outside the scientific community? This fundamental question was most explicitly addressed in Susan Owens paper on the British Royal Commission on Environmental Pollution: Drawing on a discourse analysis, she was able to show that the commission has contributed significantly to the development and acceptance of modern concepts such as integrated pollution control. Following Owens, effectiveness of scientific policy advice can be best described in terms of policy learning (cf. also Owens 1999). Further evidence was given by Peter D. Blair’s comparison of the NRC-study process and the OTA-assessment process: The strength of an OTA style analysis lies in a broad policy context, paralleling that of congressional deliberation, where (a.) questions involve broader societal, economical and environmental issues, (b.) a consensus view is highly unlikely and where (c.) many legitimate courses of policy action are possible. However, the demise of the OTA in 1994 shows the limitation of the Mertonian norms in guiding the production of scientific expertise and policy advice (cf. for a further critique Hamlin 2005: 402ff).

But where do we go from here? The symposium revealed several impediments on the way to a responsible practice of scientific policy advice:

Firstly, the responsiveness of organised science to public concerns about contentious issues will have to be increased. In order to restore and enhance the credibility of its advice, organised science will have to engage with the public in an open and transparent discourse over these issues. This is not only a question of enlightening and improving the public understanding of science but one of taking science’s role as a dialogue partner seriously and making this process really become a two-way discussion process. In particular, as Heather Douglas has pointed out, one has to be careful about bias in expert judgement as well as to somehow capture the different societal, experiential and epistemic perspectives on the problem at issue (cf. also Brown / Lentsch / Weingart 2006, with regard to citizen participation Brown 2006, and with regard to the role of values in science Douglas 2004 and, still the locus classicus, Rudner 1953). The issue of quality control pertaining to the use as well as the provision of scientific expertise by NGOs was broached by Paul Johnston from Greenpeace Research Laboratories, Exeter. In particular, he pointed out that precaution has to be necessarily science-based, but inevitably requires that decisions be made in the face of unresolved uncertainties (cf. also Johnston / Santillo 1999). In general, as was argued by Sheila Jasanoff, the quality of scientific expertise and regulatory information will be improved by coupling procedures of scientific analysis and deliberation.

Secondly, the question was raised whether scientific policy advice is in need of professional standards of conduct. In his opening lecture, Sir David King, UK Chief Scientific Advisor, presented his “Rigour, Respect and Responsibility: A Universal Ethical Code for Scientists” as well as the British Chief Scientific Advisor’s Guidelines as instructive examples of how such codes for ethical as well as professional conduct might look like.

Thirdly, the symposium highlighted the importance of taking into account the difference between policy advice and political advice: Policy Advice is about using science to broaden the range of choices available to decision makers. In order to fulfil this task, horizon-scanning, mitigating the effects of departmentalisation of scientific expert advising as well as first identifying and articulating effectively the needs and the place for scientific advice in policy development will be important measures as Sir David King pointed out. Moreover, as the OTA legacy as presented in Blair’s paper shows, it is often essential that science takes a partisan stance towards politics on behalf of the public interest; for this purpose, mechanisms will have to be developed that integrate and accommodate scientific and policy debates (cf. also Hamlin 2005: 16). Whereas policy advice broadens the range of choices, political advice goes along with a reduction of choices – preferable to one single option. However, often both aspects are related to one another: Abstaining from settling on one ”best option“ in advising policy sometimes means to take the risk that political decision or regulation about scientifically or technologically complex issues will be made on the basis of a poor risk analysis (cf. Pielke Jr. 2003).

Finally, the symposium has highlighted the importance of what Sheila Jasanoff has called the “three body problem of expertise” (Jasanoff 2005), namely, that accountability measures of scientific policy advice have to take into account all three bodies relevant to the effective integration of science and politics: Firstly, the bodies of knowledge that the experts represent (“good science”), secondly, the bodies of the experts themselves (“unbiased experts”) and, thirdly, the bodies through which experts offer judgements in the policy domain (“balanced committees”) (ibid.: 211). In order to succeed in establishing an effective and responsible system of scientific policy advice, it is crucial how the different lines of responsibility and accountability to both, academic science and politics, are organised and institutionalised. This point was nicely illustrated by Frank den Butter’s paper on the Dutch “polder model” and the institutional set-up of (economic) policy preparation in the Netherlands. The polder model goes back to Jan Tinbergen, the first Nobel prize laureate in economics. It relies on a clear separation of lines of accountability in policy preparation: firstly, a consensus on the mechanisms of the economy as formalised in econometric models; secondly, a compromise on policy goals between the different parties and, thirdly, on an independent and uncontroversial collection of data by an autonomous agency, the Central Bureau of Statistics. The aim of the polder model is to ensure the scientific quality of the policy preparation on the one hand and to gain public acceptance of the policy measures on the other. Since the foundation of the Central Planning Bureau by Tinbergen in 1948 this model has been serving as a blue print for the institutional set up of policy advice in the Netherlands (cf. also den Butter / Mosch 2003 and den Butter / Morgan 2000). However, the indubitable success of the Dutch Polder Model is dependent upon the specifics of the Netherlands: a relatively small number of key actors, a corporatists culture and a high mobility between academia, planning bureaus, advisory councils and think tanks at ministries.

To sum up, it is not by coincidence that this expert symposium was convened by the Berlin-Brandenburg Academy of Sciences and Humanities: Being one of the most outstanding bodies of organised science in Germany, responding to the societal need for reliable and credible expert advice is of highest priority to the Academy. The symposium has disclosed some of the most pertinent questions and difficulties attached to issues of quality in scientific policy advice. Moreover, it became apparent that a standardised and universal answer to the quality question in scientific policy advice is highly unlikely and, in many cases, will even not be appropriate and desirable. However, the symposium has convincingly demonstrated how science studies scholarship can very well contribute to a kind of benchmarking by which “best practices” within the institutional landscape of scientific policy advice can be identified.

Bibliography

Butter, F. A. G. den / Morgan, Mary S. (Eds.) 2000. Empirical Models and Policy Making: Interaction and Institutions. London u.a.: Routledge.

Butter, F. A. G. den / Mosch, Robert H. J.. 2003. The Dutch Miracle: Institutions, Networks, and Trust. In: Journal of Institutional and Theoretical Economics 159: pp. 362-391.

Brown, Mark B. 2006. Citizen Panels and the Concept of Representation. In: The Journal of Political Philosophy, forthcoming.

Brown, Mark B. / Lentsch, Justus / Weingart, Peter. 2006. Politikberatung und Parlament. Opladen: Budrich.

Chubin, Daryl E. / Hackett, Edward J. 1990. Peerless science: Peer Review and US Science Policy. Albany: State Univ. of New York Press.

Douglas, Heather. 2004. Border Skirmishes Between Science and Policy: Autonomy, Responsibility, and Values. In: Machamer, Peter / Wolters, Gereon (Eds.). Science, Values and Objectivity. Pittsburgh: University of Pittsburgh Press.

Funtowicz, S. O. 2001. Peer Review and Quality Control. In: Smelser,

Neil J. / Baltes, Paul B. (Eds.). International Encyclopedia of the Social and Behavioral Sciences. Amsterdam et al.: Elsevier: pp. 11179-11183.

Hamlin, Christopher. 2005. A Virtue-Free Science for Public Policy? In: Minerva 43: pp. 397-418.

Hemlin, Sven / Soren B. Rasmussen. 2006. The Shift in Academic Quality Control. In: Science, Technology & Human Values 31/2: pp. 173-198.

Jacob, Merle. 2005. Boundary Work in Contemporary Science Policy: A Review. In: Prometheus 23/2: pp. 195-207.

Jasanoff, Sheila. 2005. Judgement Under Siege. In: Maasen, Sabine / Weingart, Peter (Eds.). Democratization of Expertise? Exploring Novel Forms of Scientific Advice in Political Decision-Making. Dordrecht: Springer: pp. 209-224.

——-. 1990. The Fifth Branch: Science Advisers as Policymakers. Cambridge, Mass: Harvard University Press.

——-. 1987. Contested Boundaries in Policy-Relevant Science. In: Social Studies of Science 17/2: pp. 195-230.

——-. 1985. Peer Review in the Regulatory Process. In: Science, Technology & Human Values 10/3: pp. 20-32.

Johnston, Paul / Santillo, David. 1999. Is There a Role for Risk Assessment within Precautionary Legislation? In: Human and Ecological Risk Assessment 5/5: pp. 923-932.

Maasen, Sabine / Weingart, Peter. 2005. What’s New in Scientific Advice to Politics? In: Maasen, Sabine / Weingart, Peter (Eds.). Democratization of Expertise? Exploring Novel Forms of Scientific Advice in Political Decision-Making. Dordrecht: Springer: pp. 1-20.

Merton, Robert K. 1973. ‘Recognition’ and ‘Excellence’: Instructive Ambiguities. In: Merton, Robert K. The Sociology of Science: Theoretical and Empirical Investigations, ed. by Storer, Norman. Chicago: University of Chicago Press: pp. 419-438.

Oreskes, Naomi. 2004. Science and public policy: what’s proof got to do with it? In: Environmental Science & Policy 7: pp. 369-383.

Owens, Susan. 1999. ‘When Knowledge Matters’: The Role and Influence of the Royal Commission on Environmental Pollution. In: Journal of Environmental Policy and Planning 1/1: pp. 7-24.

Pielke, R. Jr. 2003. Another Epidemic of Politics? In: Science 300/5622: pp. 1092-1093.

Rudner, Richard. 1953. The Scientist ‘Qua’ Scientist Makes Value Judgments. In: Philosophy of Science 20/1: pp. 1-6.

Waterton, Claire / Wynne, Brian. 2004. Knowledge and political order in the European Environment Agency. In: Jasanoff, Sheila (Ed.). States of Knowledge: The Co-Production of Science and Social Order. London: Routledge: pp. 87-108.

Weingart, Peter. 2005. The Impact of Bibliometrics Upon the Science System: Inadvertent Consequences? In: Scientometrics 62/1: pp. 117-131.