A Report from the international STS conference in Japan (March 1998)
As in most social sciences, qualitative and quantitative approaches in STS are developing along differing trajectories. In the interdisciplinary field of STS, there is a large gap between these two approaches. How can we develop a theory to link the the qualitative and quantitative? Can scientometrics mediate between more fundamental research in STS and science policy issues? What are the underlying mechanisms when qualitative theory is operationalized into empirical data through simplification or abstraction? Analyzing the gap between quantitative and qualitative approaches in STS is also related to the question of what are the underlying mechanisms in scientific activity itself, in the operationalization into quantitative data, reflexively.
At the recent STS Conference in Japan (16-22 March 1998), the issue of how to bridge the gap between the qualitative traditions of STS and scientometrics was made central to two sessions. One of these sessions focused on the further development of STS and scientometrics in Japan, the other brought a number of leading figures from the two relevant arenas together for a discussion on future perspectives at the international level. A number of scholars in STS accepted our invitation to contribute to above discussions. Arie Rip chaired the international session which had contributions by Michel Callon, Steve Fuller, Loet Leydesdorff, Shin-ichi Kobayashi (one of the organizers of the conference and, among other things, the Japanese translator of the Gibbons et al. 1994 book), and myself.
Michel Callon formulated the challenge that confronts scientometrics as follows: “Scientometrics will face a crisis if it restricts itself to its concerns of further developing as a discipline per se. In order to be able to understand ‘Mode 2’ research, scientometrics itself needs to develop some characteristics of Mode 2 research” (Callon, 1998). He argued that the performative functions in relations to users have to be understood and internalized in the construction of the discipline. “We have encountered problems in making explicit the conceptual linkages between qualitative STS and scientometrics in courses at my university. Our students are not interested in scientometrics, because they do not see how this approach to studying scientific development can be made more reflexive and more consistent with qualitative methods in STS,” Sheila Jasanoff noted in the discussion. Indeed, in Japan we have also noted difficulties in linking these two different kinds of educational traditions. Both in relation to the policy processes and in relation to their educational roles, scientometricians have to engage in new discussions with STS scholars and others, providing their methodological expertise as a service to the relevant audiences.
The following is a report on the symposium on future perspectives of STS and scientometrics at the international level. As noted, Michel Callon first summarized the problems in disciplinary oriented scientometrics. He indicated a thin layer of exchange with other STS traditions and a lack of goals in the use of mathematics as internal problems of scientometrics. Furthermore, difficulties in the usage of scientometric tools in STS research and policy processes were signaled as external problems. However, Callon envisaged an application context for scientometrics in relation to so-called techno-economic networks. He discerned a future role for scientometrics in providing STS insights a point of entry to the discussions of the new economics of science because of the specialty’s affinity with econometrics. As a future perspective, he further recommended that scientometricians in the academic context should exchange more intensively with other traditions in STS, and that scientometrics in practice should reflect more qualitatively on its performative dimensions.
Following Callon’s recommendation for a Mode 2 like scientometrics, Shin-ichi Kobayashi showed the difference of the application contexts of scientometrics in Mode 1 R&D from scientometrics in Mode 2. For example, he raised the following questions: “Can measurements for Mode 1 research indicate effectively and sufficiently the activities in Mode 2?” and “Can scientometrics measure impacts towards policy goals rather than output in terms of disciplinary contributions?” These questions related to Callon’s insistence on a reflection on the application context: how can one analyze the impact of S&T indicators in different socio-political configurations (cf. Van der Meulen, 1997)?
Steve Fuller pointed out that “measurement” itself has a risk to produce a tendency that “bigger is better”, that is, appreciation only of large numbers like economic products (Fuller, 1998). For example, there is a tendency that the papers which were cited many times are considered as the good ones. Based on this tendency, Fuller insisted that “citation” can be considered as “voting” towards the papers. He claimed that although it is said that now is a “post-industrial” phase, whereby the endless production of material goods are no longer the driving forces of human condition, we are not free from the old industrial mindset towards our intellectual outputs. We are likely to deal our papers or books as “products”, using analogy of manufacture of product, and there is a tendency towards the more, the faster, the better. “Citation” is also utilized based on the implicit premise relates to this manufacturing mindset. The analogy between citing and voting can make this point clear, and this analogy is also useful to reveal the strategic characteristics of citation, explicitly.
I, Yuko Fujigaki, claimed that citation is not a voting process, but a focusing process on differences from previous papers. Citation itself is an orientation of the paper, in a map of the previous papers. Therefore, the papers which are cited frequently, can be considered as utilized in this mapping process through repetition and exaggeration of the underlying differences (Fujigaki, forthcoming). Applying autopoiesis system theory to scientific knowledge, she analyzed the process that subsequent bodies of papers exaggerate and re- evaluate the previous papers selectively. These processes can be considered as self-referential processes of the “journal system”. The study of the aggregated citation process thus potentially bridges a gap between measurement and epistemology. These aggregated citations built a disciplinary “validation boundary” (Fujigaki 1998), which reflect an embedded cognitive boundary of scientists, as well as a social one that restricts the behavior of scientists. Furthermore, these boundaries also can become empirical ones, based on journal system theory.
In a final contribution, Loet Leydesdorff showed results of citation networks from reflexive analysis among STS journals, indicating communication networks in the field of STS. His results suggest that since the later half of the 1980’s, various core journals of STS (Social Studies of Science, Scientometrics, and Research Policy) have grown apart, with less mutual citation relations between each other. He also induced the dimensions of communication in this interdisciplinary field, such as, “utilization” (Research Policy), “codification” (Social Studies of Science) and “formalization” (Scientometrics), and he explained exchange in different traditions in STS in terms of citation behaviour.
Arie Rip, who chaired this symposium and also played a role as the discussant, summarized by focusing on two points. One is how to construct a theory for linking science studies and measurement, and the other is the usage of scientometrics in politics. With reference to the former point, he elaborated on concepts like “citing as voting” introduced by Fuller and “validation boundaries” as mentioned by Fujigaki, since these concepts link the citation (quantitative) and cognitive (social) aspects. With reference to the policy use of scientometrics, he argued that there is an “eternal triangle,” constructed between scientists, science (policy) analysts, and science politicians (Rip, 1997). Following his summary, there was a discussion of these two points.
Different opinions were expressed on the present state of exchange between different kinds of research/educational traditions in STS, as illustrated by reflexive citation analysis in this area (Leydesdorff & Van den Besselaar 1997). That science studies is eventually a single field was a feeling shared strongly by the audience. For enhancement of exchange of different traditions, self-consciousness on each validation boundary in each tradition can be helpful. It can be said that “translation” between different traditions as integration method is considered as replacement of meanings of issues that across the different validation boundaries. It was also pointed out that researchers show the adaptation process towards validation boundaries of each journal which construct the tradition. The validation boundary affects the action of scientists, however, this boundary does not exist at the beginning, it is also constructed based on the action of scientists. Therefore, the validation boundary, which plays as a cognitive and social restriction, can be changed based on continuous next operation (scientists futures behaviour). We do not need to consider that boundary is fixed. It is possible that we can construct new boundaries which include exchanging between different kind of research/educational traditions, based on our future’s next behaviour.
In the discussion of the use of scientometrics in politics, several participants pointed to the risk of fixing one interpretation of quantitative data among the many possible by policy makers as well as by other scientists. The “rationality” in the interpretation process of the output data of scientometrics was discussed. In considering “rationality”, two points of entry for qualitative theorizing can be distinguished: operationalization process and interpretation process. First, qualitative theorizing can help direct quantitative measurement through operationalization process from qualitative theory to quantitative measurement. We should consider methodologically what it means when qualitative characteristics are operationalized into empirical data through simplification and abstraction. Secondly, in interpretation process, there is a feedback from the quantitative results to the qualitative questions. Here, the quantitative data operationalized in the first direction should be interpreted given the insights about the mentioned abstraction process. Otherwise, the data itself can be miss-interpreted without a context of producing these data and there emerge the risk of fixing one interpretation mentioned above. Therefore, we can say that there is a need to enhance the interaction between these operationalization process and interpretation process. The “rationality” in interpretation process of the output data of scientometrics can be re-considered in these interaction process. Furthermore, through these integration between operationalization process and interpretation process, qualitative theorizing itself will have an insight for further improving. In these interaction, quantitative approach can be made more reflexive and more consistent with qualitative methods in STS. Then we can overcome the problem that Sheila Jasanoff indicated above concerning educational traditions.
Additionally, a focus on the role of scientometrics in STS allows me and my colleagues in Japan to consider different research traditions in STS. Which are the methodological issues involved? How are conclusions from STS research warranted? These questions are important for the construction of STS in the Japanese context because of the functions of STS at the interface between the policy context (indicators) and because of the potential relations of scientometrics as a discipline with systems theory and metrics in the other sciences, like econometrics, psychometrics, and sociometrics.
Hence, we organized another session with a focus on the development of scientometrics in Japan. It included the application of co-words analysis towards policy documents (Fujigaki and Nagata, forthcoming), analysis on the operationalization process in making science indicator using the concept of meso data (Tomizawa and Niwa, 1996), publication analysis and citation analysis in astro physics (Makino, et al, 1997) and in psychiatry, application of interindustry relations analysis towards citing cited matrix (Shirabe, forthcoming), and a study on the possibility of using scientometrics method for cultural studies. The classification of methodology in scientometrics (publication, citation, words/co words analysis, institutional, co authors, etc.) and application fields was also discussed in a three dimensional space with three axis; texts, scientists, and cognitive (cf. Leydesdorff, 1995).
The two sessions were rich in terms of suggestions for the future perspectives in STS and scientometrics, indicating the ways and possibilities of exchange between the different research traditions in STS. The necessity of constructing a theory linking science dynamics and measurements accords with the possibility of integrating these traditions, that is, integration of qualitative studies and quantitative studies. Perhaps, STS has itself experienced a conflict between two cultures (natural science versus humanities and social sciences), reflected as a conflict between methodologies. Then, the analysis of dis communication among different traditions in STS and the efforts of integration among them can be utilized reflexively in analyzing the conflict between two cultures, as well as for studying communication difficulties among inter disciplinary and trans- disciplinary research.
Acknowledgment: The International Conference on STS (March 16-22, 1998, Tokyo, Hiroshima and Kyoto, Japan) was organized by Yoichiro Murakami (chair of organizing committee), Hideto Nakajima (chair of program committee) and Shin-ichi Kobayashi (chair of steering committee). I would like to show my gratitude towards these chairs for giving me the opportunity to organize above sessions with fruitful discussion. In total, about 380 people attended the conference. The conference information is available on: http://hostcinf.shinshu-u.ac.jp/stsconfjp.html. Those who wish to obtain the book of abstract, please contact: nakajima@sv.hss.titech.ac.jp.
References:
Callon, M. (1998) Is It Any Future for Scientometrics? And If Yes, Which One? Book of Abstracts of International Conference on STS, p26.
Fujigaki, Y. (1998) Filling the Gap Between the Discussion on Science and Scientist’s Everyday’s Activity: Applying the Autopoiesis System Theory to Scientific Knowledge, Social Science Information, Vol. 37, No. 1, 5-22.
Fujigaki,Y. (forthcoming, 1998) The Citation System : Citation Networks as repeatedly focusing on difference, continuous re-evaluation, and as persistent knowledge accumulation. Scientometrics.
Fujigaki, Y. and Nagata, A. (forthcoming, 1998) A Concept Evolution in Science and Technology Policy: The Process of Change in Relationship among University, Industry and Government, (forthcoming) Science and Public Policy.
Fuller, S. (1998) Science as a Vocation Circa 2000, in Brown, R.H. (ed.), Academic Knowledge and Political Practice, New York: Teachers College Press.
Gibbons, M., et al. (1994). The New Production of Knowledge: The Dynamics of Science and Research in Contemporary Societies. London: Sage. Kobayashi, S., Wen J.(forthcoming, 1998) A New Configuration of R&D Collaboration both in Public and Private Sectors: Applying Audition System in Performing Art, Triple Helix Conference Book.
Leydesdorff, L.(1995) The Challenge of Scientometrics: The development, measurement, and self organization of scientific communications, Leiden: DSWO Press, Leiden University.
Leydesdorff, L and Van den Besselaar,P. (1997) Scientometrics and Communication Theory: Towards Theoretical Informed Indicators, Scientometrics, 38(1), 155-174.
Makino, J. Fujigaki, Y. and Imai, (1997) Productivity of Research Groups: Relation between citation analysis and reputation within research community, Japan Journal for Science, Technology & Society, 6,85-100.
Van der Muelen, B.J.R. (1997) The use of S&T Indicators in Science Policy, Scientometrics, 38(1), 87-102.
Rip, A. (1997) Qualitative Conditions of Scientometrics: The New Challenge, Scientometrics, 38(1), 7-26.
Shirabe, M (forthcoming, 1998) Introduction of Economic Methods to Scientometrics: Centering around the citing cited table and the autopoietic system of citations.
Tomizawa, H. and Niwa, F.(1996) “Evaluating overall national science and technology activity: General Indicator of Science and Technology (GIST) and its implications for S&T policy”, Research Evaluation, 6(August), 83-92.
author’s address: fujigaki@nistep.go.jp