Editorial
While performing his military service, my son must have learned to fold and store his clean laundry with precise edges, tellekanter (counting edges) stacked in perfect alignment on designated shelves so any inspecting officer could simply run a finger down the edges to check that the requisite number of clean shirts, underpants, socks, towels, etc. were present and accounted for. Immediately on demission he forgot this skill. In civilian discourse, the term tellekanter is associated with pedantry bordering on insanity, with a compulsion for order so strong as to be a compulsive dis-order. Thus it must be in self-ironic jest that the Norwegian Research Council (NFR) has given the title “Tellekanter” to one page of the forms we with NFR grants must submit for our annual reports. There we are to report the number (and only the number; not the titles, authors, content, length, journal, publisher etc.) of our project’s publications in various categories in the past year – articles in refereed journals, articles in other journals, chapters in anthologies, books, textbooks, invited lectures, conference papers, … and so on down the shelves.
NFR can allow itself this self-irony. The counting of publications is a well-known ritual in academe, and in this particular context the ritual is without formal consequences. In other contexts, however, there are serious formalized consequences. Our departmental budgets are now comprised of three elements, all of them consisting of tellekanter – the number of courses of varying intensity levels (i.e. formats requiring different teacher:student ratios) in our instructional programs, the number of passing-grade credits we produce, and the number of publications we produce in various status categories (articles in refereed journals, articles in other journals, chapters in anthologies, books, textbooks, … and so on down the shelves). Logically, our departments must therefore hire those who draw students, pass students, and publish. Of these, our ability to publish is apparently deemed easiest to predict by counting. Thus when we apply for positions, tenure, promotions, and raises, we are instructed to structure our CVs according to the same shelving system, and evaluating committees are instructed first and foremost to run their fingers down the shelves counting publications on each and calculating the rate of publications per year.
Furthermore, once hired, we are encouraged to act strategically. After all, departmental budgets affect even those of us on tenure. If budgets are tight, we have less money to hire research and teaching assistants, to buy books, to replace our aging computers, to travel to conferences, etc. So even those of us with tenure, though no longer perishable must continue to publish. And we must not only publish, but must publish in the right places. The shelf-counts carry different values in the budget system. In Norway, the top shelf (the foremost 20% of refereed journals in a given field) counts two points, other refereed journals one point, and other publications fractions of a point. That “top 20%” is also identified by counting on shelves. Roughly speaking, a national committee starts with the most-cited ISI-abstracted journals in the field and works their way down the ISI shelf until they reach a cumulative total of 20% of published articles in that field’s journals. Since our budgets are competitive within a zero-sum “game” (i.e. we compete with all other departments for a proportion of the university funds on the national budget, as opposed to gaining some fixed sum per publication), a recent departmental memo instructed us to act strategically by publishing not only in our own field’s top journals, but also in other fields’ journals. A publication in another field’s journal effectively counts double – two points to us, and two points “stolen” from them.
Of course, all of this also leads to less explicit, often unintended consequences. It probably changes how and where we publish. Since journals have different preferences, that also affects what we write. It may change how editors choose among well-reviewed manuscripts (Is our primary goal to provide reading for our field, or to provide publication points?). It certainly changes the dynamics of the journal market, probably also other qualities of our discourses. In other words, it changes us as scientists and as a community of scientists. For us in science studies, it even changes the dynamics of our object of study. Like Schroedinger checking on the cat, we affect the publication data we observe by the very act of observing them … and that effect is magnified because powerful others are observing our observations. Furthermore, we ourselves are the cat. If our discourse decays, we die.
As editor of EASST Review, where am I going with this theme? I’m trying to come to grips with the implications of this for our thoughts regarding taking the Review on-line and opening up for peer-reviewed articles. What might the consequences of that move be for discourse within the field? Is a fractional-point journal nevertheless in some way invaluable, and would that value be lost if we move up a shelf? Might we elevate another journal to the top shelf by filling in from below? Would there nevertheless be manuscripts left for us worth our reviewers’ and readers’ time? Might we get more manuscripts than now? (As of now, filling in issue can be a struggle!) Might our readers pay us more attention? (As of now, we get very little response!) Can we provide a level of service that might contribute to maintaining a constructive discourse within the field, or would we be adding to writers’ frustrations and tensions? Would we add to the field’s pool of reviewers, or merely add to the workload on those reviewers the field has?
Four other items in this issue focus on this theme: One is a commentary by Aksel Tjora, ignited by what he experienced as an unacceptably slow and disrespectful review process. In her response, Ulrike Felt, editor of Science, Technology & Human Values, describes how pressure to publish has created an unsustainable situation for editors and reviewers. The third is a report on member responses to our recent on-line survey about the future of the Review. In addition, note that the use of quantitative indicators was among the aspects of health research management studied by Inge van der Weijden in a recent dissertation presented in this issue. We invite you to read and comment on all of these texts.
Within the social science departments in Norwegian universities there is now a move towards article-based PhD dissertations instead of the monographic thesis. At present, relatively early in their research project PhD students have to choose between writing either a monograph or a selection of articles (four to six articles and an overview chapter). In addition they need to reflect on which language they will use: Norwegian or English. Some supervisors, myself included, have in recent years advised PhD students to go for the article-based dissertation, because we believe there are some advantages.
First, the PhD students receive external (and anonymous) advice from experts in the field on the various parts (article manuscripts) of the research. Journal referees may provide really helpful advice and corrections, in an early enough stage of the dissertation work so that students can improve their manuscripts and be better prepared for the dissertation defence. For us as supervisors, referees’ and editors’ suggestions provide helpful second (and third, and so on) opinions. Second, supervisors and other members of the research group, or other colleagues, may contribute as co-authors to one or more of the papers to strengthen the manuscripts. In this sense, there is a major potential for stronger collaboration between researchers (it is nevertheless expected that the PhD students are main contributors and first authors of their articles). Third, if PhD students succeed in publishing articles, it is not only of great value for themselves, making their work visible, but also promotes their research group and/or university department. They will actually also contribute to the university funding, since publication points are part of the budget basis from the Norwegian Ministry of Education. Fourth, PhD students may write some articles in English and some in Norwegian, thereby getting training in both international publishing and contributing to the development and maintenance of a Norwegian academic language.
But, and there is a major ‘but’: My own recent experiences have led me in the direction of a more sceptical reflection, although I have been one of those in my department who has been most sympathetic to the article version of the PhD. My scepticism is related to the editorial processes in those journals in which we would like to publish our work. Let me very briefly outline a very recent experience with one of the major journals for EASST members, Science, Technology & Human Values (hereafter ST&HV).
In December 2003, I submitted a paper to ST&HV. It went into the reviewing process and I received a revise/resubmit response August 24th 2004, nine months later. During this period (in June 2004) I had e-mailed an inquiry about the review process and got a response from ST&HV that it was in for editorial decision; fair enough, although it had already been quite a long time. I submitted a revised version of the paper December 10th 2004, and received a response (accepted with major changes) December 4th 2005, one year after the re-submission! After a fair bit of changes on the paper, I submitted a second revised manuscript to ST&HV the 16th of March 2006. And I waited. I emailed an inquiry about the paper in August: No response. And waited (well, of course I was doing a “few” other things). And sent another request for status of the paper in September: No response. I sent a fax: No response. And I waited, but more or less trying to forget about the whole thing. In the beginning of February 2007 I sent another e-mail: No response. After a few unsuccessful phone calls, I managed to get the editor on the line the 16th of March 2007, exactly one year (!) after the submission of the second revised version of the paper. The editor then told me that she had rejected the paper.
I was really angry. But I wasn’t angry because of the rejection itself. I was angry because of the process. I have had articles rejected from various academic journals before, without my getting nasty on the phone or writing letters of frustration. I have learnt that academic publishing is a cut-throat business (although it would be less painful with quicker cutting). Being in a tenured position, a rejection does not exactly put me out of work. I am happy (well, not exactly ’happy’, but can readily accept and move on) to find other outlets for my research. My anger was (and still is) related to ST&HV’s editorial process, which took over three years with this paper. Of course I was also re-writing the paper in this period, but ST&HV was actually “sitting on” the paper for 32 months altogether.
And what am I supposed to tell my PhD students, who may consider submitting a paper to ST&HV and cannot wait this long for a decision? Well, you might guess. This leads me back to my starting point: Do I still consider the article-based PhD a good idea? Yes, actually, I do. But only if we as supervisors have fairly good knowledge about the relevant journals’ editorial processes. And this, it seems, we have to learn the hard way. In my department I am now establishing a sort of a ‘blacklist’ of journals that people in the department have found too slow in the review process, or that do not respond to inquiries. This way we can learn from each others’ (hard-learnt) experiences of publishing, and become better supervisors for PhD-students who write articles. On the constructive side of this, and related to the discussion about turning the EASST Review into a peer reviewed journal: Yes, it might be a good idea, but only if one is able to develop editorial processes where respect for authors and responsiveness to their inquiries are maintained. There is a fair chance that these qualities might be more important than the impact factor for many of us, when we decide where to submit our next papers.
As editor of Science, Technology, and Human Values for nearly five years, I got an e-mail from EASST review asking me to respond to an upcoming piece by Aksel Tjora “What kind of editorial practices can we live with?” In particular I was asked to give some account of the way publishing functions from my experiences as editor. Before doing so two short comments:
While much could be said about the concrete case at hand, I will not discuss editorial decisions here. His paper was indeed one of the few taking much too long and I did apologise to him for this.
The relation between running academic peer-reviewed journals, publishing papers and academic accountability is pushing the existing systems to their limits, particularly in a growing, multidisciplinary and institutionally not too well established field like STS. This has so far not been discussed in a systematic and sustained way, but comes up mainly when somebody feels “mistreated”. Last year we had a debate about editorial policies in SSS, this time it is the handling of papers in ST&HV.
ST&HV has been the journal of the Society for Social Studies of Science for about 20 years. Editors are appointed by the society for a five year period, getting some very limited financial support for doing the work (financing editorial help from a PhD student). When I took over the journal in July 2002, it was for many reasons in a rather difficult situation. In order to keep quality high and to assure a balanced judgement it was decided to have three review reports as a decision basis. We put in place a database of papers, built a reviewer pool, participated in several sessions at 4S and EASST organised by PhD students in order to discuss publication procedures and constructed a web-based submission system to increase transparency and visibility. We set up a group of contributing editors in order to help with difficult cases. However many of the initial idea(l)s were quickly confronted with the reality of publishing.
Firstly, finding reviewers proved increasingly difficult. Figures for 2005: 649 reviewers were approached, of whom only 352 wrote a review – 200 did not even respond. In short, reviewing seems to be a not very attractive activity and this is a primary cause of time delays. The review system also reveals other structural problems: reviewers get the same (or only marginally changed) paper from various journals, “self-plagiarism” (parts of papers already published in non STS-journals), sloppy manuscripts etc. To be concrete: while some papers quite quickly get the reviews necessary (often with a unanimous verdict) others need up to 14 (!) review requests to get the minimum 3 reviews. In domains marginal to STS, these problems get accentuated.
Secondly, the number of submitted papers has risen steeply. Starting with 59 papers in 2002 and reaching a peak in 2005 with 108, we are “back” to 90 papers for 2006. This has put the whole reviewing system under pressure. Tensions appear at three levels: For the authors it has meant waiting longer to get their response. For the editor it has - between teaching obligations and research – meant nearly doubling the expected workload. For the journal the back-log increased substantially. Two years of negotiations with SAGE lead in 2006 to an increase in page numbers and an expansion from four to six issues per year. And yet, this is not a solution as the number of papers remains high. Our acceptance rate, which was about 25% of submissions until 2004, has decreased to 20% in 2006.
Thirdly, decisions on papers are mainly based on the verdicts of reviewers, ranging from acceptance, minor/major revisions, to revise and resubmit (which opens a whole new review process) and reject. Things become difficult when reviewers do not agree at all and frequently advice is asked from editorial board members. Acceptance pending revisions is always conditional upon the quality of the revisions made, and if reviewers plead for rejection after revisions were made, I tend to follow their advice. This seems fair in the light that we now have to reject papers much earlier in the process, a fact I feel somehow ambivalent about. In the past sometimes good papers emerged after a quite lengthy and multistage process of revisions - under given pressures this seems hardly possible.
Fourthly, time has become a crucial factor. A quickly reviewed paper with clear recommendations takes around 3-4 months turn over. When time delays occur in ST&HV they are mainly due to three factors and sometimes their coincidence: Different sorts of problems in the reviewing process, coincidence of many papers and decisions to take on the editors desk in particular while the semester starts or ends, problems with e-mail communication (both technical and sheer overload problems). If paper numbers remain high editorial teams might be a solution.
Finally, the submissions of PhD students whose theses will be a collection of articles is a central point in the argument of Aksel Tjora. While it is true that submitting for publication is an important part of the academic learning process, there are times when authors seem to assume that reviewers and editors will do what they and their supervisors should have done: put a paper in good shape before submitting it. Along with very good student papers, we also get regular complaints from reviewers who do not see their task as taking part in a new form of “distributed PhD supervision”. Being an editor is a challenging job. Given current changes in academia, the pressure on authors and editors will rise. The majority of authors whom you have contact with will end with not having their paper published – a piece to which they are attached and which they want or need to have published. Thus a reflection on the changes in the publication system is more than timely and EASST is in a very good situation to be the platform of such a debate. It will be essential to move away from a very personal approach to a much more structural debate over what publication system a field like STS needs.