Open Process academic publishing: towards the peer production of academic peer review

Open access, which only considers the availability of the final product, is not enough to insure true openness, the process of peer review itself needs to be made open, participative and transparent, argue the authors.

Toni Prug (with assistance from Benjamin Geer) (excerpts):

“Publishing and peer review processes in academia are currently closed models. In my view, at least in the areas i operate in (social sciences and humanities), these processes should be far more, if not entirely, open, with a provision for privacy in special cases. I call this model Open-process academic publishing. The name deliberately distinguishes it from Open Access, which refers to only the final outcome of academic knowledge production being open. The suggestion is not to open the processes in random ways, but in ways in which this openness — fundamentally based on volunteer participation — brings/enables more structure, more internalized working discipline, more commitment, and more ability to improve cooperation/collaboration with deliberate precision – all with the goal of improving the outcomes. “[…] culture of open processes was essential in enabling the Internet to grow and evolve as spectacularly as it has”, hence, we could call it The Internet Model (software/FS + networking/IETF). Its potential screams for being reused, hacked, for other areas of production. Academia, especially its publishing side, seems to me capable of embracing such volunteer-core open-process cooperation.

Primary limitation of OA is focusing on only one part of the Open Source paradigm: openness of the final product. Which is not a surprise, given that this was the most dominant concept signifying the success of the software and networking communities at the time of creation of the OA ideas.

Today, i claim, we need a paradigm shift. Even if OA did incorporate most of the main methodological points about the collaboration that Open Source was representing, it still would not have been enough. Open Source is a very limited subset of methodology that made software and networking communities so successful. Hence, to re-capture what was lost in the Open Source, we need an Open Process and The Internet Model to replace it, and thus to expose the world to the revolutionary potential of the re-use of these models in many spheres of society, particularly in science. I will develop in detail the shortcoming of the Open Source model, and reasons for adopting new concepts in a paper i’m currently writing, with the provisional title Open Process & The Internet Model. As soon an alpha version of the paper is ready, i’ll publish it here on the blog and keep improving it live on the blog, increasing the version number with each improvement, following the practice i started with this text. Here, i’ll focus on what i think ought to done, to improve what academic publishing already does, with the focuse on the work of journals”

“The following benefits could be gained with open-process publishing and peer reviewing:

1) Quality of submissions would increase a lot over time – because new authors would see the history of the entire process and learn from it (archive of all submissions, peer reviews, editorial board comments, etc), and because they would be less likely to submit badly written texts with no adjustments to publicly stated journal guidelines (a big problem for editors, i get told over and over, is the large amount of low quality initial submissions). In the current system, with externally invisible submissions, the cost of submission for authors it too low: they can submit any rubbish without adjusting it to journal’s guidelines. The only people who see these disrespectful (towards volunteer work of editors) acts, and who associate it with author’s name, are editors. If submissions were openly visible, the cost of submitting random, unadjusted, low quality, undeveloped papers would be far higher, since such disrespectful behavior would be publicly linked to the author.

2) Quality of texts published would increase in general – because of a ) point 1, b) opening of the whole, or most, of the publishing process would also improve the quality of peer and editorial board reviews, for the same reasons like in point 1). Doing low quality, superficial peer or editorial reviews would be publicly exposed and vice versa – possibility of lost, or gained reputation as an editor or peer reviewer would be a motivating factor. In the current model, all of that work is visible only to those few who participate. The logic of reputation works well in life in general, it can work well via online tools too – Ebay is a good example of quite a successful model of attaching behavior to a name closely.

3) Journals who do this process well would attract more agile and risk taking authors – because through open-process publishing it makes more sense for authors to take more risks (might sounds counter-intuitive at first), be less within the known/accepted knowledge boundaries, since they can rely on the peer and editorial assessments of their work done in public – which in turn can lead to less politically correct, career-opportunist position taking from both authors and reviewers, and to an opportunity for more bold, leaps taking steps from both sides. In short, openness would steer reviewing assessment to be more focused on the merit (of course, different academic communities will have different notion of merit in their fields) of the work assessed, hence authors can be more confident in submitting such, more risk staking, less compromise driven works. Which would lead us away from “The modern academic system has become almost a training ground for conformity.” (Whitworth and Friedman, 2009), and away from the publish and perish devaluing model. Model whose low-risk, but well-referenced style of writing has made overall research difficult to asses. It would encourage ground-breaking authors to publish their new research early and suppress mediocre authors who often, by the sheer number of low-risk publications prosper in the current play-it-safe system, and develop careers by such — for the knowledge production suffocating (clogs the production, editors, reviewers publishers, all waste time) and for invidal careers thriving (get’s authors jobs and research grants), volume publishing. If open-process publishing was widely spread, re-writing of the same papers for different journals, again for the sake of careerism, to get research points and another publication, would be far easier to spot and expose. The current opaque system makes it easy for low-risk careerists, although Open Access is contributing to that changing for better. Open Process would reduce it drastically: if mailing lists were an early implementation model (submissions, editorial and peer reviews, revisions, everything gets sent to an open mailing list), spotting a submission which is a rewritten version of an already published paper would be trivial: one could use any good web search engine to check for key paragraphs, concepts with author’s name and it would be in no time clear whether the author has already published on the topic, where and what.

4) Journals who do this process well would significantly raise the dynamics/pace of research – because some of the most in-depth debates that now happen on academic blogs, could, thanks to the faster and open-process peer reviewing and commenting, move to journals. The form could be shorter, still referenced like academic papers are, and argument even more focused that those in an average 8000 paper are. My impression is that most long journal papers revolve around few core ideas, often not necessarily connected as closely as to necessarily require a single longer paper. Today, i believe that some of these ideas originate in blog posts. We could enable those high quality 700-800 words blog posts to be submitted in a fully referenced short, burst alike, form of 1500-2000 words. Because the argument would be shorter and focused, it would be easier to evaluate it, which would mean shorter turn around peer reviewing and publishing, and hence sooner possibility of those whose work relates to it to respond. The cycle of publishing would thus follow more closely how we research, especially for senior academics for whom: “research is often done when a few precious hours can be salvaged from a deluge of other responsibilities.”(Weber, 1999). It would also contribute to possibly avoiding the destiny of: “Many journal papers are out of date before they are even published. “; with a rather frustrating truth that many experience personally: “In the glacial world of academic publishing one rejection can delay publication by two–four years” (Whitworth and Friedman, 2009).

5) Journals would gain readership and reputation – because of all the above and because of below internal benefits and their public visibility

Internal benefits for journals

In addition, there are enormous internal benefits for journals, that would contribute to their increased organizational health and development:

1) Clearer structure and visibility of tasks and processes contributes to recognizing own most important workers – because more precise (due to breaking it down in defined and openly recorded smaller steps) and more transparent allocation of tasks and responsibilities exposes who does what and how, it rewards those who do more and better work – and in volunteer organizations (most editorial boards/collectives), recognizing contribution, and lack of it, is one of the keys for survival and improvement of the organization. Often, it happens that recognition falls to wrong people i.e. to those who have better social connections, who are in the more visible position. And that kills the spirit, rightly, of harder working, most important, participants.

2) Increased focus on implementation work and continuously carried out processes – because defining workflow steps and stages exposes what is the necessary implementation work that has to be continuously carried out – it puts emphasis on an organization/group/collective as a set of ongoing processes. It also exposes other kind of work as less important, and hence those who do it as less essential for the existence of the group/organization.

3) Easier project management – because increased task modularity and status (full status of submission = stage + state. see below) real-time visibility (anyone can anytime check the stage&state of any submission on the web system used ) allows for better project management, easier allocation/delegation of tasks, and a more precise sense of progress and problems. Which is all good for the general work spirit, time/resource assessments, and to keep authors who submit papers, and all other parties involved, informed correctly at all times about the stage&state of the submission.

4) Decision making into the hands of the people who matter most – because who does what and how becomes visible, and because those who carry out continuously implementation work matter most for the organization, decision making can be more in their hands.

For example, Marxists Internet Archive (MIA) addresses this by defining a volunteer, and hence defining decision makers, through work contributions: “MIA volunteers are people who have, in the most recent six-month period, made at least three separate contributions over a period of three weeks to six months”.

In the Open Organizations project, we defined this similarly: “Anyone who is doing implementation work in the group, or has done such work in the recent past (e.g. within the past two months), can participate in its decision-making.”

5) Attract new volunteers and reduce impact of the existing counter-productive internal participants – utilizing the above task/process openness and visibility, journal editorial boards could use decision making rules similar to MIA to attract volunteers. Through linking of decision making rights and defined implementation work, it would be recognized that certain type of work that could be done by external participant matters more than mere presence of existing internal talk&communication intensive participants. To reduce risk, only certain decision making rights can be given to new participants to start with, until existing board is not assured they are fit to carry out journal’s long term goals and strategies.

This opens up the organizations for the new participants who would from the beginning adopt the culture (habits) of doing the implementation work and it reduces detrimental influence, and eventually leads to the exclusion of, existing internal talk&communication intensive participants. Which is (exclusion habits and processes) also a positive culture to develop.

Existing software, like the Open Journal System (OJS) could be extended to enable this process to happen. An option for privacy, with reasons stated, could be added to the open-process workflow.”

3 Comments Open Process academic publishing: towards the peer production of academic peer review

Leave A Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.