Publication and Peer Review

Transcription

1 137 8 Publication and Peer Review Read this chapter if you would like to have the following questions addressed: Why do research students need to publish their work, and what are the available publication forums? What are the key issues in planning for publication, and how many papers need to be published? How should research students interpret and act upon resulting peer reviews?

2 138 Dr. D.J. Toncich - Key Factors in Postgraduate Research - A Guide for Students 8.1 Basic Objectives of Publication In Chapter 7 of this text, the focus was upon how research students should interpret and balance the work published by other researchers. In this chapter, the focus is upon the research student as a publisher of work and the significance of the publication process. A basic tenet of professional research is that, in order to have credibility, it needs to be subjected to independent critical review by peers. A typical research student may have one or more supervisors who can provide peer review but, because they are generally part of the same research process, they cannot act in an independent role. For this reason, publication forms a systematic basis upon which groups of experts can work, on an international level, to peer review presented research findings. This presents an excellent opportunity and challenge for research students to subject their work to the highest international standards of evaluation. If a postgraduate research dissertation is to be independently reviewed as the final test of a research program, one may well ask why there needs to be some intermediate peer review through publication. There are, in fact, numerous reasons for research students to publish their work, including: The fact that publication of a paper, in a refereed forum, provides an intermediate feedback mechanism which can be used to enhance research or correct deficiencies before a final dissertation is submitted for examination Unlike private reviews of work, research that is in the public domain can provide ongoing peer evaluation from other researchers The importance of circulating ideas in an international community with similar interests - publication can form the basis of ongoing interaction with other research groups The fact that publication of a paper can, in perpetuity, link a researcher to a particular piece of work, in an international forum.

3 Chapter 8 - Publication and Peer Review 139 Over and above these factors are the pragmatic benefits associated with publication - principally that publications are viewed as the outputs of a university or research centre and that they add prestige to the research organisation. In terms of the research student, however, the publication of research not only acts as a feedback mechanism but it also acts as a positive stimulus to lift the quality of the student's research process and writing style to the highest international levels. Moreover, a professional who assesses a postgraduate dissertation is more likely to be reassured by a research student who has already subjected his/her work to international peer review than by one who has not. Writing for publication in an international forum is not, however, a trivial matter for a novice research student. It is also important that the publication process is carefully aligned with the research work in order to maximise the benefits accruing from both. In this chapter, the key factors that need to be considered in terms of publication are discussed and strategies for addressing common issues are put forward. These may serve to simplify the task of publishing.

4 140 Dr. D.J. Toncich - Key Factors in Postgraduate Research - A Guide for Students 8.2 Types of Publications There are numerous forums in which research work can be presented for publicity and peer review. These include: (i) (ii) (iii) (iv) (v) (vi) (vii) International refereed journals (with no publication fee) International refereed journals (with a publication fee) International non-refereed journals International refereed conferences (where the author is an invited guest) International refereed conferences (with a conference fee for authors) National refereed/non-refereed journals National refereed/non-refereed conferences (viii) Trade/Professional Society journals (ix) Internet publication. These publication forms can essentially be divided into four categories: Refereed (with fee for publication) Refereed (with no publication fee) Non-Refereed (with fee for publication) Non-Refereed (with no publication fee). From a research student's perspective, the imperative must be to seek refereed forms of publication in order to make the process worthwhile. Nonrefereed publications and conferences achieve little more than providing a mechanism for publicising the work that has been conducted. With this in mind, the remaining decision relates to the specific journals/conferences in

5 Chapter 8 - Publication and Peer Review 141 which the work is to be published and whether or not a publication/attendance fee is required. Leaving aside the issue of publication and attendance fees, the only key factor that needs to be considered in terms of publication is the calibre of the forum and the calibre of the editorial board and review committee. In order to assess the calibre of a publication forum, it then becomes evident that a research student needs to be a studied researcher. Journals and conferences cannot be selected on an ad hoc basis by a novice. The key factor is determining where peer researchers, in closely related fields, have published their work and where landmark researchers have published their findings - sometimes, researchers, who have previously published landmark research, will be on the editorial board of a journal. Conversely, it is unwise to submit papers for publication in journals which are only peripherally related to the specific field in question - even if they are accepted, the likelihood is that referees are working at the boundaries of their knowledge base rather than within their narrow field. In other words, the rigour of the review is limited. Ideally, a journal or conference that is most tightly related to the field of work will produce the most rigorous reviews of that work - moreover, the credibility of the work will be greatly enhanced under such circumstances. The issue of fees for publication or conference attendance also require consideration on the part of the research student because they can be significant and may need to be provided by the university - hence, preplanning for publication is important. Some researchers (perhaps, innocently) believe that conference organisers pay for authors to attend conferences - although this was a common practice in the past, the majority of modern conferences require that authors pay the conference attendance fee before their work is printed into conference proceedings. Similarly, a number of journals (even some that are highly reputed) require authors to pay a fee per page for publication - after editorial selection. It is difficult to judge whether the fee element has a positive or negative effect on the quality of the work in a journal or conference - or, for that matter, on the quality of the work that has been omitted because of unpaid fees. In general, each publication needs to be assessed on its own merits and upon the history of the research that has already been published within it.

6 142 Dr. D.J. Toncich - Key Factors in Postgraduate Research - A Guide for Students 8.3 Publication Time-Frames In order to effectively publish work associated with a postgraduate research program, one needs to be aware of the processes and times involved. In this section, it is assumed that publication will be based upon refereed international conferences and international refereed journals or, in the other words, the highest levels of possible peer review. In the case of conferences, publication time-frames are relatively straightforward because they are, ultimately, bounded by the dates on which a conference is held. Normally, there are a number of phases associated with publication in international conferences. These include: A call for papers Submission of Abstracts Review of Abstracts Submission of draft paper Review of paper Submission of final paper Publication of preliminary proceedings Presentation of paper Publication of final proceedings. The final number of phases varies from conference to conference but the basic procedures are essentially similar. The process begins with a conference committee issuing publicity material, which calls upon researchers to submit papers, and ends with the publication of those papers, including additional key-note conference speeches and issues raised at the conference. A research student that wishes to submit papers for conferences needs to have a basic understanding of the typical conference forums that are available and the regularity of their occurrence - for example, some

7 Chapter 8 - Publication and Peer Review 143 conferences are one-off, while others are annual or biennial. Conferences may also commonly be held in one period of the year (e.g., the spring) in order to maximise the potential number of attendees. Conference organisers generally circulate calls for papers through various avenues including universities, government research organisations, industry research laboratories, industry groups and industry user groups, etc. Given the times required for reviewing and editing papers, the time between a call for papers and the actual date of the conference can range from 6-18 months. This means that a student, involved in a two-year research program, for example, may need to start planning for conference publication within the first three months of the research program - perhaps, long before any detailed experimentation or methodology has been developed. This planning will involve a determination or estimation of: The methodologies to be developed during the course of the research program The testing/analysis that will occur during the course of the research program The number of publications which would be expected to arise out of the research over the entire period of the program The times at which there will be sufficient information and test results to generate a publishable paper. Hence, a student who commenced a two-year research program in January, say, and wished to publish impending research findings through one conference paper and one journal paper, would need to have a clear picture of the suitable conferences that would arise over the coming months in order to fit in with editorial board deadlines. This may mean that abstracts for conference papers are submitted well in advance of any meaningful developments or experiments, and that refinements to the abstracts and final papers may need to be made as the conference time-line progresses.

8 144 Dr. D.J. Toncich - Key Factors in Postgraduate Research - A Guide for Students Many research students are particularly surprised to learn of the timeframes associated with publication in international journals. As a result, those who do not plan for publication can be left with no published work to cite in their own dissertations when they are submitted for assessment - in other words, no independent peer review can be cited and no critical feedback of key elements has been received prior to examination. Computer and networking technologies have not greatly improved the pace of review and publication in international refereed journals because the bottleneck in the process is the actual reviewing, which is carried out by expert peers, rather than the publication and printing. Publication in international journals is generally composed of a number of phases, including: Submission of a draft paper Review of draft paper Submission of revised draft paper Final review of paper Submission of final paper Formal acceptance of final paper Inclusion of paper in journal publication list Development of proof version for printing Evaluation and correction of proof version Publication. The review (or refereeing) process can be particularly lengthy because the editorial board of a journal needs to select appropriate reviewers, send them copies of the draft submission, await their availability and response and then contact the original authors to provide feedback, and so on. Given the limited time availability of expert peers to carry out paper reviews, the reviewing process can take between 2-12 months, depending upon the journal.

9 Chapter 8 - Publication and Peer Review 145 Typically, international journals are published monthly or quarterly, and each edition contains only a limited number of papers - generally in the order of five to ten. Hence, over the period of a year, a journal may only be able to publish in the order of papers. On the other hand, a journal may accept, for publication, papers in a particular year. The end result is that a publication bottleneck occurs and, even after formal acceptance of a paper, it may take one or two years for that paper to appear in fully published form. Given the lengthy delays in publication, a research student needs to accept that his/her research program will most likely have ended before some submitted papers are available in print. For this reason, those who assess research dissertations tend to be sympathetic enough to recognise that a paper which has been formally accepted for publication is, for all intents and purposes, a published work - even though the printing and distribution may still be in the pipeline. The key issue is, after all, that the work has been independently refereed at an international level. For this reason, if publication time-frames cannot accommodate the research program timeframe, the research student can still cite that papers are "in press", and incorporate the feedback from journal referees into the appendices of the dissertation as a formal comment upon (or proof of acceptance of) the postgraduate research work.

10 146 Dr. D.J. Toncich - Key Factors in Postgraduate Research - A Guide for Students 8.4 The Relationship Between Publication and Dissertation A common misconception amongst research students is that the publication of research papers, during the course of a postgraduate program, is a minor peripheral annoyance that must be endured prior to the submission of a formal dissertation for publication. In fact, the publication of research papers is an integral part of a research program and the final dissertation. A research dissertation represents the professional documentation for a systematic and impartial investigation in which a methodology has been put forward and the boundaries of that methodology have been evaluated. However, a professional investigation is only partially complete after a research student and his/her supervisor have completed the experimentation, analysis and studies. Even with the best of intentions, researchers have a natural vested interest in the outcomes of the research - in simple terms, most people would like to have their research appear as though it generated "the correct answer". For this reason, after a researcher has made the best possible attempt at impartially evaluating the boundaries of a research methodology, the next logical step is to submit the work to an independent external source to determine whether the entire process that was employed was valid. This serves to validate or negate the findings and to assess whether or not the employed experimental techniques were indeed impartial. A research dissertation, without independent external review, is essentially an incomplete work because the experimental process is incomplete - there is no external and impartial validation of the process. It is therefore important, in presenting a dissertation, to demonstrate a complete investigation by incorporating the findings of external reviewers, and this is achieved through publication - traditionally in refereed journals or conference proceedings. The research dissertation is, ultimately, a superset of the published work. In a well-planned research program, there may be several aspects to an investigation and each of these can form the basis of a refereed publication which validates the merits of the work. The complete dissertation then embraces the individual aspects and demonstrates how these were both internally (i.e., through experimentation and/or analysis) and externally (i.e., through refereed publication) validated. From the research student's perspective, the final dissertation should technically have been examined through publication of constituent parts, before submission as an

11 Chapter 8 - Publication and Peer Review 147 encapsulating work. This should alleviate much of the stress associated with the final examination of the overall postgraduate research program. Despite the importance of publishing research prior to the submission of a dissertation, in many instances, it is difficult for research students to publish their work. This may arise because the work is part of a confidential industrial research program or, perhaps, because there is insufficient time available to go through the traditional publication channels. However, the importance of the peer review to the thesis is so critical that there are alternatives available. One simple alternative is to author a paper, to the same level that would be required by a key international refereed journal in the field. However, rather than submitting the paper for review in an open forum, the objective is to seek a confidential review from two or three landmark researchers in the field. Confidentiality can be maintained (through written agreements, if necessary) and some of the important benefits of publication can still be derived. In the absence of a formally published output, the reviewers comments, and the confidential paper can be included as appendices in a dissertation to validate the work that was undertaken. This confidential review can provide a compromise alternative for those whose formal publication paths have been blocked.

12 148 Dr. D.J. Toncich - Key Factors in Postgraduate Research - A Guide for Students 8.5 Assessment of Reviews and Response Many people who are asked to review a paper or a thesis would not feel as though they had completed their task well unless they included some corrective actions and suggestions for improvement. Moreover, unlike research students, who, as humble, impartial learners, work under the restrictions of impartiality and objectivity, the referees in international publication forums, as experts, have the luxury of subjectivity and personal opinion. The key issue for the research students is then how to interpret the reviews that are received from published submissions. In assessing the feedback that is derived from reviewers (whether positive or negative), a research student needs to keep in mind that there are a number of possible reasons for the response - that is, the reviewer has: (i) (ii) (iii) (iv) Understood the work and the comments are objective and valid Understood the work but the comments are subjective Not understood the work but the comments are still objective and valid Not understood the work and the comments are subjective. It is only after receiving numerous reviews that one can comprehend the difficulty involved in communicating ideas, much less convincing others of their merits, while still retaining objectivity and impartiality in the presented arguments. However, given that the purpose of receiving reviews is to validate and improve the quality of the research work, then it is important to understand how to review a review. The starting point for reviewing feedback is to gain an understanding of whether or not the reviewer has understood what was presented. Quite often, poor phrasing or emphasis can lead a reader to a completely different understanding of what was intended by an author. It may also be that a reviewer is not an expert in the narrow field of study - this can occur if an editorial board is unable to find a tight match between the field of the paper and the expertise of its pool of referees. Many research students fret when they receive negative reviews for their papers, only to later find that these have occurred because their writing style or emphasis is poor, rather than

13 Chapter 8 - Publication and Peer Review 149 because of the research itself. Sometimes, negative reviews occur because a referee is partial to a particular theory or line of reasoning and because subjectivity causes a harsher review than would otherwise be received if the student had followed the referee's preferred line of reasoning. It must also be evident that a good review, from someone with an obvious lack of understanding of the presented work is equally unhelpful as a bad review. A negative (but objective) review on a paper, from a reviewer who is highly knowledgeable in the particular field, can also be particularly useful because it serves to correct deficiencies in the research program before the work is submitted, in totality, as a dissertation. A positive review, from a reviewer who is clearly an expert in the field, can be cited in the final dissertation as an independent validation of the quality of the research and analysis. A key issue that commonly arises in postgraduate research programs is how to tackle reviews that are clearly subjective and negative. In fact, these reviews are also particularly useful to the preparation of the final dissertation because they provide a clear indication of the arguments against which a defence must be mounted. The fact that a review is subjective does not mean that it should be dismissed out of hand, because it still represents a body of opinion. The real issue is how to counter such reviews within the confines of objective and impartial arguments. The other important feedback information that emanates from subjective reviews relates to the sensitivity and controversy of various issues and their impact upon the research program. It may be, for example, that a research student raises a minor peripheral issue in a research paper, which leads to a torrential subjective response from a reviewer. The research student then needs to evaluate whether the issue is central enough to the main theme of the research to warrant defence - if it is not, and the issue is controversial, then perhaps it is better removed from the final dissertation to avoid diluting the defence of the central theme. If, on the other hand, a strong, subjective and negative response is received to an issue which is central to the research, then the problem is clearly more serious. A defence has to be mounted against every subjective point by using incontestable data, analyses and external expert opinion. All feedback from papers, submitted to expert peers, is therefore of value to the final research dissertation because it can be analysed and used as

14 150 Dr. D.J. Toncich - Key Factors in Postgraduate Research - A Guide for Students a basis for strengthening the defence of the final arguments put forward in the dissertation. Ultimately, a research student who is confined to using the weapons of impartiality and objectivity, against the strength of arguments which can be both partial and subjective, needs to have interim guides as to how those arguments will be mounted. In the absence of such guides, the final defence (i.e., the dissertation) of the research is difficult to mount.

15 Chapter 8 - Publication and Peer Review Number of Publications Per Research Program One of the most contentious issues in academia relates to the number of publications that need to be produced by a research student prior to the submission of a dissertation for assessment. This issue is particularly sensitive because there exists a potential for a research supervisor to abuse his/her position by using research students as a research and publishing machine for his/her own ends (or to boost university outputs), rather than for the specific benefit of the student. A simple rule-of-thumb is that a research student needs to subject each significant contribution (or group of contributions) of the research program to external peer review through publication. If, for example, during the course of a Doctoral research program, there are three major contribution areas for a research program, then the student should generally publish three papers - one in each area. While this appears to be relatively straightforward, it is sometimes difficult to differentiate between areas of contribution. Several issues complicate the number of publications that should be produced. Firstly, it is generally considered unethical to publish the same paper (or minor variations of the same paper) in multiple forums. It is also considered, by some academics, to be unethical to divide a broad research contribution into a number of smaller portions and to publish individual papers covering Part A, Part B, and so on. Needless to say, there are many different shades between variations or divisions of publishable work and so, ethical guidance may need to be sought from supervisors, departmental or university research and ethics committees. In the event that a dispute does occur, between a research student and his/her supervisors, in regard to the number of required publications, and the dispute cannot be amicably resolved, then it is important that the research student refers to the appropriate university ethics guidelines in regard to publication. Given that publication issues can have serious ethical consequences, an intractable dispute between a student and a supervisor generally also needs to be formally referred to a higher decision-making body, within a department or university, for arbitration. However, the starting point needs to be a frank exchange of views between the supervisor and the student in regard to the specific objectives of publishing papers, as they relate to the specific outcomes of the postgraduate research program.

16 152 Dr. D.J. Toncich - Key Factors in Postgraduate Research - A Guide for Students There are also numerous research students who choose not to publish their research and prefer, instead, to rely solely upon the quality and integrity of their dissertations as the final (and only) external peer assessment of their work. Increasingly, however, the difficulty of finding directly relevant peer reviewers, for research thesis examination, means that such reviewers naturally feel more comfortable about reviewing a thesis if the constituent contributions have already been subjected to peer review through publication. The consequences of failing to provide preliminary review, through publication, can therefore be serious and, even in the best-case scenario, can make the final assessment phases of the program more arduous for the student.