Newcastle University. Cross Faculty Learning and Teaching and Student Experience Committee [CFLTSEC] October 2014 meeting

Transcription

1 Newcastle University Cross Faculty Learning and Teaching and Student Experience Committee [CFLTSEC] Action point arising: October 2014 meeting INTO revised marking criteria used for joint-marked assessments to ensure consistency and simplicity of language (M43.7) Section A Context Joint-marked assessments affected the following modules in the following INTO Newcastle University programmes: Foundation Business and Humanities Criteria affected INU0102/0502 EAP 40 credits Writing/ speaking INU0107/0507 Study Skills presentation INU0113/0503 Introduction to Writing Sociology INU0110/0510 Introduction to Business Writing Foundation Physical Sciences and Criteria affected Engineering INU0103/0503/0101/0501 EAP 40 credits Writing / speaking INU0116/0516 Physics Writing INU0122/0522 Study Skills and ICT presentation Foundation Biology and Biomedical Criteria affected Sciences INU0103/0503/0101/0501 EAP 40 credits Writing / speaking INU0118/0518 Biology Writing INU0122/0522 Study Skills and ICT presentation International Diploma in Business INU1101/1501 Criteria affected Writing / speaking EAP 20 credits [Diploma] INU1110/1510 Economics writing INU1107/1507 Study Skills [Diploma] Presentation

2 The issue raised by the Committee is premised by the accepted need for students to be able to improve and progress as a result of the feedback/feed-forward from an assessed piece of work (summative or formative). This suggests that students receive clear and unambiguous feedback/feed-forward messages on an assessed piece of work. The criteria used to assess submitted work, and from which these messages derive, are made available to students in module and programme handbooks. They are attached to assessment instructions and are most often used and attached as part of assessment feedback. There is widespread experience within INTO Newcastle of the assessment for learning culture, often found in mainstream secondary and post-16/post-compulsory sector (see work by Race and Black). INTO module staff often use these in class as teaching tools from the beginning of the programme and refer to them regularly in class as part of the students learning experience. The issue raised by the Committee was outlined in the minutes from March 2014 meeting: The Committee felt that some of the language used in the criteria could be difficult for students learning English to understand. The difference in the terminology used across the modules, for example, limited or occasionally could also lead to confusion for students particularly if these were not quantified. It was highlighted that students are provided with oral updates on what is required of them prior to the submission of assessment. That for next academic year INTO Newcastle University should review the criteria to ensure consistency and simplicity of language. There seem to be two issues/requests arising from this: Consistency ensuring modules use similar phrasing in their criteria Simplicity ensuring the language used by all the criteria is at a level which students can understand

3 Section B Some concerns and considerations: English language / broad comparability with IELTS All the EAP modules cited in section A, for both speaking and writing, use marking criteria based on practice which reflects that within the IELTS testing organisation. In order to achieve internal accuracy and reliability and external consistency, these involve using 8 bands across 4-5 separate criterial elements. (See appendix 1 INTO EAP writing criteria). All programmes at INTO Newcastle either English-only (Pre-sessional, EUS, etc) or Academic Pathway (Foundation, Diploma) use these criteria as core EAP criteria. This also applies to other INTO centres for EUS. Large programmes in the summer Pre-sessional 860 students employ 50+ new EAP teachers each year. Many of these colleagues are new to EAP and new to the required level of assessment skill. The more detailed criteria in Appendix 1 are actually essential for these colleagues looking and marking a piece of writing. IELTS does not reveal its detailed marking criteria. It releases a Public Version see appendix 2 for writing task 2. These are not used for marking. IELTS does not provide a feedback/feed forward option. It would be unrealistic in our view to embark on rewriting our internal criteria which markers use. This would cause significant reliability and validity issues in terms of outward facing IELTS comparability and significant internal marking standardisation issues. Proposal I propose we commit to preparing a student-facing public version to be used for feedback / feed forward purposes for both speaking and writing. Handbooks would include both sets and students would be told how they fit together. Or We commit to providing existing criteria in association with precise, student-facing feedback/ forward documents, see appendix 5 Academic Modules The number of academic modules cited in section A is significant. These cut across 4 programme areas. Foundation Programmes (broadly NQF 3) should have marking criteria and scales at a different level to International Diploma in Business (NQF 4). A number of programme colleagues (PMs and APMs) have consulted with University colleagues in Schools when updating / creating marking criteria. From these liaisons there does appear variation in actual criteria used in the University even within Schools between degree programmes.

4 There is variation between criteria in, for example, Introduction to Business and Sociology appendix 3 and 4 respectively. This involves consistency and language simplicity. There is variation between the mark schemes for Physics Lab Reports and Biology essays. This involves consistency and language. Request for clarification: 1. We would like to commit to using more student-facing language where appropriate, especially for criteria which are used to support assessment feedback / feed-forward? 2. We would seem to retain inter-module variation in criteria structure and content, to reflect subject and School-relevance variation? 3. We would assume that Diploma criteria should be different to Foundation criteria based on the NQF level difference. Chris Heady Director Studies English/EAP INTO Newcastle University October 2 nd 2014

5

6 appendix 1 INTO Newcastle University - Writing Assessment Criteria 2014 JOINT- MARKING EAP CRITERIA (FDN BUS/HUM/DIP) Grade TASK (50%) LANGUAGE (50%) (+ CEF/ IELTS approx. equivalent) overall communication and task fulfilment thematic development (organisation and coherence) incorporation of sources (where applicable) vocabulary grammar & cohesion; writing conventions 80 and above CEF C2 / IELTS 8+ the overall message is well focused, clear and fluent an outstanding example of the genre the requirements of the task (including incorporation of feedback comments if relevant) are fully covered and developed beyond normal expectations excellent organisation at whole-text and paragraph level; significant ideas easily identifiable links between ideas very clear and logical ideas very well developed: clarified, with evidence, examples or explanation; sophisticated elaboration in well- constructed paragraphs full and appropriate use of a variety of organisational structures at whole-text and paragraph level extremely proficient control of in-text citation and bibliographical referencing skills, completely adhering to academic conventions excellent paraphrases and summaries; highly competent representation of original author s meaning writer has integrated other source material seamlessly with her/his style and syntax proficient, broad lexical range fluent, consistently correct and appropriate use of vocabulary including: collocation, accurate low frequency items and idiomatic expressions errors are unusual excellent control over word formation and spelling shows excellent command of grammar with a wide range of appropriate grammatical structures and complex language consistently maintains a very high degree of grammatical accuracy, variety and complexity errors are unusual overall excellent control of cohesive features writing / presentation conventions and punctuation are used accurately

7 70-79 CEF C1 / IELTS the overall message is focused, clear and fluent a very good example of the genre the requirements of the task (including incorporation of feedback comments if relevant) are fully covered coherent and logical organisation at whole text and paragraph level; significant ideas identifiable links between ideas clear and logical ideas well-developed: clarified with evidence, examples or explanation; good elaboration in well- constructed paragraphs well- controlled use of organisational structures at whole-text and paragraph level mostly proficient control of intext citation and bibliographical referencing skills, adhering to academic conventions very good paraphrases and summaries; author s original meaning well-conveyed writer has integrated other source material well with her/his style and syntax broad lexical range fluent, mostly correct and appropriate use of vocabulary including: collocation, accurate low frequency items and idiomatic expressions though with some gaps no significant errors; but occasional minor slips good control over word formation and spelling shows good command of grammar with a range of appropriate grammatical structures and complex language usually maintains a high degree of grammatical accuracy, variety and complexity errors are occasional overall good control of cohesive features writing / presentation conventions and punctuation are generally used accurately CEF B2.2/ IELTS the overall message is mostly focused, clear and fluent a good example of the genre the requirements of the task (including incorporation of feedback comments if relevant) are satisfactorily covered; some elements may be dealt with better than others coherent and logical organisation at whole text and paragraph level; though occasional lapses are possible; most significant ideas are identifiable links between ideas mostly clear and logical ideas adequately developed; supported with some evidence, examples or details; however, there may be occasional lack of focus controlled use of organisational structures at whole text and paragraph level generally proficient control of in-text citation and bibliographical referencing skills, usually adhering to academic conventions; some minor features may be inaccurate or missing good attempts to paraphrase and summarise; original author s meaning for the most part wellconveyed; some evidence of ability to integrate other source material with own style and syntax good lexical range for the task appropriate use of vocabulary but lexical/collocation/idiomatic gaps sometimes obvious errors only cause slight strain; some confusion and incorrect word choice may occur, especially in low frequency items occasional spelling and word formation lapses do not inhibit intelligibility shows a range of appropriate grammatical structures and some complex language maintains a good degree of grammatical accuracy, variety and complexity errors and flaws in sentence structure do not usually impede intelligibility adequate but not consistent control of cohesive features writing / presentation conventions and punctuation are mostly used accurately

8 50-59 CEF B / IELTS the overall message is identifiable but not always clear, focussed or fluent some features of the genre present but others omitted or unclear the requirements of the task (including incorporation of feedback comments if relevant) are mainly but not fully met evidence of organisation at whole-text and paragraph level, though some parts may lack coherence; significant ideas may lack a clearly defined focus links between ideas are not always clear and logical ideas not always sufficiently well-developed and clarified; most ideas usually supported with reasons, evidence, examples or details inconsistent use of organisational structures at whole text and paragraph level Reliance on sources (including use of quotation) at expense of author s own writing to partially affect ability to assess writing inconsistent control of in-text citation and bibliographical referencing skills, not always adhering to academic conventions; important features may be inaccurate or missing some attempts to paraphrase and summarise; but insufficient changes have been made to original lexis and structure a little evidence of ability to integrate other source material with own style and syntax lexical range limited to intermediate, higher frequency vocabulary; vocabulary may not be adequate for an academic task restricted use of vocabulary: noticeable inaccuracy and inappropriacy when dealing with complex ideas or unfamiliar topics errors occasionally cause strain; some confusion and incorrect word choice may occur, especially in low frequency items; there may be collocation errors frequent word formation and spelling errors occasionally inhibit intelligibility shows some command of grammar but with a restricted range of grammatical structures and little complex language maintains a reasonable degree of grammatical accuracy and variety errors occur, especially in more complex structures; and some may impede intelligibility evidence of cohesive features though some problems may be evident. writing / presentation conventions and punctuation may not be fully controlled CEF A2.2- B1.1 / IELTS the overall message may be unclear in places or lack focus some attempt at the features of the genre but mostly ineffective the requirements of the task (including incorporation of feedback comments if relevant) are minimally met evidence of organisation at whole-text and paragraph level, though many parts may lack coherence; significant ideas may not be easily identifiable or lack a clearly defined focus links between ideas are unclear and not always logical ideas may be poorly argued; ideas not adequately supported with reasons, evidence, examples or details rare use of organisational structures at whole text and paragraph level Reliance on sources (including use of quotation) at expense of author s own writing to affect to significantly affect ability to assess writing very little control of in-text citation and bibliographical referencing skills, does not adhere to academic conventions; important features are inaccurate or missing little or no attempt to paraphrase and summarise; minimal changes have been made to original lexis and structure; ability to integrate other source material with own style and syntax is rarely evident evidence of poor academic practice or plagiarism lexical range limited to low intermediate/ preintermediate, higher frequency vocabulary; vocabulary may not be adequate for an academic task very restricted use of vocabulary: noticeable inaccuracy and inappropriacy when dealing with general ideas or unfamiliar topics errors often cause strain; confusion and incorrect word choice occur in high and low frequency items; noticeable collocation errors frequent word formation and spelling errors often inhibit intelligibility some accurate use of a restricted range of grammatical structures; mainly simple sentences intrusive errors occur frequently use of a limited range of basic cohesive features limited awareness and use of basic writing / presentation conventions and punctuation

9 30-39 CEF A1.2- A2.1 / IELTS the overall message is hard to follow little or no attempt at genre the task requirements (including incorporation of feedback comments if relevant) are not met organisation at whole-text and paragraph level hindered by inadequate linguistic resources ideas are unclear and not logical ideas often undeveloped; ideas may be inappropriately and/or insufficiently supported or connected use of organisational structures lacking Reliance on sources (including use of quotation) at expense of author s own writing to completely affect ability to assess writing; OR no sources are used little or no control of in-text citation and bibliographical referencing skills, does not adhere to academic conventions; important features are inaccurate or missing; or no in-text citation and/or bibliographic referencing is included extremely limited or no attempt to paraphrase and summarise; very few or no changes have been made to original lexis / structure; ability to integrate other source material with own style and syntax is not evident at all lexical range limited to low pre-intermediate/ elementary, high frequency vocabulary; vocabulary not be adequate for an academic task very restricted use of vocabulary: sufficient for the expression only of basic communicative needs errors cause strain; confusion and incorrect word choice occur in high items; noticeable collocation errors word formation and spelling errors may seriously inhibit intelligibility use of some basic simple structures systematic basic mistakes impede intelligibility use of only the most frequently occurring cohesive devices to link simple sentences extremely limited awareness and use of basic writing / presentation conventions and punctuation

10 appendix 2 Cross-FLTSEC 16 October 2014

11 Appendix 3 - Marking Criteria for Academic Assignments Overall description Mark range Excellent Very good Good Pass Fail Why are you in this band? content / structure / references An outstanding piece of work throughout with excellent analysis, synthesis and evaluation of material and concise, logical thought. Where appropriate, work shows originality and critical ability. Demonstrates comprehensive understanding of topic with evidence of substantial additional study and with virtually no errors. Extremely well presented and structured work. Could not be bettered at this Stage in the time available. Outstanding in most elements but minor deficiencies in some, compensated by excellence in others. Extremely well presented and structured work. Work overall excellent with respect to synthesis, originality, critical ability and logical argument. Thorough understanding of the topic and evidence of significant additional study, although may contain minor errors. Extremely well presented and structured work. Shows thorough understanding of topic. Substantial detail supported by reasoned argument, application and critical analysis, with evidence of further study. Very well presented and structured work. Work provides substantial information that addresses the aims and objectives of the module/topic. May contain minor errors of understanding. Some evidence of additional study. Very well presented and structured work. Work indicates understanding of the topic, largely factually correct, but lacking in critical analysis and in evidence of further study. May contain significant errors of understanding compensated by very good work in other areas. Well-presented and structured. Work that is relevant to the module/topic aims and objectives but not a full treatment. Relies almost entirely on course material and may contain significant errors of understanding and some errors of fact. Generally well presented and adequately structured. Limited but acceptable understanding of material. Omission of much relevant material and/or use of irrelevant material. May contain significant errors of understanding and fact. Adequately structured and presented. Barely acceptable with limited grasp of material. Significant omissions, errors of understanding and factual errors. Generally poorly presented and structured. Borderline fail. Demonstrates minimum acceptable understanding in some though not all areas. Many factual errors and omissions. Generally poorly presented and structured. Some material of relevance, but generally irrelevant approach and failure to understand basic requirements of module/topic. Significant factual errors and omissions. Little or no structure and poorly presented. Limited work showing an inability to deal with the requirements of the module/topic. Some factually relevant material. Extremely limited work with very little factually relevant material. Fail. Little or no attempt to complete the work.

12 INTO Newcastle University Foundation Programme - Marking framework and feedback form* (*Adapted from HEFC grading guidelines and Biggs and Collis taxonomy) Distinction (Excellent) 70%+ - An excellent piece of work which provides evidence of extensive independent research, synthesis and analysis - Comprehensive understanding of the subject is demonstrated & links are made with wider, relevant aspects of the subject - Knowledge is confidently communicated & critically discussed - A coherent & focused argument is sustained throughout the work Credit (Very good) 60-69% - Extended knowledge and understanding of the subject & key terms and concepts are demonstrated. - Key terms & concepts are used accurately & with confidence - There is evidence of independent research carried out from a range of academically credible sources, outside of the classroom notes & directed reading. - A clear line of thought - A balanced argument is provided which also demonstrates understanding of differing perspectives and their relationship to each other - Work is consistently well & highly focussed to answer the given question/title Merit (Good) 50-59% - A wider knowledge and understanding of the subject, key terms and concepts through definitions, examples & discussions of the terms, concepts & arguments. Relationships between information are not fully explored - Basic elements of theoretical perspectives are included in the work in support of discussions & arguments - Work is more coherently structured with transition paragraphs & academic signpost language which guides the reader through the various parts of the work. - Work is largely descriptive. However, there are attempts made to undertake basic analyses. - Work is clearly presented & Organised with front page, pagination, double-line spacing and appendices (where appropriate) and main text is also well presented. - Oral: audible & confident delivery. Answers are well developed & demonstrate understanding of the subject - Where appropriate, images, tables & graphs are well-labelled - Student demonstrates a good understanding of the use of the referencing system in-text & the bibliography - There is evidence of research having been undertaken from a variety of relevant academic sources. Pass 40-49% - A basic level of knowledge & understanding of one aspect of the task or subject is demonstrated. - Evidence of key terms & concepts are understood, but not fully explained or discussed in the student s own words. - Facts are presented logically & coherently with a basic descriptive commentary - Accurate answers are provided. However, they lack depth through examples & further discussion concerning their relevance to the question. - Satisfactory organisation with front page, pagination & double-line spacing. However, within the main text there are inaccuracies in the presentation of the work. - Oral: audible & clearly communicated with appropriate body language & good eye contact. Questions invited and basic responses - Student demonstrates some ability to reference work. However, there are a range of inaccuracies in the use of the referencing system in-text & the bibliography. - Evidence of some background research using minimal resources.

13 Fail <39% Grade and criteria - Student demonstrates poor knowledge & understanding of the subject - Key terms & concepts are not correctly used or defined. - The work is not organised with clear signpost vocabulary or paragraphs - The work is an assembly of often irrelevant information - Work is not organised or presented coherently - There is no front page, pagination or double-line spacing - Oral: inaudible & not clearly communicated with inappropriate body language & lack of eye contact. Questions not invited or unsuitable responses provided. Content and discussion Structural cohesion Presentation Referencing - Poor, or no, research evident - Very little, or no evidence of use of referencing system - Issues concerning plagiarism Feedback comments for: Overall mark: Sign: Date: Sign: Date:

14 Appendix 5 example of criteria + feedback comment sheet Foundation Pathways Student Speaking Coursework Assessment Feedback sheet Student: Date: Overall final mark: On this sheet you will find the final mark for this assessment (10%) of your overall Semester 2 EAP mark. You can see how you have achieved this overall mark from the three sections in the criteria: Spoken Discourse, Grammar and Vocabulary, and Pronunciation. Your assessor has highlighted/ circled the appropriate criteria for each section and has ticked some suggestions for improvement. SPOKEN DISCOURSE GRAMMAR & VOCABULARY PRONUNCIATION >80 CEF C2 Speaks naturally at length about the topic without any obvious effort or any strain on the listener. Fully coherent & highly fluent speech. Manages interaction extremely skilfully (questions, planned engagement) A wide range of grammatical structures with highly efficient use of complex sentences. Use & choice of vocabulary is consistently appropriate to academic context, function & intention. Virtually error free Slight foreign accent; does not affect intelligibility Natural and effective use of stress and intonation CEF C1 Can speak at length about the topic with very little strain on the listener. Coherent & very fluent speech. Coherence only rarely lost in complex sentences, stretches of speech. Manages interaction very skilfully (questions, planned engagement) A very good range of grammatical structures with efficient use of complex sentences. Use & choice of vocabulary is appropriate to academic context, function & intention. Occasional minor errors may occur but do not impede communication. Shows generally appropriate & effective use of stress & intonation. Minor errors in pronunciation rarely impede communication.

15 60-69 CEF B2.2 Can speak at length about the topic with only occasional strain on the listener. Coherent & fluent speech on most topics. Some occasional hesitation when dealing with complex sentences or unfamiliar topics. Manages interaction skilfully (questions, planned engagement) A good range of grammatical structures with generally efficient use of complex sentences. Use & choice of vocabulary is mainly appropriate to academic context, function & intention & adequate for discussing issues at length & making meaning clear. Some minor errors in vocabulary & complex sentences occasionally occur; however, communication is rarely affected. A moderate range of grammatical structures with some use of complex sentences. Most use & choice of vocabulary is appropriate to academic context, function & intention & mostly adequate for discussing issues at length & making meaning clear. Errors in simple sentences do not usually impede communication. Frequent errors in vocabulary sometimes cause strain for the listener. Errors made in complex sentences may affect communication. May at times over rely on memorised chunks. A limited range of grammatical structures: mostly simple & occasional compound sentences. Vocabulary selection & use is generally sufficient to deal with a limited range of issues & ideas. Frequent grammatical & vocabulary errors in simple & complex sentences. Errors may regularly impede communication. May often over rely on memorised chunks. Stress & intonation generally convey meaning well. Mispronunciation of individual words may occasionally cause strain for the listener CEF B CEF A2.2- B1.1 Can speak about the topic but with occasional strain on the listener. Generally coherent in general & familiar topics. Coherence occasionally lost, mainly when dealing with unfamiliar topics. Some loss of fluency, unnatural delays or hesitation while organising thoughts/searching for language. Manages most interaction skilfully (questions, planned engagement) Can speak about the topic but with some difficulty. Mostly coherent in general & familiar topics. Frequent loss of fluency, unnatural delays or hesitation while organising thoughts/searching for language. Loss of coherence may cause regular strain for the listener. Has severe problems managing interaction (questions, planned engagement) Problems with stress, rhythm & intonation but rarely affect intelligibility. Mispronunciation of individual words may often cause strain for the listener Problems with stress, rhythm & intonation may regularly affect intelligibility. Mispronunciation of individual words may regularly cause strain for the listener CEF A1.2- A2.1 Can speak about the topic but with difficulty. Rarely coherent in general & familiar topics. Lack of coherence causes frequent strain on the listener. Unnatural hesitations & long pauses cause frequent breakdowns in communication/fluency. Cannot manage any interaction Very limited range of structures: mostly repetitive simple sentences. Breakdown in cohesion is frequent. Vocabulary selection enough to deal with only simple information. Vocabulary not adequate for the task. Vocabulary & grammatical errors in most utterances. Errors impede communication frequently. Problems with stress, rhythm & intonation frequently affect intelligibility. Mispronunciation of individual words may cause significant strain for the listener

16 How can you improve your presentation speaking skills in the future? We have highlighted some key areas: Spoken Discourse Maintain a logical flow to your presentation Link the different sections together effectively Achieve an appropriate balance between key points and detail Provide definitions of specialist terms Use an effective strategy to deal with questions/ bring the presentation time to an end Grammar and Vocabulary Go over the grammar of functional language phrases useful in a presentation e.g. Let s move on to Use an appropriate style of language Use accurate language Use a range of grammar structures: simple to complex Make notes or prompts to help you remember important vocabulary or grammar structures Pronunciation Speak more English in and out of class- the more you speak the more confident you will feel Use your voice to emphasise points- this will keep your audience interested Practice the pronunciation of important words and phrases Don t speak too softly or loudly- make sure that your audience can follow you Don t speak too quickly or too slowly- make sure that your audience are following your points A good place to start working on your presentation skills is and for pronunciation. We also recommend going to the Language Resource Centre to use their extensive English Language materials.