Report on the functioning of the AMEE committees John Cookson MD FRCP Emeritus Professor, Hull York Medical School, UK

Report on the functioning of the AMEE committees John Cookson MD FRCP Emeritus Professor, Hull York Medical School, UK Provenance; first submitted draft 4.12.14 Draft v. 2 submitted 8.12.14 Draft v.3 with additional PG material submitted 24.12.14 Draft v.4 with additional elearning material 14.01.15 1

Introduction and scope I was asked by Professor Ronald Harden at the end of August 2014 to prepare a report for the Executive Committee of AMEE on the functioning of the four committees set up in 2011. They are: E-Learning Committee Postgraduate Committee Research Committee Simulation Committee My remit was set out in an email from Prof. Harden on 9.9.14 where he asked me to try and answer seven questions: 1. Are the AMEE committees achieving the specified aims? 2. Should the initiative and concept of AMEE committees be continued and further developed or should it be abandoned? 3. If in principle AMEE committees are to be continued should the four current committees continue and should any further committees be added? 4. Given the experience to date what should be the brief for and work expected of a committee? 5. What should be the membership of the committee and how should members and the chair be appointed and for how long? 6. How should AMEE committees be represented on or report to the AMEE Executive committee? 7. What central support should the AMEE office provide to the committees? Mode of working Documents held by the AMEE office were scanned particularly for information relating to 1, 4 and 5. Chairs and secretaries of each committee were then approached by email to explain the project (Appendix A). A questionnaire, tailored to each committee, was attached which explored in more detail issues arising from the documents and also addressed particularly 2, 3, 6 and 7 (Appendix B). Follow up email correspondence sometimes followed. The results are set out under each of the seven headings above. Subsections relating to each committee separately are used where appropriate. Activity relating to the annual conference has been expressed largely in relation to that of 2014 as reflecting the most up to date position. Remit of the Committees The remit of the Committees as stated in Paper 23 for AMEE Executive Committee February 2013 is: to contribute to AMEE with regard to their field of interest on matters relating to: o AMEE s annual conference including the content and speakers, the selection of short communications, posters and workshops for inclusion in the programme and the recruitment of chairpersons and opening discussants for the short communication and poster sessions. It is proposed for the 2012 AMEE conference that the papers submitted relating to the Committee s area of responsibility are identified and passed to the Committee for a recommendation as to which should be included in the slots assigned to the topic in the programme. 2

o Projects, initiatives, collaborations and other AMEE activities that contribute to professionalism and scholarship in medical education and to AMEE s mission of promoting excellence in teaching and learning in the healthcare professions. This will include liaison with specialist bodies and meetings with an education component devoted to the topic. o Engagement with the wide range of stakeholders and promoting and publicising AMEE s activities in the area. o The development of resources to support the practising teacher including AMEE guides and other materials. o Reports on AMEE s activities in the area including the AMEE website, newsletters to members and reports to the General Assembly. o Contributions to MedEdWorld to assist the teacher in the healthcare professions to keep up to date with developments in the area. It is proposed that each committee will have about eight members. Question 1 - Are the AMEE committees achieving the specified aims? It should be noted at the start that it is difficult to separate out the activity of committees from that of their members. At least one chair was uncertain whether she was acting as an individual or as a committee member. There is therefore some uncertainty about what activity would remain if the committees did not exist. elearning In relation to AMEE s annual conference, members have assessed abstracts on elearning and have made recommendations. There have been symposia and workshops. A member, on behalf of the Committee, has prepared two reports (2013, 2014) for the AMEE Executive on the use of electronic media at conferences. Support has been provided for ESME conference sessions on line The open meeting attracted 31 delegates in 2014. E-Poster sessions at the AMEE conference has been a focus of activity since 2012. Three successful sessions have now taken place. Selection of posters is by members of the committee using a process of moderation. A prize is given for the best poster and awarded at the end of the conference. However a replacement for the technology supplier used at Milan may be sought to achieve greater interactivity. A Hakathon was arranged for students and junior doctors in Milan 2014 Contributions by way of projects, initiatives, collaborations have so far been limited to the annual conference. Engagement with the wide range of stakeholders is at an early stage but the Hakathon is a notable example. The success of this format for the future will need to be tested in Glasgow The development of resources to support the practising teacher is not yet a priority. Members have not so far contributed significantly to AMEE Guides. Contribution to reports on AMEE s activities and MedEdWorld are limited to two SIGs on the MedEdWorld platform, one for committee members and one, e-learning with the e- learning committee, for other interested persons. As of 16.9.14 the first had 5 posts and the second 4 posts. An elearning blog on MedEdWorld is planned. Simulation In relation to AMEE s annual conference, members have assessed abstracts on simulation and made recommendations for conferences. There have been symposia and workshops 3

some of which have been oversubscribed. The symposium in 2014 attracted about 200 people and the open committee about 60. Contributions by way of projects, initiatives, collaborations have been limited to the annual conference. Engagement with the wide range of stakeholders is at an early stage but there have been links with practitioners in other disciplines. The development of resources to support the practising teacher is not yet a priority. Members have not so far contributed significantly to AMEE Guides. Contribution to reports on AMEE s activities and MedEdWorld are described as minimal. There is a SIG for the committee but there were no posts as of 16.9.14. The annual reports have been prepared out with the committee. Postgraduate The Postgraduate Committee has constructed a series of Aims somewhat different to those originally proposed for these committees as given above. They are: Raise the profile of PGME Co-operate with other committees on research and innovative developments at the AMEE conference Propose symposia/workshops that focus on PGME Assist countries where PGME is underdeveloped At AMEE conferences, the chair and some members have assessed abstracts for AMEE conference and chaired sessions and been discussants. The committee has organised and run symposia and workshops at recent conferences and plan to run two workshops and one symposium at each annual conference in the future. A prize in being considered for the best paper. The open meeting in 2014 attracted 20 people Projects, initiatives, collaborations; a recent symposium was a collaboration with the research committee In the development of resources to support the practising teacher, the committee have not yet contributed to any AMEE guides Engagement with the wide range of stakeholders has proved difficult beyond an advisory group made up of those who attend the open meeting and who receive mailings. In particular engagement with junior doctors is difficult are they are rarely supported in attending AMEE conferences. Reports on AMEE s activities have been limited to the Annual Report. There is a SIG for the committee but it had no posts as of 16.9.14 So far there have been no contributions to MedEdWorld Research The committee has been very active in improving the processes for research submission to AMEE s annual conference. There is a Research Stream Network sub-committee with five members with rules for length of service. It oversees the process of review of about 100 submissions each year with clear deadlines and structured, written feedback. Sessions and chairpersons are identified. The committee believes that as a result, acceptance of a paper in the research stream is regarded as prestigious. The committee is considering awards for the best papers. The committee members also offer one-to-one poster mentoring sessions but note that the processes for these could be improved. There are pre-conference and inconference workshops. About 20 attended the 2014 open meeting. Activity in the development of projects, initiatives, collaborations is evidenced by a significant grant programme totalling 20,000/year with around 60 applications from across the globe each year. 4

There are clear criteria for the type of study that will be supported and for what the funds can be used. The process has an application form with clear criteria which is reviewed by a sub-committee consisting of members of the Research Committee, AMEE Executive committee and others. It is a two stage process requiring a pre-proposal and a full proposal. Activity is summarised below year pre-proposal full proposal awarded amounts 2012 114 6 2 10K 10K 2013 62 4 2 8.6K 9.4K 2014 Successful applicants need to provide annual reports on progress and present the results of the research at an AMEE conference. Engagement with the wide range of stakeholders is through the wide membership and the contacts with many researchers who submit their work. In addition two SIGs have been set up on MedEdWorld one on Research Interests (2 posts as of 23.9.14) and one for the Research Committee (6 posts as of 23.9.14) The committee offers Dialogues in Medical Education (DIME), a web-based discussion forum, to assist the development of resources to support the practising teacher. Six have been offered to July 2014 and the programme is continuing into 2015. There have been difficulties with the platform for this which has now been changed. Committee members are active in writing and reviewing the AMEE Research Guides, which now number 8. Two members are writing a textbook on research in medical education. Reports on AMEE s activities are through the Annual Report. Contributions to MedEdWorld are through DIME (above) and the SIG. A Webinar in Dec 2014 on research in postgraduate education has been re-scheduled. Commentary In general the committees seem to be well engaged with AMEE s annual conference, indeed for some it is their sole activity. Although it is difficult to quantify activity, particularly in distinguishing between the activity of individuals acting alone or as members of a committee, all have assessed abstracts, made recommendations and organised symposia and workshops. Research has been particularly active with explicit procedures acting through a sub-committee and in providing structured written feedback to authors. Activity out with the annual conference is very limited with the exception of Research. For this committee the need to organise the disbursement of research funds provides a clear focus and discipline for members; a process with which most are already familiar. The Specialty Interest Groups (SIGs) set up on MedEdWorld although a potentially interesting development do not seem to be attracting much activity. Only Research seems to have contributed to the Webinar series. Although individuals have contributed to AMEE guides, these are not a committee responsibility. This lack of activity may reflect an uncertainty in the committees about purpose which may in turn reflect an uncertainty in the organisation. Some examples are given under Question 4. Recommendations That it is made clear which delegated activities are the responsibility of individuals and which of the whole committee 5

That it is made clear that the responsibilities lie with the whole membership, not just the chair and secretary. That AMEE articulates a clearer vision of what is intended as its wider activities beyond the annual meeting Question2 Should the initiative and concept of AMEE committees be continued and further developed or should it be abandoned? This question was addressed by means of a SWOT analysis. Responses below are a summary: Strengths The committees have wide international membership expertise with many global leaders in their fields (Research & Simulation); the annual meeting is a focus for activity and robust committee procedures ensure sustainability (Research). Elearning feel in control of their contributions to the annual conference. Weaknesses Too much falls on the chair to deliver, perhaps because the process is not mature. Discussion can take place but no action follows. A secretary and vice-chair are important. Many initiatives have come from the centre rather than the committee (PG). There is a lack of clarity about the mandate and governance is weak (Simulation) Lack of administrative support and teleconferencing facilities (Research & Simulation) Opportunities Specific portfolios of responsibility for members would promote accountability. There should be more opportunity for committees to suggest chairs, opening discussants and markers of abstracts. Committees (or at least the chair) should be involved in decisions about workshops relevant to their expertise (PG). Key Performance Indicators may help (Simulation) Improved administrative support and teleconferencing facilities, particularly relating to MedEdWorld; better engagement with other committees; research mentoring between annual meetings (Research) AMEE conferences have a much broader attendance than Simulation meetings so showcasing should be a priority (Simulation) Threats Inertia of members (PG) probably related to multiple commitments of members and the reluctance of employing organisations to value a contribution to AMEE (Research, Elearning & Simulation). Funding (Research). There is a European simulation group (planning a journal) which may compete (Simulation). Commentary The size and reach of AMEE must be a major strength which should be exploited to the full. However the individuals who would drive this are all very busy in their own institutions and those in turn have other priorities. Further, as described earlier, the mandate is not always clear, although this varies between committees, and administrative problems exist. Despite problems there is no suggestion that the system should be abandoned. Recommendations That the major international strengths of AMEE should be recognised and built upon. 6

That delegation of responsibility for sections of the annual conference should be more explicit with clearer accountability. Since there is an element of risk in this, clear governance frameworks need to be in place and appropriate resource provided. That AMEE articulates a clearer vision of what is intended as its wider activities beyond the annual meeting-see above. Question 3 - If in principle AMEE committees are to be continued should the four current committees continue and should any further committees be added? Simulation believe that the committee should continue and with better structures should be more productive. Both Simulation and Elearning believe that they should not merge with the other. A CME group has been formed within Postgraduate but it should be separate in the future. Perhaps there should be a research stream in each committee (Research) Other suggestions to consider are selection into medicine and a history of medical education groups. The formation of short-term task and finish groups should be considered by AMEE when appropriate. Commentary Overall there was little appetite to either increase or decrease the number of committees. The suggestion for a research stream in each committee is interesting. However it might lead to a lack of focus for all committees. More co-operation between committees might be a way forward. Recommendation That consideration is given to the advantages and disadvantages of a research stream in each committee. Question 4 - Given the experience to date what should be the brief for and work expected of a committee? Committees were asked to consider this under two headings, conference-related activity and other activity. They were asked to consider if the Remit was still fit for purpose, whether it should be changed and whether the expectations in it were realistic given the constraints. They were also asked if the expectations should go further to engage more with health care providers, resourcepoor areas and the rest of the tertiary education sector. Research consider that they have a wide spread of activities, that the Remit is fit for purpose but find the IT aspects of MedEdWorld a constraint. The grants programme could better target resource-poor areas. Activities of Simulation have been confined to the conference and that the wider aspects of the Remit, though fit for purpose, have not been achieved but may be in the future. Further development should await better achievement of core aspects. Elearning take a similar view, that most activity is related to the conference because that is the main activity of AMEE. Further they cannot do everything all the time; otherwise multiprofessional learning would be an obvious activity. If the Remit is to be revised the committees should have a hand in doing so. Postgraduate also consider that the Remit is fit for purpose but that their main activity is confined to the conference largely because of the lack of resources. They would like to do 7

more particularly in resource-poor countries and in encouraging applicants to medicine from diverse backgrounds. Commentary Most thought that the Remit remained fit for purpose. This contrasts with the actual lack of activity (with the exception of Research) in those aspects outside the annual conference. It was notable that there were few responses to the triggers in the questionnaire about the needs of those in resourcepoor areas, dialogue with health service providers and engagement with the rest of the tertiary education sector. Most thought that they could do better in areas outside the annual conference with time. Recommendation That the Remit remains unchanged at present. If however committees continue to struggle with activity other than that related to the annual conference, despite a clearer mandate and better governance described above, then they should, with the exception of Research, confine their activities to the annual conference. Question 5 - What should be the membership of the committee and how should members and the chair be appointed and for how long? The membership numbers of each committee (Executive Committee 14-16 February 2014-paper 33) was: Committee Number of members elearning 9 Postgraduate Committee 12 Research Committee 18 Simulation Committee 13 Other documents suggest different numbers And the country of origin of members was: Country Number Country Number Country Number USA 11 Singapore 2 Brazil 1 UK 7 Israel 2 Japan 1 Canada 4 Finland 1 Germany 1 Netherlands 4 S Korea 1 Belgium 1 Australia 4 Norway 1 Sweden 1 Denmark 4 UAE 1 In the interval 2011-2014, 9 persons seem to have joined the committees. None appear to have left. Elearning has not expanded and is considering that membership should be for 3 years with the possibility of renewal. They note that the field of potential members is not large. The chair should be appointed for 3+3+1 years with an extra year to assist the new chair. The international spread is wide and represents the spread of expertise. The committee meets 1-2 times a month via an on-line conferencing tool and uses email and summaries of discussions rather than agendas and minutes. Postgraduate recognise the need for wide representation. The term of office is 3 years renewable once for a further 3 but only one-third should demit annually to maintain continuity. Failure to attend twice leads to demitting. The chair is in position for 6 years. There is a secretary and vice- 8

chair. There is a trainee representative. The nomination process is planned to have a nomination committee and an application form. Agendas and minutes are kept. Research have expanded their membership significantly because of increased activity and the need to spread the load. Selection is mostly based on research expertise with geography a secondary consideration. Members serve for a 3 year term, renewable once for 3 more years. From now on potential new members will go through a nominations committee. Potential officers are discussed with the Executive Committee. Officers (chair, vice chair, secretary) are 2 year terms with one renewable cycle (up to 4 yrs. total).there is a calendar of committee activities month by month. There are sub-committees for grants, research papers, PhD reports, DIME and Guides and explicit roles for members. The main committee meets monthly and agendas and minutes are kept. Simulation have a chair and co-chair each appointed for three years. Otherwise membership and terms of reference are not well developed as yet but are under review. Agendas and minutes are not available. There have been some suggestions for a student representative but this has not been carried through Commentary The number of members seems to have comfortably exceeded that recommended in the Remit; however this may be in response to workload and the need for wide expertise. Committees are beginning to have similar rules for appointment of members and chairs. Working practices vary considerably which may in part reflect the nature of their work; nevertheless for some the lack of formal agendas and minutes may reduce their effectiveness both for the members themselves and for the wider dissemination of their activity within AMEE. Although members should be appointed primarily for their expertise in the field, a number of European countries remain unrepresented. This may impact on their ability to engage widely with stakeholders. Recommendations That committees set their membership numbers broadly as they see fit. That clear rules for selection of and duration of office of members and chairs be further developed and applied. That the strengths and weaknesses of student/trainee representatives be considered That consideration be given to whether a reasonably uniform system of agendas and minutes is required. That, while the requisite expertise is paramount, consideration be given to ensuring, across Question 6 - How should AMEE committees be represented on or report to the AMEE Executive committee? All agree that good communication with AMEE Executive is vital. Simulation believe that their current liaison with a committee member works well but also, with Elearning, Postgraduate and Research generally support an ex-officio arrangement probably filled by the chair of each committee. Some consider that a template for the Annual Report would help but others that the current arrangements are just fine. There is an opinion that the arrangement whereby all the committees meet at the same time in the conference prevents any cross-reactivity between them. 9

Recommendations That consideration be given to the chairs being ex-officio members of the Executive That consideration be given to re-scheduling the committee meeting at the annual conference Question 7 - What central support should the AMEE office provide to the committees? Elearning, Simulation and Research support greater availability of secretarial support. However details about how this might work are so far lacking. Simulation would find the creation of a database of the wider simulation network helpful. There is a suggestion that Basecamp or something similar could be used as a central repository of documents. Recommendation That secretarial support to the committees is investigated both in what is needed and what can be supplied. Conclusion Overall, the contribution of the committees to the annual meeting seems sound. However there is scope for some committees to play a larger role than at present. This may need greater clarity about the governance arrangements between the central office and the committees so that delegation downwards and responsibilities upwards are clearly understood. Some Standard Operating Procedures may help. Activity outside that related to the annual meeting has largely been confined to Research where the grants programme functions satisfactorily as a path well-travelled by many others. Otherwise, activity so far is muted or absent. The reasons are probably two-fold; lack of resource, particularly of time and an uncertainty about the role. Resources will always be limited but perhaps the uncertainty is a reflection of a wider problem. Of the various roles in the committees Remit above, support to practicing teachers is provided by AMEE Guides and excellence is promoted by Aspire. Neither are the responsibility of the committees. What is left therefore are projects, initiatives, collaborations and other AMEE activities that contribute to professionalism and scholarship and relationships with stakeholders. Perhaps there needs to be more clarity about what these mean in practice before the committees can play a major role. 10

Appendix A; letter to chairs Dear I m writing to you because, as you probably know, I have been asked by the Executive of AMEE to report on the functioning of the four Committees. In particular Ronald Harden asked if I could try and answer 7 questions: Are the AMEE committees achieving the specified aims? Should the initiative and concept of AMEE committees be continued and further developed or should it be abandoned? If in principle AMEE committees are to be continued should the four current committees continue and should any further committees be added? Given the experience to date what should be the brief for and work expected of a committee? What should be the membership of the committee and how should members and the chair be appointed and for how long? How should AMEE committees be represented on or report to the AMEE Executive committee? What central support should the AMEE office provide to the committees? I ve approached the task firstly by going through the documentation held centrally by AMEE and this has been very helpful. I would now like to seek your help in answering a number of questions. Most of these questions are broad and apply to all committees and generally will need a paragraph or two in response. Since many of are about the future direction of AMEE, it may be helpful for members of your committee to be able to reflect on them, so perhaps I could ask you to circulate it at some point. In order to save you digesting large amounts of material from them it might be easier to formulate your own response and then circulate it for comments. Others questions relate to matters of fact which I have either not been able to find in the documentation or which I have not been able to fully understand. These generally relate specifically to your committee and are indicated as such. When I have these replies back my aim is to write and submit a draft report. I will, if I may, come back to you at that point if I need clarification or new issues arise. With best wishes John Cookson Emeritus Professor, Hull York Medical School, UK. 11

Appendix B Questionnaire about the workings and future direction of the AMEE Committees 1. Are the AMEE committees achieving the specified aims? These aims are set out in the Remit, Paper 23 for AMEE Executive Committee February 2013. Extracts in italics are given below. How successful has the committee been in the selection of short communications, posters and workshops and that papers submitted. are identified and passed.. for a recommendation? promoting activities that contribute to professionalism and scholarship? engagement with the wide range of stakeholders? providing resources to support the practising teacher? reporting on activities via the website, newsletters to members and reports contributions to MedEdWorld 2. Should the initiative and concept of AMEE committees be continued and further developed or should it be abandoned? This may be helped by a SWOT analysis Strengths? Weaknesses? Opportunities? Threats? 3. If in principle AMEE committees are to be continued should the four current committees continue and should any further committees be added? There are several possibilities; committees could be disbanded, merged with another, undergo division or new ones created 4. Given the experience to date what should be the brief for and work expected of a committee? An answer will need a review of the Remit for the committees as summarised above. So Is work done in relation to the annual conference the main or only activity? Most of the other activities in the Remit are about issues beyond the conference. So Is the Remit still fit for purpose? If not in what way should it be changed? How many of the activities are realistic given known constraints? Should they go further in encouraging activity beyond the membership? Particularly with those in resource-poor areas? In encouraging dialogue with health service providers? In engaging with the rest of the tertiary education sector? 5. What should be the membership of the committee and how should members and the chair be appointed and for how long? Generally committees are now much larger than the original membership of 8. What have been the drivers behind this? Has your committee been actively considering term of membership, recruitment of new members, appointment and term of office of chair and vice-chair? 12

Do you take account of the geographical spread of members? Can you explain the reasoning behind this decision? 6. How should AMEE committees be represented on or report to the AMEE Executive committee? Currently the arrangements are mixed with two chairs on the Executive by election, one by appointment and in one case by having a member of the Executive on the committee. So Can this arrangement be improved? Would an ex-officio arrangement for each chair on the Executive work better? Is the current Annual Report to the Executive working well? Would a structured report template be an improvement? 7. What central support should the AMEE office provide to the committees? Would the availability of a secretariat help? 8. Are there any other points you would like to make not covered by the above? Questions specific to each committee 13