Study of Job Skills-Set Required of IS Graduates for Work in Instructional Design

Transcription

1 Study of Job Skills-Set Required of IS Graduates for Work in Instructional Design YOLANDA HANNA, VINCENT YAP, KOK WAI FONG, JAVIER FLETCHER, AND CRYSTAL BANCROFT Florida State University USA EXECUTIVE SUMMARY In our study of ID (instructional design) job skills gained by IS graduates, we were concerned with the factors that contribute to success in the field of instructional design and whether we and our fellow classmates were gaining the skills we would need upon graduation. We know the courses in the FSU IS program provide a solid, theoretical foundation on Instructional Design and Technology but we were concerned about what practical skills we might be missing. We embarked on a research study to try to determine answers to that basic question and we came up with five research questions to guide our study and they are: 1. Are the skills and knowledge attained by FSU IS graduates matched with job requirements? 2. What are the main requirements most often listed in ID job announcements? 3. What media and technology skills are most often listed in ID job announcements? 4. What are the top ID skills FSU IS alumni wish they had attained while in the FSU IS program? 5. Do FSU IS courses cover the top skills needed in the ID job market? To answer these questions, we decided on two main data sources, FSU IS alumni and current online ID job announcements. The alumni would be surveyed and the announcements would be analyzed for content. The research survey of FSU IS Masters program graduates, had respondents with graduation years from 1975 through The literature on the topic of competencies for instructional design reflects a field that is constantly changing and growing as an effect of large technological advances in education and training and production of multimedia. Our literature was as old as 1998 with a study from Liu, Jones and Hemstreet called Interactive Multimedia Design and Production Processes. We were very interested in offering knowledge on the topic of competencies and we used technology to perform our research as well. We distributed a simple survey to the FSU IS alumni on the program distribution list and their feedback was automatically fed and collected by a spreadsheet document on Google. Forty-four responses were received over a two week period, with 93% of the respondents offering specific things they wish they had learned while in the FSU IS program. For the job announcement content analysis we created two samples and two methods for documenting the frequency of keywords reflecting job qualifications or skills needed to get the job of an instructional designer. Of a large sample of job announcements, the types of keywords that we searched and tallied included soft skills, skills with media, the names of software packages and anything else we were able to creatively consider. After collecting the data we embarked on an attempt to learn everything we could about the skills needed in the field of instructional design from the job announcements in combination with the online survey responses received from the FSU IS graduates. To analyze the data, we went back and forth from the three data sources looking for answers and themes. The results indicate the categories of skill-sets and knowledge that respondents hope they had learned were project management, technology and business skills, and how to use software

2 tools for creating instruction. Communication, collaboration and project management were the top ranked skills on the job announcements. This was in direct agreement with much of the literature on instructional design. For example, where Liu et al. (1998) state Throughout the creation of a multimedia project, the people responsible for each of the roles within the company must be in constant communication to ensure consistency in the design and focus of the end product. and the project manager had the primary responsibility of negotiating with the client and outlining the objectives, scope, schedule, and funding for the project. Our data showed many FSU IS alumni surprised by the expectation that they manage these aspects of projects. Another finding as described in survey answers, showed instructional designers challenged by and wishing for strategies for working with subject matter experts. This also was seen in literature such as by Liu et al. (1998) where they state The findings suggest that because many clients lack a familiarity with multimedia, the importance of educating the client as to the design, capabilities, and costs involved in multimedia development is critical. We learned that Flash, Captivate, HTML, and PowerPoint were highly ranked in job announcements as well as Microsoft Office products. We learned that there many alumni who felt unprepared as far as being able to use new media and technology. Some FSU IS alumni hate IT (information technology) and others wish they had learned more IT because of its importance in instructional design work. Job announcements and job titles were often very technologically inclined. Our final recommendations and conclusions were that for the most part the FSU IS alumni were extremely satisfied with their preparation and did not have trouble finding employment upon graduation but most mentioned there were practical skills they wish they had attained. This was no surprise in that our data showed more job announcements calling for a bachelor s than a master s degrees and a bachelor s program will generally provide students with more practical skills, than theoretical knowledge. As far as surprises we learned that conducting needs assessment was singled out on most job announcements. Knowledge of developing courseware for adults and knowledge of adult learning theories was a finding from our data that we did not expect but became evident when we considered the consumers of education and training. The main thing we found is that the field is organic, in that it has changed over the years as alumni reflected with us on what they had learned and what they need to know now. Lifelong learning and initiative to get their hands wet (Liu, 1998) and to not be afraid of technology and to please become proficient in analysis, evaluation, technology, communicating and collaboration is what we recommend to aspiring and graduating IS students. Page 2 of 37

3 PURPOSE INTRODUCTION The purpose of this study is to summarize current job requirements for instructional design (ID) jobs and to find if current instructional systems FSU graduates feel the IS program prepared them for work in the field of ID and to share what they feel may be lacking. RATIONALE The rationale for undertaking the study was that to see if there is a mismatch between what is needed and what is gained by graduates preparing for ID work. As a team of five researchers in our last year of graduate studies in the FSU IS program, we wanted to find what the data said. We wanted to find what are the practical skills needed in the real world of ID and how past and new graduates reflect on their education versus the need. The knowledge could lead to a win-win situation for all involved, including employers. We may learn that there are simple things a student can do during their academic preparation to supplement their skills. But in not knowing what skills are needed, we and other FSU IS graduates could be in for a surprise, if upon graduation, we may not be able to find employment in the field for lack of the right skills. It is not only the responsibility of the school to prepare students with practical skills but students must have the initiative to learn what they need to learn. With our findings and recommendations, we hope to help future IS students and teachers understand the ID job requirements. AIMS & OBJECTIVES We aim to understand the qualifications, skills, knowledge, competencies, and success factors being required by hiring managers, in relation to ID jobs, to summarize the requirements for interested stakeholders; graduate students, and universities working to align their curriculum with current job needs. An objective of the research was to discover if hiring managers were confusing the computer science programmer with the instructional designer and, if so, is an instructional designer now expected to know how to use programming languages to create products? Regardless, even if that were the case, a student could not hope to debate with a hiring manager about what a real instructional designer does, but instead she should be prepared to fit the picture the industry has of what an instructional designer does. We want to help ensure graduates have the right credentials upon graduation. RESEARCH QUESTIONS The research questions that guided our research study are: 1. Are the skills and knowledge attained by FSU IS graduates matched with job requirements? 2. What are the main requirements most often listed in ID job announcements? 3. What media and technology skills are most often listed in ID job announcements? 4. What are the top ID skills FSU IS alumni wish they had attained while in the FSU IS program? 5. Do FSU IS courses cover the top skills needed in the ID job market? Page 3 of 37

4 BACKGROUND Our research study looks for the reality of jobs and credentials required in the ID field. Instructional designers need skills with prototyping, storyboarding, creating online courseware. Instructional designers use graphics, video, and audio to create interactive media. In order to create instructional solutions instructional designers communicate and collaborate with clients, subject matter experts and project team members. The instructional designer must have a toolbox from which to choose from to do her job effectively. The instructional designer, in the field today, knows they need to be able to use technology to effectively communicate solutions to clients, and consulting skills to negotiate contracts and solutions. The need and scope of our research and concerns is most vividly highlighted by statements made by Liu et al. (1998) with, Upon graduation, a student majoring in instructional technology accepts a job as an instructional designer for a multimedia company. Yet she finds herself ill-prepared for the job because what she learned in the classroom reflects little of what is actually used in the multimedia field. As Liu et al. (1998) continued, A professor is asked to teach courses on multimedia development and finds that the literature on instructional design provides little insight into the actual design and production process used by multimedia professionals. PARTICIPANTS & DATA SOURCES METHOD To collect the data we needed to support the answers to our research questions, we depended on two main sources. One was the collection of job announcements (JAs) from an online job site where we aimed to identify descriptive information of required skills for ID jobs. We selected an online job site, monster.com, to source our data for Instructional Design skills. Through the process, a total of 306 samples were identified and collected. The search was deliberately done using only one job site in order to eliminate having replicated JAs in our data samples. The second method was from participants of the survey we designed to collect qualitative feedback data. They consisted of IS graduates from FSU and to reach out to them, we decided the best approach was through the IS Alumni Listserv mailing list since the alumni represented the sample population we were targeting. An alternative channel using the Nole s IS Alumni Network was considered but we eventually felt the IS Alumni Listserv would be a better option to assure a better response success rate with its larger membership listing. The current membership of IS Alumni under Listserv has about 1140 graduates compared to the Nole s IS Alumni Connection of 200 membership. DATA COLLECTION Survey In seeking to provide data evidence and validate our data collected through JAs, we designed an online survey using Google Docs to collect the data. Specifically, the survey aimed to address the research question What are the top ID skills FSU-IS alumni wish they had attained while in the FSU-IS program? The survey consisted of six items that sought to gather basic information on FSU IS graduates academic and work background and their perceptions about the value of the FSU IS program in preparing them for their work. The survey questionnaire is in Appendix F. To ensure the survey was effective in soliciting the data we wanted, we deliberately designed it to keep it simple and focused with a minimum number of items. For ease of response, the first 5 items were structured as close-ended items with answer choices provided and served as lead-in questions to the main open-ended, last item. The design proved useful, as it helped respondents to be more reflective and prepared in providing their feedback to the last item, the crux of our Page 4 of 37

5 qualitative data source. Likewise, the open-ended format of the last item enabled respondents to write their responses without space constraints and this proved useful in gathering the in-depth, quality data we were striving to collect. To ensure usability, the online survey was pilot-tested two weeks before the actual administration using a convenience sampling of respondents; FSU classmates from a face-to-face class. The feedback and suggestions from the pilot test were considered and incorporated into the final version of the survey. See Appendix A for details of the pilot-testing feedback and recommendations offered to us. The online survey was sent to FSU IS graduates using the FSU Alumni Listserv with the assistance of Mary Kate McKee, who is the FSU IS Program Administrator. This was done after seeking approval from Dr. Vanessa Dennen, the IS Program Faculty Coordinator. The survey was left opened for over two weeks, from March 16 through March 29th, and a reminder was sent a few days prior to closing the survey. In all, we gathered a total of 47 replies, but 3 were subsequently omitted from the sample as they were from PhD graduates who are not our intended population. With that, we have an estimated response rate of 4% (44/1140) from the survey. We felt this was a rather good response rate based on the two week timeframe for the return, and that participation was voluntary. In addition, the quality of the feedback from the 44 respondents obtained was constructive to our data collection effort. Since the online survey was supported by the Google Documents engine, the survey responses were automatically tracked and managed by the software. This allowed us to monitor the progress during the two week administration period. This was useful and efficient in that we were able to instantly read the data captured in the spreadsheet, as well as create summary bar charts of the data collected. A sample of the data captured on the spreadsheet is shown in Table 1 below: Table 1: Survey Items and Excerpt of Responses Collected by Google Docs 1. Which 2. Which job field are you 3. Is your 4. How many years have you 5. My academic 6. List the top three things you wish you had learned year currently current been working in training in as part of your academic did working in? job IS related job the FSU IS preparation in you related field since program has Instructional Systems at gradu to your graduation? made it easy FSU. ate IS to find from training employment the IS? in the field progr of IS. am in FSU? Performance More than Yes Consulting years Yes 1. Consulting skills, 2. Linking the "language of performance improvement" to the "language of business" 1983 Management Consulting Petroleum Industry No More than 10 years Yes 1. Project management 2. Web-based course design (not available in my day) 3. Multi-media programming 2001 Retail Yes 6-10 years Yes Flash Graphics creation and editing Sound editing Video editing 2004 Human Resource not Technology, technology, Yes directly technology!! Page 5 of 37

6 1. Which year did you gradu ate from the IS progr am in FSU? 2. Which job field are you currently working in? 3. Is your current job related to your IS training? 4. How many years have you been working in IS related job field since graduation? 1991 Education Yes More than 10 years 5. My academic training in the FSU IS program has made it easy to find employment in the field of IS. Yes 6. List the top three things you wish you had learned as part of your academic preparation in Instructional Systems at FSU. Diffusion of innovation and change Leadership and management Interpersonal communication and skills Job Announcements Online Documents We adopted an unobtrusive data collection methodology, where we searched available documents, to find the instructional designer skills-set required for employment in the field. We searched job boards and collected job announcements (JA), analyzed the contents and derived at a list of ID skills that we later ranked in order of frequency. Sample48 We collected a sample of forty-eight job announcements for the manual content analysis. We piloted two methods for collecting the data. Both involved searching for online instructional design announcements and creating a static file, for subsequent reference. One method was a manual search and cutting and pasting, while the other was an automated search with the use of an RSS feeder to aggregate the announcements. During piloting, we found the RSS feed and aggregating tool resulted in many duplicates and we decided not to use it. 1. Of the original job announcement only the qualifications were copied and then fragmented into rows. An excerpt of the end product can be seen in Fig Keywords were created by researchers and put into the 4 th column (E) and 5 th columns (F) over two sessions to rephrase the qualification. 3. A tally was done to indicate the frequency for the skills within the 48 announcements. 4. An X was placed for skills like personality characteristics, bilingual skills, citizenship, security clearance, years of experience and degree requirements. 5. An X was placed when rephrasing the Qualification would lead to repeating a keyword we had already used for the same job on a prior row. 6. Each announcement was stored on Page 6 of 37

7 Figure 1 Conversion of Job Announcement Online Text to Excel Rows Manual Analysis Method of Sample48 During the analysis, each excel row containing a job qualification was read out loud by a researcher and we spoke about how to rephrase the qualification until we had consensus. The job qualification was rephrased into a skill or marked with an X. An X was used for bilingual skills, citizenship, and years of experience. During the pilot, we learned that there was much consistency in how we rephrased qualifications. The manual content analysis only involved the qualification section of the document (or text that resembled a qualification section), cut and pasted from the original JA into rows of an Excel worksheet. Refer to Figure 1 for an excerpt. The 48 jobs were fragmented into 727 rows. Two columns were added to contain a first and second iteration of rephrasing the job requirement. This was done over the course of one month, with team members discussing and coming to consensus of the standardized content. After the analysis, we ended up with over 150 keywords based on sample48. During this process of consolidating keywords, we combined items like Adobe Captivate and Captivate, CBT and Computer Based Training, and others meaning the same thing. Sample258 We collected a sample of 258 JAs for the automated content analysis (sample258). This larger sample size of JAs were copied and pasted into one text file after searching instructional design on monster.com. For the sample size to meet the sampling error of ±5%, we selected 258 job announcements. This was based on the population size of job announcements being equal to 761, which was the total number of JA returns for the search conducted on the job site on March 3, We chose one job site for our selection as we thought an employer might advertise the same job on two different sites but it was unlikely they would list the same job twice on the same job board. We ensured the sample of JAs selected for the automated content analysis were distinct and different from those selected for the manual content analysis by deliberately selecting JAs that were posted on the job board after Feb 6th, 2009, which was the last day when the items were selected for sample48. We accessed the monster job board simultaneously from different locations, during an online chat and found the announcements have a Date Job Posted and are returned in the same order on Page 7 of 37

8 every computer. We concluded that sorting the JAs by Date Job Posted and selecting from different dates would enable us to concurrently collect job announcements from the same job board without selecting duplicates. Two researchers subsequently used the piloted method and collaboratively collected the 258 random samples from the online job board then copied them all into one file for content analysis by an automated method. After collecting the documents and storing them we pulled out the details that we needed from the documents. A pilot was done to confirm the feasibility of both the automated and manual analysis methods. The methodologies were revised and implemented. Automated Analysis Method of Sample258 The macro, written to automate the content analysis or keyword searches was used on sample258. On sample258, with the full contents of 258 job announcements in a word file we ran the macro for over 100 keywords. Using upper or lowercase when searching did not affect the results. 1. The macro is run for keywords and the frequency for each is recorded. 2. Variations of the keyword are considered and searched as well. See Table On multiple searches, a fraction of the word was searched, such as communicat in order to capture communications and communicate. For the automated analysis of sample258 the entire text of the JA is included. See Appendix D for an excerpt of sample258 keyword frequencies. In a stated limitation of the macro, some keywords had to be manually counted for verification of results, such as interview which shows in the context of a company s hiring practices and also within the context of interviewing of a subject matter expert, an ID skill. When we thought the frequency for the keyword could be incorrect, we manually counted within the 600 page file using the Microsoft Word Find function. In Table 2 we present findings for the Project Management skill across the three data sources. Percentages are not applicable to sample258 because we do not know, by using the macro, how many times the same keyword may be reflected within the text of one job. Table 2: Tally of Keywords from Three Data Sources Data Source Skill or Keyword Identified f % Sample48 Project Management % Sample258 Manage Project 8 n/a Sample258 Project Management 97 n/a Survey Project Management 13 30% We resolved disagreements in keyword frequency when the word was within another word, such as story counted within storyboard. See Figure 4. We discussed the nuances and limitations of the automated search, such as including a space or not, combining totals when the same skill was spelled differently and being careful to not over count a skill. When searching java the keyword appears 34 times, and when searching for Java Script, keyword java_script appears 20 times. In this case the final frequency for Java Script is recorded as 20 and Java is recorded as 14, because Java and Java Script are different things. Other differences were discussed and resolved and some keywords were searched using the Find function, not the macro. When Acrobat has 11 frequencies in sample258 and Adobe Acrobat has 10 frequencies, Adobe Acrobat is recorded as 11 frequencies. Figure 2: Tally for keywords related to Adobe Acrobat on sample258 Page 8 of 37

9 Acrobat 11 Adobe Acrobat 10 When Macromedia has 9 frequencies and Macromedia Flash has 2 frequencies, the frequencies are left as is because there is a program called Macromedia Director and a Macromedia Flash. Figure 3: Tally for keywords related to Macromedia Flash on sample258 Macromedia 9 Macromedia Flash 2 Figure 4 illustrates how Story Board appears 2 times and Storyboard appears 22 times. In data collating, the skill for storyboarding will be recorded as 24 frequencies. Figure 4: Tally for keywords related to Storyboarding skill on sample258 Story Board 2 Storyboard 22 DATA ANALYSIS Our approach for answering research question # 1, Are the skills and knowledge attained by FSU IS graduates matched with job requirements, is as follows. We sent an online survey to IS graduate alumni and reviewed responses for the following survey item, My academic training in the FSU IS program has made it easy to find employment in the field of IS. The answers provided were a yes or no. We also relied on survey item, List the top three things you wish you had learned as part of your academic preparation in Instructional Systems at FSU to answer research question # 1. We compared those answers to the top ranked skills on the two job announcement samples. For research question # 2, What are the requirements most often listed in ID job announcements? we analyzed the contents of two samples of job announcements. On the first (smaller) sample (sample48) we manually rephrased the qualifications and standardized them into one, two or three word phrases. On the second (larger) sample (sample258), we used a macro program to automatically count the frequency for keywords. Then we compared the frequency of keywords on both samples to determine the top skills required for instructional design jobs. For answering research question # 3, What types of new media and technology skills are most often listed in ID job announcements? we used a similar methodology as we did for answering question #2. We generated a list of keywords of popular new media, specific software programs and technology-related keywords (i.e. web-design, authoring tools, and multimedia ) and noted the frequency counts of each as we analyzed the contents of sample48. We noted the frequencies as well on sample258 for same keywords searched in an automated fashion. Through the analysis we were able to generate a list and come up with the top ranked. To answer research question # 4, What are the top ID skills FSU IS alumni wish they had attained while in the FSU IS program? we analyzed the contents of the responses to survey item # 6 List the top three things you wish you had learned as part of your academic preparation in Instructional Systems at FSU. We noted the themes that emerged among the graduates who participated in the survey. Finally, for research question # 5, Do FSU IS courses cover the top skills needed in the ID job market, we combined our analysis of survey responses with findings on the previous questions and acknowledged that we could not give a definite answer except for performing a second research project specifically focusing on the course curriculum. Page 9 of 37

10 How We Answered Each Research Question In answering research question # 1, Are the skills and knowledge attained by FSU IS graduates matched with job requirements? we believe the answer is yes and no. Although the data reflects an alumni body satisfied with what they learned in the IS program with 42 out of 44 respondents answering Yes (to the survey item that FSU helped them find employment in IS), 41 of the same 44 listed one or more skills they wish they had attained while in the program, indicating the preparation may be incomplete. In that there was little feedback reporting a gap in core ID skills and competencies, this means the majority of respondents felt the IS program curriculum adequately prepared them with general core ID practices and theory. There are skills graduates do not possess and were not expecting to need in their work, such as project management, business knowledge and IT skills. Many alumni expressed surprise and frustration over having to know information technology, manage projects and negotiate budgets and product specifications with subject matter experts. A 2007 FSU graduate doing consulting work with the federal government as a client said it best with: More project management -- more business classes (financial management, org design, and oddly enough intro to business and economics). I find myself managing IT related projects -- although I hate IT, I wish I would have learned more about the industry in school. Also, certifications (real ones like PMP, ITIL, six sigma) or info on those certifications would have helped. I also think professors with professional backgrounds -- not just academia -- would have helped. The item on the survey on whether the FSU IS preparation made it easy to find employment begs research on whether it is the reputation or the preparation by the program that aided 42 out of 44 respondents in finding employment. To answer research question # 2, what are the requirements most often listed in ID job announcements? the data shows the top required skills are communication, collaboration and project management. Following are technology skills especially with software for developing courseware. Communication skills include the ability to network, negotiate, facilitate, and influence others, with strong interpersonal skills, especially when working within a team and with subject matter experts. To answer research question #3, what types of technology skills are listed most in ID job announcements? Flash, PowerPoint, Captivate and HTML were the top ranked technology skills in sample258. Captivate, Flash, HTML and SCORM were the top ranked on sample48, with PowerPoint scoring slightly less than SCORM. See Figure 6. For research question # 4, what are the top ID skills FSU IS alumni wish they had attained while in the FSU IS program, the survey respondents rated project management, communication, information technology, software multimedia programs and business acumen at the top. Next ranked were business skills, skills in instructional design specifically with conducting needs assessments, with office productivity software, such as Microsoft Suite and excellent writing skills following. Alumni statements and job announcement data were mostly, in agreement. Except for some observations, we do not have a definitive answer for research question # 5, on whether the FSU IS courses cover the top skills needed in the ID job market. In reviewing the data a degree from FSU does seem to make it easy for graduates to have gained employment in the field, (except for two newer graduates, who stated otherwise) and most of the alumni expressed satisfaction and appreciation for their training by the FSU IS program. Still we did broach the question delicately letting the respondents express their satisfaction with the program but still able to give suggestions for improvement. 93% of respondents offered suggestions for improvement. Page 10 of 37

11 Again, because we did not do an in-depth analysis of the FSU IS curriculum to determine the alignment of courses to job needs we can not answer this question but only offer some observations. Based on our experience as FSU IS graduate students, and in looking at current course descriptions it is evident that FSU offers courses to develop collaboration skills, that are so widely needed in the field, with EME 6635 Design of Online Collaborative Learning (currently not a required course) and EME 5604 Designing Instructor-Led Courses. Project management skills are addressed in EME 6631 Managing Instructional Development. Based on our experience, Adult Learning Theory is covered in EDP 5216 Theories of Learning and while many of the learning theories covered in this course do apply to both adults and children, only one week is spent specifically on Adult Learning Theories. This may not be enough education on the topic if compared to our data findings. DATA ANALYSIS LIMITATIONS A limitation is that while we state the frequency that a skill is listed, we do not know, other than by looking at the corresponding job title and job description, the depth of knowledge required for that skill. If the job announcement says SCORM, the instructional designer may only need to know what SCORM is and his job function may require him to ensure someone on the development team is building SCORM into the design of the product. However, if he is the developer he needs to know how to build SCORM functionality into the design of the product. Although we state the frequency the skill is listed in our study and data sources, we can only approximate that the frequency is reflecting its level of importance in the minds of hiring managers. In a study by Brill, Bishop, and Walker, (2006), the researchers list 78 success factors for project management and later rank each in level of importance by asking an expert panel until the experts reach consensus. We did not do this type of analysis. Another limitation is that some keywords are overrepresented on sample258 if and when the keyword is found in several sections of one JA. Sample258 has job announcements aggregated into one file, without cutting and pasting only the qualifications sections. Sometimes the job announcement will repeat the keyword searched in several sections. However, the automated analysis is an efficient method that allows us to access a bigger sample. We sometimes performed a manual find, such as for keyword interview when looking for data on interviewing the subject matter expert skill. We did this to ensure we counted the skill for interview correctly, when we were looking for interview within the context of interviewing a subject matter expert and not within the context of the company s hiring and interviewing process. Another example is that keyword cultur (as in culture) and divers (as in diversity) are listed 80 and 109 times in sample258 however, we are not to say the words show within the context of an ID understanding how culture and diversity affects their development and team work. Many times the keywords show within the JA in reference to the human resource and company s position that they embrace the idea of diversity in the workplace. In our analysis we did not analyze these keywords in the context of skills. During our analysis we did not differentiate for requirements based on job industry. Also, we notice that in our sample of 258, 23 of the jobs were from an online university called ITT and although the job title was for instructional design, most were advertising for an online (adjunct) professor. None of these announcements had keywords that would skew our new media results. However, the following keywords show up on most of ITT job announcements interpersonal, presentation, influencing, teach, motivate, organizing, critical thinking, writing, Microsoft Office, and management system software. FINDINGS Page 11 of 37

12 While we did not ask in the survey for the top 3 things alumni were glad they had learned while in the program, many of the survey respondents volunteered positive remarks about their learning experience with the FSU IS program. One shared that The theory and the print development class have supported my endeavors very well and a 1975 wrote At the time, my education was very comprehensive - I took courses in Ed Psych, Measurement and Evaluation, as well as ISD. However, I would have liked some guidance on the different possibilities for a career in ISD. A summary of the Survey Respondents answers is seen in Table 3 below. Table 3: Survey Respondent Summary IS Graduates Survey Respondent Summary Reply Received till date: 44 (4.1%) Population Size Reached: Approx Years: 1975 to /44: 29% 05/44: 11% 05/44: 11% 09/44: 20% 02/44: 5% 02/44: 5% 11/44: 25% 42/44: 95% 04/44: 9% 14/44: 32% 5/44: 11% 5/44: 11% 23/44: 52% Year of Graduation from IS Program Current Job Field Education Human Resources Defense Industry Consulting Firm IT Industry HealthCare Other Current Job Related to IS Training Yes No Years of Experience in IS related Job 1-3 Years 4-6 Years 6-10 Years More than 10 years 42/44: 95% 02/44: 5% Usefulness of FSU IS Training in Finding Employment Yes No 1 Communication and Collaboration The analysis we did on job announcements has communication and collaboration as indispensable and at the top of skills needed by an instructional designer. Collaboration means the ability to work successfully as part of a team. Communication refers to written, oral and other types of communication. On sample48 keywords collaboration and communication were reflected on 52% of the cases. On sample258, the frequencies for communication, team and (partial word search) collaborat were 294, 242 and 140 respectively. In the job announcements skills analysis, written, as in written communications, was counted 113 (minus 23 if adjusting for the number of ITT jobs in sample258) and reflects a very high need in the field of ID as also discovered by Rowley in Keyword writing skills was represented 31% within sample48. Specifically, technical writing accounted for 4% in sample48 and appeared 15 times in sample258. Page 12 of 37

13 On communication, a Florida State University class of 1997 alumni said a speech class would have helped him: A speech class (probably called something else now.) This would have prepared me to more confident in front of groups of people. This is a skill I've built over the years, but things like this likely would have advanced my career faster because I would have been more visible, especially in front of senior management. While the data indicated collaboration and team work to be important, the data also showed a need for workers to be able to work independently, with independent on 22% of sample258 and 10% of sample48. Other keywords reflecting the importance of independence are: self-motivated, self-directed, self-starter and starter accounting for 7% of sample258 and 10% in sample48. 2 Project Management In all three data sources we found a high demand for project management skills. Keyword project management was found on 52% of sample48 and project management and manage projects shows 105 times on sample258. In addition, 13 of 44 FSU IS graduates wrote they wished they had developed solid project management skills. PMP, Project Management Professional certifications are in some job announcements, and a 2007 FSU IS graduate wrote: More project management...also, certifications (real ones like PMP, ITIL, Six Sigma) or info on those certifications would have helped. The PMP certification is a professional certification obtained after formally logging in a large number of project management hours and passing the PMP exam. In relation to project management, keyword deadline, had a high frequency of 61 in sample Desire for Business Management Skills A total of 8, or 18 %, of the 44 FSU graduates who responded highlighted that they wished they had acquired more business skills. Keywords marketing and budgeting had similar frequencies in both job announcement samples, being mentioned 47 and 53 times respectively, in sample258 and 1% in sample48. Also, 2 survey respondents stated they wish they had gained skills in budgeting because when they are put in charge of a project they are sometimes also required to handle the financial side of the project. The graph in Figure 5 shows the frequency for keywords which were consolidated into major topics. When combining keywords like PMP, project management, manage projects and PM in sample48 we found 25/48. The data findings are that among the skills listed in qualifications, project management was by far much more numerous than for other topics. Figure 5: Business Skills and Project Management Frequency on sample48 Page 13 of 37

14 4 Technology, E-Learning Tools, Programming Skills Technology A survey respondent implored, technology, technology, technology, on what he wish he would have learned while in the FSU IS program and he is right. The respondent may be referring to office productivity software or multimedia and simulation software. Another respondent who stated he hates IT is probably in the wrong line of work because, as we had suspected, there is some overlap in IT and ID. ID jobs require strong technology skills. In sample258, 26 jobs preferred or required a degree in computer science and other JA s called for strong computer science skills. Our data shows job titles for instructional designers vary greatly with some job titles and job descriptions being extremely technical in nature. For those jobs calling for Structured Query Language (SQL) skills, the job titles were Analyst: Academic Technology, Learning Management Systems Administrator, Technical Curriculum Developer, Custom Solutions Architect, Technical Trainer/ ID, Director of Engineering, Data Systems Architect, DOE Web Applications Developer, and Director of Engineering and all were categorized as instructional design jobs. Aptitude with Developing Skills and using elearning Tools ID job announcement qualifications mentioned multimedia (79 times on sample258), interactive (79 times on sample258), elearning (73 times on sample258), simulation (66 times on sample258), and video production (44 times on sample258). Table 4 shows the frequency for aptitude with ID skills. Refer to Figure 6 for the frequency of skills with specific software or programming languages. As seen in Figure 6, Flash, PowerPoint, Captivate and HTML were the top ranked technology skills in sample258 and sample48 was similar with Captivate, Flash, HTML, SCORM, Lectora, and PowerPoint. Table 4: Sample258 keyword search and Sample48 manual content analysis findings FSU graduates mentioned elearning, multimedia, technology (5 times), authoring tools (4 times), simulation (2 times), web-based instruction (7 times), social media or Web 2.0 (3 times), graphics creation (2 times) and specific types of software in response to survey item # 6, list the top three Page 14 of 37

15 things you wish you had learned as part of your academic preparation in Instructional Systems at FSU. It was not only those who graduated long ago who wished they had more technology skills. Even more recent graduates mentioned their need for stronger technology skills. Their feedback corroborates findings on the job announcements skills review. From the sample data job titles we can see the need for expertise with technology. Some job titles that show evidence to this are: E-Learning Developer Associate Interactive Designer Instructional Systems Designer II/Web-Based Designer Gaming Producer Education Technologist Learning Architect Professional Innovative Instructional Designer (blended learning) Instructional Systems Designer The big need is for course curriculum that allows students to learn and use multimedia software. The instructional designer is ultimately going to use sound, images and text for developing instructional products. In preparing for that, assignments within an IS course should require a student to creatively use sound, images and text to represent ideas, over and above concepts on ISD. The idea is for students to develop skills using a variety of media to express their ideas so that when they need to use multimedia to represent a company s instructional ideas, they have a framework and experience in doing so. Programming Skills We found some jobs call for programming skills, and not just mark-up languages like HTML, DHTML, XML or website design structure like CSS (Cascading Sheets) with 12 mentions in sample258. But there were requirements for programming (31 times in sample258) skills with Java, Java Script and Ruby and multiple jobs (around 25) listed a degree in computer science as an alternative to a degree in instructional systems. Figure 6: Frequency of Technology Keywords on Three Data Sources Page 15 of 37

16 5 Adult Learning Theory Writers of ID job announcements have included the requirement that an instructional designer understand the adult learner. The adult learner makes up a large proportion of the target audience for receiving training by new technological methods. On sample48, 19% of the jobs asked for knowledge of adult learning theories. On sample258 keyword adult learning was listed 69 times and keyword adult was listed 100 times. Learning theories, in general, were mentioned 34 times in sample258. The job announcements on sample48, indicating adult learning theory did so with the following excerpts: job14.html Expertise and application in Adult Learning Theory. job15.html Experience with instructional design, adult education, graphics or media design, or related fields is a plus, but not required. job20.html Knowledge and application of instructional design and principles of adult learning. job23.html Proficiency in utilizing instructional design methodologies and adult learning theories job24.html Knowledge of training methods, tools, and techniques for adult learners job25.html Knowledge of adult learning theory. job31.html knowledge of adult learning principles job48.html Experience with Adult learning theory In sample258 we know the keyword adult can be within a phrase, stating anything other than adult learning theory, such as experience working with adults. However, keyword adult learning has a frequency of 69 on sample258, showing this is an important qualification to have. 6 - More Practical Experience in ID Skills such as Needs Assessment and Others A prominent theme emerged for more practical experience in ISD. Keyword ADDIE appeared 16 times in the sample258 document. A 1993 graduate stated: I only have one thing for my wish list...i would have liked more experiences in how to conceptualize, plan, conduct and report various forms of rigorous research related to human and instructional design. There were high tallies for instructional development from our job announcement analysis and this is expected as instructional development is a fundamental skill for an instructional designer. However, we noticed the tallies on each of the specific processes within the ISD model were fairly equal. Needs analysis was the only step within the ISD process that was singled out and specifically mentioned in the survey responses and job announcements. JA text indicating needs assessment was found 42 times in sample258 (manually counted) and assess, incorporating needs assessment and the ability to assess in general, showed 184 times. Sample48 had a 16% result for needs assessments. A 1985 FSU graduate, wrote she wish she had learned practical needs assessment skills. Specific job announcement text related to the duties or qualification of needs assessment included:...conduct needs analysis......comprehensive program assessment reviews......conduct client needs assessments including system and process analysis......works closely with stakeholders to conduct needs assessment......conduct instructional needs assessments......learning/training needs analysis... Page 16 of 37

17 7- Bachelors versus Masters Job Requirements On sample258 the data showed more announcements preferred, or required, a bachelor s degree. Table 5 shows the spelling variations we used for our searches. A manual hand count was done on keyword MA to ensure the text was referring to a master s degree, and not in a word, such as Massachusetts. Table 5: Keywords Searched for Degree Requirements on sample258 Keywords Searched on Sample258 B.A. 7 Masters 50 BA 4 MBA 15 Bachelor 210 Master's 69 Bachelor 221 M.A. 2 MA 3 Masters Office Productivity Software The job announcement skills analysis indicated a high need for hires to be well-versed in office productivity software, such as the Microsoft Suite of products. Keyword Microsoft shows 93 times in sample258 and MS Suite resulted in 33% of sample48. A survey respondent stated: I wish that I had more training in technology- such has using Microsoft applications more efficiently and another said Simple word, power point and excel skills. Of the office productivity software, PowerPoint was specified the most times, with 72 mentions in sample258, 8% in sample48 and twice by survey respondents. Related to skills with PowerPoint is the wish respondents have for learning how to create storyboards. One PowerPoint use is creating storyboards. Storyboard showed 4% of the time on the survey data, 24 times in sample258, and 2% in sample48. A 2007 FSU graduate stated: Also, a main focus of my job has been writing storyboards for Web Based Trainings and I didn't feel that I had received any courses on this. A 1995 FSU graduate stated: I wish I had learned to really storyboard the way most companies do - in a Word or PowerPoint template. That would have been so helpful to see how different companies storyboard and practice filling out those templates for a mini course. Keyword for Microsoft Excel, Excel shows 19 times on sample258 or 7%. However, analysis, which can often be done using Excel showed as follows: Keyword analysis shows 133 times on sample258, Analyze shows 55 times on sample258 and Analytical Skills, as rephrased by the research team, shows 25% of the time on sample48. Knowing the tool aids in the skill. 9 - Consulting Skills, Change Management, Negotiating with SMEs and Clients More than 40% of survey respondents hoped to have learned more project management skills, consulting, client/stake holder s management, negotiation and leadership. Apart from the knowledge and skills to perform the main ID tasks, many felt that these other competencies were equally important in being effective in their jobs. While many of the respondents who provided the feedback on this particular area are working in Page 17 of 37

18 the consulting industry, there are a wide array of respondents from other job fields like Defense, Education, IT, HR and Healthcare. This shows the trend for demand of business and management skills in IS related jobs are not only confined to those in a consulting career, but also relevant to those working in other industries. These skills are equally essential for managing internal organizational projects and stakeholders. We found survey respondents expressing frustration with their lack of skills in working with clients and subject matter experts (SME). Their responses triggered an investigation of the job analysis data and we found a match in what respondents were saying and qualifications sought after by hiring managers. Keywords associated with this function include consulting (25 times in sample258), interviewing (16 times in sample258), facilitating (114 times on sample258), leadership (145 times on sample258), negotiating, and others like needs assessment. Sample258 has keyword consulting 25 times and 16% of survey respondents wrote it. Sample258 showed keyword interview 16 times but because interview could be referring to details about a company s hiring process, we counted interview manually. Interviewing skills showed in 6% of jobs in sample48 and 11% of survey respondents mentioned interviewing skills. A 1994 FSU graduate working in Human Performance said he could benefit from learning: Consulting skills and linking the language of performance improvement to the language of business. The need to possess skills for working with subject matter experts showed approximately 6% on both sample258 and sample48 and 11% of FSU IS respondents specifically expressed an interest in more training for working with subject matter experts saying: How to facilitate Subject Matter Experts (SMEs) meetings to get basic ISD components: objectives, content, etc., SME interviewing techniques Other skills that aid in consulting include interpersonal skills. Interpersonal skills are mentioned twice by FSU graduates, 81 times in sample258 and 29% in sample48. FSU graduates wished they had learned better negotiation skills, including contract negotiation skills. Negotiation was mentioned 9 times in sample258 and 4% in sample48. While individually, these keywords may not reflect high frequencies, taken together, there is a clear need for instructional designers to learn how to work with, negotiate and communicate with the client and the subject matter expert in a consulting role. A 1983 graduate said he wished for More on interpersonal relationships in business, like contract negotiation with clients. Quality Assurance is a skill needed if the instructional designer or consultant wants repeat customers and a good reputation in the industry. In the ID industry Quality Assurance is rarely mentioned (in our data) but evaluation is a mentioned 275 in sample258, referring to formative, summative and continuous evaluation. Liu et al, in 1988, reflected that many interviewees stress the importance of having such continuous evaluation as a key factor for ensuring the quality of the product. Evaluation takes on different forms. Quality products should be the final outcome of ID work. The challenge of collecting information from a client or SME was noticed in the data. One survey respondent wrote about how important it is to collect information from the SME. He states: Learning shorthand (or buy a digital tape recorder). I cannot tell you how many times I have sat with an SME and learned a lot of great information, only to have lost it because I can't read my own handwriting! As an outside consultant or vendor, the instructional designer does not have the luxury to meet multiple times with an SME (before coming across as inept) and often has to be able to get the Page 18 of 37