*Net Perceptions, Inc West 78th Street Suite 300 Minneapolis, MN

Size: px
Start display at page:

Download "*Net Perceptions, Inc West 78th Street Suite 300 Minneapolis, MN"

Transcription

1 From: AAAI Technical Report WS Compilation copyright 1998, AAAI ( All rights reserved. Recommender Systems: A GroupLens Perspective Joseph A. Konstan *t, John Riedl *t, AI Borchers, * and Jonathan L. Herlocker* *GroupLens Research Project Dept. of Computer Science and Engineering University of Minnesota Minneapolis, MN *Net Perceptions, Inc West 78th Street Suite 300 Minneapolis, MN ABSTRACT In this paper, we review the history and research findings of the GroupLens Research project I and present the four broad research directions that we feel are most critical for recommender systems. INTRODUCTION: A History of the GroupLens Project The GroupLens Research project began at the Computer Supported Cooperative Work (CSCW) Conference in One of the keynote speakers at the conference lectured on a his vision of an emerging information economy, in which most of the effort in the economy would revolve around production, distribution, and consumption of information, rather than physical goods and services. Paul Resnick, then a student at MIT, and now a professor at the University of Michigan, and one of us (Riedl) were moved by the talk consider the technical challenges that would have to be overcome to enable the information economy. We realized that as the amount of information increased enormously, while people s ability to process information remained stable, one of the critical challenges would be technology that would automate matching people with the information they would find most valuable. There were two main thrusts of research activity in this area that we knew of: (1) Artificial Intelligence (AI) research develop tools that would serve as a "knowledge robot", or knowbot, continually seeking out information, reading and understanding it, and returning with the information that the knowbot determined would be most valuable to its user. (2) Information Filtering (IF) research to develop even more efficient tools for selecting documents that contain keywords of interest to a user. These techniques were, and continue to be fruitful, but we felt they each have one serious weakness. In the case of the knowbot, the weakness is that we are still a significant distance from technology that can understand articles in the way a human does. In the case of Information Filtering, the weakness is that 1 GroupLensT M is a trademark of Net Perceptions, Inc, which develops and markets the GroupLens Recommendation Engine. Net Perceptions allows the University of Minnesota to use the name "GroupLens Research" for continuity. The ideas and opinions expressed in this paper are those of the authors and do not represent opinions of Net Perceptions, Inc. identifying sets of articles by keyword does not scale to a situation in which there are thousands of articles that contain any imaginable set of keywords. Taken together, these two weaknesses represented an opportunity for a new type of filtering, that would focus on finding which available articles match huma notions of quality and taste. Such a system would be able to produce a list of articles that each user would like, independent of their content. We decided to apply our ideas in the domain of Usenet news. Usenet screams for better information filtering, with hundreds of thousands of articles posted daily. Many of the articles in each Usenet newsgroup are on the same topic, so syntactic techniques that identify topic are much less valuable in Usenet. Further, different people value very different sets of articles, with some people participating in long discussion threads that other people couldn t imagine even reading. We developed a system that falls into the class that is now called automatic collaborative filtering. It collects ratings from people on articles, combines the ratings statistically, and produces recommendations for other people of how much they are likely to like each article. We invited people to participate in using GroupLens from all over the Internet, and studied the effect of the system on users. Users resisted our early attempts to establish multidimensional rating schemes, including characteristics such as quality of the writing, and suitability of the topic for the newsgroup. Rating on multiple dimensions was too much work. We changed to single-dimension ratings, with the dimension being "What score would you have liked GroupLens to predict for you for this article?" We found that users did change behavior in response to the recommendations, reading a much higher percentage of the articles that GroupLens predicted they would like than of either randomly selected articles, or articles GroupLens predicted they would not like. However, there were many articles for which GroupLens was unable to provide ratings, because even with a two to three hundred users, there were simply too many articles in the six newsgroups we were studying. A greater density of ratings by article would have improved the usability of the system for most users. The low ratings density was compounded by the first rater problem, which is the problem that a pure collaborative filtering system cannot possibly make recommendations to 60

2 the first person that reads each article. One effect of these two problems is that some beginning users of the system saw little value from GroupLens initially, and hence never developed the habit of contributing ratings, though they continued to use GroupLens-enabled news readers. Because most users did not like most articles, and because GroupLens was effective at identifying articles users would like, users requested the ability to scan a newsgroup for the articles that were predicted to be of high interest to them. This led to our exploring a different style of interface to a collaborative filtering system, the TopN interface. Rather than predicting a score for each article, a TopN interface greedily seeks articles that are likely to have high scores for an individual user, and recommends those articles to that user. Eventually, such an interface might be able to present each of us with a list of the most interesting articles for us from all of Usenet each morning. Our key lesson learned was that a very high volume, low quality system like Usenet would require a very large number of users for collaborative filtering to be successful. For our research purposes, we needed a lower volume, higher density testbed. Our colleagues from Digital Equipment Corporation were closing down their research system on movie recommendations, and offered us the data to jump-start a similar system using GroupLens. We launched our system in the summer of 1997, and have been running it since at MovieLens is entirely web-based, and has several thousand regular users. Users rate movies, and MovieLens recommends other movies to them. Over the past six years of research, we have learned that people are hungry for effective tools for information filtering, and that collaborative filtering is an exciting complemento existing filtering systems. Users value both the taste-based recommendations, and the sense of community they get by participating in a group filtering process. However, there are many open research problems still in collaborative filtering. Below we discuss our early results on some of these problems, and outline the remaining problems we feel to be most important to the evolution of the field of collaborative filtering. CURRENT RESEARCH RESULTS Our recent research has focused on improving the quality and efficiency of collaborative filtering systems. We have taken a broad approach, seeking solutions that improve the efficiency, accuracy, and coverage of the system. Specifically, we ve examined partitioning users and items, incorporating filtering agents into the collaborative filtering framework, and using existing data sets to start up a collaborative filtering recommendation system. Partitioning Users and Items Both performance and accuracy concerns led us to explore the use of partitioning. If user tastes are more consistent within a partition of items, and therefore, user agreement is more consistent, then partitioning the items in the system may yield more accurate recommendations. Even if the increased accuracy is offset by the smaller number of items available to establish user correlations, partitioning may be valuable because it can help scale the performance of the system; each partition can be run in parallel on a separate server. To explore the potential of item partitioning, we considered three partitioning strategies for MovieLens: random partitions, partitions by movie genre, and partitions generated algorithmically by clustering based on ratings. Clustering-based partitions produced a slight loss in prediction accuracy as partitions grew smaller, but showed promise for a reasonable trade-off between performance and accuracy. Movie genre partitions yielded less accurate recommendations than cluster-based ones, though some genres were much more accurate, and others much less so). Random partitions were slightly worse still. The value of item partitions clearly depends on the domain of the recommendation system and the density of ratings within and across potential partitions (our earlier Usenet work found that mixing widely different newsgroups together reduced accuracy). One advantage of the clustering result is that it may be more broadly applicable in domains where items lack obvious attributes for partitioning. We also looked at the value of user partitioning, starting with the extreme case of pre-computed symmetric neighborhoods based on our clustering algorithm; these were small partitions of about 200 users. If symmetric neighborhoods yield good results, time per recommendation can be reduced dramatically, since substantial perneighborhood computation can be performed incrementally and amortized across the neighbors. We found that the accuracy of recommendations was almost as good as using the full data set, but that the coverage (i.e., the number of movies for which we could compute a recommendation) fell by 14%. To restore coverage we introduced a two level hierarchy of users. Users from each other neighborhood were collapsed into a single composite user. Each neighborhood then had all users represented, similar users were represented at full resolution and the more distant users were represented at the much lower resolution of one composite user per neighborhood. This restored full coverage and the quality of predictions was only slightly degraded by about 1% from the unpartitioned case. We are continuing to explore these hierarchical approaches. Filterbots One problem with pure collaborative filtering is that users cannot receive recommendations for an item until enough other users have rated it. Content-based information filtering approaches, by contrast, avoid this problem by establishing profiles that can be used to evaluate items (e.g., keyword preferences). To combine the best of both approaches, we developed filterbots--rating agents that use content information to generate ratings systematically. These ratings are entered into the recommendation system 61

3 by treating the filterbots as additional users. This approach has the benefit of allowing us to use simple-minded or controversial filterbots; if a user agrees with a particular filterbot, that filterbot becomes part of the user s neighborhood and gains influence in recommendations. If a user does not agree with the filterbot, it does not become part of that user s neighborhood and is therefore ignored. To test this concept, we created several simple-minded filterbots for Usenet news. We found that a spell checking filterbot not only increased coverage dramatically (as much as 514%), but also increased accuracy as much as 74%. In tee.humor, a notoriously high noise group, all three of our simple filterbots (spell checker, percentage of included text, and message length) improved coverage and quality. In other newsgroups, some filterbots helped while others did not. Fortunately, the cost of filterbots is quite low, particularly since simple ones appear to have significant value. We plan to continue exploring filterbots, looking both at simple content filtering algorithms and at learning agents. Jump-Starting a Recommendation Engine Collaborative filtering systems face a start up problem: until a critical mass of ratings has been collected, there is not enough data to compute recommendations. Accordingly, early users receive little value for their contribution. In our MovieLen system we had the good fortune of starting our system seeded with a database over 2.8 million ratings from the earlier EachMovie recommender system. For privacy reasons the database we received from EachMovie had only anonymous users; although we could not associate these users with our own, they could still serve as recommenders for our users. We call this "dead data." We took advantage of this rare opportunity to evaluate the experience of new users in systems with and without the dead data. We retrospectively evaluated the recommendation accuracy, coverage, and user satisfaction for early users of EachMovie and MovieLens. For our accuracy and coverage experiments, we held the recommendation algorithm constant, and found that the jump-started case (MovieLens) had better coverage (nearly 100%, as compared with 89%) and higher accuracy (increases as high as 19%, depending on the metric used). To assess user satisfaction, we retrospectively compared user retention and participation in our current MovieLens system with that of the early EachMovie system. By looking at the session, rating, and overall length of active use of corresponding early EachMovie and MovieLens users (all of which could be reconstructed from logs), were able to measure indicators of user satisfaction. We found that early MovieLens users were more active than early EachMovie users in all categories, with dramatic increases in the number of ratings and number of sessions. Accordingly, it appears that the start-up problem is a real one--user retention and participation improves when users receive value--and using historical or "dead" data may be a useful technique for improving start-up. WHAT S NEXT: A RESEARCH AGENDA Based on our prior work, we ve identified four broad problem areas that are particularly critical to the success of recommender systems. We discuss these in general, highlighting work know to be underway, but also presenting open questions that are ripe for research. Supporting Users and Decision-Making Early work in recommender systems focused on the technology of making recommendations. Papers cited measures of system accuracy such as mean absolute error, measures of throughput and latency, and occasionally a general metric indicating the people used the system, or perhaps that they said they liked it. Now that the technological feasibility of recommender systems is well established, we must face the challenge of designing and evaluating systems to support users and their decisionmaking processes. While there are many factors that affect the decisionmaking value of a recommendation system for users, three critical issues have arisen in each system we ve studied: accuracy, confidence, and user interface. Accuracy is the measure of how closely the recommendations generated by the system predict the actual preferences of the user. Measurement of accuracy is itself a challenging issue that we discuss below. However, for any sensible definition of accuracy, recommender systems still are far from perfect. Both Maes work on Ringo and our own work suggest that today s pure collaborative filtering systems typically achieve at best an accuracy of plus-orminus one on a seven-point scale. Further research is needed to determine the theoretical limit of accuracy, based on user rate/re-rate differences and empirically determined variances. Then, significant work is needed on a wide range of approaches to improve accuracy for user tasks. These approaches include those discussed below and special filtering models tuned for precision and for recall. Confidence is a measure of how certain the recommendation system is of its recommendation. While statistical confidence measures often are expressed as confidence intervals or expected distributions, current recommendation systems generally provide no more than a simple "high, medium, or low" confidence score. Part of the difficulty with expressing confidence as an interval, distribution, or variance is the complexity of the statistics underlying collaborative filtering. Unlike dense analytic techniques such as multiple regression, collaborative filtering lacks a well-understood measure of error or variance. Sources of error include: the user s own variance in rating, the imperfect matching of neighbors, the degree to which past agreement really does predict future agreement, the number of items rated by the user, and the number of items in common with each neighbor, rounding effects, and many others. 62

4 At the same time, measures of confidence are critical to users trying to determine whether to rely upon a recommendation. Without confidence measures, it is extremely difficult to provide recommendations in situations where users are risk averse. Perhaps even worse is the poor reputation that a recommendation engine will receive if it delivers low-confidence recommendations. Accordingly, a key research priority is the development of computable and usable confidence measures. When algorithms permit, analytic solutions are desirable, but we are also investigating empirical confidence measures that can be derived from and applied to an existing system. User interface issues in collaborative filtering span a range of questions including: When and how to use multi-dimensional ratings When and how to use implicit ratings How a recommendation should be displayed Multi-dimensional ratings (and therefore predictions) seem natural in certain applications. Restaurants are often evaluated separately on food, service, and value. Tasks are often rated for importance and urgency. Today s recommendation engines can accept these dimensions separately, but further research is needed on crossdimension correlation and recommendation. Implicit ratings are observational measures of a user s preference for an item. For example, in Usenet news we found that time spent reading an article is a good measure of preference for that article. Similarly, listening to music, viewing art, and purchasing consumer goods are all good indicators of preference. Today s systems lack automated means for evaluating and calibrating implicit ratings; further research would be valuable. There are many ways to display recommendations, ranging from simply listing recommended items (without order), marking recommended items in a list of items, to displaying a predicted rating for items. Many research questions need to be answered through real user studies. In one study we ve conducted, we saw that the correctness of user decision making is directly affected by the type of display used. In other work, we are examining the question of whether the expected value of a predicted rating distribution is more or less valuable than the probability of the rating exceeding a "worthwhile" cutoff. For example, is a user more interested that a movie is likely to be three-anda-half stars, or that it has a 40% chance of being four or five stars? Many similar questions remain unanswered. Beyond Collaborative Filtering The second major research issue for recommender systems is integrating technologies other than collaborative filtering into recommendation systems. Content analysis, demographics, data mining, machine learning, and other approaches to learning from data each have advantages that can help offset some of the fundamental limitations of collaborative filtering (e.g., sparsity and the early rater problem). Some of the interesting open research questions include: How to integrate content analysis techniques from information retrieval, information filtering, and agents research into recommendation systems. Our filterbot work is a first step in this direction, as is MIT s collaborating agent work and Stanford s Fab system. More research is needed to discover which techniques work for which applications. How to take advantage of user demographics and rulebased knowledge in recommendation systems. How to integrate the power of data mining with the real-time capabilities of recommender systems. Particularly interesting questions include identifying temporal trends in preferences (e.g., people who prefer A at time t are more likely to prefer B at time t+l). How to take advantage of machine learning techniques in recommender system. In all of these cases, a key question will be whether one technology can be incorporated into the framework of another, or whether a new architecture is needed to merge the two types of knowledge. Scale, Sparsity, and Algorithmlcs There fundamental problem of producing accurate recommendations efficiently from a sparse set of ratings is inherent to collaborative filtering. Indeed, if the ratings set were dense, there would be little value in producing recommendations. Accordingly, there is still great need for continued research into fundamental issues in performance, scalability, and applicability. One particularly interesting research area that we, along with others, are actively investigating is techniques for reducing the computational complexity of recommendation. As discussed above, the complexity of recommendations grows generally with the size of the database, which is the product of the number of users and the number of items. Neighborhoods, partitioning, and factor analysis all attempt to reduce this size by limiting the elements considered along the user dimension, the item dimension, or both. Neighborhood techniques use only a subset of users in the computation of recommendations. Many variants have been proposed and implemented; research is needed to assess the varying merits of symmetric vs. asymmetric neighborhoods, on-demand vs. longer-term neighborhoods, threshold vs. size limit, etc. Partitioning is discussed above; factor analysis is a different approach that tries to decompose users or items into a combination of shared "taste" vectors. Each of these techniques has promise for reducing storage and computation. A second critical issues is to continue to address the challenge of ratings sparsity, particularly for applications where few users ever should rate an item. Clustering, factor 63

5 analysis, and hierarchical techniques that combine individual items with clusters or factors can provide one set of solutions. Integration with other technologies will provide others. Finally, there is still significant work needed on the central algorithmics of collaborative filtering to improve performance and accuracy. The use of Pearson correlations for neighbor selection and weighting is common in recommendation systems, yet many alternatives may be more suitable, depending on the distribution of ratings. Similarly, the results from RINGO, along with some of our own, suggest that the common Pearson algorithm overvalues neighbors with low correlations. Further work, particularly including empirical work, is needed to evaluate candidate algorithms. Metrics and Benchmarks Sadly, evaluation is one of the weakest parts of current recommender system research. Systems are not compared against each other directly, and published results use a variety of metrics, often incomparable ones. Both research and collaboration is needed to establish an accepted set of metrics and benchmarks for evaluating recommendation systems. Three areas are particularly important and promising. Accuracy metrics. There are nearly a dozen different metrics that have been used to measure recommendation system accuracy. Statistical measures of error include the mean absolute error between predicted and actual rating, the root mean squared error (to more heavily weigh large errors), and the correlation between predicted and actual ratings. Other metrics attempt to assess the prevalence of large errors. Reversal measures tally the frequency with which "embarrassingly bad" recommendations are made. A different set of metrics attempts to assess the effectiveness of the recommendation engine in filtering items. Precision and recall statistics, borrowed from information retrieval, together with receiver operating characteristic measurements from signal processing discount errors that do not affect usage, and more heavily weigh errors near the decision point. For example, the difference between 1.0 and 2.5 on a five point scale may be unimportant, since both are rejected, while the difference between 3.0 and 4.5 is extremely relevant. Finally, ordering metrics assess the effectiveness of top-n algorithms in identifying true "top" items. The different metrics in each category should be evaluated with the most useful ones identified as expected for publication and comparison. Coverage combined with accuracy. Accuracy metrics alone are not useful for many types of comparison. By setting the confidence threshold high, most systems can increase accuracy at the expense of coverage. Meaningful combined metrics are needed to allow meaningful evaluation of coverage/accuracy trade-offs. We are working on a combined decision-support metric, but others are needed as well. Full-system benchmarks. The greatest need right now is for corpora and benchmarks that can be used for comparison. In the future, these should be integrated with economic models to evaluate, in monetary terms, the value added by a recommender system. ACKNOWLEDGMENTS We would like to acknowledge the financial support of the National Science Foundation and Net Perceptions, Inc. We also would like to thank the dozens of individuals, mostly students, who have contributed their effort to the GroupLens Research project. Finally, we would like to thank our users, all FOR ADDITIONAL INFORMATION Several good sources of bibliographic information already exist in the area of collaborative information filtering and recommender systems. Rather than duplicate that work here, we refer the user to: 1. The March 1997 issue of Communications of the ACM, edited by Hal Varian and Paul Resnick. In addition to containing articles on several relevant systems, the section introduction and articles contain extensive bibliographic information. 2. The Collaborative Filtering Resources web page, at this page grew out of a March 1996 workshop on collaborative filtering held at Berkeley. It includes pointers to other reference pages. 64

A Case Study: News Classification Based on Term Frequency

A Case Study: News Classification Based on Term Frequency A Case Study: News Classification Based on Term Frequency Petr Kroha Faculty of Computer Science University of Technology 09107 Chemnitz Germany kroha@informatik.tu-chemnitz.de Ricardo Baeza-Yates Center

More information

The Good Judgment Project: A large scale test of different methods of combining expert predictions

The Good Judgment Project: A large scale test of different methods of combining expert predictions The Good Judgment Project: A large scale test of different methods of combining expert predictions Lyle Ungar, Barb Mellors, Jon Baron, Phil Tetlock, Jaime Ramos, Sam Swift The University of Pennsylvania

More information

Python Machine Learning

Python Machine Learning Python Machine Learning Unlock deeper insights into machine learning with this vital guide to cuttingedge predictive analytics Sebastian Raschka [ PUBLISHING 1 open source I community experience distilled

More information

Lecture 1: Machine Learning Basics

Lecture 1: Machine Learning Basics 1/69 Lecture 1: Machine Learning Basics Ali Harakeh University of Waterloo WAVE Lab ali.harakeh@uwaterloo.ca May 1, 2017 2/69 Overview 1 Learning Algorithms 2 Capacity, Overfitting, and Underfitting 3

More information

Rule Learning With Negation: Issues Regarding Effectiveness

Rule Learning With Negation: Issues Regarding Effectiveness Rule Learning With Negation: Issues Regarding Effectiveness S. Chua, F. Coenen, G. Malcolm University of Liverpool Department of Computer Science, Ashton Building, Ashton Street, L69 3BX Liverpool, United

More information

Evidence for Reliability, Validity and Learning Effectiveness

Evidence for Reliability, Validity and Learning Effectiveness PEARSON EDUCATION Evidence for Reliability, Validity and Learning Effectiveness Introduction Pearson Knowledge Technologies has conducted a large number and wide variety of reliability and validity studies

More information

Learning From the Past with Experiment Databases

Learning From the Past with Experiment Databases Learning From the Past with Experiment Databases Joaquin Vanschoren 1, Bernhard Pfahringer 2, and Geoff Holmes 2 1 Computer Science Dept., K.U.Leuven, Leuven, Belgium 2 Computer Science Dept., University

More information

On-Line Data Analytics

On-Line Data Analytics International Journal of Computer Applications in Engineering Sciences [VOL I, ISSUE III, SEPTEMBER 2011] [ISSN: 2231-4946] On-Line Data Analytics Yugandhar Vemulapalli #, Devarapalli Raghu *, Raja Jacob

More information

Reinforcement Learning by Comparing Immediate Reward

Reinforcement Learning by Comparing Immediate Reward Reinforcement Learning by Comparing Immediate Reward Punit Pandey DeepshikhaPandey Dr. Shishir Kumar Abstract This paper introduces an approach to Reinforcement Learning Algorithm by comparing their immediate

More information

NCEO Technical Report 27

NCEO Technical Report 27 Home About Publications Special Topics Presentations State Policies Accommodations Bibliography Teleconferences Tools Related Sites Interpreting Trends in the Performance of Special Education Students

More information

Notes on The Sciences of the Artificial Adapted from a shorter document written for course (Deciding What to Design) 1

Notes on The Sciences of the Artificial Adapted from a shorter document written for course (Deciding What to Design) 1 Notes on The Sciences of the Artificial Adapted from a shorter document written for course 17-652 (Deciding What to Design) 1 Ali Almossawi December 29, 2005 1 Introduction The Sciences of the Artificial

More information

Assignment 1: Predicting Amazon Review Ratings

Assignment 1: Predicting Amazon Review Ratings Assignment 1: Predicting Amazon Review Ratings 1 Dataset Analysis Richard Park r2park@acsmail.ucsd.edu February 23, 2015 The dataset selected for this assignment comes from the set of Amazon reviews for

More information

ADVANCED MACHINE LEARNING WITH PYTHON BY JOHN HEARTY DOWNLOAD EBOOK : ADVANCED MACHINE LEARNING WITH PYTHON BY JOHN HEARTY PDF

ADVANCED MACHINE LEARNING WITH PYTHON BY JOHN HEARTY DOWNLOAD EBOOK : ADVANCED MACHINE LEARNING WITH PYTHON BY JOHN HEARTY PDF Read Online and Download Ebook ADVANCED MACHINE LEARNING WITH PYTHON BY JOHN HEARTY DOWNLOAD EBOOK : ADVANCED MACHINE LEARNING WITH PYTHON BY JOHN HEARTY PDF Click link bellow and free register to download

More information

What s in a Step? Toward General, Abstract Representations of Tutoring System Log Data

What s in a Step? Toward General, Abstract Representations of Tutoring System Log Data What s in a Step? Toward General, Abstract Representations of Tutoring System Log Data Kurt VanLehn 1, Kenneth R. Koedinger 2, Alida Skogsholm 2, Adaeze Nwaigwe 2, Robert G.M. Hausmann 1, Anders Weinstein

More information

Rule Learning with Negation: Issues Regarding Effectiveness

Rule Learning with Negation: Issues Regarding Effectiveness Rule Learning with Negation: Issues Regarding Effectiveness Stephanie Chua, Frans Coenen, and Grant Malcolm University of Liverpool Department of Computer Science, Ashton Building, Ashton Street, L69 3BX

More information

WE GAVE A LAWYER BASIC MATH SKILLS, AND YOU WON T BELIEVE WHAT HAPPENED NEXT

WE GAVE A LAWYER BASIC MATH SKILLS, AND YOU WON T BELIEVE WHAT HAPPENED NEXT WE GAVE A LAWYER BASIC MATH SKILLS, AND YOU WON T BELIEVE WHAT HAPPENED NEXT PRACTICAL APPLICATIONS OF RANDOM SAMPLING IN ediscovery By Matthew Verga, J.D. INTRODUCTION Anyone who spends ample time working

More information

CROSS-LANGUAGE INFORMATION RETRIEVAL USING PARAFAC2

CROSS-LANGUAGE INFORMATION RETRIEVAL USING PARAFAC2 1 CROSS-LANGUAGE INFORMATION RETRIEVAL USING PARAFAC2 Peter A. Chew, Brett W. Bader, Ahmed Abdelali Proceedings of the 13 th SIGKDD, 2007 Tiago Luís Outline 2 Cross-Language IR (CLIR) Latent Semantic Analysis

More information

BENCHMARK TREND COMPARISON REPORT:

BENCHMARK TREND COMPARISON REPORT: National Survey of Student Engagement (NSSE) BENCHMARK TREND COMPARISON REPORT: CARNEGIE PEER INSTITUTIONS, 2003-2011 PREPARED BY: ANGEL A. SANCHEZ, DIRECTOR KELLI PAYNE, ADMINISTRATIVE ANALYST/ SPECIALIST

More information

Diploma in Library and Information Science (Part-Time) - SH220

Diploma in Library and Information Science (Part-Time) - SH220 Diploma in Library and Information Science (Part-Time) - SH220 1. Objectives The Diploma in Library and Information Science programme aims to prepare students for professional work in librarianship. The

More information

The Internet as a Normative Corpus: Grammar Checking with a Search Engine

The Internet as a Normative Corpus: Grammar Checking with a Search Engine The Internet as a Normative Corpus: Grammar Checking with a Search Engine Jonas Sjöbergh KTH Nada SE-100 44 Stockholm, Sweden jsh@nada.kth.se Abstract In this paper some methods using the Internet as a

More information

College Pricing. Ben Johnson. April 30, Abstract. Colleges in the United States price discriminate based on student characteristics

College Pricing. Ben Johnson. April 30, Abstract. Colleges in the United States price discriminate based on student characteristics College Pricing Ben Johnson April 30, 2012 Abstract Colleges in the United States price discriminate based on student characteristics such as ability and income. This paper develops a model of college

More information

Entrepreneurial Discovery and the Demmert/Klein Experiment: Additional Evidence from Germany

Entrepreneurial Discovery and the Demmert/Klein Experiment: Additional Evidence from Germany Entrepreneurial Discovery and the Demmert/Klein Experiment: Additional Evidence from Germany Jana Kitzmann and Dirk Schiereck, Endowed Chair for Banking and Finance, EUROPEAN BUSINESS SCHOOL, International

More information

Module 12. Machine Learning. Version 2 CSE IIT, Kharagpur

Module 12. Machine Learning. Version 2 CSE IIT, Kharagpur Module 12 Machine Learning 12.1 Instructional Objective The students should understand the concept of learning systems Students should learn about different aspects of a learning system Students should

More information

Multi-Lingual Text Leveling

Multi-Lingual Text Leveling Multi-Lingual Text Leveling Salim Roukos, Jerome Quin, and Todd Ward IBM T. J. Watson Research Center, Yorktown Heights, NY 10598 {roukos,jlquinn,tward}@us.ibm.com Abstract. Determining the language proficiency

More information

Axiom 2013 Team Description Paper

Axiom 2013 Team Description Paper Axiom 2013 Team Description Paper Mohammad Ghazanfari, S Omid Shirkhorshidi, Farbod Samsamipour, Hossein Rahmatizadeh Zagheli, Mohammad Mahdavi, Payam Mohajeri, S Abbas Alamolhoda Robotics Scientific Association

More information

Australian Journal of Basic and Applied Sciences

Australian Journal of Basic and Applied Sciences AENSI Journals Australian Journal of Basic and Applied Sciences ISSN:1991-8178 Journal home page: www.ajbasweb.com Feature Selection Technique Using Principal Component Analysis For Improving Fuzzy C-Mean

More information

Visit us at:

Visit us at: White Paper Integrating Six Sigma and Software Testing Process for Removal of Wastage & Optimizing Resource Utilization 24 October 2013 With resources working for extended hours and in a pressurized environment,

More information

Learning Optimal Dialogue Strategies: A Case Study of a Spoken Dialogue Agent for

Learning Optimal Dialogue Strategies: A Case Study of a Spoken Dialogue Agent for Learning Optimal Dialogue Strategies: A Case Study of a Spoken Dialogue Agent for Email Marilyn A. Walker Jeanne C. Fromer Shrikanth Narayanan walker@research.att.com jeannie@ai.mit.edu shri@research.att.com

More information

CONCEPT MAPS AS A DEVICE FOR LEARNING DATABASE CONCEPTS

CONCEPT MAPS AS A DEVICE FOR LEARNING DATABASE CONCEPTS CONCEPT MAPS AS A DEVICE FOR LEARNING DATABASE CONCEPTS Pirjo Moen Department of Computer Science P.O. Box 68 FI-00014 University of Helsinki pirjo.moen@cs.helsinki.fi http://www.cs.helsinki.fi/pirjo.moen

More information

Machine Learning and Development Policy

Machine Learning and Development Policy Machine Learning and Development Policy Sendhil Mullainathan (joint papers with Jon Kleinberg, Himabindu Lakkaraju, Jure Leskovec, Jens Ludwig, Ziad Obermeyer) Magic? Hard not to be wowed But what makes

More information

Preliminary Report Initiative for Investigation of Race Matters and Underrepresented Minority Faculty at MIT Revised Version Submitted July 12, 2007

Preliminary Report Initiative for Investigation of Race Matters and Underrepresented Minority Faculty at MIT Revised Version Submitted July 12, 2007 Massachusetts Institute of Technology Preliminary Report Initiative for Investigation of Race Matters and Underrepresented Minority Faculty at MIT Revised Version Submitted July 12, 2007 Race Initiative

More information

Hardhatting in a Geo-World

Hardhatting in a Geo-World Hardhatting in a Geo-World TM Developed and Published by AIMS Education Foundation This book contains materials developed by the AIMS Education Foundation. AIMS (Activities Integrating Mathematics and

More information

Probabilistic Latent Semantic Analysis

Probabilistic Latent Semantic Analysis Probabilistic Latent Semantic Analysis Thomas Hofmann Presentation by Ioannis Pavlopoulos & Andreas Damianou for the course of Data Mining & Exploration 1 Outline Latent Semantic Analysis o Need o Overview

More information

Organizational Knowledge Distribution: An Experimental Evaluation

Organizational Knowledge Distribution: An Experimental Evaluation Association for Information Systems AIS Electronic Library (AISeL) AMCIS 24 Proceedings Americas Conference on Information Systems (AMCIS) 12-31-24 : An Experimental Evaluation Surendra Sarnikar University

More information

A Game-based Assessment of Children s Choices to Seek Feedback and to Revise

A Game-based Assessment of Children s Choices to Seek Feedback and to Revise A Game-based Assessment of Children s Choices to Seek Feedback and to Revise Maria Cutumisu, Kristen P. Blair, Daniel L. Schwartz, Doris B. Chin Stanford Graduate School of Education Please address all

More information

WHY GRADUATE SCHOOL? Turning Today s Technical Talent Into Tomorrow s Technology Leaders

WHY GRADUATE SCHOOL? Turning Today s Technical Talent Into Tomorrow s Technology Leaders WHY GRADUATE SCHOOL? Turning Today s Technical Talent Into Tomorrow s Technology Leaders (This presentation has been ripped-off from a number of on-line sources) Outline Why Should I Go to Graduate School?

More information

On-the-Fly Customization of Automated Essay Scoring

On-the-Fly Customization of Automated Essay Scoring Research Report On-the-Fly Customization of Automated Essay Scoring Yigal Attali Research & Development December 2007 RR-07-42 On-the-Fly Customization of Automated Essay Scoring Yigal Attali ETS, Princeton,

More information

Speech Recognition at ICSI: Broadcast News and beyond

Speech Recognition at ICSI: Broadcast News and beyond Speech Recognition at ICSI: Broadcast News and beyond Dan Ellis International Computer Science Institute, Berkeley CA Outline 1 2 3 The DARPA Broadcast News task Aspects of ICSI

More information

Edexcel GCSE. Statistics 1389 Paper 1H. June Mark Scheme. Statistics Edexcel GCSE

Edexcel GCSE. Statistics 1389 Paper 1H. June Mark Scheme. Statistics Edexcel GCSE Edexcel GCSE Statistics 1389 Paper 1H June 2007 Mark Scheme Edexcel GCSE Statistics 1389 NOTES ON MARKING PRINCIPLES 1 Types of mark M marks: method marks A marks: accuracy marks B marks: unconditional

More information

Purdue Data Summit Communication of Big Data Analytics. New SAT Predictive Validity Case Study

Purdue Data Summit Communication of Big Data Analytics. New SAT Predictive Validity Case Study Purdue Data Summit 2017 Communication of Big Data Analytics New SAT Predictive Validity Case Study Paul M. Johnson, Ed.D. Associate Vice President for Enrollment Management, Research & Enrollment Information

More information

UK Institutional Research Brief: Results of the 2012 National Survey of Student Engagement: A Comparison with Carnegie Peer Institutions

UK Institutional Research Brief: Results of the 2012 National Survey of Student Engagement: A Comparison with Carnegie Peer Institutions UK Institutional Research Brief: Results of the 2012 National Survey of Student Engagement: A Comparison with Carnegie Peer Institutions November 2012 The National Survey of Student Engagement (NSSE) has

More information

How People Learn Physics

How People Learn Physics How People Learn Physics Edward F. (Joe) Redish Dept. Of Physics University Of Maryland AAPM, Houston TX, Work supported in part by NSF grants DUE #04-4-0113 and #05-2-4987 Teaching complex subjects 2

More information

TUESDAYS/THURSDAYS, NOV. 11, 2014-FEB. 12, 2015 x COURSE NUMBER 6520 (1)

TUESDAYS/THURSDAYS, NOV. 11, 2014-FEB. 12, 2015 x COURSE NUMBER 6520 (1) MANAGERIAL ECONOMICS David.surdam@uni.edu PROFESSOR SURDAM 204 CBB TUESDAYS/THURSDAYS, NOV. 11, 2014-FEB. 12, 2015 x3-2957 COURSE NUMBER 6520 (1) This course is designed to help MBA students become familiar

More information

Chapter 10 APPLYING TOPIC MODELING TO FORENSIC DATA. 1. Introduction. Alta de Waal, Jacobus Venter and Etienne Barnard

Chapter 10 APPLYING TOPIC MODELING TO FORENSIC DATA. 1. Introduction. Alta de Waal, Jacobus Venter and Etienne Barnard Chapter 10 APPLYING TOPIC MODELING TO FORENSIC DATA Alta de Waal, Jacobus Venter and Etienne Barnard Abstract Most actionable evidence is identified during the analysis phase of digital forensic investigations.

More information

AUTHOR COPY. Techniques for cold-starting context-aware mobile recommender systems for tourism

AUTHOR COPY. Techniques for cold-starting context-aware mobile recommender systems for tourism Intelligenza Artificiale 8 (2014) 129 143 DOI 10.3233/IA-140069 IOS Press 129 Techniques for cold-starting context-aware mobile recommender systems for tourism Matthias Braunhofer, Mehdi Elahi and Francesco

More information

Evaluation of Usage Patterns for Web-based Educational Systems using Web Mining

Evaluation of Usage Patterns for Web-based Educational Systems using Web Mining Evaluation of Usage Patterns for Web-based Educational Systems using Web Mining Dave Donnellan, School of Computer Applications Dublin City University Dublin 9 Ireland daviddonnellan@eircom.net Claus Pahl

More information

Evaluation of Usage Patterns for Web-based Educational Systems using Web Mining

Evaluation of Usage Patterns for Web-based Educational Systems using Web Mining Evaluation of Usage Patterns for Web-based Educational Systems using Web Mining Dave Donnellan, School of Computer Applications Dublin City University Dublin 9 Ireland daviddonnellan@eircom.net Claus Pahl

More information

DYNAMIC ADAPTIVE HYPERMEDIA SYSTEMS FOR E-LEARNING

DYNAMIC ADAPTIVE HYPERMEDIA SYSTEMS FOR E-LEARNING University of Craiova, Romania Université de Technologie de Compiègne, France Ph.D. Thesis - Abstract - DYNAMIC ADAPTIVE HYPERMEDIA SYSTEMS FOR E-LEARNING Elvira POPESCU Advisors: Prof. Vladimir RĂSVAN

More information

Web as Corpus. Corpus Linguistics. Web as Corpus 1 / 1. Corpus Linguistics. Web as Corpus. web.pl 3 / 1. Sketch Engine. Corpus Linguistics

Web as Corpus. Corpus Linguistics. Web as Corpus 1 / 1. Corpus Linguistics. Web as Corpus. web.pl 3 / 1. Sketch Engine. Corpus Linguistics (L615) Markus Dickinson Department of Linguistics, Indiana University Spring 2013 The web provides new opportunities for gathering data Viable source of disposable corpora, built ad hoc for specific purposes

More information

Activities, Exercises, Assignments Copyright 2009 Cem Kaner 1

Activities, Exercises, Assignments Copyright 2009 Cem Kaner 1 Patterns of activities, iti exercises and assignments Workshop on Teaching Software Testing January 31, 2009 Cem Kaner, J.D., Ph.D. kaner@kaner.com Professor of Software Engineering Florida Institute of

More information

2 nd grade Task 5 Half and Half

2 nd grade Task 5 Half and Half 2 nd grade Task 5 Half and Half Student Task Core Idea Number Properties Core Idea 4 Geometry and Measurement Draw and represent halves of geometric shapes. Describe how to know when a shape will show

More information

A Note on Structuring Employability Skills for Accounting Students

A Note on Structuring Employability Skills for Accounting Students A Note on Structuring Employability Skills for Accounting Students Jon Warwick and Anna Howard School of Business, London South Bank University Correspondence Address Jon Warwick, School of Business, London

More information

THEORY OF PLANNED BEHAVIOR MODEL IN ELECTRONIC LEARNING: A PILOT STUDY

THEORY OF PLANNED BEHAVIOR MODEL IN ELECTRONIC LEARNING: A PILOT STUDY THEORY OF PLANNED BEHAVIOR MODEL IN ELECTRONIC LEARNING: A PILOT STUDY William Barnett, University of Louisiana Monroe, barnett@ulm.edu Adrien Presley, Truman State University, apresley@truman.edu ABSTRACT

More information

Success Factors for Creativity Workshops in RE

Success Factors for Creativity Workshops in RE Success Factors for Creativity s in RE Sebastian Adam, Marcus Trapp Fraunhofer IESE Fraunhofer-Platz 1, 67663 Kaiserslautern, Germany {sebastian.adam, marcus.trapp}@iese.fraunhofer.de Abstract. In today

More information

ECE-492 SENIOR ADVANCED DESIGN PROJECT

ECE-492 SENIOR ADVANCED DESIGN PROJECT ECE-492 SENIOR ADVANCED DESIGN PROJECT Meeting #3 1 ECE-492 Meeting#3 Q1: Who is not on a team? Q2: Which students/teams still did not select a topic? 2 ENGINEERING DESIGN You have studied a great deal

More information

Proficiency Illusion

Proficiency Illusion KINGSBURY RESEARCH CENTER Proficiency Illusion Deborah Adkins, MS 1 Partnering to Help All Kids Learn NWEA.org 503.624.1951 121 NW Everett St., Portland, OR 97209 Executive Summary At the heart of the

More information

STUDIES WITH FABRICATED SWITCHBOARD DATA: EXPLORING SOURCES OF MODEL-DATA MISMATCH

STUDIES WITH FABRICATED SWITCHBOARD DATA: EXPLORING SOURCES OF MODEL-DATA MISMATCH STUDIES WITH FABRICATED SWITCHBOARD DATA: EXPLORING SOURCES OF MODEL-DATA MISMATCH Don McAllaster, Larry Gillick, Francesco Scattone, Mike Newman Dragon Systems, Inc. 320 Nevada Street Newton, MA 02160

More information

Process Evaluations for a Multisite Nutrition Education Program

Process Evaluations for a Multisite Nutrition Education Program Process Evaluations for a Multisite Nutrition Education Program Paul Branscum 1 and Gail Kaye 2 1 The University of Oklahoma 2 The Ohio State University Abstract Process evaluations are an often-overlooked

More information

Word Segmentation of Off-line Handwritten Documents

Word Segmentation of Off-line Handwritten Documents Word Segmentation of Off-line Handwritten Documents Chen Huang and Sargur N. Srihari {chuang5, srihari}@cedar.buffalo.edu Center of Excellence for Document Analysis and Recognition (CEDAR), Department

More information

Learning Methods for Fuzzy Systems

Learning Methods for Fuzzy Systems Learning Methods for Fuzzy Systems Rudolf Kruse and Andreas Nürnberger Department of Computer Science, University of Magdeburg Universitätsplatz, D-396 Magdeburg, Germany Phone : +49.39.67.876, Fax : +49.39.67.8

More information

Networks and the Diffusion of Cutting-Edge Teaching and Learning Knowledge in Sociology

Networks and the Diffusion of Cutting-Edge Teaching and Learning Knowledge in Sociology RESEARCH BRIEF Networks and the Diffusion of Cutting-Edge Teaching and Learning Knowledge in Sociology Roberta Spalter-Roth, Olga V. Mayorova, Jean H. Shin, and Janene Scelza INTRODUCTION How are transformational

More information

Software Maintenance

Software Maintenance 1 What is Software Maintenance? Software Maintenance is a very broad activity that includes error corrections, enhancements of capabilities, deletion of obsolete capabilities, and optimization. 2 Categories

More information

Mathematics subject curriculum

Mathematics subject curriculum Mathematics subject curriculum Dette er ei omsetjing av den fastsette læreplanteksten. Læreplanen er fastsett på Nynorsk Established as a Regulation by the Ministry of Education and Research on 24 June

More information

Copyright Corwin 2015

Copyright Corwin 2015 2 Defining Essential Learnings How do I find clarity in a sea of standards? For students truly to be able to take responsibility for their learning, both teacher and students need to be very clear about

More information

Alpha provides an overall measure of the internal reliability of the test. The Coefficient Alphas for the STEP are:

Alpha provides an overall measure of the internal reliability of the test. The Coefficient Alphas for the STEP are: Every individual is unique. From the way we look to how we behave, speak, and act, we all do it differently. We also have our own unique methods of learning. Once those methods are identified, it can make

More information

P. Belsis, C. Sgouropoulou, K. Sfikas, G. Pantziou, C. Skourlas, J. Varnas

P. Belsis, C. Sgouropoulou, K. Sfikas, G. Pantziou, C. Skourlas, J. Varnas Exploiting Distance Learning Methods and Multimediaenhanced instructional content to support IT Curricula in Greek Technological Educational Institutes P. Belsis, C. Sgouropoulou, K. Sfikas, G. Pantziou,

More information

Linking Task: Identifying authors and book titles in verbose queries

Linking Task: Identifying authors and book titles in verbose queries Linking Task: Identifying authors and book titles in verbose queries Anaïs Ollagnier, Sébastien Fournier, and Patrice Bellot Aix-Marseille University, CNRS, ENSAM, University of Toulon, LSIS UMR 7296,

More information

Reducing Features to Improve Bug Prediction

Reducing Features to Improve Bug Prediction Reducing Features to Improve Bug Prediction Shivkumar Shivaji, E. James Whitehead, Jr., Ram Akella University of California Santa Cruz {shiv,ejw,ram}@soe.ucsc.edu Sunghun Kim Hong Kong University of Science

More information

Best Practices in Internet Ministry Released November 7, 2008

Best Practices in Internet Ministry Released November 7, 2008 Best Practices in Internet Ministry Released November 7, 2008 David T. Bourgeois, Ph.D. Associate Professor of Information Systems Crowell School of Business Biola University Best Practices in Internet

More information

Inquiry Learning Methodologies and the Disposition to Energy Systems Problem Solving

Inquiry Learning Methodologies and the Disposition to Energy Systems Problem Solving Inquiry Learning Methodologies and the Disposition to Energy Systems Problem Solving Minha R. Ha York University minhareo@yorku.ca Shinya Nagasaki McMaster University nagasas@mcmaster.ca Justin Riddoch

More information

Corpus Linguistics (L615)

Corpus Linguistics (L615) (L615) Basics of Markus Dickinson Department of, Indiana University Spring 2013 1 / 23 : the extent to which a sample includes the full range of variability in a population distinguishes corpora from archives

More information

learning collegiate assessment]

learning collegiate assessment] [ collegiate learning assessment] INSTITUTIONAL REPORT 2005 2006 Kalamazoo College council for aid to education 215 lexington avenue floor 21 new york new york 10016-6023 p 212.217.0700 f 212.661.9766

More information

Automating the E-learning Personalization

Automating the E-learning Personalization Automating the E-learning Personalization Fathi Essalmi 1, Leila Jemni Ben Ayed 1, Mohamed Jemni 1, Kinshuk 2, and Sabine Graf 2 1 The Research Laboratory of Technologies of Information and Communication

More information

CS Machine Learning

CS Machine Learning CS 478 - Machine Learning Projects Data Representation Basic testing and evaluation schemes CS 478 Data and Testing 1 Programming Issues l Program in any platform you want l Realize that you will be doing

More information

Reading Horizons. Organizing Reading Material into Thought Units to Enhance Comprehension. Kathleen C. Stevens APRIL 1983

Reading Horizons. Organizing Reading Material into Thought Units to Enhance Comprehension. Kathleen C. Stevens APRIL 1983 Reading Horizons Volume 23, Issue 3 1983 Article 8 APRIL 1983 Organizing Reading Material into Thought Units to Enhance Comprehension Kathleen C. Stevens Northeastern Illinois University Copyright c 1983

More information

ReinForest: Multi-Domain Dialogue Management Using Hierarchical Policies and Knowledge Ontology

ReinForest: Multi-Domain Dialogue Management Using Hierarchical Policies and Knowledge Ontology ReinForest: Multi-Domain Dialogue Management Using Hierarchical Policies and Knowledge Ontology Tiancheng Zhao CMU-LTI-16-006 Language Technologies Institute School of Computer Science Carnegie Mellon

More information

On the Combined Behavior of Autonomous Resource Management Agents

On the Combined Behavior of Autonomous Resource Management Agents On the Combined Behavior of Autonomous Resource Management Agents Siri Fagernes 1 and Alva L. Couch 2 1 Faculty of Engineering Oslo University College Oslo, Norway siri.fagernes@iu.hio.no 2 Computer Science

More information

TheCenter. The Myth of Number One: Indicators of Research University. Performance. The Top American Research Universities.

TheCenter. The Myth of Number One: Indicators of Research University. Performance. The Top American Research Universities. TheCenter The Myth of Number One: Indicators of Research University John V. Lombardi Diane D. Craig Elizabeth D. Capaldi Denise S. Gater Performance July 2000 The Top American Research Universities An

More information

PIRLS. International Achievement in the Processes of Reading Comprehension Results from PIRLS 2001 in 35 Countries

PIRLS. International Achievement in the Processes of Reading Comprehension Results from PIRLS 2001 in 35 Countries Ina V.S. Mullis Michael O. Martin Eugenio J. Gonzalez PIRLS International Achievement in the Processes of Reading Comprehension Results from PIRLS 2001 in 35 Countries International Study Center International

More information

University of Toronto

University of Toronto University of Toronto OFFICE OF THE VICE PRESIDENT AND PROVOST 1. Introduction A Framework for Graduate Expansion 2004-05 to 2009-10 In May, 2000, Governing Council Approved a document entitled Framework

More information

AGENDA LEARNING THEORIES LEARNING THEORIES. Advanced Learning Theories 2/22/2016

AGENDA LEARNING THEORIES LEARNING THEORIES. Advanced Learning Theories 2/22/2016 AGENDA Advanced Learning Theories Alejandra J. Magana, Ph.D. admagana@purdue.edu Introduction to Learning Theories Role of Learning Theories and Frameworks Learning Design Research Design Dual Coding Theory

More information

Susan K. Woodruff. instructional coaching scale: measuring the impact of coaching interactions

Susan K. Woodruff. instructional coaching scale: measuring the impact of coaching interactions Susan K. Woodruff instructional coaching scale: measuring the impact of coaching interactions Susan K. Woodruff Instructional Coaching Group swoodruf@comcast.net Instructional Coaching Group 301 Homestead

More information

Improving Fairness in Memory Scheduling

Improving Fairness in Memory Scheduling Improving Fairness in Memory Scheduling Using a Team of Learning Automata Aditya Kajwe and Madhu Mutyam Department of Computer Science & Engineering, Indian Institute of Tehcnology - Madras June 14, 2014

More information

Machine Learning and Data Mining. Ensembles of Learners. Prof. Alexander Ihler

Machine Learning and Data Mining. Ensembles of Learners. Prof. Alexander Ihler Machine Learning and Data Mining Ensembles of Learners Prof. Alexander Ihler Ensemble methods Why learn one classifier when you can learn many? Ensemble: combine many predictors (Weighted) combina

More information

By Laurence Capron and Will Mitchell, Boston, MA: Harvard Business Review Press, 2012.

By Laurence Capron and Will Mitchell, Boston, MA: Harvard Business Review Press, 2012. Copyright Academy of Management Learning and Education Reviews Build, Borrow, or Buy: Solving the Growth Dilemma By Laurence Capron and Will Mitchell, Boston, MA: Harvard Business Review Press, 2012. 256

More information

ABET Criteria for Accrediting Computer Science Programs

ABET Criteria for Accrediting Computer Science Programs ABET Criteria for Accrediting Computer Science Programs Mapped to 2008 NSSE Survey Questions First Edition, June 2008 Introduction and Rationale for Using NSSE in ABET Accreditation One of the most common

More information

Higher Education Six-Year Plans

Higher Education Six-Year Plans Higher Education Six-Year Plans 2018-2024 House Appropriations Committee Retreat November 15, 2017 Tony Maggio, Staff Background The Higher Education Opportunity Act of 2011 included the requirement for

More information

A NOTE ON UNDETECTED TYPING ERRORS

A NOTE ON UNDETECTED TYPING ERRORS SPkClAl SECT/ON A NOTE ON UNDETECTED TYPING ERRORS Although human proofreading is still necessary, small, topic-specific word lists in spelling programs will minimize the occurrence of undetected typing

More information

AQUA: An Ontology-Driven Question Answering System

AQUA: An Ontology-Driven Question Answering System AQUA: An Ontology-Driven Question Answering System Maria Vargas-Vera, Enrico Motta and John Domingue Knowledge Media Institute (KMI) The Open University, Walton Hall, Milton Keynes, MK7 6AA, United Kingdom.

More information

Aviation English Training: How long Does it Take?

Aviation English Training: How long Does it Take? Aviation English Training: How long Does it Take? Elizabeth Mathews 2008 I am often asked, How long does it take to achieve ICAO Operational Level 4? Unfortunately, there is no quick and easy answer to

More information

Postprint.

Postprint. http://www.diva-portal.org Postprint This is the accepted version of a paper presented at CLEF 2013 Conference and Labs of the Evaluation Forum Information Access Evaluation meets Multilinguality, Multimodality,

More information

OVERVIEW OF CURRICULUM-BASED MEASUREMENT AS A GENERAL OUTCOME MEASURE

OVERVIEW OF CURRICULUM-BASED MEASUREMENT AS A GENERAL OUTCOME MEASURE OVERVIEW OF CURRICULUM-BASED MEASUREMENT AS A GENERAL OUTCOME MEASURE Mark R. Shinn, Ph.D. Michelle M. Shinn, Ph.D. Formative Evaluation to Inform Teaching Summative Assessment: Culmination measure. Mastery

More information

REQUEST FOR PROPOSALS SUPERINTENDENT SEARCH CONSULTANT

REQUEST FOR PROPOSALS SUPERINTENDENT SEARCH CONSULTANT REQUEST FOR PROPOSALS SUPERINTENDENT SEARCH CONSULTANT Saint Paul Public Schools Independent School District # 625 360 Colborne Street Saint Paul MN 55102-3299 RFP Superintendent Search Consultant, St.

More information

OFFICE OF ENROLLMENT MANAGEMENT. Annual Report

OFFICE OF ENROLLMENT MANAGEMENT. Annual Report 2014-2015 OFFICE OF ENROLLMENT MANAGEMENT Annual Report Table of Contents 2014 2015 MESSAGE FROM THE VICE PROVOST A YEAR OF RECORDS 3 Undergraduate Enrollment 6 First-Year Students MOVING FORWARD THROUGH

More information

Alignment of Australian Curriculum Year Levels to the Scope and Sequence of Math-U-See Program

Alignment of Australian Curriculum Year Levels to the Scope and Sequence of Math-U-See Program Alignment of s to the Scope and Sequence of Math-U-See Program This table provides guidance to educators when aligning levels/resources to the Australian Curriculum (AC). The Math-U-See levels do not address

More information

Procedia - Social and Behavioral Sciences 141 ( 2014 ) WCLTA Using Corpus Linguistics in the Development of Writing

Procedia - Social and Behavioral Sciences 141 ( 2014 ) WCLTA Using Corpus Linguistics in the Development of Writing Available online at www.sciencedirect.com ScienceDirect Procedia - Social and Behavioral Sciences 141 ( 2014 ) 124 128 WCLTA 2013 Using Corpus Linguistics in the Development of Writing Blanka Frydrychova

More information

Early Warning System Implementation Guide

Early Warning System Implementation Guide Linking Research and Resources for Better High Schools betterhighschools.org September 2010 Early Warning System Implementation Guide For use with the National High School Center s Early Warning System

More information

Mastering Team Skills and Interpersonal Communication. Copyright 2012 Pearson Education, Inc. publishing as Prentice Hall.

Mastering Team Skills and Interpersonal Communication. Copyright 2012 Pearson Education, Inc. publishing as Prentice Hall. Chapter 2 Mastering Team Skills and Interpersonal Communication Chapter 2-1 Communicating Effectively in Teams Chapter 2-2 Communicating Effectively in Teams Collaboration involves working together to

More information

A Study of Metacognitive Awareness of Non-English Majors in L2 Listening

A Study of Metacognitive Awareness of Non-English Majors in L2 Listening ISSN 1798-4769 Journal of Language Teaching and Research, Vol. 4, No. 3, pp. 504-510, May 2013 Manufactured in Finland. doi:10.4304/jltr.4.3.504-510 A Study of Metacognitive Awareness of Non-English Majors

More information