Lecture 6: Applications
|
|
- Christiana Leonard
- 6 years ago
- Views:
Transcription
1 Lecture 6: Applications Michael L. Littman Rutgers University Department of Computer Science Rutgers Laboratory for Real-Life Reinforcement Learning What is RL? Branch of machine learning concerned with sequential behavior: tries to remove human activities from the inner loop of the learning process. makes systems that improve a performance metric via interaction with their environment. Much in common with goals of autonomic computing
2 Reinforcement-Learning Hypothesis Intelligent behavior arises from the actions of an individual seeking to maximize its received reward signals in a complex and changing world. Research program: identify where reward signals come from, develop algorithms that search the space of behaviors to maximize reward signals. Example: Find The Ball Learn: which way to turn to minimize steps to see goal (ball) from camera input given experience.
3 Localization: The Garden Path To teach the robot which way to turn, easier if the robot knows where it is. Teach robot to recognize where it is facing. Facing east wall, facing NW corner, etc. From an RL perspective, we ve shot ourselves in the foot. Need labels, training data. No longer autonomously learnable. Human input required. Counterintuitive Alternative? Instead, don t tell robot where it is. Give robot two things: Ability to recognize when goal is achieved. Measure of cost en route (time, in this case). Now, robot can define locations implicitly--- how do they relate to the goal? Less direct learning problem. But, no human intervention needed during learning process. Ideal setting for RL.
4 Formulation is Key RL agents can either be a big win or a nonstarter depending on the problem formulation. I ll describe several attempts I ve been involved with, good and bad. Network Repair: Diagnosis There s a failure in the network. If the computer can identify the problem, it should be easier to repair. Learn mapping from symptoms to diagnosis. Again, need to train with labeled examples. Uses our notion of an ontology of problems.
5 Network Repair: Full Connectivity repair (Littman, Ravi, Fenson, Howard 04). Recover from corrupted network interface config. Minimize time to repair. Info. gathering actions: PluggedIn, PingIp, PingLhost, PingGateway, DnsLookup, Repair actions: RenewLease, UseCachedIP, FixIP. Additional information helps to make the right choice. Needed extra code for: detecting restored connectivity (doable) keeping time (easy) Learned Policy Recovery from corrupted network interface configuration. Java/Windows XP: Minimize time to repair.
6 Spam Filtering Machine learning crucial in development of commercial-grade spam filters. Problem: Input: bag of words and other features Output: likelihood the message is spam Learning: lots of data, always changing human already in the loop (don t get feedback on suppressed messages) Adaptive Filtering A version of spam filtering amenable to RL. reward for delivering non-spam message (!10) punishment for delivering spam (+1) learn from (sparse) human feedback Pitfalls: If spam/non-spam distinction easy, encouraged to right behavior by opportunity costs. If distinction is hard, either deliver all or no messages (depending on how common spam is). Must encourage smart exploration early on so the system has a good chance to learn the distinction.
7 Spam Tagging as RL Messages arrive at a server. Server has a set of filter programs. Message is spam if fail any filter in set. Cost: Computation time to process message. Try to run cheap / likely-to-fail filters first Non-spam fixed cost, can tag spam quickly Output always same! Sorting and SAT, also (Lagoudakis, Littman, Parr) Other Relevant Applications Deadlock detection interval selection How often should we check for deadlock to balance overhead and wasted time? [Earlier talk] Network routing in changing conditions How do we decide when to find new routes? Wireless network rate selection Rate adjustment depends on whether delays are due to congestion or noise.
8 Sticky One: Network Security Recognize intrusions. Prevent intrusion symptoms. Hard to define rewards here. system needs to see both sides of the tradeoff so it doesn t solve security problems by turning off the network for legitimate use,!1 for unauthorized use Rewards (not just the policy) seems to require intrusion detection! Algorithms Discussed problems that are better/worse. Let s say we have a problem we re ready to attack, what algorithms are appropriate?
9 Families of RL Approaches policy search s value-function based model based s a s a " Q T, R More direct use, less direct learning a Search for action that maximizes value v Solve Bellman equations s r More direct learning, less direct use Some Algorithms Model-based Estimate T, R; solve approximate MDP. Prioritized sweeping, Dyna Value-function-based Use observed transitions to modify Q itself. Q-learning, SARSA Policy search Try out different policies to find the best. policy gradient, genetic approaches
10 Mixed Bag Of the three, model-based approaches appear to be most data efficient. Model-based approaches still have the problem of solving the model. In some cases, useful to cast the modelsolving problem as an RL problem! Backgammon (Tesauro): Model known, valuefunction-based learning used to solve it. Helicopter (Ng et al.): Model acquired via expert experience, policy search used to solve it. Summary Thoughts RL formulation requires computable rewards. time to goal, if goal detectable Future work: How do RL when reward function must be learned autonomically?
11 Some Robot Videos! Ng Abbeel, Helicopter Navigation #1 Nouri
12 Navigation #2 Nouri Creative Learning Walsh
13 Terrain Learning #2 Leffler, Mansley, Edmunds!"#$%&'()$! *(%)+,-.(/()$!0(&-)%)' Multiagent Reinforcement Learning Pinky and The Brain
14 The RL Way Reward optimization is a black box. If you want to influence the learning process, do it by manipulating the reward function! Examples: shaping rewards (give hints about optimal policy) (Ng, Harada, Russell 99) intrinsic motivation (rewards associated with the learning process itself---like learning new things) (Barto, Singh, Chentanez, 04) exploration bonus (encourage exploration via rewards for uncertainty) (Brafman & Tennenholtz 02) Evolutionary Perspective Chapman Cohen ( ): Human life, in line with animal life in general, has to develop not merely a dislike for such things as threaten life, but also a liking for their opposite. The development of this capacity means that in the long run the actions which promote pleasure, and those which preserve life, roughly coincide.
15 Multiagent RL What is there to talk about? Nothing: It'll just work itself out (other agents are a complex part of the environment). A bit: Without a boost, learning to work with other agents is just too hard. A lot: Must be treated directly because it is fundamentally different from other learning. Claim: Multiagent problems addressed via specialized shaping rewards. Shaping Rewards We re smart, but evolution doesn t trust us to plan all that far ahead. Evolution programs us to want things likely to bring about what we need: taste/nutrition pleasure/procreation eye contact/care generosity/cooperation
16 Shaping Rewards in RL Real task: Escape. One definition of reward function: -1 for each step, +100 for escape. Learning is too slow. If survival depends on escape, would not survive. Alternative: Additional +10 for pushing any button. We call these Shaping rewards. Pros and Cons of Shaping Can be really helpful. Not really the main task, but serve to encourage learning of pertinent parts of the model. Example: Babies like standing up. Somewhat risky. Can distract the learner so it spends all its time gathering easy-to-find, but task-irrelevant rewards. Learner can t tell a real reward from a shaping reward.
17 Why Have Social Rewards? Big advantages for (safe) cooperation. For reciprocal altruism, a species needs: repeated interactions recognize conspecifics; discriminate against defectors incentive towards long-term over short-term gain Necessary, but not sufficient: Must learn how. Drives Linked with Altruism To lead individuals to reap the benefits of reciprocal altruism, it s critical to: want to be around others, feel obligated to return favors, feel obligated to punish a defector. Evidence that the reward centers of our brains urge precisely this behavior.
18 Does Rejection Hurt? (Eisenberger et al. 03) In snubbing condition, brain centers associated with physical pain become active. Pain evident even when subjects barred from participation by technical difficulties. From Time Magazine
19 Is Cooperation Pleasurable? fmri during repeated Prisoner s Dilemma Payoffs: $3 (tempt), $2 (coop), $1 (defect), $0 (sucker) (Rilling et al. 02) Mutual cooperation most common (rational). Activation in reward center (area known to respond to desserts, pictures of pretty faces, money, cocaine) brighter for $2 (cooperative) payoff than for $3 (cheating) payoff. Is Revenge Sweet? Getting Even: Ultimatum Game Proposer is given $10. Proposer offers x! X to Responder. Responder can take it or leave it. Take it: Responder gets x, Proposer gets $10-x Leave it: Both get nothing. X = {2,8} or {2,5} or {2,2} or {2,0}
20 What Should Responder Do? Fraction of time accepting x=2 X! one-shot# repeated# human # {2,8}: 100%# 33%# 70% # {2,5}: 100%# 0%# 55% # {2,2}: 100%# 100%# 80% # {2,0}: 100%# 100%# 90% Repeated game analysis (Littman & Stone 03) Human results (Falk et al. 03) Ultimatum: Discussion Human results not rational (maximize utility). Common elements with maximizing utility assuming a repeated setting. But, not quite. Suggests other motivations/influences: reward for revenge.
21 Other Reward Functions Evidence that we have internal reward functions for some specific human-nature events appear in the popular press about once a month. Some recent ones: Love at First Sight Cuteness : Images of adorable kids and animals activates reward center. Schadenfreude Eye Contact Love at first sight. A research team led by Knut Kampe of the Institute of Cognitive Neuroscience at University College, London, has determined that eye contact with a pretty face (one judged to be attractive by the viewer [on variables such as radiance, empathy, cheerfulness, motherliness, and conventional beauty]) activates a pleasure center of the brain called the ventral striatum. Kampe's research, published in the journal Nature (2001), found that the brain-imaged pleasure response (which appears in a matter of seconds after viewing the face) only shows when mutual eye-contact is established, and does not show when looking into an attractive face whose eyes are averted or turned away.
22 Ha ha Tania Singer at University College London and her colleagues, who published a schadenfreude paper in Nature, were not actually searching for schadenfreude when they used functional magnetic resonance imaging to watch the brains of subjects in action. Their primary interest was variation in levels of empathy, which can be detected by the activity in "pain-related areas" like the "fronto-insular and anterior cingulate cortices" of the brain when a person is watching someone else in pain. The empathy circuits lighted up in both men and women when bad things happened to good people. When bad things happened to bad people, the women in the study were still empathic. But not the men. Not only did they show less empathy toward bad people, but the reward center in the left nucleus accumbens lighted up. All that translates as "Serves him right!" Evolutionary RL (Ackley & Littman 90) Evolution valued health positively, predators negatively. Tree senility: Value trees positively (defense against predators), negative long-term effects (no food). Need sophisticated intelligence for rewards (emotions!)
23 Ackley s Video
Exploration. CS : Deep Reinforcement Learning Sergey Levine
Exploration CS 294-112: Deep Reinforcement Learning Sergey Levine Class Notes 1. Homework 4 due on Wednesday 2. Project proposal feedback sent Today s Lecture 1. What is exploration? Why is it a problem?
More informationModule 12. Machine Learning. Version 2 CSE IIT, Kharagpur
Module 12 Machine Learning 12.1 Instructional Objective The students should understand the concept of learning systems Students should learn about different aspects of a learning system Students should
More informationReinforcement Learning by Comparing Immediate Reward
Reinforcement Learning by Comparing Immediate Reward Punit Pandey DeepshikhaPandey Dr. Shishir Kumar Abstract This paper introduces an approach to Reinforcement Learning Algorithm by comparing their immediate
More informationRover Races Grades: 3-5 Prep Time: ~45 Minutes Lesson Time: ~105 minutes
Rover Races Grades: 3-5 Prep Time: ~45 Minutes Lesson Time: ~105 minutes WHAT STUDENTS DO: Establishing Communication Procedures Following Curiosity on Mars often means roving to places with interesting
More informationAxiom 2013 Team Description Paper
Axiom 2013 Team Description Paper Mohammad Ghazanfari, S Omid Shirkhorshidi, Farbod Samsamipour, Hossein Rahmatizadeh Zagheli, Mohammad Mahdavi, Payam Mohajeri, S Abbas Alamolhoda Robotics Scientific Association
More informationThe dilemma of Saussurean communication
ELSEVIER BioSystems 37 (1996) 31-38 The dilemma of Saussurean communication Michael Oliphant Deparlment of Cognitive Science, University of California, San Diego, CA, USA Abstract A Saussurean communication
More informationLaboratorio di Intelligenza Artificiale e Robotica
Laboratorio di Intelligenza Artificiale e Robotica A.A. 2008-2009 Outline 2 Machine Learning Unsupervised Learning Supervised Learning Reinforcement Learning Genetic Algorithms Genetics-Based Machine Learning
More informationWhile you are waiting... socrative.com, room number SIMLANG2016
While you are waiting... socrative.com, room number SIMLANG2016 Simulating Language Lecture 4: When will optimal signalling evolve? Simon Kirby simon@ling.ed.ac.uk T H E U N I V E R S I T Y O H F R G E
More informationIAT 888: Metacreation Machines endowed with creative behavior. Philippe Pasquier Office 565 (floor 14)
IAT 888: Metacreation Machines endowed with creative behavior Philippe Pasquier Office 565 (floor 14) pasquier@sfu.ca Outline of today's lecture A little bit about me A little bit about you What will that
More informationLecture 10: Reinforcement Learning
Lecture 1: Reinforcement Learning Cognitive Systems II - Machine Learning SS 25 Part III: Learning Programs and Strategies Q Learning, Dynamic Programming Lecture 1: Reinforcement Learning p. Motivation
More informationGetting Started with Deliberate Practice
Getting Started with Deliberate Practice Most of the implementation guides so far in Learning on Steroids have focused on conceptual skills. Things like being able to form mental images, remembering facts
More informationSoftware Maintenance
1 What is Software Maintenance? Software Maintenance is a very broad activity that includes error corrections, enhancements of capabilities, deletion of obsolete capabilities, and optimization. 2 Categories
More informationRadius STEM Readiness TM
Curriculum Guide Radius STEM Readiness TM While today s teens are surrounded by technology, we face a stark and imminent shortage of graduates pursuing careers in Science, Technology, Engineering, and
More informationB. How to write a research paper
From: Nikolaus Correll. "Introduction to Autonomous Robots", ISBN 1493773070, CC-ND 3.0 B. How to write a research paper The final deliverable of a robotics class often is a write-up on a research project,
More informationRed Flags of Conflict
CONFLICT MANAGEMENT Introduction Webster s Dictionary defines conflict as a battle, contest of opposing forces, discord, antagonism existing between primitive desires, instincts and moral, religious, or
More informationA Pipelined Approach for Iterative Software Process Model
A Pipelined Approach for Iterative Software Process Model Ms.Prasanthi E R, Ms.Aparna Rathi, Ms.Vardhani J P, Mr.Vivek Krishna Electronics and Radar Development Establishment C V Raman Nagar, Bangalore-560093,
More informationSeminar - Organic Computing
Seminar - Organic Computing Self-Organisation of OC-Systems Markus Franke 25.01.2006 Typeset by FoilTEX Timetable 1. Overview 2. Characteristics of SO-Systems 3. Concern with Nature 4. Design-Concepts
More informationLaboratorio di Intelligenza Artificiale e Robotica
Laboratorio di Intelligenza Artificiale e Robotica A.A. 2008-2009 Outline 2 Machine Learning Unsupervised Learning Supervised Learning Reinforcement Learning Genetic Algorithms Genetics-Based Machine Learning
More informationAGENDA LEARNING THEORIES LEARNING THEORIES. Advanced Learning Theories 2/22/2016
AGENDA Advanced Learning Theories Alejandra J. Magana, Ph.D. admagana@purdue.edu Introduction to Learning Theories Role of Learning Theories and Frameworks Learning Design Research Design Dual Coding Theory
More informationExperience Corps. Mentor Toolkit
Experience Corps Mentor Toolkit 2 AARP Foundation Experience Corps Mentor Toolkit June 2015 Christian Rummell Ed. D., Senior Researcher, AIR 3 4 Contents Introduction and Overview...6 Tool 1: Definitions...8
More informationOn the Combined Behavior of Autonomous Resource Management Agents
On the Combined Behavior of Autonomous Resource Management Agents Siri Fagernes 1 and Alva L. Couch 2 1 Faculty of Engineering Oslo University College Oslo, Norway siri.fagernes@iu.hio.no 2 Computer Science
More informationLecturing in a Loincloth
THE CHRONICLE REVIEW Lecturing in a Loincloth Griffin Kenemer, NG Studios By Bill Schindler MARCH 13, 2016 Ifashioned from brain-tanned deerskins. The am alone, shivering, bobbing in a dugout canoe off
More informationCritical Thinking in Everyday Life: 9 Strategies
Critical Thinking in Everyday Life: 9 Strategies Most of us are not what we could be. We are less. We have great capacity. But most of it is dormant; most is undeveloped. Improvement in thinking is like
More informationReinForest: Multi-Domain Dialogue Management Using Hierarchical Policies and Knowledge Ontology
ReinForest: Multi-Domain Dialogue Management Using Hierarchical Policies and Knowledge Ontology Tiancheng Zhao CMU-LTI-16-006 Language Technologies Institute School of Computer Science Carnegie Mellon
More informationHow To Take Control In Your Classroom And Put An End To Constant Fights And Arguments
How To Take Control In Your Classroom And Put An End To Constant Fights And Arguments Free Report Marjan Glavac How To Take Control In Your Classroom And Put An End To Constant Fights And Arguments A Difficult
More informationTwo Futures of Software Testing
WWW.QUALTECHCONFERENCES.COM Europe s Premier Software Testing Event World Forum Convention Centre, The Hague, Netherlands The Future of Software Testing Two Futures of Software Testing Michael Bolton,
More informationThe Strong Minimalist Thesis and Bounded Optimality
The Strong Minimalist Thesis and Bounded Optimality DRAFT-IN-PROGRESS; SEND COMMENTS TO RICKL@UMICH.EDU Richard L. Lewis Department of Psychology University of Michigan 27 March 2010 1 Purpose of this
More informationLearning Optimal Dialogue Strategies: A Case Study of a Spoken Dialogue Agent for
Learning Optimal Dialogue Strategies: A Case Study of a Spoken Dialogue Agent for Email Marilyn A. Walker Jeanne C. Fromer Shrikanth Narayanan walker@research.att.com jeannie@ai.mit.edu shri@research.att.com
More informationHentai High School A Game Guide
Hentai High School A Game Guide Hentai High School is a sex game where you are the Principal of a high school with the goal of turning the students into sex crazed people within 15 years. The game is difficult
More informationKnowledge based expert systems D H A N A N J A Y K A L B A N D E
Knowledge based expert systems D H A N A N J A Y K A L B A N D E What is a knowledge based system? A Knowledge Based System or a KBS is a computer program that uses artificial intelligence to solve problems
More informationSIMPLY THE BEST! AND MINDSETS. (Growth or fixed?)
SIMPLY THE BEST! AND MINDSETS (Growth or fixed?) SIMPLY THE BEST Why American Schools are the Best in the World! Kindergarten through High School EVERYONE! No exceptions. No disclaimers. So why all the
More informationGoing to School: Measuring Schooling Behaviors in GloFish
Name Period Date Going to School: Measuring Schooling Behaviors in GloFish Objective The learner will collect data to determine if schooling behaviors are exhibited in GloFish fluorescent fish. The learner
More informationThe Flaws, Fallacies and Foolishness of Benchmark Testing
Benchmarking is a great tool for improving an organization's performance...when used or identifying, then tracking (by measuring) specific variables that are proven to be "S.M.A.R.T." That is: Specific
More informationStrategy Study on Primary School English Game Teaching
6th International Conference on Electronic, Mechanical, Information and Management (EMIM 2016) Strategy Study on Primary School English Game Teaching Feng He Primary Education College, Linyi University
More informationPREP S SPEAKER LISTENER TECHNIQUE COACHING MANUAL
1 PREP S SPEAKER LISTENER TECHNIQUE COACHING MANUAL IMPORTANCE OF THE SPEAKER LISTENER TECHNIQUE The Speaker Listener Technique (SLT) is a structured communication strategy that promotes clarity, understanding,
More informationSTUDENT MOODLE ORIENTATION
BAKER UNIVERSITY SCHOOL OF PROFESSIONAL AND GRADUATE STUDIES STUDENT MOODLE ORIENTATION TABLE OF CONTENTS Introduction to Moodle... 2 Online Aptitude Assessment... 2 Moodle Icons... 6 Logging In... 8 Page
More informationNo Parent Left Behind
No Parent Left Behind Navigating the Special Education Universe SUSAN M. BREFACH, Ed.D. Page i Introduction How To Know If This Book Is For You Parents have become so convinced that educators know what
More informationLEGO MINDSTORMS Education EV3 Coding Activities
LEGO MINDSTORMS Education EV3 Coding Activities s t e e h s k r o W t n e d Stu LEGOeducation.com/MINDSTORMS Contents ACTIVITY 1 Performing a Three Point Turn 3-6 ACTIVITY 2 Written Instructions for a
More informationThis map-tastic middle-grade story from Andrew Clements gives the phrase uncharted territory a whole new meaning!
A Curriculum Guide to The Map Trap By Andrew Clements About the Book This map-tastic middle-grade story from Andrew Clements gives the phrase uncharted territory a whole new meaning! Alton Barnes loves
More informationThesis-Proposal Outline/Template
Thesis-Proposal Outline/Template Kevin McGee 1 Overview This document provides a description of the parts of a thesis outline and an example of such an outline. It also indicates which parts should be
More informationISFA2008U_120 A SCHEDULING REINFORCEMENT LEARNING ALGORITHM
Proceedings of 28 ISFA 28 International Symposium on Flexible Automation Atlanta, GA, USA June 23-26, 28 ISFA28U_12 A SCHEDULING REINFORCEMENT LEARNING ALGORITHM Amit Gil, Helman Stern, Yael Edan, and
More informationAn Introduction to Simio for Beginners
An Introduction to Simio for Beginners C. Dennis Pegden, Ph.D. This white paper is intended to introduce Simio to a user new to simulation. It is intended for the manufacturing engineer, hospital quality
More informationA BOOK IN A SLIDESHOW. The Dragonfly Effect JENNIFER AAKER & ANDY SMITH
A BOOK IN A SLIDESHOW The Dragonfly Effect JENNIFER AAKER & ANDY SMITH THE DRAGONFLY MODEL FOCUS GRAB ATTENTION TAKE ACTION ENGAGE A Book In A Slideshow JENNIFER AAKER & ANDY SMITH WING 1: FOCUS IDENTIFY
More informationWhy Pay Attention to Race?
Why Pay Attention to Race? Witnessing Whiteness Chapter 1 Workshop 1.1 1.1-1 Dear Facilitator(s), This workshop series was carefully crafted, reviewed (by a multiracial team), and revised with several
More informationThe Foundations of Interpersonal Communication
L I B R A R Y A R T I C L E The Foundations of Interpersonal Communication By Dennis Emberling, President of Developmental Consulting, Inc. Introduction Mark Twain famously said, Everybody talks about
More informationALL-IN-ONE MEETING GUIDE THE ECONOMICS OF WELL-BEING
ALL-IN-ONE MEETING GUIDE THE ECONOMICS OF WELL-BEING LeanIn.0rg, 2016 1 Overview Do we limit our thinking and focus only on short-term goals when we make trade-offs between career and family? This final
More informationP a g e 1. Grade 4. Grant funded by: MS Exemplar Unit English Language Arts Grade 4 Edition 1
P a g e 1 Grade 4 Grant funded by: P a g e 2 Lesson 1: Understanding Themes Focus Standard(s): RL.4.2 Additional Standard(s): RL.4.1 Estimated Time: 1-2 days Resources and Materials: Handout 1.1: Details,
More informationSpeeding Up Reinforcement Learning with Behavior Transfer
Speeding Up Reinforcement Learning with Behavior Transfer Matthew E. Taylor and Peter Stone Department of Computer Sciences The University of Texas at Austin Austin, Texas 78712-1188 {mtaylor, pstone}@cs.utexas.edu
More informationPython Machine Learning
Python Machine Learning Unlock deeper insights into machine learning with this vital guide to cuttingedge predictive analytics Sebastian Raschka [ PUBLISHING 1 open source I community experience distilled
More informationAn OO Framework for building Intelligence and Learning properties in Software Agents
An OO Framework for building Intelligence and Learning properties in Software Agents José A. R. P. Sardinha, Ruy L. Milidiú, Carlos J. P. Lucena, Patrick Paranhos Abstract Software agents are defined as
More informationCurriculum Design Project with Virtual Manipulatives. Gwenanne Salkind. George Mason University EDCI 856. Dr. Patricia Moyer-Packenham
Curriculum Design Project with Virtual Manipulatives Gwenanne Salkind George Mason University EDCI 856 Dr. Patricia Moyer-Packenham Spring 2006 Curriculum Design Project with Virtual Manipulatives Table
More informationSOFTWARE EVALUATION TOOL
SOFTWARE EVALUATION TOOL Kyle Higgins Randall Boone University of Nevada Las Vegas rboone@unlv.nevada.edu Higgins@unlv.nevada.edu N.B. This form has not been fully validated and is still in development.
More informationArtificial Neural Networks written examination
1 (8) Institutionen för informationsteknologi Olle Gällmo Universitetsadjunkt Adress: Lägerhyddsvägen 2 Box 337 751 05 Uppsala Artificial Neural Networks written examination Monday, May 15, 2006 9 00-14
More informationExtending Learning Across Time & Space: The Power of Generalization
Extending Learning: The Power of Generalization 1 Extending Learning Across Time & Space: The Power of Generalization Teachers have every right to celebrate when they finally succeed in teaching struggling
More informationA Reinforcement Learning Variant for Control Scheduling
A Reinforcement Learning Variant for Control Scheduling Aloke Guha Honeywell Sensor and System Development Center 3660 Technology Drive Minneapolis MN 55417 Abstract We present an algorithm based on reinforcement
More informationTop Ten Persuasive Strategies Used on the Web - Cathy SooHoo, 5/17/01
Top Ten Persuasive Strategies Used on the Web - Cathy SooHoo, 5/17/01 Introduction Although there is nothing new about the human use of persuasive strategies, web technologies usher forth a new level of
More informationUnderstanding and Changing Habits
Understanding and Changing Habits We are what we repeatedly do. Excellence, then, is not an act, but a habit. Aristotle Have you ever stopped to think about your habits or how they impact your daily life?
More informationLearning Prospective Robot Behavior
Learning Prospective Robot Behavior Shichao Ou and Rod Grupen Laboratory for Perceptual Robotics Computer Science Department University of Massachusetts Amherst {chao,grupen}@cs.umass.edu Abstract This
More informationHow to make an A in Physics 101/102. Submitted by students who earned an A in PHYS 101 and PHYS 102.
How to make an A in Physics 101/102. Submitted by students who earned an A in PHYS 101 and PHYS 102. PHYS 102 (Spring 2015) Don t just study the material the day before the test know the material well
More informationOFFICE OF ENROLLMENT MANAGEMENT. Annual Report
2014-2015 OFFICE OF ENROLLMENT MANAGEMENT Annual Report Table of Contents 2014 2015 MESSAGE FROM THE VICE PROVOST A YEAR OF RECORDS 3 Undergraduate Enrollment 6 First-Year Students MOVING FORWARD THROUGH
More informationAdaptations and Survival: The Story of the Peppered Moth
Adaptations and Survival: The Story of the Peppered Moth Teacher: Rachel Card Subject Areas: Science/ELA Grade Level: Fourth Unit Title: Animal Adaptations Lesson Title: Adaptations and Survival: The Story
More informationThe Good Judgment Project: A large scale test of different methods of combining expert predictions
The Good Judgment Project: A large scale test of different methods of combining expert predictions Lyle Ungar, Barb Mellors, Jon Baron, Phil Tetlock, Jaime Ramos, Sam Swift The University of Pennsylvania
More informationA Case-Based Approach To Imitation Learning in Robotic Agents
A Case-Based Approach To Imitation Learning in Robotic Agents Tesca Fitzgerald, Ashok Goel School of Interactive Computing Georgia Institute of Technology, Atlanta, GA 30332, USA {tesca.fitzgerald,goel}@cc.gatech.edu
More informationOn Human Computer Interaction, HCI. Dr. Saif al Zahir Electrical and Computer Engineering Department UBC
On Human Computer Interaction, HCI Dr. Saif al Zahir Electrical and Computer Engineering Department UBC Human Computer Interaction HCI HCI is the study of people, computer technology, and the ways these
More informationNotes on The Sciences of the Artificial Adapted from a shorter document written for course (Deciding What to Design) 1
Notes on The Sciences of the Artificial Adapted from a shorter document written for course 17-652 (Deciding What to Design) 1 Ali Almossawi December 29, 2005 1 Introduction The Sciences of the Artificial
More informationRETURNING TEACHER REQUIRED TRAINING MODULE YE TRANSCRIPT
RETURNING TEACHER REQUIRED TRAINING MODULE YE Slide 1. The Dynamic Learning Maps Alternate Assessments are designed to measure what students with significant cognitive disabilities know and can do in relation
More informationFormative Assessment in Mathematics. Part 3: The Learner s Role
Formative Assessment in Mathematics Part 3: The Learner s Role Dylan Wiliam Equals: Mathematics and Special Educational Needs 6(1) 19-22; Spring 2000 Introduction This is the last of three articles reviewing
More informationConversation Starters: Using Spatial Context to Initiate Dialogue in First Person Perspective Games
Conversation Starters: Using Spatial Context to Initiate Dialogue in First Person Perspective Games David B. Christian, Mark O. Riedl and R. Michael Young Liquid Narrative Group Computer Science Department
More informationPart I. Figuring out how English works
9 Part I Figuring out how English works 10 Chapter One Interaction and grammar Grammar focus. Tag questions Introduction. How closely do you pay attention to how English is used around you? For example,
More informationCS 100: Principles of Computing
CS 100: Principles of Computing Kevin Molloy August 29, 2017 1 Basic Course Information 1.1 Prerequisites: None 1.2 General Education Fulfills Mason Core requirement in Information Technology (ALL). 1.3
More informationCAFE ESSENTIAL ELEMENTS O S E P P C E A. 1 Framework 2 CAFE Menu. 3 Classroom Design 4 Materials 5 Record Keeping
CAFE RE P SU C 3 Classroom Design 4 Materials 5 Record Keeping P H ND 1 Framework 2 CAFE Menu R E P 6 Assessment 7 Choice 8 Whole-Group Instruction 9 Small-Group Instruction 10 One-on-one Instruction 11
More informationSpeech Recognition at ICSI: Broadcast News and beyond
Speech Recognition at ICSI: Broadcast News and beyond Dan Ellis International Computer Science Institute, Berkeley CA Outline 1 2 3 The DARPA Broadcast News task Aspects of ICSI
More informationTEAM-BUILDING GAMES, ACTIVITIES AND IDEAS
1. Drop the Ball Time: 10 12 minutes Purpose: Cooperation and healthy competition Participants: Small groups Materials needed: Golf balls, straws, tape Each small group receives 12 straws and 18 inches
More informationMENTORING. Tips, Techniques, and Best Practices
MENTORING Tips, Techniques, and Best Practices This paper reflects the experiences shared by many mentor mediators and those who have been mentees. The points are displayed for before, during, and after
More informationReduce the Failure Rate of the Screwing Process with Six Sigma Approach
Proceedings of the 2014 International Conference on Industrial Engineering and Operations Management Bali, Indonesia, January 7 9, 2014 Reduce the Failure Rate of the Screwing Process with Six Sigma Approach
More informationMajor Milestones, Team Activities, and Individual Deliverables
Major Milestones, Team Activities, and Individual Deliverables Milestone #1: Team Semester Proposal Your team should write a proposal that describes project objectives, existing relevant technology, engineering
More informationHigh-level Reinforcement Learning in Strategy Games
High-level Reinforcement Learning in Strategy Games Christopher Amato Department of Computer Science University of Massachusetts Amherst, MA 01003 USA camato@cs.umass.edu Guy Shani Department of Computer
More informationResults In. Planning Questions. Tony Frontier Five Levers to Improve Learning 1
Key Tables and Concepts: Five Levers to Improve Learning by Frontier & Rickabaugh 2014 Anticipated Results of Three Magnitudes of Change Characteristics of Three Magnitudes of Change Examples Results In.
More informationConducting an interview
Basic Public Affairs Specialist Course Conducting an interview In the newswriting portion of this course, you learned basic interviewing skills. From that lesson, you learned an interview is an exchange
More informationWhat s in Your Communication Toolbox? COMMUNICATION TOOLBOX. verse clinical scenarios to bolster clinical outcomes: 1
COMMUNICATION TOOLBOX Lisa Hunter, LSW, and Jane R. Shaw, DVM, PhD www.argusinstitute.colostate.edu What s in Your Communication Toolbox? Throughout this communication series, we have built a toolbox of
More informationSpeak Up 2012 Grades 9 12
2012 Speak Up Survey District: WAYLAND PUBLIC SCHOOLS Speak Up 2012 Grades 9 12 Results based on 130 survey(s). Note: Survey responses are based upon the number of individuals that responded to the specific
More informationMastering Team Skills and Interpersonal Communication. Copyright 2012 Pearson Education, Inc. publishing as Prentice Hall.
Chapter 2 Mastering Team Skills and Interpersonal Communication Chapter 2-1 Communicating Effectively in Teams Chapter 2-2 Communicating Effectively in Teams Collaboration involves working together to
More informationLearning Lesson Study Course
Learning Lesson Study Course Developed originally in Japan and adapted by Developmental Studies Center for use in schools across the United States, lesson study is a model of professional development in
More informationArchitecting Interaction Styles
- provocation facilitation leading empathic interviewing whiteboard simulation judo tactics when in an impasse: provoke effective when used sparsely especially recommended when new in a field: contribute
More informationMAILCOM Las Vegas. October 2-4, Senior Director, Proposal Management BrightKey, Inc.
MAILCOM Las Vegas October 2-4, 2017 CRS#: LD250 Session: Mystery Solved! Cracking the Case on Productivity Day/Date: Tuesday, October 3, 2017 Round/Time: Round 5, 11:30am-12:30pm Presented By: Sally S.
More informationTD(λ) and Q-Learning Based Ludo Players
TD(λ) and Q-Learning Based Ludo Players Majed Alhajry, Faisal Alvi, Member, IEEE and Moataz Ahmed Abstract Reinforcement learning is a popular machine learning technique whose inherent self-learning ability
More informationPh.D. in Behavior Analysis Ph.d. i atferdsanalyse
Program Description Ph.D. in Behavior Analysis Ph.d. i atferdsanalyse 180 ECTS credits Approval Approved by the Norwegian Agency for Quality Assurance in Education (NOKUT) on the 23rd April 2010 Approved
More informationFirms and Markets Saturdays Summer I 2014
PRELIMINARY DRAFT VERSION. SUBJECT TO CHANGE. Firms and Markets Saturdays Summer I 2014 Professor Thomas Pugel Office: Room 11-53 KMC E-mail: tpugel@stern.nyu.edu Tel: 212-998-0918 Fax: 212-995-4212 This
More informationWriting the Personal Statement
Writing the Personal Statement For Graduate School Applications ZIA ISOLA, PHD RESEARCH MENTORING INSTITUTE OFFICE OF DIVERSITY, GENOMICS INSTITUTE Overview: The Parts of a Graduate School Application!
More informationWORK OF LEADERS GROUP REPORT
WORK OF LEADERS GROUP REPORT ASSESSMENT TO ACTION. Sample Report (9 People) Thursday, February 0, 016 This report is provided by: Your Company 13 Main Street Smithtown, MN 531 www.yourcompany.com INTRODUCTION
More informationIntel-powered Classmate PC. SMART Response* Training Foils. Version 2.0
Intel-powered Classmate PC Training Foils Version 2.0 1 Legal Information INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE,
More informationBeyond Classroom Solutions: New Design Perspectives for Online Learning Excellence
Educational Technology & Society 5(2) 2002 ISSN 1436-4522 Beyond Classroom Solutions: New Design Perspectives for Online Learning Excellence Moderator & Sumamrizer: Maggie Martinez CEO, The Training Place,
More informationWhat is Teaching? JOHN A. LOTT Professor Emeritus in Pathology College of Medicine
What is Teaching? JOHN A. LOTT Professor Emeritus in Pathology College of Medicine What is teaching? As I started putting this essay together, I realized that most of my remarks were aimed at students
More informationCourse Objectives Upon completion of this course, you will: Have a clear grasp of organic gardening techniques and methods
Organic Gardening Instructor: Fiona Doherty, fcd9@cornell.edu Purpose This 6-week online course is intended to examine the basics of small-scale organic gardening. The topics and depth of information offered
More informationUniversity of Toronto Physics Practicals. University of Toronto Physics Practicals. University of Toronto Physics Practicals
This is the PowerPoint of an invited talk given to the Physics Education section of the Canadian Association of Physicists annual Congress in Quebec City in July 2008 -- David Harrison, david.harrison@utoronto.ca
More informationEXECUTIVE SUMMARY. Online courses for credit recovery in high schools: Effectiveness and promising practices. April 2017
EXECUTIVE SUMMARY Online courses for credit recovery in high schools: Effectiveness and promising practices April 2017 Prepared for the Nellie Mae Education Foundation by the UMass Donahue Institute 1
More informationRunning head: THE INTERACTIVITY EFFECT IN MULTIMEDIA LEARNING 1
Running head: THE INTERACTIVITY EFFECT IN MULTIMEDIA LEARNING 1 The Interactivity Effect in Multimedia Learning Environments Richard A. Robinson Boise State University THE INTERACTIVITY EFFECT IN MULTIMEDIA
More informationTime Management. To receive regular updates kindly send test to : 1
Time Management CA. Rajkumar S Adukia B.Com (Hons), FCA, ACS, ACWA, LLB, DIPR, DLL &LP, IFRS(UK), MBA email id: rajkumarradukia@caaa.in Mob: 09820061049/9323061049 To receive regular updates kindly send
More informationWELCOME! Of Social Competency. Using Social Thinking and. Social Thinking and. the UCLA PEERS Program 5/1/2017. My Background/ Who Am I?
Social Thinking and the UCLA PEERS Program Joan Storey Gorsuch, M.Ed. Social Champaign Champaign, Illinois j.s.gorsuch@gmail.com WELCOME! THE And Using Social Thinking and the UCLA PEERS Program Of Social
More informationProcess improvement, The Agile Way! By Ben Linders Published in Methods and Tools, winter
Process improvement, The Agile Way! By Ben Linders Published in Methods and Tools, winter 2010. http://www.methodsandtools.com/ Summary Business needs for process improvement projects are changing. Organizations
More information