Reinforcement Learning. Reinforcement learning and HMMs. Hidden Markov Models (HMMs) are appropriate when our agent models the world as follows
|
|
- Harvey Shepherd
- 5 years ago
- Views:
Transcription
1 Reinforcement Learning Reinforcement learning and HMMs We now examine: some potential shortcomings of hidden Markov models, and of supervised learning; an extension know as the Markov Decision Process (MDP); the way in which we might learn from rewards gained as a result of acting within an environment; specific, simple algorithms for performing such learning, and their convergence properties. Reading: Russell and Norvig, chapter 21. Mitchell chapter 13. Copyright c Sean Holden Hidden Markov Models (HMMs) are appropriate when our agent models the world as follows Pr(S0) S0 S1 S2 S3 E1 Pr(St St 1) Pr(Et St) and only wants to infer information about the state of the world on the basis of observing the available evidence. This might be criticised as un-necessarily restricted, although it is very effective for the right kind of problem. E2 E3 Reinforcement learning and supervised learning Reinforcement learning: the basic case Supervised learners learn from specifically labelled chunks of information: We now begin to model the world in a more realistic way as follows: x??? S0 S1 S2 S3 (x1, 1) (x2, 1) (x3, 0). In any state: Perform an action a to move to a new state. (There may be many possibilities.) Receive a reward r depending on the start state and action. The agent can perform actions in order to change the world s state. This might also be criticised as un-necessarily restricted: there are other ways to learn. If the agent performs an action in a particular state, then it gains a corresponding reward.
2 Deterministic Markov Decision Processes Formally, we have a set of states S = {s 1, s 2,...,s n } and in each state we can perform one of a set of actions We also have a function A = {a 1, a 2,...,a m }. S : S A S such that S(s, a) is the new state resulting from performing action a in state s, and a function R : S A R such that R(s, a) is the reward obtained by executing action a in state s. Deterministic Markov Decision Processes From the point of view of the agent, there is a matter of considerable importance: The agent does not have access to the functions S and R. It therefore has to learn a policy, which is a function p : S A such that p(s) provides the action a that should be executed in state s. What might the agent use as its criterion for learning a policy? Measuring the quality of a policy Say we start in a state at time t, denoted s t, and we follow a policy p. At each future step in time we get a reward. Denote the rewards r t, r t+1,... and so on. A common measure of the quality of a policy p is the discounted cumulative reward V p (s t ) = ǫ i r t+i = r t + ǫr t+1 + ǫ 2 r t+2 + where 0 ǫ 1 is a constant, which defines a trade-off for how much we value immediate rewards against future rewards. Measuring the quality of a policy Other common measures are the average reward 1 T lim r t+i T T and the finite horizon reward T In these notes we will only address the discounted cumulative reward. r t+i The intuition for this measure is that, on the whole, we should like our agent to prefer rewards gained quickly.
3 Two important issues Note that in this kind of problem we need to address two particularly relevant issues: The temporal credit assignment problem: that is, how do we decide which specific actions are important in obtaining a reward? The exploration/exploitation problem. How do we decide between exploiting the knowledge we already have, and exploring the environment in order to possibly obtain new (and more useful) knowledge? The optimal policy Ultimately, our learner s aim is to learn the optimal policy p opt = argmaxv p (s) p for all s. We will denote the optimal discounted cumulative reward as V opt (s) = V p opt (s). How might we go about learning the optimal policy? We will see later how to deal with these. Learning the optimal policy The only information we have during learning is the individual rewards obtained from the environment. We could try to learn V opt (s) directly, so that states can be compared: Consider s as better than s if V opt (s) > V opt (s ). However we actually want to compare actions, not states. Learning V opt (s) might help as p opt (s) = argmax [R(s, a) + ǫv opt (S(s, a))] a but only if we know S and R. The Q function The trick is to define the following function: Q(s, a) = R(s, a) + ǫv opt (S(s, a)) This function specifies the discounted cumulative reward obtained if you do action a in state s and then follow the optimal policy. As p opt (s) = argmax Q(s, a) a then provided one can learn Q it is not necessary to have knowledge of S and R to obtain the optimal policy. As we are interested in the case where these functions are not known, we need something slightly different.
4 The Q function Note also that V opt (s) = max Q(s,) and so Q(s,a) = R(s, a) + ǫ max Q(S(s, a),) which suggests a simple learning algorithm. Let Q be our learner s estimate of what the exact Q function is. That is, in the current scenario Q is a table containing the estimated values of Q(s,a) for all pairs (s, a). Q-learning Start with all entries in Q set to 0. (In fact we will see in a moment that random entries will do.) Repeat the following: 1. Look at the current state s and choose an action a. (We will see how to do this in a moment.) 2. Do the action a and obtain some reward R(s,a). 3. Observe the new state S(s, a). 4. Perform the update Q (s, a) = R(s, a) + ǫ max Q (S(s, a),) Note that this can be done in episodes. For example, in learning to play games, we can play multiple games, each being a single episode. This looks as though it might converge! Note that, if the rewards are at least 0 and we initialise Q to 0 then, n,s, a Q n+1(s, a) Q n(s,a) and n,s, a Q(s, a) Q n(s, a) 0 However, we need to be a bit more rigorous than this... If: 1. the agent is operating in an environment that is a deterministic MDP; 2. rewards are bounded in the sense that there is a constant δ > 0 such that s,a R(s,a) < δ 3. all possible pairs s and a are visited infinitely often; then the Q-learning algorithm converges, in the sense that as n. a,s Q n(s, a) Q(s,a)
5 This is straightforward to demonstrate. Using condition 3, take two stretches of time in which all s and a pairs occur: All s, a occur All s, a occur Define ξ(n) = max s,a Q n(s, a) Q(s,a) the maximum error in Q at n. We have, Q n+1(s, a) Q(s,a) = (R(s,a) ǫ max = ǫ max Q n(s(s, a),) max Q n(s(s, a),) Q(S(s,a),) ǫ max ǫ max s,a Q n(s, a) Q(s,a) = ǫξ(n). Convergence as described follows. Q n(s(s, a),)) (R(s, a) ǫ max Q(S(s, a),)) Q(S(s,a),) What happens when Q n(s, a) is updated to Q n+1(s, a)? Choosing actions to perform We have not yet answered the question of how to choose actions to perform during learning. One approach is to choose actions based on our current estimate Q. For instance action chosen in current state s = argmaxq (s, a). a However we have already noted the trade-off between exploration and exploitation. It makes more sense to: explore during the early stages of training; exploit during the later stages of training. Choosing actions to perform One way in which to choose actions that incorporates these requirements is to introduce a constant λ and choose actions probabilistically according to Note that: Pr(action a state s) = if λ is small this promotes exploration; if λ is large this promotes exploitation. We can vary λ as training progresses. λq (s,a) a λq (s,a) This seems particularly important in the light of condition 3 of the convergence proof.
6 Improving the training process There are two simple ways in which the process can be improved: 1. If training is episodic, we can store the rewards obtained during an episode and update backwards at the end. This allows better updating at the expense of requiring more memory. 2. We can remember information about rewards and occasionally re-use it by re-training. Nondeterministic MDPs The Q-learning algorithm generalises easily to a more realistic situation, where the outcomes of actions are probabilistic. Instead of the functions S and R we have probability distributions and Pr(new state current state, action) Pr(reward current state, action). and we now use S(s, a) and R(s, a) to denote the corresponding random variables. We now have ( ) V p = E ǫ i r t+i and the best policy p opt maximises V p. We now have Q-learning for nondeterministic MDPs Q(s, a) = E(R(s,a)) + ǫ σ = E(R(s,a)) + ǫ σ Pr(σ s,a)v opt (σ) Pr(σ s, a) max Q(σ,) and the rule for learning becomes ] Q n+1 = (1 θ n+1 )Q n(s, a) + θ n+1 [R(s, a) + max Q n(s(s, a),) with θ n+1 = v n+1 (s, a) where v n+1 (s, a) is the number of times the pair s and a has been visited so far. If: for nondeterministic MDPs 1. the agent is operating in an environment that is a nondeterministic MDP; 2. rewards are bounded in the sense that there is a constant δ > 0 such that s,a R(s,a) < δ 3. all possible pairs s and a are visited infinitely often; 4. n i (s, a) is the ith time that we do action a in state s; and also...
7 ...we have for nondeterministic MDPs 0 θ n < 1 θ ni (s,a) = i=1 θn 2 i (s,a) < i=1 then with probability 1 the Q-learning algorithm converges, in the sense that a,s Q n(s, a) Q(s,a) as n. 26 Alternative representation for the Q table But there s always a catch... We have to store the table for Q : even for quite straightforward problems it is HUGE!!! - certainly big enough that it can t be stored; a standard approach to this problem is, for example, to represent it as a neural network; one way might be to make s and a the inputs to the network and train it to produce Q (s, a) as its output. This, of course, introduces its own problems, although it has been used very successfully in practice. It might be covered in Artificial Intelligence III, which unfortunately does not yet exist!
Lecture 10: Reinforcement Learning
Lecture 1: Reinforcement Learning Cognitive Systems II - Machine Learning SS 25 Part III: Learning Programs and Strategies Q Learning, Dynamic Programming Lecture 1: Reinforcement Learning p. Motivation
More informationReinforcement Learning by Comparing Immediate Reward
Reinforcement Learning by Comparing Immediate Reward Punit Pandey DeepshikhaPandey Dr. Shishir Kumar Abstract This paper introduces an approach to Reinforcement Learning Algorithm by comparing their immediate
More informationGeorgetown University at TREC 2017 Dynamic Domain Track
Georgetown University at TREC 2017 Dynamic Domain Track Zhiwen Tang Georgetown University zt79@georgetown.edu Grace Hui Yang Georgetown University huiyang@cs.georgetown.edu Abstract TREC Dynamic Domain
More informationImproving Action Selection in MDP s via Knowledge Transfer
In Proc. 20th National Conference on Artificial Intelligence (AAAI-05), July 9 13, 2005, Pittsburgh, USA. Improving Action Selection in MDP s via Knowledge Transfer Alexander A. Sherstov and Peter Stone
More informationRegret-based Reward Elicitation for Markov Decision Processes
444 REGAN & BOUTILIER UAI 2009 Regret-based Reward Elicitation for Markov Decision Processes Kevin Regan Department of Computer Science University of Toronto Toronto, ON, CANADA kmregan@cs.toronto.edu
More informationFF+FPG: Guiding a Policy-Gradient Planner
FF+FPG: Guiding a Policy-Gradient Planner Olivier Buffet LAAS-CNRS University of Toulouse Toulouse, France firstname.lastname@laas.fr Douglas Aberdeen National ICT australia & The Australian National University
More informationExploration. CS : Deep Reinforcement Learning Sergey Levine
Exploration CS 294-112: Deep Reinforcement Learning Sergey Levine Class Notes 1. Homework 4 due on Wednesday 2. Project proposal feedback sent Today s Lecture 1. What is exploration? Why is it a problem?
More informationAxiom 2013 Team Description Paper
Axiom 2013 Team Description Paper Mohammad Ghazanfari, S Omid Shirkhorshidi, Farbod Samsamipour, Hossein Rahmatizadeh Zagheli, Mohammad Mahdavi, Payam Mohajeri, S Abbas Alamolhoda Robotics Scientific Association
More informationLaboratorio di Intelligenza Artificiale e Robotica
Laboratorio di Intelligenza Artificiale e Robotica A.A. 2008-2009 Outline 2 Machine Learning Unsupervised Learning Supervised Learning Reinforcement Learning Genetic Algorithms Genetics-Based Machine Learning
More informationHigh-level Reinforcement Learning in Strategy Games
High-level Reinforcement Learning in Strategy Games Christopher Amato Department of Computer Science University of Massachusetts Amherst, MA 01003 USA camato@cs.umass.edu Guy Shani Department of Computer
More informationTD(λ) and Q-Learning Based Ludo Players
TD(λ) and Q-Learning Based Ludo Players Majed Alhajry, Faisal Alvi, Member, IEEE and Moataz Ahmed Abstract Reinforcement learning is a popular machine learning technique whose inherent self-learning ability
More informationArtificial Neural Networks written examination
1 (8) Institutionen för informationsteknologi Olle Gällmo Universitetsadjunkt Adress: Lägerhyddsvägen 2 Box 337 751 05 Uppsala Artificial Neural Networks written examination Monday, May 15, 2006 9 00-14
More informationModule 12. Machine Learning. Version 2 CSE IIT, Kharagpur
Module 12 Machine Learning 12.1 Instructional Objective The students should understand the concept of learning systems Students should learn about different aspects of a learning system Students should
More informationDiscriminative Learning of Beam-Search Heuristics for Planning
Discriminative Learning of Beam-Search Heuristics for Planning Yuehua Xu School of EECS Oregon State University Corvallis,OR 97331 xuyu@eecs.oregonstate.edu Alan Fern School of EECS Oregon State University
More informationLecture 1: Machine Learning Basics
1/69 Lecture 1: Machine Learning Basics Ali Harakeh University of Waterloo WAVE Lab ali.harakeh@uwaterloo.ca May 1, 2017 2/69 Overview 1 Learning Algorithms 2 Capacity, Overfitting, and Underfitting 3
More informationISFA2008U_120 A SCHEDULING REINFORCEMENT LEARNING ALGORITHM
Proceedings of 28 ISFA 28 International Symposium on Flexible Automation Atlanta, GA, USA June 23-26, 28 ISFA28U_12 A SCHEDULING REINFORCEMENT LEARNING ALGORITHM Amit Gil, Helman Stern, Yael Edan, and
More informationAMULTIAGENT system [1] can be defined as a group of
156 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS PART C: APPLICATIONS AND REVIEWS, VOL. 38, NO. 2, MARCH 2008 A Comprehensive Survey of Multiagent Reinforcement Learning Lucian Buşoniu, Robert Babuška,
More informationSpeeding Up Reinforcement Learning with Behavior Transfer
Speeding Up Reinforcement Learning with Behavior Transfer Matthew E. Taylor and Peter Stone Department of Computer Sciences The University of Texas at Austin Austin, Texas 78712-1188 {mtaylor, pstone}@cs.utexas.edu
More informationIAT 888: Metacreation Machines endowed with creative behavior. Philippe Pasquier Office 565 (floor 14)
IAT 888: Metacreation Machines endowed with creative behavior Philippe Pasquier Office 565 (floor 14) pasquier@sfu.ca Outline of today's lecture A little bit about me A little bit about you What will that
More informationLearning Prospective Robot Behavior
Learning Prospective Robot Behavior Shichao Ou and Rod Grupen Laboratory for Perceptual Robotics Computer Science Department University of Massachusetts Amherst {chao,grupen}@cs.umass.edu Abstract This
More informationTask Completion Transfer Learning for Reward Inference
Machine Learning for Interactive Systems: Papers from the AAAI-14 Workshop Task Completion Transfer Learning for Reward Inference Layla El Asri 1,2, Romain Laroche 1, Olivier Pietquin 3 1 Orange Labs,
More informationAn Online Handwriting Recognition System For Turkish
An Online Handwriting Recognition System For Turkish Esra Vural, Hakan Erdogan, Kemal Oflazer, Berrin Yanikoglu Sabanci University, Tuzla, Istanbul, Turkey 34956 ABSTRACT Despite recent developments in
More informationTesting A Moving Target: How Do We Test Machine Learning Systems? Peter Varhol Technology Strategy Research, USA
Testing A Moving Target: How Do We Test Machine Learning Systems? Peter Varhol Technology Strategy Research, USA Testing a Moving Target How Do We Test Machine Learning Systems? Peter Varhol, Technology
More informationGenerative models and adversarial training
Day 4 Lecture 1 Generative models and adversarial training Kevin McGuinness kevin.mcguinness@dcu.ie Research Fellow Insight Centre for Data Analytics Dublin City University What is a generative model?
More informationLaboratorio di Intelligenza Artificiale e Robotica
Laboratorio di Intelligenza Artificiale e Robotica A.A. 2008-2009 Outline 2 Machine Learning Unsupervised Learning Supervised Learning Reinforcement Learning Genetic Algorithms Genetics-Based Machine Learning
More informationSession 2B From understanding perspectives to informing public policy the potential and challenges for Q findings to inform survey design
Session 2B From understanding perspectives to informing public policy the potential and challenges for Q findings to inform survey design Paper #3 Five Q-to-survey approaches: did they work? Job van Exel
More informationProbabilistic Model Checking of DTMC Models of User Activity Patterns
Probabilistic Model Checking of DTMC Models of User Activity Patterns Oana Andrei 1, Muffy Calder 1, Matthew Higgs 1, and Mark Girolami 2 1 School of Computing Science, University of Glasgow, G12 8RZ,
More informationCorrective Feedback and Persistent Learning for Information Extraction
Corrective Feedback and Persistent Learning for Information Extraction Aron Culotta a, Trausti Kristjansson b, Andrew McCallum a, Paul Viola c a Dept. of Computer Science, University of Massachusetts,
More informationEvolutive Neural Net Fuzzy Filtering: Basic Description
Journal of Intelligent Learning Systems and Applications, 2010, 2: 12-18 doi:10.4236/jilsa.2010.21002 Published Online February 2010 (http://www.scirp.org/journal/jilsa) Evolutive Neural Net Fuzzy Filtering:
More informationTask Completion Transfer Learning for Reward Inference
Task Completion Transfer Learning for Reward Inference Layla El Asri 1,2, Romain Laroche 1, Olivier Pietquin 3 1 Orange Labs, Issy-les-Moulineaux, France 2 UMI 2958 (CNRS - GeorgiaTech), France 3 University
More informationTruth Inference in Crowdsourcing: Is the Problem Solved?
Truth Inference in Crowdsourcing: Is the Problem Solved? Yudian Zheng, Guoliang Li #, Yuanbing Li #, Caihua Shan, Reynold Cheng # Department of Computer Science, Tsinghua University Department of Computer
More informationLanguage Acquisition Fall 2010/Winter Lexical Categories. Afra Alishahi, Heiner Drenhaus
Language Acquisition Fall 2010/Winter 2011 Lexical Categories Afra Alishahi, Heiner Drenhaus Computational Linguistics and Phonetics Saarland University Children s Sensitivity to Lexical Categories Look,
More informationAn OO Framework for building Intelligence and Learning properties in Software Agents
An OO Framework for building Intelligence and Learning properties in Software Agents José A. R. P. Sardinha, Ruy L. Milidiú, Carlos J. P. Lucena, Patrick Paranhos Abstract Software agents are defined as
More informationTeachable Robots: Understanding Human Teaching Behavior to Build More Effective Robot Learners
Teachable Robots: Understanding Human Teaching Behavior to Build More Effective Robot Learners Andrea L. Thomaz and Cynthia Breazeal Abstract While Reinforcement Learning (RL) is not traditionally designed
More informationReinForest: Multi-Domain Dialogue Management Using Hierarchical Policies and Knowledge Ontology
ReinForest: Multi-Domain Dialogue Management Using Hierarchical Policies and Knowledge Ontology Tiancheng Zhao CMU-LTI-16-006 Language Technologies Institute School of Computer Science Carnegie Mellon
More informationPlanning with External Events
94 Planning with External Events Jim Blythe School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213 blythe@cs.cmu.edu Abstract I describe a planning methodology for domains with uncertainty
More informationLearning Methods in Multilingual Speech Recognition
Learning Methods in Multilingual Speech Recognition Hui Lin Department of Electrical Engineering University of Washington Seattle, WA 98125 linhui@u.washington.edu Li Deng, Jasha Droppo, Dong Yu, and Alex
More informationPython Machine Learning
Python Machine Learning Unlock deeper insights into machine learning with this vital guide to cuttingedge predictive analytics Sebastian Raschka [ PUBLISHING 1 open source I community experience distilled
More informationLecture 1: Basic Concepts of Machine Learning
Lecture 1: Basic Concepts of Machine Learning Cognitive Systems - Machine Learning Ute Schmid (lecture) Johannes Rabold (practice) Based on slides prepared March 2005 by Maximilian Röglinger, updated 2010
More informationLearning Optimal Dialogue Strategies: A Case Study of a Spoken Dialogue Agent for
Learning Optimal Dialogue Strategies: A Case Study of a Spoken Dialogue Agent for Email Marilyn A. Walker Jeanne C. Fromer Shrikanth Narayanan walker@research.att.com jeannie@ai.mit.edu shri@research.att.com
More informationUsing focal point learning to improve human machine tacit coordination
DOI 10.1007/s10458-010-9126-5 Using focal point learning to improve human machine tacit coordination InonZuckerman SaritKraus Jeffrey S. Rosenschein The Author(s) 2010 Abstract We consider an automated
More informationFinding Your Friends and Following Them to Where You Are
Finding Your Friends and Following Them to Where You Are Adam Sadilek Dept. of Computer Science University of Rochester Rochester, NY, USA sadilek@cs.rochester.edu Henry Kautz Dept. of Computer Science
More informationLearning Human Utility from Video Demonstrations for Deductive Planning in Robotics
Learning Human Utility from Video Demonstrations for Deductive Planning in Robotics Nishant Shukla, Yunzhong He, Frank Chen, and Song-Chun Zhu Center for Vision, Cognition, Learning, and Autonomy University
More informationTransfer Learning Action Models by Measuring the Similarity of Different Domains
Transfer Learning Action Models by Measuring the Similarity of Different Domains Hankui Zhuo 1, Qiang Yang 2, and Lei Li 1 1 Software Research Institute, Sun Yat-sen University, Guangzhou, China. zhuohank@gmail.com,lnslilei@mail.sysu.edu.cn
More informationarxiv: v1 [cs.se] 20 Mar 2014
Probabilistic Model Checking of DTMC Models of User Activity Patterns Oana Andrei 1, Muffy Calder 1, Matthew Higgs 1, and Mark Girolami 2 1 School of Computing Science, University of Glasgow, G12 8RZ,
More informationA Neural Network GUI Tested on Text-To-Phoneme Mapping
A Neural Network GUI Tested on Text-To-Phoneme Mapping MAARTEN TROMPPER Universiteit Utrecht m.f.a.trompper@students.uu.nl Abstract Text-to-phoneme (T2P) mapping is a necessary step in any speech synthesis
More informationCSL465/603 - Machine Learning
CSL465/603 - Machine Learning Fall 2016 Narayanan C Krishnan ckn@iitrpr.ac.in Introduction CSL465/603 - Machine Learning 1 Administrative Trivia Course Structure 3-0-2 Lecture Timings Monday 9.55-10.45am
More information2/15/13. POS Tagging Problem. Part-of-Speech Tagging. Example English Part-of-Speech Tagsets. More Details of the Problem. Typical Problem Cases
POS Tagging Problem Part-of-Speech Tagging L545 Spring 203 Given a sentence W Wn and a tagset of lexical categories, find the most likely tag T..Tn for each word in the sentence Example Secretariat/P is/vbz
More informationA Reinforcement Learning Variant for Control Scheduling
A Reinforcement Learning Variant for Control Scheduling Aloke Guha Honeywell Sensor and System Development Center 3660 Technology Drive Minneapolis MN 55417 Abstract We present an algorithm based on reinforcement
More informationUnsupervised Acoustic Model Training for Simultaneous Lecture Translation in Incremental and Batch Mode
Unsupervised Acoustic Model Training for Simultaneous Lecture Translation in Incremental and Batch Mode Diploma Thesis of Michael Heck At the Department of Informatics Karlsruhe Institute of Technology
More informationAcquiring Competence from Performance Data
Acquiring Competence from Performance Data Online learnability of OT and HG with simulated annealing Tamás Biró ACLC, University of Amsterdam (UvA) Computational Linguistics in the Netherlands, February
More informationThe Good Judgment Project: A large scale test of different methods of combining expert predictions
The Good Judgment Project: A large scale test of different methods of combining expert predictions Lyle Ungar, Barb Mellors, Jon Baron, Phil Tetlock, Jaime Ramos, Sam Swift The University of Pennsylvania
More informationAction Models and their Induction
Action Models and their Induction Michal Čertický, Comenius University, Bratislava certicky@fmph.uniba.sk March 5, 2013 Abstract By action model, we understand any logic-based representation of effects
More informationA simulated annealing and hill-climbing algorithm for the traveling tournament problem
European Journal of Operational Research xxx (2005) xxx xxx Discrete Optimization A simulated annealing and hill-climbing algorithm for the traveling tournament problem A. Lim a, B. Rodrigues b, *, X.
More informationIntroduction to the Practice of Statistics
Chapter 1: Looking at Data Distributions Introduction to the Practice of Statistics Sixth Edition David S. Moore George P. McCabe Bruce A. Craig Statistics is the science of collecting, organizing and
More informationLEARNING TO PLAY IN A DAY: FASTER DEEP REIN-
LEARNING TO PLAY IN A DAY: FASTER DEEP REIN- FORCEMENT LEARNING BY OPTIMALITY TIGHTENING Frank S. He Department of Computer Science University of Illinois at Urbana-Champaign Zhejiang University frankheshibi@gmail.com
More informationA Case Study: News Classification Based on Term Frequency
A Case Study: News Classification Based on Term Frequency Petr Kroha Faculty of Computer Science University of Technology 09107 Chemnitz Germany kroha@informatik.tu-chemnitz.de Ricardo Baeza-Yates Center
More informationAI Agent for Ice Hockey Atari 2600
AI Agent for Ice Hockey Atari 2600 Emman Kabaghe (emmank@stanford.edu) Rajarshi Roy (rroy@stanford.edu) 1 Introduction In the reinforcement learning (RL) problem an agent autonomously learns a behavior
More informationProof Theory for Syntacticians
Department of Linguistics Ohio State University Syntax 2 (Linguistics 602.02) January 5, 2012 Logics for Linguistics Many different kinds of logic are directly applicable to formalizing theories in syntax
More informationOn-the-Fly Customization of Automated Essay Scoring
Research Report On-the-Fly Customization of Automated Essay Scoring Yigal Attali Research & Development December 2007 RR-07-42 On-the-Fly Customization of Automated Essay Scoring Yigal Attali ETS, Princeton,
More informationHow do adults reason about their opponent? Typologies of players in a turn-taking game
How do adults reason about their opponent? Typologies of players in a turn-taking game Tamoghna Halder (thaldera@gmail.com) Indian Statistical Institute, Kolkata, India Khyati Sharma (khyati.sharma27@gmail.com)
More informationSemi-Supervised GMM and DNN Acoustic Model Training with Multi-system Combination and Confidence Re-calibration
INTERSPEECH 2013 Semi-Supervised GMM and DNN Acoustic Model Training with Multi-system Combination and Confidence Re-calibration Yan Huang, Dong Yu, Yifan Gong, and Chaojun Liu Microsoft Corporation, One
More informationEvolution of Collective Commitment during Teamwork
Fundamenta Informaticae 56 (2003) 329 371 329 IOS Press Evolution of Collective Commitment during Teamwork Barbara Dunin-Kȩplicz Institute of Informatics, Warsaw University Banacha 2, 02-097 Warsaw, Poland
More informationOn the Combined Behavior of Autonomous Resource Management Agents
On the Combined Behavior of Autonomous Resource Management Agents Siri Fagernes 1 and Alva L. Couch 2 1 Faculty of Engineering Oslo University College Oslo, Norway siri.fagernes@iu.hio.no 2 Computer Science
More informationAn empirical study of learning speed in backpropagation
Carnegie Mellon University Research Showcase @ CMU Computer Science Department School of Computer Science 1988 An empirical study of learning speed in backpropagation networks Scott E. Fahlman Carnegie
More informationDecision Analysis. Decision-Making Problem. Decision Analysis. Part 1 Decision Analysis and Decision Tables. Decision Analysis, Part 1
Decision Support: Decision Analysis Jožef Stefan International Postgraduate School, Ljubljana Programme: Information and Communication Technologies [ICT3] Course Web Page: http://kt.ijs.si/markobohanec/ds/ds.html
More informationPredicting Future User Actions by Observing Unmodified Applications
From: AAAI-00 Proceedings. Copyright 2000, AAAI (www.aaai.org). All rights reserved. Predicting Future User Actions by Observing Unmodified Applications Peter Gorniak and David Poole Department of Computer
More informationWhile you are waiting... socrative.com, room number SIMLANG2016
While you are waiting... socrative.com, room number SIMLANG2016 Simulating Language Lecture 4: When will optimal signalling evolve? Simon Kirby simon@ling.ed.ac.uk T H E U N I V E R S I T Y O H F R G E
More informationSegmental Conditional Random Fields with Deep Neural Networks as Acoustic Models for First-Pass Word Recognition
Segmental Conditional Random Fields with Deep Neural Networks as Acoustic Models for First-Pass Word Recognition Yanzhang He, Eric Fosler-Lussier Department of Computer Science and Engineering The hio
More informationLearning and Transferring Relational Instance-Based Policies
Learning and Transferring Relational Instance-Based Policies Rocío García-Durán, Fernando Fernández y Daniel Borrajo Universidad Carlos III de Madrid Avda de la Universidad 30, 28911-Leganés (Madrid),
More informationChinese Language Parsing with Maximum-Entropy-Inspired Parser
Chinese Language Parsing with Maximum-Entropy-Inspired Parser Heng Lian Brown University Abstract The Chinese language has many special characteristics that make parsing difficult. The performance of state-of-the-art
More informationA Version Space Approach to Learning Context-free Grammars
Machine Learning 2: 39~74, 1987 1987 Kluwer Academic Publishers, Boston - Manufactured in The Netherlands A Version Space Approach to Learning Context-free Grammars KURT VANLEHN (VANLEHN@A.PSY.CMU.EDU)
More informationA Comparison of Annealing Techniques for Academic Course Scheduling
A Comparison of Annealing Techniques for Academic Course Scheduling M. A. Saleh Elmohamed 1, Paul Coddington 2, and Geoffrey Fox 1 1 Northeast Parallel Architectures Center Syracuse University, Syracuse,
More informationContinual Curiosity-Driven Skill Acquisition from High-Dimensional Video Inputs for Humanoid Robots
Continual Curiosity-Driven Skill Acquisition from High-Dimensional Video Inputs for Humanoid Robots Varun Raj Kompella, Marijn Stollenga, Matthew Luciw, Juergen Schmidhuber The Swiss AI Lab IDSIA, USI
More informationDISCOVERY Loyalty Programme
DISCOVERY Loyalty Programme Facilitator Guide How to Use the Facilitator s Guide This Facilitator s Guide is designed to aid you in guiding students through the GHA DISCOVERY Loyalty Programme effectively
More informationRule Learning With Negation: Issues Regarding Effectiveness
Rule Learning With Negation: Issues Regarding Effectiveness S. Chua, F. Coenen, G. Malcolm University of Liverpool Department of Computer Science, Ashton Building, Ashton Street, L69 3BX Liverpool, United
More informationLanguage properties and Grammar of Parallel and Series Parallel Languages
arxiv:1711.01799v1 [cs.fl] 6 Nov 2017 Language properties and Grammar of Parallel and Series Parallel Languages Mohana.N 1, Kalyani Desikan 2 and V.Rajkumar Dare 3 1 Division of Mathematics, School of
More informationCognitive Thinking Style Sample Report
Cognitive Thinking Style Sample Report Goldisc Limited Authorised Agent for IML, PeopleKeys & StudentKeys DISC Profiles Online Reports Training Courses Consultations sales@goldisc.co.uk Telephone: +44
More informationAn Introduction to Simulation Optimization
An Introduction to Simulation Optimization Nanjing Jian Shane G. Henderson Introductory Tutorials Winter Simulation Conference December 7, 2015 Thanks: NSF CMMI1200315 1 Contents 1. Introduction 2. Common
More informationLiquid Narrative Group Technical Report Number
http://liquidnarrative.csc.ncsu.edu/pubs/tr04-004.pdf NC STATE UNIVERSITY_ Liquid Narrative Group Technical Report Number 04-004 Equivalence between Narrative Mediation and Branching Story Graphs Mark
More informationCOMPUTER-AIDED DESIGN TOOLS THAT ADAPT
COMPUTER-AIDED DESIGN TOOLS THAT ADAPT WEI PENG CSIRO ICT Centre, Australia and JOHN S GERO Krasnow Institute for Advanced Study, USA 1. Introduction Abstract. This paper describes an approach that enables
More informationNCEO Technical Report 27
Home About Publications Special Topics Presentations State Policies Accommodations Bibliography Teleconferences Tools Related Sites Interpreting Trends in the Performance of Special Education Students
More informationEvolution of Symbolisation in Chimpanzees and Neural Nets
Evolution of Symbolisation in Chimpanzees and Neural Nets Angelo Cangelosi Centre for Neural and Adaptive Systems University of Plymouth (UK) a.cangelosi@plymouth.ac.uk Introduction Animal communication
More informationBSP !!! Trainer s Manual. Sheldon Loman, Ph.D. Portland State University. M. Kathleen Strickland-Cohen, Ph.D. University of Oregon
Basic FBA to BSP Trainer s Manual Sheldon Loman, Ph.D. Portland State University M. Kathleen Strickland-Cohen, Ph.D. University of Oregon Chris Borgmeier, Ph.D. Portland State University Robert Horner,
More informationQuickStroke: An Incremental On-line Chinese Handwriting Recognition System
QuickStroke: An Incremental On-line Chinese Handwriting Recognition System Nada P. Matić John C. Platt Λ Tony Wang y Synaptics, Inc. 2381 Bering Drive San Jose, CA 95131, USA Abstract This paper presents
More information10.2. Behavior models
User behavior research 10.2. Behavior models Overview Why do users seek information? How do they seek information? How do they search for information? How do they use libraries? These questions are addressed
More informationUsing Synonyms for Author Recognition
Using Synonyms for Author Recognition Abstract. An approach for identifying authors using synonym sets is presented. Drawing on modern psycholinguistic research, we justify the basis of our theory. Having
More informationKnowledge based expert systems D H A N A N J A Y K A L B A N D E
Knowledge based expert systems D H A N A N J A Y K A L B A N D E What is a knowledge based system? A Knowledge Based System or a KBS is a computer program that uses artificial intelligence to solve problems
More informationAutomatic Discretization of Actions and States in Monte-Carlo Tree Search
Automatic Discretization of Actions and States in Monte-Carlo Tree Search Guy Van den Broeck 1 and Kurt Driessens 2 1 Katholieke Universiteit Leuven, Department of Computer Science, Leuven, Belgium guy.vandenbroeck@cs.kuleuven.be
More informationBusiness. Pearson BTEC Level 1 Introductory in. Specification
Pearson BTEC Level 1 Introductory in Business Specification Pearson BTEC Level 1 Introductory Certificate in Business Pearson BTEC Level 1 Introductory Diploma in Business Pearson BTEC Level 1 Introductory
More informationunderstand a concept, master it through many problem-solving tasks, and apply it in different situations. One may have sufficient knowledge about a do
Seta, K. and Watanabe, T.(Eds.) (2015). Proceedings of the 11th International Conference on Knowledge Management. Bayesian Networks For Competence-based Student Modeling Nguyen-Thinh LE & Niels PINKWART
More informationNatural Language Processing. George Konidaris
Natural Language Processing George Konidaris gdk@cs.brown.edu Fall 2017 Natural Language Processing Understanding spoken/written sentences in a natural language. Major area of research in AI. Why? Humans
More informationUNIVERSITY OF CALIFORNIA SANTA CRUZ TOWARDS A UNIVERSAL PARAMETRIC PLAYER MODEL
UNIVERSITY OF CALIFORNIA SANTA CRUZ TOWARDS A UNIVERSAL PARAMETRIC PLAYER MODEL A thesis submitted in partial satisfaction of the requirements for the degree of DOCTOR OF PHILOSOPHY in COMPUTER SCIENCE
More informationSCHOOL WITHOUT CLASSROOMS BERLIN ARCHITECTURE COMPETITION TO
SCHOOL WITHOUT CLASSROOMS BERLIN ARCHITECTURE COMPETITION 01.04.2017 TO 30.06.2017 www.archasm.in MISSION STATEMENT What if we lived in an age where school and learning was not systemized but optimized?
More informationCOMPUTATIONAL COMPLEXITY OF LEFT-ASSOCIATIVE GRAMMAR
COMPUTATIONAL COMPLEXITY OF LEFT-ASSOCIATIVE GRAMMAR ROLAND HAUSSER Institut für Deutsche Philologie Ludwig-Maximilians Universität München München, West Germany 1. CHOICE OF A PRIMITIVE OPERATION The
More informationCONCEPT MAPS AS A DEVICE FOR LEARNING DATABASE CONCEPTS
CONCEPT MAPS AS A DEVICE FOR LEARNING DATABASE CONCEPTS Pirjo Moen Department of Computer Science P.O. Box 68 FI-00014 University of Helsinki pirjo.moen@cs.helsinki.fi http://www.cs.helsinki.fi/pirjo.moen
More informationMGT/MGP/MGB 261: Investment Analysis
UNIVERSITY OF CALIFORNIA, DAVIS GRADUATE SCHOOL OF MANAGEMENT SYLLABUS for Fall 2014 MGT/MGP/MGB 261: Investment Analysis Daytime MBA: Tu 12:00p.m. - 3:00 p.m. Location: 1302 Gallagher (CRN: 51489) Sacramento
More informationThe Strong Minimalist Thesis and Bounded Optimality
The Strong Minimalist Thesis and Bounded Optimality DRAFT-IN-PROGRESS; SEND COMMENTS TO RICKL@UMICH.EDU Richard L. Lewis Department of Psychology University of Michigan 27 March 2010 1 Purpose of this
More informationEvery curriculum policy starts from this policy and expands the detail in relation to the specific requirements of each policy s field.
1. WE BELIEVE We believe a successful Teaching and Learning Policy enables all children to be effective learners; to have the confidence to take responsibility for their own learning; understand what it
More informationAgent-Based Software Engineering
Agent-Based Software Engineering Learning Guide Information for Students 1. Description Grade Module Máster Universitario en Ingeniería de Software - European Master on Software Engineering Advanced Software
More information