Web and Internet Economics Introduction to Machine Learning Matteo Papini a.a. 2017/2018
Internet Commerce vs Regular Commerce Efficiency Pull driven marketing and advertising Trust and reputation Personalization New opportunities
Internet Commerce vs Regular Commerce Efficiency Pull driven marketing and advertising Trust and reputation Personalization New opportunities
Internet Commerce vs Regular Commerce Efficiency Pull driven marketing and advertising Trust and reputation Personalization New opportunities
Internet Commerce vs Regular Commerce Efficiency Pull driven marketing and advertising Trust and reputation Personalization New opportunities
Internet Commerce vs Regular Commerce Efficiency Pull driven marketing and advertising Trust and reputation Personalization New opportunities
Problem Problem: Models are based on several unknown parameters Examples: Online advertising Website optimization Pricing Sponsored search Targeting Recommendations Solution: Learn from data!
Problem Problem: Models are based on several unknown parameters Examples: Online advertising Website optimization Pricing Sponsored search Targeting Recommendations Solution: Learn from data!
Problem Problem: Models are based on several unknown parameters Examples: Online advertising Website optimization Pricing Sponsored search Targeting Recommendations Solution: Learn from data!
Problem Problem: Models are based on several unknown parameters Examples: Online advertising Website optimization Pricing Sponsored search Targeting Recommendations Solution: Learn from data!
Problem Problem: Models are based on several unknown parameters Examples: Online advertising Website optimization Pricing Sponsored search Targeting Recommendations Solution: Learn from data!
Problem Problem: Models are based on several unknown parameters Examples: Online advertising Website optimization Pricing Sponsored search Targeting Recommendations Solution: Learn from data!
Problem Problem: Models are based on several unknown parameters Examples: Online advertising Website optimization Pricing Sponsored search Targeting Recommendations Solution: Learn from data!
Problem Problem: Models are based on several unknown parameters Examples: Online advertising Website optimization Pricing Sponsored search Targeting Recommendations Solution: Learn from data!
Problem Problem: Models are based on several unknown parameters Examples: Online advertising Website optimization Pricing Sponsored search Targeting Recommendations Solution: Learn from data!
What is Machine Learning? The real question is: what is learning? Mitchell (1997): A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E
What is Machine Learning? The real question is: what is learning? Mitchell (1997): A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E
What is Machine Learning? The real question is: what is learning? Mitchell (1997): A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E
What is ML useful for? Computer vision and robotics Speech recognition Biology and medicine Financial industry Information retrieval, Web search,... Video gaming Many application and many jobs...
Why Machine Learning? We need computers to make informed decisions on new, unseen data Often it is too difficult to design a set of rules by hand Machine learning allows to automatically extract relevant information from data applying it to analyze new data
Why Machine Learning? We need computers to make informed decisions on new, unseen data Often it is too difficult to design a set of rules by hand Machine learning allows to automatically extract relevant information from data applying it to analyze new data
Why Machine Learning? We need computers to make informed decisions on new, unseen data Often it is too difficult to design a set of rules by hand Machine learning allows to automatically extract relevant information from data applying it to analyze new data
Machine Learning
Machine Learning Models Supervised Learning Learn the model Unsupervised Learning Learn the representation Reinforcement Learning Learn to control
Machine Learning Models Supervised Learning Learn the model Unsupervised Learning Learn the representation Reinforcement Learning Learn to control
Machine Learning Models Supervised Learning Learn the model Unsupervised Learning Learn the representation Reinforcement Learning Learn to control
Supervised Learning Goal Estimating the unknown model that maps known inputs to known outputs Training set: D = { x, y } y = f(x) Problems Classification Regression Techniques Artificial Neural Networks Support Vector Machines Decision trees
Supervised Learning Goal Estimating the unknown model that maps known inputs to known outputs Training set: D = { x, y } y = f(x) Problems Classification Regression Techniques Artificial Neural Networks Support Vector Machines Decision trees
Supervised Learning Goal Estimating the unknown model that maps known inputs to known outputs Training set: D = { x, y } y = f(x) Problems Classification Regression Techniques Artificial Neural Networks Support Vector Machines Decision trees
Supervised Learning Goal Estimating the unknown model that maps known inputs to known outputs Training set: D = { x, y } y = f(x) Problems Classification Regression Techniques Artificial Neural Networks Support Vector Machines Decision trees
Supervised Learning Goal Estimating the unknown model that maps known inputs to known outputs Training set: D = { x, y } y = f(x) Problems Classification Regression Techniques Artificial Neural Networks Support Vector Machines Decision trees
Supervised Learning Goal Estimating the unknown model that maps known inputs to known outputs Training set: D = { x, y } y = f(x) Problems Classification Regression Techniques Artificial Neural Networks Support Vector Machines Decision trees
Supervised Learning Goal Estimating the unknown model that maps known inputs to known outputs Training set: D = { x, y } y = f(x) Problems Classification Regression Techniques Artificial Neural Networks Support Vector Machines Decision trees
Supervised Learning: Example
Unsupervised Learning Goal Learning a more efficient representation of a set of unknown inputs Training set: D = {x}? = f(x) Problems Compression Clustering Techniques K-means Self-organizing maps Principal Component Analysis
Unsupervised Learning Goal Learning a more efficient representation of a set of unknown inputs Training set: D = {x}? = f(x) Problems Compression Clustering Techniques K-means Self-organizing maps Principal Component Analysis
Unsupervised Learning Goal Learning a more efficient representation of a set of unknown inputs Training set: D = {x}? = f(x) Problems Compression Clustering Techniques K-means Self-organizing maps Principal Component Analysis
Unsupervised Learning Goal Learning a more efficient representation of a set of unknown inputs Training set: D = {x}? = f(x) Problems Compression Clustering Techniques K-means Self-organizing maps Principal Component Analysis
Unsupervised Learning Goal Learning a more efficient representation of a set of unknown inputs Training set: D = {x}? = f(x) Problems Compression Clustering Techniques K-means Self-organizing maps Principal Component Analysis
Unsupervised Learning Goal Learning a more efficient representation of a set of unknown inputs Training set: D = {x}? = f(x) Problems Compression Clustering Techniques K-means Self-organizing maps Principal Component Analysis
Unsupervised Learning Goal Learning a more efficient representation of a set of unknown inputs Training set: D = {x}? = f(x) Problems Compression Clustering Techniques K-means Self-organizing maps Principal Component Analysis
Unsupervised Learning: Example
Unsupervised Learning: Example
Reinforcement Learning Goal Learning the optimal policy Training set: D = { x, u, x, r } π (x) = arg max u {Q (x, u)}, where Q (x, u) must be estimated. Problems Markov Decision Process (MDP) Partially Observable MDP (POMDP) Stochastic Games (SG) Techniques Q-learning SARSA Fitted Q-iteration
Reinforcement Learning Goal Learning the optimal policy Training set: D = { x, u, x, r } π (x) = arg max u {Q (x, u)}, where Q (x, u) must be estimated. Problems Markov Decision Process (MDP) Partially Observable MDP (POMDP) Stochastic Games (SG) Techniques Q-learning SARSA Fitted Q-iteration
Reinforcement Learning Goal Learning the optimal policy Training set: D = { x, u, x, r } π (x) = arg max u {Q (x, u)}, where Q (x, u) must be estimated. Problems Markov Decision Process (MDP) Partially Observable MDP (POMDP) Stochastic Games (SG) Techniques Q-learning SARSA Fitted Q-iteration
Reinforcement Learning Goal Learning the optimal policy Training set: D = { x, u, x, r } π (x) = arg max u {Q (x, u)}, where Q (x, u) must be estimated. Problems Markov Decision Process (MDP) Partially Observable MDP (POMDP) Stochastic Games (SG) Techniques Q-learning SARSA Fitted Q-iteration
Reinforcement Learning Goal Learning the optimal policy Training set: D = { x, u, x, r } π (x) = arg max u {Q (x, u)}, where Q (x, u) must be estimated. Problems Markov Decision Process (MDP) Partially Observable MDP (POMDP) Stochastic Games (SG) Techniques Q-learning SARSA Fitted Q-iteration
Reinforcement Learning Goal Learning the optimal policy Training set: D = { x, u, x, r } π (x) = arg max u {Q (x, u)}, where Q (x, u) must be estimated. Problems Markov Decision Process (MDP) Partially Observable MDP (POMDP) Stochastic Games (SG) Techniques Q-learning SARSA Fitted Q-iteration
Reinforcement Learning Goal Learning the optimal policy Training set: D = { x, u, x, r } π (x) = arg max u {Q (x, u)}, where Q (x, u) must be estimated. Problems Markov Decision Process (MDP) Partially Observable MDP (POMDP) Stochastic Games (SG) Techniques Q-learning SARSA Fitted Q-iteration
Reinforcement Learning Goal Learning the optimal policy Training set: D = { x, u, x, r } π (x) = arg max u {Q (x, u)}, where Q (x, u) must be estimated. Problems Markov Decision Process (MDP) Partially Observable MDP (POMDP) Stochastic Games (SG) Techniques Q-learning SARSA Fitted Q-iteration
But Who s Counting? But Who s Counting?
Successful Stories: AlphaGo
Successful Stories: Parkour Parkour
Successful Stories: Atari Atari
Teaching Material Sutton and Barto, Reinforcement Learning: an Introduction, MIT Press, 1998. http://incompleteideas.net/book/the-book-2nd.html Look for the second edition Chapters 1-6 S. Bubeck and N. Cesa-Bianchi, Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems. In Foundations and Trends in Machine Learning, Vol 5: No 1, 1-122, 2012. http://www.princeton.edu/ sbubeck/surveybcb12.pdf Sections 1, 2, and 3.
Teaching Material Sutton and Barto, Reinforcement Learning: an Introduction, MIT Press, 1998. http://incompleteideas.net/book/the-book-2nd.html Look for the second edition Chapters 1-6 S. Bubeck and N. Cesa-Bianchi, Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems. In Foundations and Trends in Machine Learning, Vol 5: No 1, 1-122, 2012. http://www.princeton.edu/ sbubeck/surveybcb12.pdf Sections 1, 2, and 3.
Teaching Material Sutton and Barto, Reinforcement Learning: an Introduction, MIT Press, 1998. http://incompleteideas.net/book/the-book-2nd.html Look for the second edition Chapters 1-6 S. Bubeck and N. Cesa-Bianchi, Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems. In Foundations and Trends in Machine Learning, Vol 5: No 1, 1-122, 2012. http://www.princeton.edu/ sbubeck/surveybcb12.pdf Sections 1, 2, and 3.
Teaching Material Sutton and Barto, Reinforcement Learning: an Introduction, MIT Press, 1998. http://incompleteideas.net/book/the-book-2nd.html Look for the second edition Chapters 1-6 S. Bubeck and N. Cesa-Bianchi, Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems. In Foundations and Trends in Machine Learning, Vol 5: No 1, 1-122, 2012. http://www.princeton.edu/ sbubeck/surveybcb12.pdf Sections 1, 2, and 3.
Teaching Material Sutton and Barto, Reinforcement Learning: an Introduction, MIT Press, 1998. http://incompleteideas.net/book/the-book-2nd.html Look for the second edition Chapters 1-6 S. Bubeck and N. Cesa-Bianchi, Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems. In Foundations and Trends in Machine Learning, Vol 5: No 1, 1-122, 2012. http://www.princeton.edu/ sbubeck/surveybcb12.pdf Sections 1, 2, and 3.
Teaching Material Sutton and Barto, Reinforcement Learning: an Introduction, MIT Press, 1998. http://incompleteideas.net/book/the-book-2nd.html Look for the second edition Chapters 1-6 S. Bubeck and N. Cesa-Bianchi, Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems. In Foundations and Trends in Machine Learning, Vol 5: No 1, 1-122, 2012. http://www.princeton.edu/ sbubeck/surveybcb12.pdf Sections 1, 2, and 3.
Teaching Material Sutton and Barto, Reinforcement Learning: an Introduction, MIT Press, 1998. http://incompleteideas.net/book/the-book-2nd.html Look for the second edition Chapters 1-6 S. Bubeck and N. Cesa-Bianchi, Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems. In Foundations and Trends in Machine Learning, Vol 5: No 1, 1-122, 2012. http://www.princeton.edu/ sbubeck/surveybcb12.pdf Sections 1, 2, and 3.