Web and Internet Economics

Transcription

1 Web and Internet Economics Introduction to Machine Learning Matteo Papini a.a. 2017/2018

2 Internet Commerce vs Regular Commerce Efficiency Pull driven marketing and advertising Trust and reputation Personalization New opportunities

3 Internet Commerce vs Regular Commerce Efficiency Pull driven marketing and advertising Trust and reputation Personalization New opportunities

4 Internet Commerce vs Regular Commerce Efficiency Pull driven marketing and advertising Trust and reputation Personalization New opportunities

5 Internet Commerce vs Regular Commerce Efficiency Pull driven marketing and advertising Trust and reputation Personalization New opportunities

6 Internet Commerce vs Regular Commerce Efficiency Pull driven marketing and advertising Trust and reputation Personalization New opportunities

7 Problem Problem: Models are based on several unknown parameters Examples: Online advertising Website optimization Pricing Sponsored search Targeting Recommendations Solution: Learn from data!

8 Problem Problem: Models are based on several unknown parameters Examples: Online advertising Website optimization Pricing Sponsored search Targeting Recommendations Solution: Learn from data!

9 Problem Problem: Models are based on several unknown parameters Examples: Online advertising Website optimization Pricing Sponsored search Targeting Recommendations Solution: Learn from data!

10 Problem Problem: Models are based on several unknown parameters Examples: Online advertising Website optimization Pricing Sponsored search Targeting Recommendations Solution: Learn from data!

11 Problem Problem: Models are based on several unknown parameters Examples: Online advertising Website optimization Pricing Sponsored search Targeting Recommendations Solution: Learn from data!

12 Problem Problem: Models are based on several unknown parameters Examples: Online advertising Website optimization Pricing Sponsored search Targeting Recommendations Solution: Learn from data!

13 Problem Problem: Models are based on several unknown parameters Examples: Online advertising Website optimization Pricing Sponsored search Targeting Recommendations Solution: Learn from data!

14 Problem Problem: Models are based on several unknown parameters Examples: Online advertising Website optimization Pricing Sponsored search Targeting Recommendations Solution: Learn from data!

15 Problem Problem: Models are based on several unknown parameters Examples: Online advertising Website optimization Pricing Sponsored search Targeting Recommendations Solution: Learn from data!

16 What is Machine Learning? The real question is: what is learning? Mitchell (1997): A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E

17 What is Machine Learning? The real question is: what is learning? Mitchell (1997): A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E

18 What is Machine Learning? The real question is: what is learning? Mitchell (1997): A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E

19 What is ML useful for? Computer vision and robotics Speech recognition Biology and medicine Financial industry Information retrieval, Web search,... Video gaming Many application and many jobs...

20 Why Machine Learning? We need computers to make informed decisions on new, unseen data Often it is too difficult to design a set of rules by hand Machine learning allows to automatically extract relevant information from data applying it to analyze new data

21 Why Machine Learning? We need computers to make informed decisions on new, unseen data Often it is too difficult to design a set of rules by hand Machine learning allows to automatically extract relevant information from data applying it to analyze new data

22 Why Machine Learning? We need computers to make informed decisions on new, unseen data Often it is too difficult to design a set of rules by hand Machine learning allows to automatically extract relevant information from data applying it to analyze new data

23 Machine Learning

24 Machine Learning Models Supervised Learning Learn the model Unsupervised Learning Learn the representation Reinforcement Learning Learn to control

25 Machine Learning Models Supervised Learning Learn the model Unsupervised Learning Learn the representation Reinforcement Learning Learn to control

26 Machine Learning Models Supervised Learning Learn the model Unsupervised Learning Learn the representation Reinforcement Learning Learn to control

27 Supervised Learning Goal Estimating the unknown model that maps known inputs to known outputs Training set: D = { x, y } y = f(x) Problems Classification Regression Techniques Artificial Neural Networks Support Vector Machines Decision trees

28 Supervised Learning Goal Estimating the unknown model that maps known inputs to known outputs Training set: D = { x, y } y = f(x) Problems Classification Regression Techniques Artificial Neural Networks Support Vector Machines Decision trees

29 Supervised Learning Goal Estimating the unknown model that maps known inputs to known outputs Training set: D = { x, y } y = f(x) Problems Classification Regression Techniques Artificial Neural Networks Support Vector Machines Decision trees

30 Supervised Learning Goal Estimating the unknown model that maps known inputs to known outputs Training set: D = { x, y } y = f(x) Problems Classification Regression Techniques Artificial Neural Networks Support Vector Machines Decision trees

31 Supervised Learning Goal Estimating the unknown model that maps known inputs to known outputs Training set: D = { x, y } y = f(x) Problems Classification Regression Techniques Artificial Neural Networks Support Vector Machines Decision trees

32 Supervised Learning Goal Estimating the unknown model that maps known inputs to known outputs Training set: D = { x, y } y = f(x) Problems Classification Regression Techniques Artificial Neural Networks Support Vector Machines Decision trees

33 Supervised Learning Goal Estimating the unknown model that maps known inputs to known outputs Training set: D = { x, y } y = f(x) Problems Classification Regression Techniques Artificial Neural Networks Support Vector Machines Decision trees

34 Supervised Learning: Example

35 Unsupervised Learning Goal Learning a more efficient representation of a set of unknown inputs Training set: D = {x}? = f(x) Problems Compression Clustering Techniques K-means Self-organizing maps Principal Component Analysis

36 Unsupervised Learning Goal Learning a more efficient representation of a set of unknown inputs Training set: D = {x}? = f(x) Problems Compression Clustering Techniques K-means Self-organizing maps Principal Component Analysis

37 Unsupervised Learning Goal Learning a more efficient representation of a set of unknown inputs Training set: D = {x}? = f(x) Problems Compression Clustering Techniques K-means Self-organizing maps Principal Component Analysis

38 Unsupervised Learning Goal Learning a more efficient representation of a set of unknown inputs Training set: D = {x}? = f(x) Problems Compression Clustering Techniques K-means Self-organizing maps Principal Component Analysis

39 Unsupervised Learning Goal Learning a more efficient representation of a set of unknown inputs Training set: D = {x}? = f(x) Problems Compression Clustering Techniques K-means Self-organizing maps Principal Component Analysis

40 Unsupervised Learning Goal Learning a more efficient representation of a set of unknown inputs Training set: D = {x}? = f(x) Problems Compression Clustering Techniques K-means Self-organizing maps Principal Component Analysis

41 Unsupervised Learning Goal Learning a more efficient representation of a set of unknown inputs Training set: D = {x}? = f(x) Problems Compression Clustering Techniques K-means Self-organizing maps Principal Component Analysis

42 Unsupervised Learning: Example

43 Unsupervised Learning: Example

44 Reinforcement Learning Goal Learning the optimal policy Training set: D = { x, u, x, r } π (x) = arg max u {Q (x, u)}, where Q (x, u) must be estimated. Problems Markov Decision Process (MDP) Partially Observable MDP (POMDP) Stochastic Games (SG) Techniques Q-learning SARSA Fitted Q-iteration

45 Reinforcement Learning Goal Learning the optimal policy Training set: D = { x, u, x, r } π (x) = arg max u {Q (x, u)}, where Q (x, u) must be estimated. Problems Markov Decision Process (MDP) Partially Observable MDP (POMDP) Stochastic Games (SG) Techniques Q-learning SARSA Fitted Q-iteration

46 Reinforcement Learning Goal Learning the optimal policy Training set: D = { x, u, x, r } π (x) = arg max u {Q (x, u)}, where Q (x, u) must be estimated. Problems Markov Decision Process (MDP) Partially Observable MDP (POMDP) Stochastic Games (SG) Techniques Q-learning SARSA Fitted Q-iteration

47 Reinforcement Learning Goal Learning the optimal policy Training set: D = { x, u, x, r } π (x) = arg max u {Q (x, u)}, where Q (x, u) must be estimated. Problems Markov Decision Process (MDP) Partially Observable MDP (POMDP) Stochastic Games (SG) Techniques Q-learning SARSA Fitted Q-iteration

48 Reinforcement Learning Goal Learning the optimal policy Training set: D = { x, u, x, r } π (x) = arg max u {Q (x, u)}, where Q (x, u) must be estimated. Problems Markov Decision Process (MDP) Partially Observable MDP (POMDP) Stochastic Games (SG) Techniques Q-learning SARSA Fitted Q-iteration

49 Reinforcement Learning Goal Learning the optimal policy Training set: D = { x, u, x, r } π (x) = arg max u {Q (x, u)}, where Q (x, u) must be estimated. Problems Markov Decision Process (MDP) Partially Observable MDP (POMDP) Stochastic Games (SG) Techniques Q-learning SARSA Fitted Q-iteration

50 Reinforcement Learning Goal Learning the optimal policy Training set: D = { x, u, x, r } π (x) = arg max u {Q (x, u)}, where Q (x, u) must be estimated. Problems Markov Decision Process (MDP) Partially Observable MDP (POMDP) Stochastic Games (SG) Techniques Q-learning SARSA Fitted Q-iteration

51 Reinforcement Learning Goal Learning the optimal policy Training set: D = { x, u, x, r } π (x) = arg max u {Q (x, u)}, where Q (x, u) must be estimated. Problems Markov Decision Process (MDP) Partially Observable MDP (POMDP) Stochastic Games (SG) Techniques Q-learning SARSA Fitted Q-iteration

52 But Who s Counting? But Who s Counting?

53 Successful Stories: AlphaGo

54 Successful Stories: Parkour Parkour

55 Successful Stories: Atari Atari

56 Teaching Material Sutton and Barto, Reinforcement Learning: an Introduction, MIT Press, Look for the second edition Chapters 1-6 S. Bubeck and N. Cesa-Bianchi, Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems. In Foundations and Trends in Machine Learning, Vol 5: No 1, 1-122, sbubeck/surveybcb12.pdf Sections 1, 2, and 3.

57 Teaching Material Sutton and Barto, Reinforcement Learning: an Introduction, MIT Press, Look for the second edition Chapters 1-6 S. Bubeck and N. Cesa-Bianchi, Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems. In Foundations and Trends in Machine Learning, Vol 5: No 1, 1-122, sbubeck/surveybcb12.pdf Sections 1, 2, and 3.

58 Teaching Material Sutton and Barto, Reinforcement Learning: an Introduction, MIT Press, Look for the second edition Chapters 1-6 S. Bubeck and N. Cesa-Bianchi, Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems. In Foundations and Trends in Machine Learning, Vol 5: No 1, 1-122, sbubeck/surveybcb12.pdf Sections 1, 2, and 3.

59 Teaching Material Sutton and Barto, Reinforcement Learning: an Introduction, MIT Press, Look for the second edition Chapters 1-6 S. Bubeck and N. Cesa-Bianchi, Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems. In Foundations and Trends in Machine Learning, Vol 5: No 1, 1-122, sbubeck/surveybcb12.pdf Sections 1, 2, and 3.

60 Teaching Material Sutton and Barto, Reinforcement Learning: an Introduction, MIT Press, Look for the second edition Chapters 1-6 S. Bubeck and N. Cesa-Bianchi, Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems. In Foundations and Trends in Machine Learning, Vol 5: No 1, 1-122, sbubeck/surveybcb12.pdf Sections 1, 2, and 3.

61 Teaching Material Sutton and Barto, Reinforcement Learning: an Introduction, MIT Press, Look for the second edition Chapters 1-6 S. Bubeck and N. Cesa-Bianchi, Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems. In Foundations and Trends in Machine Learning, Vol 5: No 1, 1-122, sbubeck/surveybcb12.pdf Sections 1, 2, and 3.

62 Teaching Material Sutton and Barto, Reinforcement Learning: an Introduction, MIT Press, Look for the second edition Chapters 1-6 S. Bubeck and N. Cesa-Bianchi, Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems. In Foundations and Trends in Machine Learning, Vol 5: No 1, 1-122, sbubeck/surveybcb12.pdf Sections 1, 2, and 3.