MIT Smart Web apps using Machine Learning

Transcription

1 MIT Smart Web apps using Machine Learning

2 Hello! I am Carlos Aguayo ~13 years at Appian Director, Software Development Master's student

3

4 What is Machine Learning?

5 What is Machine Learning Machine learning is the subfield of computer science that "gives computers the ability to learn without being explicitly programmed" - Arthur Samuel, 1959

6 What is Machine Learning A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E. - Tom Mitchell

7 Let's start with a demo!

8 Gender Recognition by Voice and Speech Analysis Given an audio, tell if the voice in the audio is male or female.

9 Acoustic Properties Measured meanfreq mean frequency (in khz) centroid frequency centroid sd standard deviation of frequency peakf peak frequency median median frequency (in khz) meanfun average of fundamental frequency Q25 first quantile (in khz) minfun minimum fundamental frequency Q75 third quantile (in khz) maxfun maximum fundamental frequency IQR interquantile range (in khz) meandom average of dominant frequency skew skewness mindom minimum of dominant frequency kurt kurtosis maxdom maximum of dominant frequency sp.ent spectral entropy dfrange range of dominant frequency sfm spectral flatness modindx modulation index mode mode frequency

10 How?

11 How?

12 How?

13 What did we do? 3,168 voice samples

14 What did we do? 3,168 voice samples Machine Learning Algorithm

15 What did we do? f(x) 3,168 voice samples Machine Learning Algorithm

16 What did we do? f(x) 3,168 voice samples Machine Learning Algorithm f(x)

17 How?

18 How?

19 How? Given an X and Y, is this point pink or blue?

20 How? Given an X and Y, is this point pink or blue?

21 How? Blue!

22 How?

23 How?

24 How?

25 How?

26 K-Nearest Neighbors One of the simplest, yet effective, machine learning algorithms.

27 How?

28 Support Vector Machine Hyperplane that represents the largest separation between classes

29 How?

30 Decision Trees Another simple, and effective, supervised learning algorithm.

31 mode minfun maxdom Q25 median meanfun skew

32 Human vs. Machine Up to 3 dimensions! High dimensional space!

33 Supervised Learning

34 Supervised Learning Supervised learning is the machine learning task of inferring a function from labeled training data. The training data consist of a set of training examples. Each example is a pair consisting of an input object and an output value. A supervised learning algorithm analyzes the training data and produces an inferred function, which can be used for mapping new examples.

35 Supervised Learning Supervised learning is the machine learning task of inferring a function from labeled training data. The training data consist of a set of training examples. Each example is a pair consisting of an input object and an output value. A supervised learning algorithm analyzes the training data and produces an inferred function, which can be used for mapping new examples. 3,168 voice samples

36 Supervised Learning Supervised learning is the machine learning task of inferring a function from labeled training data. The training data consist of a set of training examples. Each example is a pair consisting of an input object and an output value. A supervised learning algorithm analyzes the training data and produces an inferred function, which can be used for mapping new examples.

37 Supervised Learning Supervised learning is the machine learning task of inferring a function from labeled training data. The training data consist of a set of training examples. Each example is a pair consisting of an input object and an output value. A supervised learning algorithm analyzes the training data and produces an inferred function, which can be used for mapping new examples.

38 Supervised Learning Neural Networks

39 Supervised Learning Will you go out to the party tonight?

40 Supervised Learning Will you go out to the party tonight? Can I wake up late tomorrow?

41 Supervised Learning Will you go out to the party tonight? Can I wake up late tomorrow? Will the person that I like be there?

42 Supervised Learning Will you go out to the party tonight? Can I wake up late tomorrow? Will the person that I like be there? Will my friends be there?

43 Supervised Learning Will you go out to the party tonight? Can I wake up late tomorrow? Will the person that I like be there? Will my friends be there? Do I have any other plans tonight? Have I gone to that party before?

44 Supervised Learning Crush? Friend? Late? 5 No Plans? New? 3 It's just a weighted decision. If the output is equal or larger than 10, I'll be there! Sum

45 Supervised Learning Yes Crush? No Friend? No Late? No No Plans? No New? 10 It's just a weighted decision. If the output is equal or larger than 10, I'll be there! Sum = 10 Yes! I'll be at the party!

46 Supervised Learning No Crush? No Friend? 10 5 Yes Late? No No Plans? No New? It's just a weighted decision. If the output is equal or larger than 10, I'll be there! Sum =5 No, raincheck.

47 Supervised Learning No Yes Crush? Friend? Yes Late? 5 No No Plans? No New? 3 It's just a weighted decision. If the output is equal or larger than 10, I'll be there! Sum = 12 Yes! I'll be at the party!

48 Supervised Learning Crush? Friend? Late? 5 No Plans? New? 3 Sum

49 Supervised Learning

50 Supervised Learning Sum Crush Friend Late Plan New Other Sum

51 Supervised Learning Sum Crush Friend Sum Sum Late Sum Plan New Other Sum

52 Supervised Learning Sum Crush Friend Sum Sum Late Sum Plan New Other Sum Sum

53 Supervised Learning Deep Learning

54 Supervised Learning Convolutional Neural Networks (CNNs)

55 Supervised Learning High Level Summary

56 Supervised Learning High Level Summary Labeled Data You get a set of samples, each of them with an answer.

57 Supervised Learning High Level Summary Labeled Data Model You get a set of samples, each of them with an answer. Learn a model that can successfully predict the seen and unseen samples.

58 Supervised Learning High Level Summary Labeled Data Model Predict You get a set of samples, each of them with an answer. Learn a model that can successfully predict the seen and unseen samples. A number, face, voice, price of a house, stock, etc.

59 Supervised Learning High Level Summary Labeled Data Model Predict You get a set of samples, each of them with an answer. Learn a model that can successfully predict the seen and unseen samples. A number, face, voice, price of a house, stock, etc.

60 The Future...

61 Deep Blue (1996) The system derived its playing strength mainly from brute force computing power. Chess knowledge was fine tuned by grandmasters. Studied thousands of games.

62 Deep Blue (1996) The system derived its playing strength mainly from brute force computing power. Chess knowledge was fine tuned by grandmasters. Studied thousands of games.

63 Go

64

65

66

67

68

69

70

71

72 How?

73 How?

74 How?

75 How?

76 Reinforcement Learning

77 Reinforcement Learning Elements of Reinforcement Learning States The agent is in a given state at all times.

78 Reinforcement Learning Elements of Reinforcement Learning States Actions The agent is in a given state at all times. The agent can perform a finite number of actions that will take it into a new state.

79 Reinforcement Learning Elements of Reinforcement Learning States Actions Rewards The agent is in a given state at all times. The agent can perform a finite number of actions that will take it into a new state. The agent is awarded a reward for each state that it is in. Typically an integer number.

80 Reinforcement Learning Elements of Reinforcement Learning States Actions Rewards The agent is in a given state at all times. The agent can perform a finite number of actions that will take it into a new state. The agent is awarded a reward for each state that it is in. Typically an integer number. Hungry Eat Not Hungry

81 Reinforcement Learning Elements of Reinforcement Learning States Actions Rewards The agent is in a given state at all times. The agent can perform a finite number of actions that will take it into a new state. The agent is awarded a reward for each state that it is in. Typically an integer number. Hungry -10 Eat Not Hungry +10

82 Reinforcement Learning Elements of Reinforcement Learning Objective The agent goal is to maximize the reward.

83 Reinforcement Learning Elements of Reinforcement Learning Objective Policy The agent goal is to maximize the reward. A policy states the action to take at each possible state.

84 Reinforcement Learning Elements of Reinforcement Learning Objective Policy Optimal Policy The agent goal is to maximize the reward. A policy states the action to take at each possible state. Maximizes the long time expected reward

85 Reinforcement Learning World - 3 by 3 grid Actions - Up, Down, Left, Right Rewards - All states have a -1 with the exception of top right

86 Reinforcement Learning World - 3 by 3 grid Actions - Up, Down, Left, Right Rewards - All states have a -1 with the exception of top right What action should we take if we are in this state?

87 Reinforcement Learning Can we teach a Taxi to pick up a passenger and drive to destination?

88 Reinforcement Learning World - 5 by 5 grid, 4 designated locations Actions - Up, Down, Left, Right, Pickup, Dropoff Rewards - All states have a -1 with the exception of being at the destination and dropping passenger which has +20

89 Reinforcement Learning World - 5 by 5 grid, 4 designated locations Actions - Up, Down, Left, Right, Pickup, Dropoff Rewards - All states have a -1 with the exception of being at the destination and dropping passenger which has +20 Taxi is on this square with the passenger, dropoff location is "G". Which action should it take?

90 Reinforcement Learning World - 5 by 5 grid, 4 designated locations Actions - Up, Down, Left, Right, Pickup, Dropoff Rewards - All states have a -1 with the exception of being at the destination and dropping passenger which has +20 Taxi is on this square with the passenger, dropoff location is "G". Which action should it take? dropoff

91 Reinforcement Learning World - 5 by 5 grid, 4 designated locations Actions - Up, Down, Left, Right, Pickup, Dropoff Rewards - All states have a -1 with the exception of being at the destination and dropping passenger which has +20 Taxi is on this square with the passenger, dropoff location is "G". Which action should it take?

92 Reinforcement Learning World - 5 by 5 grid, 4 designated locations Actions - Up, Down, Left, Right, Pickup, Dropoff Rewards - All states have a -1 with the exception of being at the destination and dropping passenger which has +20 Taxi is on this square with the passenger, dropoff location is "G". Which action should it take? right

93 Reinforcement Learning World - 5 by 5 grid, 4 designated locations Actions - Up, Down, Left, Right, Pickup, Dropoff Rewards - All states have a -1 with the exception of being at the destination and dropping passenger which has +20 How many states can we possibly have?

94 Reinforcement Learning World - 5 by 5 grid, 4 designated locations Actions - Up, Down, Left, Right, Pickup, Dropoff Rewards - All states have a -1 with the exception of being at the destination and dropping passenger which has +20 How many states can we have? 5x5 grid = 25 Passenger can be at either of 4 locations or on board = 5 Destination = 4 25 * 5 * 4 = 500 states

95 Reinforcement Learning World - 5 by 5 grid, 4 designated locations Actions - Up, Down, Left, Right, Pickup, Dropoff Rewards - All states have a -1 with the exception of being at the destination and dropping passenger which has +20 What if we create a table and learn what action to take at each state?

96 Reinforcement Learning World - 5 by 5 grid, 4 designated locations Actions - Up, Down, Left, Right, Pickup, Dropoff Rewards - All states have a -1 with the exception of being at the destination and dropping passenger which has +20 What if we create a table and learn what action to take at each state?

97 Reinforcement Learning What if the state space is really big (continuous)?

98 Reinforcement Learning What if the state space is really big (continuous)?

99 Reinforcement Learning Balance a pole Keep a pole standing for as long as possible

100 Reinforcement Learning Land in the moon! Fire the spaceship engines to land in the moon!

101 MIT Smart Web apps using Machine Learning

102 Sentiment & Text Analysis Extract Information about Text and understand Sentiment

103 Image classification Detect object within image

104 Chatbots

105 Thank you! Questions? Carlos Aguayo