Support Vector Machines!

The Sorting Hat is sick today We need to help it sort the students!

But all we know is: Intelligence Bravery House (Gryffindor or Ravenclaw) 5.5 10.5? 8 16? 12.5 4.5? 4 12? 10.5 6? 15 7? 10.5 13? 10 3?

Let s look at what the Sorting Hat did before...

Instructions for how to plot data You have a list of people, each with an intelligence and bravery value, and house. For each person, write their intelligence and bravery value on a sticky note (use a pink sticky if they re from Gryffindor and a blue sticky if they re from Ravenclaw) Plot each sticky note on your paper

Previous students

Add new students to your plot (yellow stickies) Intelligence Bravery House (Gryffindor or Ravenclaw) 5.5 10.5? 8 16? 12.5 4.5? 4 12? 10.5 6? 15 7? 10.5 13? 10 3?

New students this year

Here are their houses Replace the yellow stickies with the correct color sticky. Now, use a yardstick to autosort new students! Students on one side will be sorted into Gryffindor. Students on the other side will be sorted into Ravenclaw.

Don t move your yardstick!

What does your yardstick tell you about these students? Intelligence: 16 Bravery: 15 Intelligence: 1 Bravery: 5.5

What does your yardstick tell you about these students?

This year s new students! Harry Potter Parvati Patil Ron Weasley Hermione Granger Dean Thomas Neville Longbottom Luna Lovegood Padma Patil Terry Boot Michael Corner

So what makes a good line?

So what makes a good line? Bad!

So what makes a good line? Good? Why?

Maximum-margins!

Next year!

Welcome to another year at Hogwarts! After being sick last year, the sorting hat is starting to think that it might be time to retire... We are the new Sorting Hat in-training! We choose the house, then the Sorting hat will tell us if we got it right

Set up your yardstick, then don t move it...

Intelligence Bravery House (Gryffindor or Ravenclaw) Time to sort some students! (yellow stickies) 6 8? 8 4? 3 11? 12 14? 7 4? 6 2? 12.5 9? 11 9? 9 1? 12 1?

Which houses do you think these students should be sorted into?

Here are the answers. How did you do?

So what makes a good line? (What should we do for next year?)

Last year of Sorting-Hat-apprenticeship

Set up your yardstick, then don t move it...

Intelligence Bravery House (Gryffindor or Ravenclaw) Time to sort some students! (yellow stickies) 6 5? 4 2.5? 10 10? 15 12? 14 9? 16 3? 14 12? 2 9? 5 1? 5 13?

Which houses do you think these students should be sorted into?

Here are the answers! How did you do?

So what makes a good line? (Which line was better?)

What if we had one more attribute (feature) of people that we were measuring? (How would we incorporate this data into our auto-sorter?) Intelligence Bravery Quidditch ability House 4 12 12 Gryffindor (Ron Weasley) 16 15 4 Gryffindor (Hermione Granger) 8 16 18 Gryffindor (Harry Potter)

What if we had two more attributes (features) of people that we were measuring? (How would we incorporate this data into our auto-sorter?) Intelligence Bravery Quidditch ability Kindness House 4 12 12 10 Gryffindor (Ron Weasley) 16 15 4 16 Gryffindor (Hermione Granger) 8 16 18 10 Gryffindor (Harry Potter)

Now that we have the power to learn patterns from data and apply that knowledge...

Let s sort beings into human versus non-human! You have a list of beings, their height, and whether they are human or nonhuman Put a piece of masking tape on your string for each being, and mark whether it is human or non-human Can you get 100% classification accuracy using your yardstick?

We can do this from 2D to 3D too! (...or 3D to 4D!...or 4D to 9D! )

Machine learning! Model patterns in data (using math!) Use models to infer information from new data

Support Vector Machines! (The math!) Support Vector Machine BIG = + margin Small errors = Balance between reducing errors versus making a bigger margin

Support Vector Machines! (The math!) Big C: Errors are REALLY BAD! Small C: Errors are ok, as long as we make the margin BIG

Support Vector Machines are powerful! Handwriting recognition! (Automated mail sorting) Image classification! (Cat or not?) Text document classification! (Automated spam filtering)

Machine learning is powerful!

Thanks for helping with sorting!

The sorting hat - Welcome to Hogwarts! - Every year the sorting hat sorts students into houses (e.g. Gryffindor and Ravenclaw). - But the sorting hat is sick today :(. - Your job is to help the sorting hat sort students. - How can we do this? - Lets look at how previous students were sorted.

How can we sort new students? - How can we use the data we plotted to sort this year s students? - We will now try and sort a few students!

How can we sort new students? Intelligence Bravery House XX XX?

How can we sort new students? Sorting-hat solution: Intelligence Bravery House XX XX Gryffindor (Harry Potter)

How can we sort new students? Sorting-hat solutions: Intelligence Bravery House XX XX Gryffindor (Ron Weasley) XX XX Ravenclaw (Luna Lovegood) XX XX Ravenclaw (Terry Boot) XX XX Ravenclaw (Padma Patil) XX XX Gryffindor (Parvati Patil) XX XX Gryffindor (Dean Thomas) XX XX Ravenclaw (Michael Corner)

Maximum margins! Here are two more students: Intelligence Bravery House XX XX? XX XX? - Does your classifier correctly sort these students? - How can we describe a best line using only the original students?

Maximum margins! Sorting-hat solutions: Intelligence Bravery House XX XX Gryffindor (Hermione Granger) XX XX Gryffindor (Neville Longbottom)

Sorting hat answers show some outliers (but still barely linearly separable) (i.e. Hermione and Neville) Make a new line We can choose to either try really hard to do what the sorting hat, or do a better job of following the sorting hat s overall pattern

Last round of 2-D classification

Humans versus non-humans

Some description of how non-linear SVM works?

How can we sort new students? Intelligence Bravery House XX XX? XX XX? XX XX? XX XX? XX XX? XX XX? XX XX?