Do we need more training data or better models for object detection?

Transcription

1 Do we need more training data or better models for object detection? Xiangxin Zhu Carl Vondrick Deva Ramanan Charless Fowlkes University of California, Irvine Appeared in BMVC 2012 Slides adapted from Charless Fowlkes 1

2 MoBvaBons 2 2

3 Current state of object recognibon Avg. AP Year Avg. AP Avg. num. of training samples per class PASCAL VOC detec=on challenge provides realisbc benchmark of object detecbon performance. Performance has steadily increased! 3

4 Current state of object recognibon Avg. AP Year Avg. AP Avg. num. of training samples per class PASCAL VOC detec=on challenge provides realisbc benchmark of object detecbon performance. Performance has steadily increased!. but so has the amount of training data?? 4

5 Bayes Risk P(X face) P(X background) Feature space may limit our ulbmate classificabon performance 5

6 Performance saturabon Ideal Performance Data 6

7 Model Bias Class of models may not be flexible enough 7

8 Model Bias 8

9 Ideal Performance Model Complexity 9

10 Experiments 10 10

11 Experiment #1 Single Face Template HOG Feature vector Train a linear classifier using SVM PosiBve examples + hard negabve mining 11

12 Performance vs #training examples 12

13 Performance vs #training examples Worse performance with more training data?!?!? 13

14 Performance vs #training examples Average precision Single template face model Fixed C=0.002 Crossval on C Num. of training samples 14

15 Average precision Need to make cross validabon easy for everyday users! C N=10 N=50 N=100 N=500 N=900 15

16 16 16

17 Experiment #2 We want to detect faces at many different viewpoints what posibve training data should we use? (a) include all viewpoints in training (b) only train on a subset of views (e.g. frontal faces) 17

18 Single template face model Average precision All Frontal Number of training samples Worse performance with more training data?!?!? 18

19 Single template face model Average precision All Frontal Number of training samples Single template trained with 200 clean frontal faces outperforms template trained with 800 images that include all views of faces. This holds true for both training and test performance 19

20 Learned templates All views Frontal views only 20

21 SVM is sensibve to outliers Single template face model Average precision All Frontal Number of training samples ALL has lower training objec3ve, but higher 0-1 loss! 21

22 Experiment #3 Increase model complexity by using mixture components to model different viewpoints. 22

23 Model wider range of variability by using a mixture of rigid templates 23

24 DiscriminaBve clustering uses mixture components to take care of outliers? AP Dataset Size 24

25 Human supervised clustering

26 Human- in- the- loop clustering can boost mixture model performance Average precision Face Human cluster, K=5 Kmeans cluster, K= Num. of training samples 26

27 Human- in- the- loop clustering can boost mixture model performance Average precision Bus 0.45 Human cluster, K=5 0.4 Kmeans cluster, K= Num. of training samples 27

28 AP 0.6 K=1 K=3 K=5 0.5 K=13 K= Number of training data AP 0.6 N= N=100 N=500 N= Number of mixtures Ideal Ideal Performance Data Model Complexity 28

29 Bus Category K=1 K=3 K=5 K=11 K=21 AP N=50 N=100 N=500 N=1000 N= Number of training data Number of mixtures 29

30 PASCAL 10x Dataset Collected 10 Bmes as much posibve training data as original PASCAL dataset Collect images from Flickr, MTurk users label images 30

31 10x training dataset with DPM Average precision Horse Bicycle 0.4 Bus Cat 0.3 Cow Diningtable 0.2 Motorbike Sheep Sofa 0.1 Train Tvmonitor Num. of training samples Cross validabon to choose opbmal regularizabon and # of mixture components for each category Performance saturates with 10 templates per category and 100 posibve training examples per template 31

32 Experiment #4: Have we reached Bayes Risk for linear classifiers with HOG features? Ideal Performance Data 32

33 Deformable part models parts deformabon model detector output Represent local part appearance with templates, connected by springs that encode relabve locabons. Trained using SVM. [Felzenszwalb, McAllester, Ramaman. 2008] 33

34 Alternate view of DPM Every placement of parts synthesizes a rigid template Dynamic programming used in DPM is a fast way to index a very large collecbon of rigid templates 34

35 Why does DPM do bejer than rigid mixtures? Part appearances are shared during training Can extrapolate to new unseen configurabons 35

36 Rigid Part Model (RPM): part appearance is learned from training only score spabal configurabons of parts seen during training very fast to test 36

37 Faces 0.9 Average precision Sup.DPM RMP 0.6 Latent DPM Mix. HoG templ. 0.5 K=1, Frontal 0.4 K=1, All Num. of training samples 37

38 State of the art face detecbon with only 100 training examples DPM with shared parameters precision [Zhu & Ramanan, 2012] 38

39 39

40 Do we need more data? 40

41 Do we need more data? More training data helps, but only if you are careful Clean training data can help SVM which is sensibve to outliers Having the proper correspondence / alignment / clustering can greatly improve model performance Bejer models might provide more bang for the buck 41

42 Dataset Bias: Distribu3ons Match 42 42