Deep search. Enhancing a search bar using machine learning. Ilgün Ilgün & Cedric Reichenbach

Transcription

1 #BaselOne7

2 Deep search Enhancing a search bar using machine learning Ilgün Ilgün & Cedric Reichenbach

3 We are not researchers

4 Outline I. Periscope: A search tool II. Goals III. Deep learning IV. Applying CNNs to Periscope

5 Periscope: A search tool

6 Periscope Open source Based on Vaadin Main project: github.com/cedricreichenbach/vaadin-periscope Magnolia integration: git.magnolia-cms.com/users/creichenbach/repos/periscope

7 Live result listing

8 Pluggable results supplier API Result ResultSupplier Result Result Periscope ResultSupplier Result Result Result Result ResultSupplier

9 ResultSupplier API String gettitle(); List<Result> search(string query) throws SearchFailedException;

10 Goals

11 Searchable images Cat???

12 Speech recognition Hello, world!??? hello world

13 Adaptive result order Car Car SBB Cargo Mobility Car Sharing??? Carson Daly SBB Cargo Mobility Car Sharing Carson Daly Disney Cartoons Disney Cartoons

14 Deep learning

15 What is deep learning? Artificial Intelligence Machine Learning Deep Learning

16 What can be done with deep learning? Image recognition Speech recognition Fraud detection Recommendation Systems

17 Learning Types Supervised Unsupervised e.g. image tagging (figure out e.g. classification of images whose picture it is) (divide picture per person) Semi-supervised e.g. Periscope

18 The no free lunch theorem

19 (Convolutional) Neural networks

20 Artificial neuron (unit)

21 Neural network input layer hidden layer output layer

22 Deep neural network input layer multiple hidden layers output layer...

23 Learning by adjusting weights input layer sample input forward pass hidden layers output layer measured error expected output backpropagation

24 Convolutional layer

25 Convolutions Kernel Original data Convolved feature

26 Applying CNNs to Periscope

27 Image recognition and automated tagging Cat Panda Dog Battleship

28 Image recognition networks commons.wikimedia.org/wiki/file:typical_cnn.png Jones, N.: The learning machines, Nature 55

29 ResNet-5 Residual network 5 layers 52-layer variant: st place on ILSVRC 25 Deep Residual Learning for Image Recognition by Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun arxiv.org/abs/

30 ImageNet database Millions of labelled images Based on WordNet nouns > 5 images per class Pre-trained networks available

31 Automated tagging Cat Database cat cat cat cat ImageNet

32 Magnolia implementation Magnolia Image Tagging Module: github.com/ilgun/magnolia-image-tagging-module Free, open source Labels image assets using JCR properties

33 [demo]

34 Speech recognition hello world Hello, world!

35 Web Speech API const recognition = new SpeechRecognition(); recognition.onresult = event => { const transcript = event.results[][].transcript; //... }; recognition.start();

36 Adaptive result order Carg Online (= dynamic) learning Anton Cargnelli Semi-supervised Carglass SBB Cargo CarGold

37 Mapping to be trained Carg SBB Cargo String Set entry

38 Input layer 5 28 code points in ASCII 5 characters max 28 2-dimensional array, 28 x 5 units:

39 c #99 a #97 r #4 g #3 28

40 Initial convolution Kernel size: 28 x 3 associate 3 adjacent characters foobar foo oob oba bar ar

41 c a r g 3 x 28

42 Output layer.5 Carglass n units (size of result set) n Output value [, ]: Estimation of relevance.6 SBB Cargo. Cargold

43 Training samples Input Output Carglass SBB Cargo Cargold...

44 [demo]

45 Setup 28 x 5 (input layer) convolution ( x) 5 fully connected 5 fully connected 2 fully connected fully connected n (output layer)

46 Take-aways UX machine learning Deep learning has matured Entry barriers are low

47 Q&A

48 Appendix

49 Residual neural network Instead of F(x), approximate F(x) + x by adding shortcuts Solves vanishing gradient problem Each building block learns something new deeper networks possible Building block Deep Residual Learning for Image Recognition, Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun, arxiv.org/abs/

50 Configuration Activation function: Hyperbolic tangent Optimization: Stochastic gradient descent Update function: Nesterovs Learning rate:.2 (high because few data points) Regularization: L2, ⁵