#BaselOne7
Deep search Enhancing a search bar using machine learning Ilgün Ilgün & Cedric Reichenbach
We are not researchers
Outline I. Periscope: A search tool II. Goals III. Deep learning IV. Applying CNNs to Periscope
Periscope: A search tool
Periscope Open source Based on Vaadin Main project: github.com/cedricreichenbach/vaadin-periscope Magnolia integration: git.magnolia-cms.com/users/creichenbach/repos/periscope
Live result listing
Pluggable results supplier API Result ResultSupplier Result Result Periscope ResultSupplier Result Result Result Result ResultSupplier
ResultSupplier API String gettitle(); List<Result> search(string query) throws SearchFailedException;
Goals
Searchable images Cat???
Speech recognition Hello, world!??? hello world
Adaptive result order Car Car SBB Cargo Mobility Car Sharing??? Carson Daly SBB Cargo Mobility Car Sharing Carson Daly Disney Cartoons Disney Cartoons
Deep learning
What is deep learning? Artificial Intelligence Machine Learning Deep Learning
What can be done with deep learning? Image recognition Speech recognition Fraud detection Recommendation Systems
Learning Types Supervised Unsupervised e.g. image tagging (figure out e.g. classification of images whose picture it is) (divide picture per person) Semi-supervised e.g. Periscope
The no free lunch theorem
(Convolutional) Neural networks
Artificial neuron (unit).7.4.6.8
Neural network input layer hidden layer output layer
Deep neural network input layer multiple hidden layers output layer...
Learning by adjusting weights input layer sample input forward pass hidden layers output layer measured error expected output backpropagation
Convolutional layer
Convolutions Kernel Original data Convolved feature 4 3 4 2 4 3 2 3 4
Applying CNNs to Periscope
Image recognition and automated tagging Cat Panda Dog Battleship
Image recognition networks commons.wikimedia.org/wiki/file:typical_cnn.png Jones, N.: The learning machines, Nature 55
ResNet-5 Residual network 5 layers 52-layer variant: st place on ILSVRC 25 Deep Residual Learning for Image Recognition by Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun arxiv.org/abs/52.3385
ImageNet database Millions of labelled images Based on WordNet nouns > 5 images per class Pre-trained networks available
Automated tagging Cat Database cat cat cat cat ResNet-5 @ ImageNet
Magnolia implementation Magnolia Image Tagging Module: github.com/ilgun/magnolia-image-tagging-module Free, open source Labels image assets using JCR properties
[demo]
Speech recognition hello world Hello, world!
Web Speech API const recognition = new SpeechRecognition(); recognition.onresult = event => { const transcript = event.results[][].transcript; //... }; recognition.start();
Adaptive result order Carg Online (= dynamic) learning Anton Cargnelli Semi-supervised Carglass SBB Cargo CarGold
Mapping to be trained Carg SBB Cargo String Set entry
Input layer 5 28 code points in ASCII 5 characters max 28 2-dimensional array, 28 x 5 units:
c #99 a #97 r #4 g #3 28
Initial convolution Kernel size: 28 x 3 associate 3 adjacent characters foobar foo oob oba bar ar
c a r g 3 x 28
Output layer.5 Carglass n units (size of result set) n Output value [, ]: Estimation of relevance.6 SBB Cargo. Cargold
Training samples Input Output.............................. Carglass SBB Cargo Cargold...
[demo]
Setup 28 x 5 (input layer) convolution ( x) 5 fully connected 5 fully connected 2 fully connected fully connected n (output layer)
Take-aways UX machine learning Deep learning has matured Entry barriers are low
Q&A
Appendix
Residual neural network Instead of F(x), approximate F(x) + x by adding shortcuts Solves vanishing gradient problem Each building block learns something new deeper networks possible Building block Deep Residual Learning for Image Recognition, Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun, arxiv.org/abs/52.3385
Configuration Activation function: Hyperbolic tangent Optimization: Stochastic gradient descent Update function: Nesterovs Learning rate:.2 (high because few data points) Regularization: L2, ⁵