1 (8) Institutionen för informationsteknologi Olle Gällmo Universitetsadjunkt Adress: Lägerhyddsvägen 2 Box 337 751 05 Uppsala Artificial Neural Networks written examination Monday, May 15, 2006 9 00-14 00 Allowed help material: Pen, paper and rubber, dictionary Telefon: 018-471 10 09 Telefax: 018 51 19 25 Hemsida: user.it.uu.se/~crwth Epost: olle.gallmo@it.uu.se Information Technology Olle Gällmo Lecturer Address: Lägerhyddsvägen 2 Box 337 SE-751 05 Uppsala SWEDEN Telephone: +46 18-471 10 09 Telefax: +46 18 51 19 25 Web site: user.it.uu.se/~crwth E-mail: olle.gallmo@it.uu.se Please, answer (in Swedish or English) the following questions to the best of your ability. Any assumptions made, which are not already part of the problem formulation, must be stated clearly in your answer. Write your name on top of each page. Don't forget to hand in the last page! (your answers to question 10) The maximum number of points is 40. To get the grade G (pass) a total of 20 points is required. The grade VG (pass with distinction) requires approximately 30 points, but also depends on the results on the lab course (labs + project). Your teacher will drop in sometime between 10.00 and 11.00 to answer questions. In this exam, some concepts may be called by different names than the ones used in the book. Here is a list of useful synonyms and acronyms: Perceptron = summation unit = SU = conventional neuron Binary perceptron = summation unit with binary (step) activation function Multilayer perceptron = MLP = Feedforward network of summation units RBF = Radial Basis Functions Standard Competitive Learning = LVQ-I without a neighbourhood function Objective function = the function to be minimzed or maximized = error function = fitness function Now, sit back, relax and enjoy the exam! Good luck!
21 students attended this exam, of which 9 failed, 7 passed (G) and 5 passed with 30 points or more (may become pass with distinction, depending on the results from lab course). The best result was 37 points. 1. Why is it impossible for a single binary perceptron to solve the OR problem?... (2) Because OR is not a linearly separable problem. Perceptrons solve classification tasks by adjusting a hyper plane in the input space (i.e. in 2D, as in this case, a line). Most students got this. Some failed to mention that the discriminant is a hyperplane/line, though, which is the main point here. 21 answers, 15 with max credit, Average: 1.5 2. Neural networks require lots of data to be trained properly. If you have too little data (too few input-target pairs) the first thing to try is to get more. However, sometimes this is simply not possible and, then, to split up the few data you have in a training set and a test set might be considered wasteful. Describe how K-fold cross validation can be used to deal with this problem! (Note: This is not early-stopping!)... (3) Split the data into K-sets, of N/K patterns each, where N is the total number of patterns. Train on all but one and test on the one left out. Do that for each of the K sets. Report the average error over the K tests. Alternative: Select N/K patterns at random, train on the rest, test on the ones selected and run this K times. Report result as above. K-Fold cross validation is not an early-stopping technique, though, as was clearly pointed out in the question. 1 point was deducted for answers which did not mention what to do with the K test results. 18 answers, 5 with max credit, Average: 1.9
3. What is weight decay? What is it good for and how can it be implemented?... (2) Weight decay is to let each weight in a neural network strive for 0 (in addition to the change given by the training algorithm, of course). There are several reasons for wanting to do this. For example: (most common answer) So that we can remove unecessary weights after training, since they will be very close to 0. By the way, if you do this you should retrain the network afterwards. To avoid numerical problems with too large weights since the weighted sum is in the exponent of the sigmoid, large weights may quickly lead to numerical problems. (related to the previous) Large weights also means that the sigmoids are likely to bottom out in either end, where the derivative is close to 0. This makes the network rigid, since this derivative is multiplied in the weight update formula. So, weight decay gives the network more flexibility and can speed up learning, since it tends to move the weighted sums closer to the region in the sigmoid where the derivative is the largest. Implementation: After updating the weights according to the update rule, update them again by w := (1-e)w where e is the decay rate. Some students associated weight decay with Ant Colony Optimization instead of Neural Networks. Indeed, the decay of pheromones can be viewed as a form of weight decay, but since the concept has not been discussed in those term on this course, only partial credit was given for such answers. 21 answers, 7 with max credit, Average: 1.1 4. Write down the back propagation algorithm! For full credit, your description must be clear enough for someone who knows what a multilayer perceptron is, to implement the algorithm.... (5) Considering that so much of this course focus on MLPs and backprop, I was very surprised to see how many students failed this question. Only two students got max credit, only two more were close. most got less than half of that. 20 answers, 2 with max credit, Average:1.9
5. How is the hidden layer of a RBF network different from the hidden layer in a MLP? Explain this difference in terms of: a) what the hidden nodes compute when feeding data to the network... (2) MLP hidden nodes compute weighted sums of the input and feed that through a sigmoid. RBF nodes compute the distance between the input vector and the weight vector and feed that through a Gaussian (or similar function). b) how this affects the shape of the discriminant when using the networks for classification... (2) MLP hidden nodes form hyperplanes, RBF nodes form hyperspheres (or hyperellipses) Sidenote: It is not the activation function which decides the shape of the discriminant. The weighted sum forms the hyperplane in MLPs (the sigmoid only decides what to output, given the distance from that hyperplane). Similarly, for RBFs, it is the distance calculation between input vector and weight vector which forms the hypersphere, not the Gaussian. c) how the hidden nodes are trained... (2) MLPs are usually trained by some form of backprop (see Q4). The hidden layer of a RBF network is usually trained by some form of unsupervised learning, e.g. competitive learning or K-Means. Some students got minor deductions for only describing RBF, not comparing to MLP. A few students described the output layer instead of the hidden layer. 21 answers, 1 with max credit, Average: 3.4
6. The course book suggests two ways to initialize the weights of unsupervised competitive learning networks: Either (a) initialize weights to small random values sampled from a uniform distribution, or (b) take the first input patterns as initial weight vectors One of these is actually not a very good idea. Which, and why?(3) (a) is the bad one. Initializing like this is to place all nodes (their weight vectors) in a region close to origo. That may be very far from where the data actually is, i.e. where the nodes are needed. This means that the one which happens to be closest from the start is likely to win for all patterns. This is the "winner-take-all" scenario. (b) is better, since the initial positions of the nodes then are close to the data. Some students pointed out that it would be even better to select vectors from the data at random instead of the N first ones. Indeed, the data may be sorted which could lead to all nodes getting intial positions in the same cluster (which may lead to the same problem as in (a) but is not as likely). All students who claimed (b) was better then (a) did so with arguments that either did not hold or which actually are good arguments, for (a)! For example, some students worried that drawing weight vectors from the data may give the network extreme weight values well, if the data is in extreme locations, then the nodes should also be there! 21 answers, 12 with max credit, Average: 1.8 7. Explain how weights are updated in Kohonen's Self Organizing Feature Map!... (3) SOFM nodes are organized in a 2D grid (usually). Each node's (i) weight vector is moved towards the input vector (instead of just the winner's, k, as in competitive learning) by an amount which decreases with distance from the winner node on the grid. (i.e. it is not a function of the distance between their weight vectors, but of the distance between node i and k on the grid): w i = ηf i, k)( x w ) ( i f is usually a Gaussian, but the important point is only that it should have its maximum (=1) för node k itself (i=k) and then decrease with distance from k. Just saying that the winner and it's neighbours are updated is not enough. I wanted to know how either in words or formally. 20 answers, 10 with max credit, Average: 1.8
8. Most reinforcement learning algorithms include a discount factor, usually denoted by the greek letter γ. What is the main reason for having this factor?... (3) The goal of RL is to maximize the sum of all future rewards. γ is there to ensure that this sum is finite, even if we have an infinite future. Partial credit was given for the answer that γ is there to ensure that blame/credit is given more to recent actions than to earlier ones. This is indeed an effect of γ, but it is a side effect and it is not always wanted. Actually, this side effect is the main disadvantage with discounted RL methods. Whether it is important to blame late decisions more than others, or not, depends on the application and should therefore be expressed in the problem formulation (for example in how rewards are payed), not be builtin into the algorithm! In lab 2, for example, there was nothing in the problem formulation that said that we wanted it to find the shortest route. Q-Learning/Sarsa did, however, find the shortest route because they have this hidden assumption builtin. There are other ways to ensure that the sum is finite, without this side effect. R-Learning, for example. But we have not discussed these kind of algorithms on this course. 19 answers, 6 with max credit, Average: 1.3 9. Explain and compare the two evolutionary computing concepts genotype and phenotype.... (3) A genotype is the encoding of a solution to a problem, for example a bit string. A phenotype is the interpretation of the genotype (its manifestation, you might say, or its semantics) necessary in order to evaluate its fitness. Operators ususally work on the genotypic level, while evaluation is done on the phenotypic level. 18 answers, 13 with max credit, Average: 2.4 10. Please mark in the table on the next page, which statements are true and which are false, and hand in the table together with your other answers. Each correct answer yields +0.5 points and each incorrect answer yields -0.5 points, so be careful do not guess! (However, the total number of points on this assigment can not become less than zero.)... (10) See comments to each individual question in the table below Average score: 5.5 Max: 8.5
Exam ANN 060515 Your name:... Statement True False a) If a state in a Hopfield network is stable, then the inverse state is not. False. A state and its reverse (mirror) is one and the same. They have the same energy. Hopfield networks really only store correlation between nodes. 19 answers, 18 correct. b) The main difference between immediate and delayed reinforcement learning is in how often the rewards are received. False. A delayed reinforcement learning task is one where the optimal solution can only be found by associating incoming rewards with a whole sequence of previous actions, instead of just the latest one. However, the reward may very well be received in every time step. 18 answers, 14 correct. c) Temporal difference learning tries to minimize the difference between successive predictions, rather than the difference between the current prediction and the true value. True. This is the main point with TD and that which makes it most different from supervised learning. 12 answers, 9 correct. d) An advantage with gradient descent based methods, such as back propagation, is that they can not get stuck in local minima. False. Gradient descent methods are very likely to get stuck in local minima, since they only move downhill. We can hope that we'll overshoot local minima, but we cannot avoid the problem. 21 answers, 20 correct. e) Weights are update more often in pattern learning than in batch (epoch) learning. True. Pattern learning updates the weights after every pattern presentation. Batch learning only accumulates the weight changes until the whole batch of patterns has been presented once. 20 answers, 19 correct. f) A Hopfield network with symmetric weights and asynchronous updating will always converge to a stable state. True. It may not be the wanted stable state, but it will be stable. 17 answers, 14 correct. g) If a classification task is such that it is possible to separate the classes with a linear discriminant, then a perceptron trained by Rosenblatt's convergence procedure is guaranteed to find it. True. It is even guaranteed to find it in finite time. This is Rosenblatt's Convergence Theorem. 8 answers, 6 correct. h) One difference between (most) evolutionary computation algorithms and (most) neural network training algorithms, is that evolutionary computation algorithms do not require the objective function to be differentiable. True. This is since the objective function in EC is only used for selection, not to compute how to change the parameters in the system. 19 answers, 18 correct. i) Increasing the number of hidden nodes in a multilayer perceptron improves generalization. False. Increasing the number of hidden nodes increases its representational ability, which increases the risk of overtraining (learning all patterns by heart in the extreme case). Some students pointed out, though, that if the initial number of hidden nodes was too small to solve the problem at all, then increasing the number should improve generalization. Up to the point where the number of nodes is exactly the required number of nodes, that is. My mistake, not to consider that. I intended to give all student +0.5 points for this question, whether they answered the question or not, but this would not have changed the grade for any student, so I didn't. 17 answers, 16 correct. j) It is impossible to overtrain a genetic algorithm. False. Any form of learning system, which trains on a training set which is not complete, can be overtrained, i.e. become too specialized on that particular training set. 9 answers, 4 correct.
k) In general, radial basis function networks perform better than multilayer perceptrons for problems with many input variables. False. RBF networks have more problems in high dimensionality than MLPs. In general. 16 answers, 15 correct. l) In radial basis function networks, one hidden layer is sufficient. True. In theory this is true also for MLPs (though more hidden layers are very rarely needed). For RBF networks it's not just theory, though, but a very practical rule. 19 answers, 18 correct. m) RPROP (Resilient backprop) only considers the sign of the gradient, not its value, when computing weight changes. True. Instead it adopts the step length on-line, individually for each weight, depending on whether we seem to keep moving downhill, or whether we have overshot a minimum. 20 answers, 19 correct. n) In general, compact input representation are better than distributed ones, when training neural networks. False. Neural networks prefer distributed representations. Neural networks are distributed processing devices, after all, and why should it have to learn how to unpack data which we packed? 12 answers, 11 correct. o) Evolutionary computation algorithms are very unlikely to get stuck in local minima. True, since it is a population method. As long as the population is spread out (maintaining diversity) it is very unlikely that they all get stuck. 18 answers, 17 correct. p) The boolean function (a AND b AND c) OR d can be implemented by a single binary perceptron. True. See Figure below. 19 answers, 14 correct. q) A self organizing feature map can be trained to separate overlapping classes. False. That would be true magic. And even if it could, it would be a failure: The whole point with SOFM is that it should preserve topology, i.e. maintain statistical distributions found in the higher dimensional space. 13 answers, 6 correct. r) Any function represented by a multilayer perceptron with a linear hidden layer, can be represented by a single layer of perceptrons (i.e. without that hidden layer). True. It can be collapsed into a network without that hidden layer. Multiplying x by 2 and then by 5 is equivalent to multiplying x by 10. 17 answers, 12 correct. s) In general, radial basis function networks are better for on-line learning tasks (continuous learning) than multilayer perceptrons. True. This is because a basis function is local. New information from some other part of the space does not affect it and, hence, is not likely to destroy what it has learnt so far. MLPs, on the other hand, tend to destroy the old information when new information comes in. 16 answers, 15 correct. t) Randomizing the order in which patterns are presented to the network has no effect in batch (epoch) learning. True, since the weight updates are accumulated (summed up) until the end of the batch, and + is commutative. 20 answers, 15 correct. a b c d 1 1 1 3-1 2.5 (a AND b AND c) OR d Proof by example to question p)