资源描述
cascade-correlation 1Cascade-correlationBy Kranthi & sudhancascade-correlation 2Contents lMotivation lRecollecting back-proplCascading architecture lLearning Algorithm lExamplelComparision with other systemscascade-correlation 3Motivation lCurse of dimensionalitylSimple networklDetermines the structure lFast learner cascade-correlation 4Recollect lWhats Back-propagation?lProblems with this alogorithm.lHow CC is going to solve these problems?cascade-correlation 5Cascade-CorrelationCC combines 2 key ideas.lCascade architecture. lLearning alogorithm.cascade-correlation 6Cascade ArchitecturelBegins with some inputs and one or more outputs.lEvery input is connected to every output.lBias is permanently set to +1.cascade-correlation 7x0 x1x2y2y1Stage -1 cascade-correlation 8x0 x1x2y2y1z1Stage-2cascade-correlation 9x0 x1x2y2y1z1z2Stage-3cascade-correlation 10AlgorithmlTrain stage 1. If error is not acceptable, proceed.lTrain stage 2. If error is not acceptable proceed.lEtccascade-correlation 11Algorithm1. Train all the connections ending at an output unit with a usual learning algorithm until the error of the net no longer decreases. 2 CC starts with a minimal network consisting only of an input and an output layer. Both layers are fully connectedcascade-correlation 12Algorithm3. Generate the so-called candidate units. Every candidate unit is connected with all input units and with all existing hidden units. Between the pool of candidate units and the output units there are no weights. cascade-correlation 13Algorithm 4. Try to maximize the correlation between the activation of the candidate units and the residual error of the net by training all the links leading to a candidate unit. Learning takes place with an ordinary learning algorithm. The training is stopped when the correlation scores no longer improves. cascade-correlation 14Algorithml In order to maximize this S,we compute partial derivative of S with respect to each candidate units incoming weight5.Choose the candidate unit with the maximum correlation, freeze its incoming weights and add it to the net. cascade-correlation 15Algorithm 6. To change the candidate unit into a hidden unit, generate links between the selected unit and all the output units. Since the weights leading to the new hidden unit are frozen, a new permanent feature detector is obtained. Loop back to step 2.7. This algorithm is repeated until the overall error of the net falls below a given valuecascade-correlation 16Example -Two spirals problem cascade-correlation 17Evolution of a 12-hidden unit solution to the two spirals problemcascade-correlation 18Evolution of a 12-hidden unit solution to the two spirals problemcascade-correlation 19Comparing CC with other learning algorithmslNo need to predict the size,depth and connectivity pattern of networklLearns fast unlike some other algorithms.lAt any time,we only train one layer of weights in network so results can be cached.cascade-correlation 20Experimental results for hand written digits data sets cascade-correlation 21Experimental results for the patients with severe head injury dataset cascade-correlation 22Experimentals results for land satellite image cascade-correlation 23Conclusion lPrinciple difference between CC and other algorithms is dynamic creation of hidden unitslSpeed up the learning process considerably cascade-correlation 24ReferencesThe Cascade Correlation Learning Architecture. Scott Fahlman and Christian Lebiere.Machine Learning, Neural and Statistical Classification byD. Michie, D.J. Spiegelhalter, C.C. Taylor (eds)cascade-correlation 25Thank you
展开阅读全文