资源描述
,单击此处编辑母版标题样式,单击此处编辑母版文本样式,第二级,第三级,第四级,第五级,*,AI&NN Notes,Chapter 9,Feedback Neural Networks,AI&NN Notes Cha,1,9.1 Basic Concepts,Attractor,:a state toward which the system evolves in,time starting from certain initial conditions.,Basin of attraction,:the set of initial conditions which,initiates the evolution terminating in the attractor.,Fixed point,:,if an attractor is in a form of a unique,point in state space.,Limit Cycle,:if an attractor consists of a periodic,sequence of states.,9.1 Basic ConceptsAttractor:,2,Hopfield Network and its Basic Assumptions,1,2,n,v,v,v,1,2,n,T,T,T,1,2,n,i,i,i,n,2,1,w,w,w,w,21,12,n1,w,w,n2,1n,2n,1.1 layer,n neurons,2.T -Threshold of,neuron i,3.w -weight from,j to i,4.v -output of,neuron j,5.i -external input to,the i-th neuron,ij,i,j,Hopfield Network and its Basic,3,The total input of the i-th neuron is,net =,i,J=1,j,i,n,w v +i -T =W V+i -T ,for I=2,n,ij,j,i,i,i,t,i,i,where,W =,w,w,w,i,i1,i2,in,.,.,.,V=,v,v,v,1,2,n,The total input of the i-th ne,4,The complete matrix description of the linear portion,of the system shown in the figure is given by,net=WV+i-t,where,net=,net,net,net,i=,i,i,i,1,2,n,1,2,n,are vectors containing activation,external input,to each neuron and threshold vector,respectively.,t=,T,T,T,1,n,2,The complete matrix descriptio,5,W is an n,n matrix containing network weights:,W=,w,w,w,=,w,w,w,w,w,w,w,w,w,w,w,w,0,0,0,0,1,2,n,t,t,t,12,13,1n,21,23,2n,31,32,3n,n1,n2,n3,w =w,ij,ji,w =0,ii,and,W is an nn matrix contain,6,9.2 Discrete-Time Hopfield Network,Assuming that the neurons activation function is sgn,the,transition rule,of the i-th neuron would be,-1,if net 0 (excitatory state),v,If,for a given time,only a single neuron is allowed to,update its output and only one entry in vector v is,allowed to change,this is an,asynchronous operation,under which each element of the output vector is,updated separately while taking into account the most,recent values for the elements that have already been,updated and remain stable.,(*),9.2 Discrete-Time Hopfield N,7,Based on(*),the,update rule,of a discrete-time,recurrent network,for one value of i at a time,becomes,v =sgn(w v +i -T)for random i,i=1,2,n,and k=0,1,2,.,i,K+1,i,t,k,i,i,where k denotes the index of recursive update.This is,referred as,asynchronous stochastic recursion,of the,Hopfield network.This update process will continue,until all n entries of v have been updated.,The recursive computation continues until the output,node vector remains unchanged with further iterations.,Similarly,for,synchronous operation,we have,i,K+1,v =TWv +i-t,for all neurons,k=0,1,.,K+1,k,where all neurons change their output simultaneously.,Based on(*),the upda,8,Geometrical Explanation,The output vector v is one of the vertices of the n-,dimensional cube-1,1 in E space.The vector,moves during recursions from vertex to vertex,until,it is stabilizes in one of the 2 vertices available.,The movement is from a vertex to an,adjacent,vertex,since the,asynchronous update mode,allows for a,single-component update of an n-tuple vector at a,time.,The,final,position of v as k,is determined by,weights,thresholds,inputs,and the initial vector v,as well as the order of transitions,.,n,n,0,n,Geometrical Explanationnn0n,9,To evaluate the stability property of the dynamical,system of interest,the,computational energy function,is defined in n-dimensional output space v .,If the increments of a certain bounded positive-valued,computational energy function under the transition,rule are found to be,non-positive,then the function,can be called a,Lyapunov function,and the system,would be asymptotically stable.,The scalar-valued energy function for the discussed,system is a quadratic form:,n,E=-1/2 V WV-i V+t V,t,t,t,To evaluate the stability prop,10,or,E=-1/2,w v v -i v +t v,i=1,n,j=1,j,i,n,ij,i,j,i=1,i=1,n,i,i,n,i,i,The energy function in asynchronous mode.,Assume that the output node I has been updated at the,k-th instant so that v -v =,v .Computing the energy,gradient vector:,i,k+1,k,i,E=-1/2(W +W)v-i +t =-Wv-i +t,v,t,t,t,t,t,W =W,t,The energy increment becomes,E=(,E,)v=(-W v-i +t )v,t,t,t,t,i,i,i,i,This is because only the i-th output is updated.,ori=1nj=1nijiji=1i=1niiniiThe,11,Therefore we have,(,v)=0,v 0,i,t,This can be rewritten as,E=-(w v +i -t)v for j i,i,i,i,j,ij,n,j=1,or briefly,E=-net v,i,i,Note that,when net 0,then,v,0,thus,(net,v)is always non-negative,.In other words,any corresponding,energy changes,E are non-positive,provided that w =w .,i,i,i,i,i,i,ij,ji,Therefore we have (v),12,Further we can show that the non-increasing energy,function has a minimum.,Since W is indefinite because of its zero diagonal,then,E has neither a minimum nor maximum in,unconstrained output space.However,E is obviously,bounded in n-dimensional space consisting of the 2,vertices of n-dimensional cube,Thus,E has to reach,its minimum finally under the update algorithm.,n,Example of recursive asynchronous update of,computed digit 4:,Further we can show that the n,13,(a)
展开阅读全文