基于RocketMQ-Connect构建全新数据流转处理平台课件

,12/20/2020,#,#,基于,RocketMQ Connect,构建全新数据 流转处理平台,技术创新，变革未来,基于RocketMQ Connect构建全新数据 流转处理平,OpenMessaging,Connect,Overview,OpenMessaging,Connect,Architecture,OpenMessaging,Connect,in,practice,OpenMessaging,Connect,Now,and,Future,目录,OpenMessaging Connect Overview,2,Concept,RocketMQ,Connect,：,A,simple,understanding,of,RocketMQ Connect,is,the,middleware,that,uses,RocketMQ,to,copy,data,from,a,source,data,storage,system,to,a,target data,store.,Mainly,composed,of,source,connector,sink,connector,runtime.,Source,Connector,：,Responsible,for,ingesting the,data,from,source,data,and,sending,it,to,RocketMQ,Sink,Connector,：,Responsible,for consuming,messages,from,RocketMQ,and,writing,the,data,to,target,storage,Runtime,：,The,runtime,is,the source,sink,connector,runtime,environment,responsible,for,loading,the,Connector,providing,the,RESTful,interface,starting,the,connector,task,service,discovery,between,cluster,nodes,configuration,synchronization,consumption,schedule,saving,failover,load,balancing,and,so,on.,ConceptRocketMQ Connect：A sim,3,File,Connect,Demo,File Connect Demo,4,RocketMQ,Connect,Overview,OLAP,Pr,o,d,u,c,e,r,Service,Sourcing,Data,Consum,e,r,Service,Traditional,stream,data,process,model,Pr,o,d,u,c,e,r,Service,Consum,e,r,Service,Sink,Data,RocketMQ Connect OverviewOLAPS,5,OpenMessaging,Connect,Overview,Challenge,How,to,copies,data,between,so,many,different,sources,Efficiency,High,Availability,failover,Scalability,OpenMessaging Connect Overview,6,OpenMessaging,Connect,Overview,OL,A,P,OpenMessaging Connect Overview,7,OpenMessaging,Connect,Overview,OL,A,P,OpenMessaging Connect Overview,8,OpenMessaging,Connect,Overview,Benefits,of,Messaging,Connect,Focus,on,data,copying,Scales,to,application,Accessible,connector,API,Streaming,and,batch,Easy,cluster,management,OpenMessaging Connect Overview,9,OpenMessaging,Connect,Overview,Whats,Messaging,Connect,Messaging,Connect,is,a,distributed,scalable,fault-tolerant,service,designed,to,reliably,stream,data between,messaging,middleware,and,other,data,system.,OpenMessaging Connect Overview,10,OpenMessaging,Connect,Overview,OL,A,P,Whats,OpenMessaging,OpenMessaging,is,a,standard,for messaging,middleware.,OpenMessaging Connect Overview,11,OpenMessaging,Connect,Overview,Relationship,of,OpenMessaging,Connect,and,RocketMQ,OpenMessaging,Connect,is,a,connect,standard,for,messaging,middleware.,Runtime,based,on,RocketMQ,provide,the,ability,of,steaming,data,between,other,data,system,with,Apache,RocketMQ,OpenMessaging Connect Overview,12,OpenMessaging,Connect,Architecture,Connect,Model,Worker,Model,Connect,Runtime,OpenMessaging Connect Architec,13,OpenMessaging,Connect,Architecture,Connect,Model,Connector:,defines,how,developers,create,connector,plugins,which,import,and,export,data,from,another,data,systems.,Two,key,components:,connector,task,OpenMessaging Connect Architec,14,OpenMessaging,Connect,Architecture,Connect,Model,OpenMessaging Connect Architec,15,OpenMessaging,Connect,Architecture,Worker,Model,OpenMessaging Connect Architec,16,OpenMessaging,Connect,Architecture,Connect,Runtime,1,2,3,4,5,OpenMessaging Connect Architec,17,Config,Management,1,2,3,4,5,OpenMessaging,Connect,Architecture,Config Management12345OpenMess,18,Rebalance,and,Failover,1,2,3,4,5,OpenMessaging,Connect,Architecture,Rebalance and Failover12345Ope,19,REST,API,POST,/connectors/connector,name,GET,/connectors/connector,name/config,GET,/connectors/connector,name/status,POST,/connectors/connector,name/stop,OpenMessaging,Connect,Architecture,REST APIOpenMessaging Connect,20,OpenMessaging,connect,in,Practice,MySQL,connector,OpenMessaging connect in Pract,21,OpenMessaging,connect,in,Practice,MySQL,connector,OpenMessaging connect in Pract,22,OpenMessaging,connect,in,Practice,MySQL,connector,OpenMessaging connect in Pract,23,OpenMessaging,connect,in,Practice,MySQL,connector,OpenMessaging connect in Pract,24,OpenMessaging,Connect,Now,and,Future,RocketMQ,O,p,e,n,M,e,ss,a,g,i,n,g,Runtime,R,o,c,k,e,t,M,Q,Runtime,OpenMessaging,Connect,OpenMessaging Connect Now and,25,OpenMessaging,Connect,Now,and,Future,OpenMessaging Connect Now and,26,O,p,e,n,M,e,ss,a,g,ing,Connect,Connect,Ecos,y,s,t,em,OpenMessaging ConnectConnect,27,2,0,谢谢聆听！,20谢谢聆听！,28,