Serengei虚拟化你的大数据应用VMWare

Click to edit Master title style,Click to edit Master text styles,Second level,Third level,Fourth level,Fifth level,4/25/2010,#,2009,VMware,Inc.,All,rights,reserved,Serengeti,-,虚,拟,化你的大数据,应,用,蔺,永,华,Vmware,Inc.,Agenda,Todays,big,data,system,Why,virtualize,hadoop?,Serengeti,introduction,Common,questions,about,virtualization,Serengeti,solution,Deep,insight,into,Serengeti,Summary,Q&A,Todays,Big,Data,System:,ETL,Unstructured,Data,(HDFS),Real,Time,Structured,Database,Big,SQL,Data,Parallel,Batch,Processing,Real,Time,Streams,Real-Time,Processing,(s4,storm),Analytics,Agenda,Todays,big,data,system,Why,virtualize,hadoop?,Serengeti,introduction,Common,questions,about,virtualization,Serengeti,solution,Deep,insight,into,Serengeti,Summary,Q&A,Challenges,To,Use,Hadoop,in,physical,infrastructure,Deployment,Difficult,to,deploy,cost,several,people,for,several,days,even,months,Difficult,to,tune,cluster,performance,Low,Efficiency,Hadoop,clusters,are,typically,not,100%,utilized,across,all,hardware,resources.,Difficult,to,share,resources,safely,between,different,workload,Single,Point,of,Failure,Single,point,of,failure,for,Name,Node,and,Job,tracker,No,HA,for,Hive,HCatalog,etc.,Why,Virtualize,Hadoop?,-,Get,your,Hadoop,cluster,in,minutes,1/1000humanefforts,LeastHadoopoperation,knowledge,Fullyautomated,process,10,minutesto,get,a,Hadoop/HBaseclusterfrom,scratch,Server,preparation,OS,installation,Automateby,Serengeti,on,vSpherewith,best,practice,Network,Configuration,Hadoop,Installation,and,Configuration,Manual,process,costdays,Why,Virtualize,Hadoop?,-,Consolidate,sprawling,clusters,Clustersshare,serverswith,strongisolation,Single,Hardware,Infrastructure,Unified,operations,Optimize,Shared,Resources,=,higher,utilization,Elastic,resources,=,faster,on-demand,access,Hadoop,Dev,Hadoop,Prod,HBase,ClusterSprawling,Single,purpose,clusters,for,various,business,applications,lead,to,cluster,sprawl.,Cluster,Consolidation,Simplify,Finance,Hadoop,Virtualization,Platform,Hadoop,Dev,Hadoop,Prod,HBase,.,Portal,Hadoop,Portal,Hadoop,30%CAPEXDown,50%+,resourcesaresitting,idlewhilehighpriorityjob,is,burningup,its,cluster.,Utilizeall,resourcesfrom,pool,on,demand.,Dynamic,elastic,scalingonshared,resourcepool,Why,Virtualize,Hadoop?,Utilize,all,your,resources,to,solve,the,priority,problem,3X,fasterto,getanalyticresults,vSphere,High,Availability,(HA),-,protection,against,unplanned,downtime,Overview,Protection,against,host,and,VM,failures,Automatic,failure,detection,(host,guest,OS),Automatic,virtual,machine,restart,in,minutes,on,any,available,host,in,cluster,OS,and,application-independent,does,not,require,complex,configuration,changes,(Coordination),Zookeepr,Management,Server,High,Availability,for,the,Hadoop,Stack,(Hadoop,Distributed,File,System),HBase,(Key-Valuestore),HDFS,MapReduce,(Job,Scheduling/Execution,System),Pig,(Data,Flow),Hive,BI,Reporting,ETLTools,RDBMS,Jobtracker,Namenode,(SQL),Hive,MetaDB,HCatalog,Hcatalog,MDB,Server,XX,HAHA,App,OS,AppApp,OSOS,App,OS,App,OS,App,OS,App,OS,VMwareESX,X,VMwareESX,Zerodowntime,zerodataloss,failoverforallvirtualmachinesin,caseofhardwarefailures,IntegratedwithVMwareHA/DRS,Nocomplexclusteringor,specializedhardwarerequired,Singlecommonmechanismforall,applicationsandoperating,FT,vSphereFaultToleranceprovidescontinuousprotection,Overview,SingleidenticalVMsrunningin,locksteponseparatehosts,systems,ZerodowntimeforNameNode,JobTrackerandothercomponentsinHadoopclusters,Agenda,Todaysbigdatasystem,Whyvirtualizehadoop?,Serengetiintroduction,Commonquestionsaboutvirtualization,Serengetisolution,DeepinsightintoSerengeti,Summary,Q&A,Easyandrapiddeploymentandmanagement,OpensourceprojectlaunchedinJune2012,0.8isreleasedatApr.,andwillrelease0.9atJun.,ToolkitthatleveragevirtualizationtosimplifyHadoopdeployment,andoperations,Deployaclusterin10Minutesfullyautomated,CustomizeHadoopandHBasecluster,Automatedclusteroperation,Comewitheco-systemcomponents,SupportallpopularHadoopDistributions,Serengeti,Demo:10minutestoaHadoopclusterwithSerengeti,Agenda,Today,sbigdatasystem,Whyvirtualizehadoop?,Serengetiintroduction,Commonquestionsaboutvirtualization,Serengetisolution,DeepinsightintoSerengeti,Summary,Q&A,Commonquestionsaboutvirtualization,LocalDisk,Canlocaldiskbeusedinvirtualizationenvironment?,FlexibilityandScalability,Howtoflexiblescheduleresourcesbetweenclustersanddifferent,applicationsasmentionedabove?,Datastability,Invirtualenvironment,howcanwedistributedataacrosshostandrack?,Datalocality,Hadoopwillschedulecomputetasksnearbythedata,toreducenetwork,IOfordataR/W.Canvirtualenvironmentgetthesameresult?,Performance,Howabouttheperformanceinvirtualenvironment?,Agenda,Today,sbigdatasystem,Whyvirtualizehadoop?,Serengetiintroduction,Commonquestionsaboutvirtualization,Serengetisolution,DeepinsightintoSerengeti,Summary,Q&A,CanIuselocaldiskeasily?,OtherVM,OtherVM,OtherVM,OtherVM,OtherVM,OtherVM,OtherVM,OtherVM,Hadoop,Hadoop,Hadoop,Hadoop,Hadoop,Hadoop,Hadoop,Hadoop,Hadoop,Hadoop,SerengetiExtendVirtualStorageArchitecturetoIncludeLocalDisk,SharedStorage:SANorNAS