资源描述
Click to edit Master text styles,Second level,Third level,Fourth level,Fifth level,Silberschatz,Korth and Sudarshan,20.,#,Click to edit Master title style,Database System Concepts-6,th,Edition,Click to edit Master text styles,Second level,Third level,Fourth level,Fifth level,Click to edit Master title style,Chapter 20:Data Analysis,Chapter 20:Data Analysis,DecisionSupport Systems,Data Warehousing,Data Mining,Classification,AssociationRules,Clustering,DecisionSupport Systems,Decision-support systemsareusedtomake business decisions,oftenbasedondatacollectedbyon,-,-linetransaction-processing systems,.,.,Examplesofbusinessdecisions,:,:,What items to stock?,What insurancepremium to change?,Towhom to sendadvertisements,?,?,Examplesofdata usedfor makingdecisions,Retailsalestransactiondetails,Customerprofiles,(,(income,age,gender,etc.),Decision-Support Systems,:,:Overview,Data analysistasksaresimplifiedbyspecializedtoolsandSQL extensions,Example tasks,Foreachproduct category andeach region,whatwere thetotalsalesinthelastquarter andhowdotheycompare withthe samequarterlast year,Asabove,for eachproductcategoryandeachcustomercategory,Statisticalanalysispackages,(,(e.g.,:,:S+,),)canbeinterfaced withdatabases,Statisticalanalysisisa large field,butnot coveredhere,Data miningseekstodiscoverknowledgeautomaticallyintheformofstatisticalrulesandpatternsfromlargedatabases,.,.,Adata warehousearchivesinformationgatheredfrom multiple sources,andstoresitunderaunified schema,atasinglesite.,Importantfor large businessesthatgeneratedata frommultipledivisions,possiblyatmultiplesites,Data mayalso be purchasedexternally,Data Warehousing,Data sourcesoftenstoreonlycurrent data,nothistorical data,Corporatedecisionmaking requires aunifiedview of allorganizationaldata,includinghistoricaldata,Adata warehouseisa repository,(,(archive,),)of information gathered frommultiplesources,stored under aunifiedschema,at asingle site,Greatly simplifiesquerying,permitsstudyofhistoricaltrends,Shiftsdecisionsupportqueryload awayfromtransactionprocessing systems,Data Warehousing,DesignIssues,When andhowtogather data,Sourcedriven architecture:datasourcestransmitnewinformationtowarehouse,eithercontinuously or periodically,(,(e.g,.,.,atnight),Destinationdrivenarchitecture:warehouseperiodicallyrequestsnew information fromdatasources,Keeping warehouseexactly synchronizedwith datasources,(,(e.g,.,.,usingtwo,-,-phasecommit)istooexpensive,Usually OK to haveslightlyout-of-datedataatwarehouse,Data/updatesare periodicallydownloaded formonline transaction processing,(,(OLTP,),)systems,.,.,What schematouse,Schemaintegration,More WarehouseDesignIssues,Data cleansing,E.g,.,.,correct mistakes in addresses,(,(misspellings,zipcodeerrors,),),Mergeaddress lists fromdifferent sourcesandpurgeduplicates,Howtopropagate updates,Warehouseschema maybea(materialized,),)viewofschema fromdatasources,What datatosummarize,Rawdatamaybetoo large to store on-line,Aggregatevalues(totals,/,/subtotals)oftensuffice,Queries on rawdata canoftenbetransformedbyqueryoptimizertouse aggregatevalues,WarehouseSchemas,Dimensionvalues areusually encodedusingsmallintegersand mappedtofull valuesviadimension tables,Resultantschema is calledastar schema,More complicated schemastructures,Snowflakeschema:multiple levelsofdimensiontables,Constellation:multiple facttables,Data WarehouseSchema,Data Mining,Data miningistheprocessofsemi-automaticallyanalyzing large databasestofind usefulpatterns,Predictionbasedonpast history,Predict if acredit cardapplicant poses agoodcreditrisk,basedonsomeattributes(income,jobtype,age,.,.,.)andpasthistory,Predict if apatternofphonecalling cardusageislikely to be fraudulent,Some examples of predictionmechanisms:,Classification,Givena newitem whose class is unknown,predicttowhichclassitbelongs,Regressionformulae,Givena setofmappingsforanunknownfunction,predictthefunctionresult fora newparametervalue,Data Mining,(,(Cont.,),),DescriptivePatterns,Associations,Find books thatare often boughtby,“,“similar”customers.Ifanewsuchcustomerbuys onesuch book,suggestthe otherstoo,.,.,Associationsmay be usedasafirststep in detectingcausation,E.g,.,.,associationbetween exposure to chemical Xand cancer,Clusters,E.g,.,.,typhoid cases wereclustered in an areasurroundingacontaminatedwell,Detectionofclustersremainsimportantindetecting epidemics,ClassificationRules,Classificationruleshelp assignnewobjectstoclasses.,E.g,.,.,givena newautomobile insuranceapplicant,shouldheorshebeclassifiedaslowrisk,medium riskorhighrisk?,Classificationrulesforaboveexamplecoulduseavariety of data,suchaseducationallevel,salary,age,etc,.,.,personP,P,.,.degree=mastersandP.income,75,000,P.credit,=,=excellent,personP,P,.,.degree=bachelorsand(P.income25,000and P,.,.income75,000,),)P.credit,=,=good,Rulesarenot necessarily exact:theremaybesomemisclassifications,Classificationrulescanbeshowncompactly as adecisiontree.,DecisionTree,ConstructionofDecisionTrees,Trainingset:adatasampleinwhichthe classification is alreadyknown,.,.,Greedytopd
展开阅读全文