并行计算概述课件

单击此处编辑母版标题样式,单击此处编辑母版文本样式,第二级,第三级,第四级,第五级,2019/10/27,并行计算-硬件基础及性能评测,#,并行计算,Parallel Computing,基本概念,2024/9/16,2,如何满足不断增长的计算力需求？,用速度更快的硬件，也就是减少每一条指令所需时间,优化算法（或者优化编译）,用多个处理机,(,器,),同时解决一个问题,并行计算,2024/9/16,3,串行计算与并行计算,2024/9/16,4,并行的层次,程序级并行,子程序级并行,语句级并行,操作级并行,微操作级并行,并行粒度,粗,细,2024/9/16,5,FLOPS,Floating point number Operations Per Second -,每个时钟周期执行浮点运算的次数,理论峰值,CPU,主频每时钟周期执行浮点运算数,CPU,数目,部分处理器每时钟周期执行浮点运算数：,2024/9/16,6,www.top500.org,2024/9/16,7,Top500,2007,年,11,月,高居榜首的依然是来自,IBM,的“蓝色基因,/L”,。自从,2004,年,11,月以来，该系统已经,连续三年遥遥领先,，而且计算能力不断提升，,Linpack,基准测试性能,478.2 TFlop/s,(,每秒,478.2,万亿次运算,),，而半年前还是,280.6 TFlop/s,拿下亚军位置的还是,IBM,，不过换成了一台落成不久的,“蓝色基因,/P”,。位于德国尤里希研究中心的这套新系统运算能力,167.3 TFlop/s,，不过按照,IBM,的设计规划，蓝色基因,/P,的性能将有望突破,1 TFlop/s,大关，即每秒一千万亿次运算。,2024/9/16,8,Top500,2007,年,11,月,第三名也是个新面孔，同时也是新,墨西哥,计算应用中心,(NMCAC),的第一套超级计算机，由,SGI,基于,Altix ICE 8200,打造，计算能力,126.9 TFlop/s,。,同时,印度史上首次杀入了,TOP10,行列,，印度计算研究实验室的,HP Cluster Platform 3000 BL460c,以,117.9 TFlop/s,的性能拿到了第四位,2024/9/16,9,供应商系统数量,2024/9/16,10,供应商计算能力,2024/9/16,11,国家分布系统数量,2024/9/16,12,国家分布计算能力,2024/9/16,13,体系结构系统数量,2024/9/16,14,体系结构计算能力,2024/9/16,15,应用领域系统数量,2024/9/16,16,应用领域计算能力,2024/9/16,17,操作系统系统数量,2024/9/16,18,操作系统计算能力,2024/9/16,19,处理器家族系统数量,2024/9/16,20,处理器家族计算能力,2024/9/16,21,系统数量,2024/9/16,22,计算能力,2024/9/16,23,2007,年中国高性能计算机性能,TOP100,2024/9/16,24,并行化方法,域分解（,Domain decomposition,）,任务分解（,Task decomposition,）,流水线（,Pipelining,）,2024/9/16,25,域分解,First, decide how data elements should be divided among processors,Second, decide which tasks each processor should be doing,Example: Vector addition,2024/9/16,26,域分解,Find the largest element of an array,2024/9/16,27,域分解,Find the largest element of an array,CPU 0,CPU 1,CPU 2,CPU 3,2024/9/16,28,域分解,Find the largest element of an array,CPU 0,CPU 1,CPU 2,CPU 3,2024/9/16,29,域分解,Find the largest element of an array,CPU 0,CPU 1,CPU 2,CPU 3,2024/9/16,30,域分解,Find the largest element of an array,CPU 0,CPU 1,CPU 2,CPU 3,2024/9/16,31,域分解,Find the largest element of an array,CPU 0,CPU 1,CPU 2,CPU 3,2024/9/16,32,域分解,Find the largest element of an array,CPU 0,CPU 1,CPU 2,CPU,3,2024/9/16,33,域分解,Find the largest element of an array,CPU 0,CPU 1,CPU 2,CPU 3,2024/9/16,34,域分解,Find the largest element of an array,CPU 0,CPU 1,CPU 2,CPU 3,2024/9/16,35,域分解,Find the largest element of an array,CPU 0,CPU 1,CPU 2,CPU 3,2024/9/16,36,域分解,Find the largest element of an array,CPU 0,CPU 1,CPU 2,CPU 3,2024/9/16,37,域分解,Find the largest element of an array,CPU 0,CPU 1,CPU,2,CPU,3,2024/9/16,38,任务（功能）分解,First, divide tasks among processors,Second, decide which data elements are going to be accessed (read and/or written) by which processors,Example: Event-handler for GUI,2024/9/16,39,任务分解,f(),s(),r(),q(),h(),g(),2024/9/16,40,任务分解,f(),s(),r(),q(),h(),g(),CPU 0,CPU 2,CPU,1,2024/9/16,41,任务分解,f(),s(),r(),q(),h(),g(),CPU 0,CPU 2,CPU 1,2024/9/16,42,任务分解,f(),s(),r(),q(),h(),g(),CPU 0,CPU 2,CPU 1,2024/9/16,43,任务分解,f(),s(),r(),q(),h(),g(),CPU 0,CPU 2,CPU 1,2024/9/16,44,任务分解,f(),s(),r(),q(),h(),g(),CPU 0,CPU 2,CPU 1,2024/9/16,45,流水线,Special kind of task decomposition,“,Assembly line,”,parallelism,Example: 3D rendering in computer graphics,Rasterize,Clip,Project,Model,Input,Output,2024/9/16,46,Processing One Data Set (Step 1),Rasterize,Clip,Project,Model,2024/9/16,47,Processing One Data Set (Step 2),Rasterize,Clip,Project,Model,2024/9/16,48,Processing One Data Set (Step 3),Rasterize,Clip,Project,Model,2024/9/16,49,Processing One Data Set (Step 4),Rasterize,Clip,Project,Model,The pipeline processes 1 data set in 4 steps,2024/9/16,50,Processing Two Data Sets (Step 1),Rasterize,Clip,Project,Model,2024/9/16,51,Processing Two Data Sets (Time 2),Rasterize,Clip,Project,Model,2024/9/16,52,Processing Two Data Sets (Step 3),Rasterize,Clip,Project,Model,2024/9/16,53,Processing Two Data Sets (Step 4),Rasterize,Clip,Project,Model,2024/9/16,54,Processing Two Data Sets (Step 5),Rasterize,Clip,Project,Model,The pipeline processes 2 data sets in 5 steps,2024/9/16,55,Pipelining Five Data Sets (Step 1),Data set 0,Data set 1,Data set 2,Data set 3,Data set 4,CPU 0,CPU 1,CPU 2,CPU 3,2024/9/16,56,Pipelining Five Data Sets (Step 2),Data set 0,Data set 1,Data set 2,Data set 3,Data set 4,CPU 0,CPU 1,CPU 2,CPU 3,2024/9/16,57,Pipelining Five Data Sets (Step 3),Data set 0,Data set 1,Data set 2,Data set 3,Data set 4,CPU 0,CPU 1,CPU 2,CPU 3,2024/9/16,58,Pipelining Five Data Sets (Step 4),Data set 0,Data set 1,Data set 2,Data set 3,Data set 4,CPU 0,CPU 1,CPU 2,CPU 3,2024/9/16,59,Pipelining Five Data Sets (Step 5),Data set 0,Data set 1,Data set 2,Data set 3,Data set 4,CPU 0,CPU 1,CPU 2,CPU 3,2024/9/16,60,Pipelining Five Data Sets (Step 6),Data set 0,Data set 1,Data set 2,Data set 3,Data set 4,CPU 0,CPU 1,CPU 2,CPU 3,2024/9/16,61,Pipelining Five Data Sets (Step 7),Data set 0,Data set 1,Data set 2,Data set 3,Data set 4,CPU 0,CPU 1,CPU 2,CPU 3,2024/9/16,62,Pipelining Five Data Sets (Step 8),Data set 0,Data set 1,Data set 2,Data set 3,Data set 4,CPU 0,CPU 1,CPU 2,CPU 3,