网格体系结构之ogsa第三章网格体系结构

单击此处编辑母版标题样式,单击此处编辑母版文本样式,第二级,第三级,第四级,第五级,*,网格计算Grid Computing 肖侬,开放网格服务体系结构,Open Grid Services Architecture,五层沙漏网格体系结构,五层沙漏,应用背景是科学研究领域EScience,以协议为中心的协议结构,一个基础平台允许访问，集成远程异构资源和服务,Application,构造层Fabric,连通层,资源层Resource,聚合层,Collective,应用层,Information,services,Data,mgmt,. . .,Resource,mgmt,Collaboration,Tools,Data Mgmt,Tools,Distributed,simulation,. . .,net,开放网格服务体系结构OGSA,OGSA（Open Grid Services Architecture),面向服务的体系结构SOA,以服务为中心的“服务结构”,一切都是服务,商业应用,分布企业计算，例如企业应用集成和B2B等,Web Services和Grid技术融合的产物,遵循Web Service标准，扩展它,Web Services: Basic Functionality,OGSA,OGSI: Interface to Grid Infrastructure,Compute, Data & Storage Resources,Distributed,Application & Integration Technology for Problem Domain X,Generic Virtual Service Access and Integration Layer,-,Structured Data,Integration,Structured Data Access,Structured Data,Relational,XML,Semi-structured,Transformation,Registry,Job Submission,Data Transport,Resource Usage,Banking,Brokering,Workflow,Authorisation,OGSA产生的动机,科学研究领域的成功商业计算,电子科研环境商业应用，如同WEB技术发展,资源集成是一个基本挑战,问题和解决方法类似,分布、可靠、可扩展和安全的分布系统挑战,向基础设施迈进支持多个应用领域,R,Discovery,R,Access,RM,RM,RM,RM,RM,OGSA产生的动机,以主机为中心企业计算以网络为中心企业计算,外部的网络、资源和服务,高度集成的内部IT基础设施分解成异构多个系统的集合，异构和小系统的集合按QoS的重新集成,现行方法是面向平台的纵向集成构件和服务,没有合适标准的IT基础架构和工具，分布计算的管理、灵活性、可扩展等的保证非常麻烦,Simulation,Pricing,Design,Design,Design Analytics,OGSA产生的动机,服务提供商,Web、存储、内容分发和应用等服务提供商,EUntility,按服务级别动态共享、分配和组织资源；,快速一致的响应时间、多级别的高可用，灵活实时的重配置,B2B计算,多企业之间的合作虚拟组织,分布异构系统的集成,Increased functionality,standardization,Custom,solutions,1990,1995,2000,2005,Open Grid,Services Arch,Real standards,Multiple implementations,Web services, etc.,Managed shared,virtual systems,Computer science research,Globus Toolkit,Defacto standard,Single implementation,Internet,standards,The Emergence ofOpen Grid Standards,2010,OGSA结构,Everything is a Grid service,计算资源、存储资源、网络、程序、数据库等,面向服务的体系结构,互操作问题分成两个子问题,服务接口的定义,协议的表示，解决服务的激活,讨论标准接口定义机制，局部/远程透明，自适应局部OS服务和单一服务语义等,虚拟化资源,将服务功能以标准的方式表达，任何实现都可以激活；,利用已有的资源和系统功能,A grid service is a web service,兼容、扩展web Service,服务的发现、描述，C/S代码的自动产生等web Service特性,有状态的服务,OGSA结构（续）,网格可视为一个网格功能服务的扩展集合,可以采用各种方式聚合这些服务，以满足VO需要,服务响应网格互操作协议的消息,网格功能服务以服务的形式实现各自的功能,对先前的层次网格体系结构的补充和完善,定义网格功能服务的基本公共接口和行为,定义服务规范交互、创建和基本公共管理等,更容易实现网格服务本身的功能,便于分布系统的集成,基础服务规范和架构建立,系统的差别不在是交互机制，而在于资源拥有者的管理、安全等策略的不同,OGSA结构（续）,核心服务的定义,服务级别、数据访问和集成、工作流、安全、策略、监测和诊断等.,GGF 正在努力的目标,支持基于上述定义服务的任何面向应用的服务实现,基于核心服务之上构建高级服务,OGSA服务模型,系统应当由少量的永久服务和大量的瞬时服务组成,所有的服务遵循网格服务接口和行为的定义,可靠的服务激活、生命周期管理、发现、授权、通知、升级、并发和可管理性；,管理服务实例的接口,网格服务,网格服务的语义,遵循所定义的一套标准接口和行为规范：命名、发现、动态服务创建，生命周期管理，通知和可管理性等,重点解决服务的管理、命名发现和使用问题,由WSDL定义的a Web Service,分布系统关键问题：命名、服务状态、生命周期和通知等；,网格服务（续）,网格服务语义没有定义服务本身的实现,没定义什么样编程模型、技术、开发和运行环境,多种Host Environment(宿主环境）,OS，J2EE, .NET, “C”, ,网格服务的交互方式与宿主环境无关,通过定义网格服务的语义,定义了所有运行环境都需具有的基本特征,保证网格服务的成功,不同环境不同的实现,网格服务（续）,基于Web Service的资源虚拟化,服务化各类资源，多个逻辑实例映射相同物理资源,位置透明、多协议绑定和服务的多种实现,组成高级服务，不关心各服务如何构成实现,网格服务（续）,网格服务语义接口和行为如下：,全局命名和引用（GSH和GSR）,创建（factory）,生命周期的管理,注册和发现,授权,通知,并发,可管理性,网格服务：瞬时服务实例,OGSA系统由持久服务和瞬时服务组成,“Web services” 强调永久 服务的发现和调用,整个企业持久状态的接口,无状态的服务,在网格环境中，必须也要支持瞬时服务实例, 动态地创建和取消,分布活动状态的接口,例如工作流, 视频会议, 远程数据分析等,网格服务：命名和绑定,每一个服务实例有一个全局唯一的、不变的名字 GSH（Grid Service Handle）,通常是一个全局唯一的 URL,没有携带网络协议和地址等信息,网格引用服务GSR（Grid Service Reference）描述了和实例交互所需的实例特殊信息,包括协议绑定信息，网络地址等，,客户程序使用它和实例交互,GSR在实例生命期内会发生变化，一个GSH可能对应多个GSR，但一个时间是一一对应,名字和实现的分离方便了服务的升级和演变,如何和grid Service通信的WSDL文档,The Mapper interface 接口将GSH转换为GSR,get on GSH also returns a GSR,网格服务：Factory,Factory 服务：,提供了创建新的网格服务实例的接口,不定义服务如何创建,Factory行为,创建所请求的网格服务实例,返回 网格服务实例 GSH和初始GSR,GSH：,网格服务句柄,Grid Service Handle,GSR：网格服务引用（Grid Service Reference）,瞬时,服务产生,Register,2. Client请求factory的,创建服务操作以创建服务实例,Factory,Service,Client,3. Factory创建服务实例,4. factory 返回服务实例的GSH,5. Client调用服务实例,服务,factory,注册，,client,查询这个服务类,GSH,网格服务：服务数据Service Data,一个网格服务实例维护一个服务数据元素的集合,基本的内部信息，接口的特殊信息和应 用的数据，使用XML模式定义信息属性,FindServiceData操作 (GridService 接口) 查询这些信息,Attributes combined into a single (logical) document within the service,网格服务：生存期管理,GS 实例由factory或手工创建;,GridService 接口支持,Destroy 操作用于显式撤消,软状态撤销,软状态生命周期管理机制,初始的生命周期在创建时和Factory协商,用户可以SetTerminationTime设置生命周期,GetTerminationTime 查询,通知 interface也允许生命周期的通知,软状态生命期管理可避免资源的无限期占用等,网格服务：服务注册和发现,网格注册服务实例维持网格服务的信息,注册管理接口允许网格服务的软状态注册,网格服务实例可以周期性地注册他们的 GSHs 到注册服务,GSH注册接口和GSH相关服务属性定义分开,服务发现接口允许客户查询服务实例的信息，可以用来发现服务实例集合,Query,operation, plus supporting operations,Extensible query language support,返回一个WS-Inspection文档包括GS集合的GSHs,也返回集合的策略,Also available through Discovery interface,网格服务：Notification,提供了统一的消息/事件服务,多种目的，允许对服务的存在和服务数据的变化进行，例如服务数据元素变化的通知或创建新的服务实例的通知,NotificationSource,消息通知发布方,接受用户的事件消息订阅Soft state subscription,可以使用第三方消息,NotificationSink,消息通知接受方,接受事件消息,异步传递通知消息,Notification，Example,Notification，Example,Notification，Example,Notification，Example,网格服务：Change 管理,服务的升级和变化,不同服务,版本管理,兼容性,网格服务：授权,在调用网格服务实例的时候协议绑定经过认证,OGSA服务实例在其所有的操作上使用授权策略,可能是Site、instance、服务等对象控制,OGSA定义远程访问控制策略的管理,OperationAuthorizationManagenment,Subjectequivalency,网格服务的组成,Service registry,Service requestor (e.g. user application),Service factory,Create Service,Grid Service Handle,Resource allocation,Service instances,Register Service,Service discovery,Interactions standardized using WSDL,Service data Keep-alives Notifications Service invocation,Authentication & authorization are applied to all requests,OGSA服务,OGSA定义服务的标准接口,基于OGSA构造网格运行环境,Factory,Registry,Service,Factory,H2R,Mapper,Service,Service,Service,.,.,(a) Simple Hosting,Environment,Factory,Registry,Service,Factory,H2R,Mapper,Service,Service,Service,.,.,F,R,F,M,S,S,S,F,R,F,M,S,S,S,(b) Virtual Hosting,Environment,E2E,Factory,E2E Reg,E2E H2R,Mapper,.,F,1,R,M,S,S,S,F,2,R,M,S,S,S,E2E S,E2E S,E2E S,(c) Collective Services,In each case, Registry handle is effectively the unique,name for the virtual organization.,应用例子1,Example:Data Mining for Bioinformatics,User,Application,BioDB n,Storage Service Provider,Mining,Factory,Community,Registry,Database,Service,BioDB 1,Database,Service,.,.,.,Compute Service Provider,“I want to create,a personal database,containing data on,metabolism”,.,.,.,Database,Factory,Example:Data Mining for Bioinformatics,User,Application,BioDB n,Storage Service Provider,Mining,Factory,Community,Registry,Database,Service,BioDB 1,Database,Service,.,.,.,Compute Service Provider,.,.,.,“Find me a data mining service, and somewhere to store data”,Database,Factory,Example:Data Mining for Bioinformatics,User,Application,BioDB n,Storage Service Provider,Mining,Factory,Community,Registry,Database,Service,BioDB 1,Database,Service,.,.,.,Compute Service Provider,.,.,.,GSHs for Mining,and Database,factories,Database,Factory,Example:Data Mining for Bioinformatics,User,Application,BioDB n,Storage Service Provider,Mining,Factory,Community,Registry,Database,Service,BioDB 1,Database,Service,.,.,.,Compute Service Provider,.,.,.,“Create a data mining service with initial lifetime 10”,“Create a,database with initial lifetime 1000”,Database,Factory,Example:Data Mining for Bioinformatics,User,Application,BioDB n,Storage Service Provider,Database,Factory,Mining,Factory,Community,Registry,Database,Service,BioDB 1,Database,Service,.,.,.,Compute Service Provider,.,.,.,Database,Miner,“Create a data mining service with initial lifetime 10”,“Create a,database with initial lifetime 1000”,Example:Data Mining for Bioinformatics,User,Application,BioDB n,Storage Service Provider,Database,Factory,Mining,Factory,Community,Registry,Database,Service,BioDB 1,Database,Service,.,.,.,Compute Service Provider,.,.,.,Database,Miner,Query,Query,Example:Data Mining for Bioinformatics,User,Application,BioDB n,Storage Service Provider,Database,Factory,Mining,Factory,Community,Registry,Database,Service,BioDB 1,Database,Service,.,.,.,Compute Service Provider,.,.,.,Database,Miner,Query,Query,Keepalive,Keepalive,Example:Data Mining for Bioinformatics,User,Application,BioDB n,Storage Service Provider,Database,Factory,Mining,Factory,Community,Registry,Database,Service,BioDB 1,Database,Service,.,.,.,Compute Service Provider,.,.,.,Database,Miner,Keepalive,Keepalive,Results,Results,Example:Data Mining for Bioinformatics,User,Application,BioDB n,Storage Service Provider,Database,Factory,Mining,Factory,Community,Registry,Database,Service,BioDB 1,Database,Service,.,.,.,Compute Service Provider,.,.,.,Database,Miner,Keepalive,Example:Data Mining for Bioinformatics,User,Application,BioDB n,Storage Service Provider,Database,Factory,Mining,Factory,Community,Registry,Database,Service,BioDB 1,Database,Service,.,.,.,Compute Service Provider,.,.,.,Database,Keepalive,OGSA Platform components,OGSI（ Open Grid Services Infrastructure,）,OGSA平台服务,OGSA模型,OGSI,OGSI（ Open Grid Services Infrastructure,）,定义可管理网格服务的核心接口和行为,定义了服务的creating, managing, and exchanging information,定义了客户如何和网格服务交互,Supported by strong open source technology & major commercial vendors,.,OGSI Specification,定义基本的Grid Service接口和行为,Naming and bindings (basis for virtualization),Lifecycle,Service instances created by,factories,Destroyed,explicitly,or via,soft state,Information model (basis for monitoring & discovery),Service data,(attributes),Operations for,querying,and,setting,this info,Asynchronous,notification,of changes to service date,OGSI Specification（续）,定义 了GWSDL,OGSI requires interface extension/composition,WSDL conventions and extensions,For describing and naming services,work within W3C WSDL working group to define standard interface extension in WSDL 1.2 that meets OGSI requirements,So defined gwsdl:portType that extends WSDL 1.1 portType with:,WSDL 1.2 portType extension,WSDL 1.2 open content model,Service Groups (basis for registries & collective svcs),Group membership rules & membership management,Base Fault type,GWSDL Example,Open Grid Services Infrastructure,Implementation,Service,data,element,Other standard interfaces:,factory,notification,collections,Hosting environment/runtime,(“C”, J2EE, .NET, ),Service,data,element,Service,data,element,GridService,(required),Data,access,Lifetime management,Explicit destruction,Soft-state lifetime,Introspection:,What port types?,What policy?,What state?,Client,Grid Service,Handle,Grid Service,Reference,handle,resolution,Example:Reliable File Transfer Service,Performance,Policy,Faults,service,data,elements,Pending,File,Transfer,Internal,State,Grid,Service,Notfn,Source,Policy,interfaces,Query &/or,subscribe,to service data,Fault,Monitor,Perf.,Monitor,Client,Client,Client,Request and manage file transfer operations,Data transfer operations,OGSI Implementations,Globus Toolkit version 3.0 (Java, C client),U Virginia OGSI.NET (.NET),LBNL pyGlobus (Python),U Edinburgh (.NET),U Manchester (PERL),Fujitsu Unicore (Java),OGSA平台服务,建立在OGSI之上的高层服务核心服务,service discovery,.,Agreement negotiation,Common management model,Data access and integration,messaging, and monitoring,Security and policy,Etc.,这些标准服务在 GGF, OASIS等组织中定义,OGSA模型,OGSA模型,定义公共资源和服务类型的公共模型，以支持网格服务接口的定义,Common Resource Model（CRM）,CIM，SNMPdeng,Transactions & Contexts,WS-Coordination & WS-Transaction,IBM/MS (not in standards org),WS-CAF (Coordinated Application Framework),Sun/Oracle/Arjuna/Fujitsu (not in standards org),WS-CTX (Context),WS-CF (Coordination Framework),WS-TXM (Transaction Management),Both take a “contextualization” approach,Context (id) threaded through SOAP header,OGSI for context creation, naming & lifecycle?,Security Standards,Many core security standards are from IETF,X.509, Kerberos, etc.,X.509 Proxy Certificates (RFC soon hopefully),Used by Globus Toolkit GSI,OASIS appears to be leader in Web services security standards,WS-Security: SOAP message security,SAML: signed assertions using XML,XACML: access control lists using XML,GGF OGSA Security WG evaluating security specifications for applicability to OGSA,IBM/MicrosoftWS Security Architecture,Large set of specifications for doing Web services security, most of which should be appropriate for OGSA,Announced April 2002,Initial spec in July 2002 (WS-Security),Submitted to OASIS,New crops of specs arrive periodically,WS-Policy*, WS-Trust, WS-Federation, etc.,But Not yet in any standards organization,WS SecurityCurrent/Proposed WSS-specs,proposed,SOAP Foundation,WS-Security,WS-Policy,WS-Trust,WS-Privacy,WS-Secure,Conversation,WS-Federation,WS-Authorization,In progress,promised,OASIS SAML & XACML,SAML: Security Assertion Markup Language,Good for asserting properties such as group membership, etc,XACML: eXtensible Access Control Markup Language,For defining access control policies,These are gaining considerable momentum, but WS-Policy* leaves them in question,Project Liberty Alliance,V1.x specifications for identity federation,Allows cross-organization identification,Privacy preserving model,Based on SAML,WS-Agreement,Recall key criteria of a Grid:,Coordinates resources that are not subject to centralized control,using standard, open, general-purpose protocols and interfaces ,to deliver non-trivial qualities of service,.,Implies need to express and negotiate agreements that govern the delivery of services to clients,Agreement = what will be done, QoS, billing, compliance monitoring,WS-Agreement Contents,Standard agreement,language,A composition of a set of terms that govern a services behavior with respect to clients,Agreement language uses WS-Policy (currently),Standard attributes for terms that express current state of negotiation,Other groups define specific terms,Standard agreement negotiation,protocol,Establish, monitor, re-negotiate agreement,Expressed using OGSI GWSDL interfaces,Each agreement represented by a service,WS-Agreement Applicability,All interesting Web/Grid services interactions will be governed by agreements!,WS-Agreement (language and interfaces) should be used by specifications that define domain-specific services,Data services,Job submission,Specialized services,Etc.,Platforms Community Scheduling Framework & WS-Agreement,Reservation Agreement Exchange,Internet,Queuing,Service,Job,Service,Reservation,Service,RM Adapter,LSF,Site A,Site B,RM Adapter,PBS,Site C,SGE,RM,Adaptor,WSDM / WSMF / CMM,OASIS Web Services Distributed Management (WSDM) technical committee,Management using/of Web Services,HP submitted its Web Services Management Framework (WSMF) to WSDM in July 2003,WS-Events: event schema, subscription, message queues,WSMF-Foundation: management,using,Web services,WSM: management,of,Web services,GGF Common Management Model (CMM) WG,IBM submission overlaps WSMF-Foundation,Working to bring WSDM & CMM together, based on OGSI foundation,Data as Service:OGSA Data Access & Integration,Service-oriented treatment of data appears to have significant advantages,Leverage OGSI introspection, lifetime, etc.,Compatibility with Web services,Standard service interfaces being defined,Service data: e.g., schema,Derive new data services from old (views),Externalize to e.g. file/database format,Perform queries or other operations,Data Services,GGF Data Access and Integration Svcs (DAIS),OGSI-compliant interfaces to access relational and XML databases,Needs to be generalized to encompass other data sources (see next slide),Generalized DAIS becomes the foundation for:,Replication: Data located in multiple locations,Federation: Composition of multiple sources,Provenance: How was data generated?,“OGSA Data Services”(Foster, Tuecke, Unger, eds.),Describes conceptual model for representing all manner of data sources as Web services,Database, filesystems, devices, programs, ,Integrates WS-Agreement,Data service is an OGSI-compliant Web service that implements one or more of base data interfaces:,DataDescription, DataAccess, DataFactory, DataManagement,These would be extended and combined for specific domains (including DAIS),1a. Request to Registry for sources of data about “x”,1b. Registry responds with Factory handle,2a. Request to Factory for access to database,2c. Factory returns handle of GDS to client,3a. Client queries GDS with XPath, SQL, etc,3b. GDS interacts with database,3c. Results of query returned to client as XML,SOAP/HTTP,service creation,API interactions,Registry,Factory,2b. Factory creates GridDataService to manage access,Grid Data Service,Client,XML / Relational database,Data Access & Integration Services,Slide Courtesy Malcolm Atkinson, UK eScience Center,统一的网格系统平台的基础支撑,OGSI,OGSA Platform services,: registry,authorization, monitoring,Data mgmt, CMM, etc., etc.,UNIX,Host.,Env,More specialized &,domain,-,specific,services,OGSI,OGSI,Windows,J2EE,Host.,Env,Host.,Env,协议,协议,协议,Web Services: Basic Functionality,OGSA,Open Grid Services Architecture,OGSI: Interface to Grid Infrastructure,Applications in Problem Domain X,Compute, Data & Storage Resources,Distributed,Application & Integration Technology for Problem Domain X,Users in Problem Domain X,Virtual Integration Architecture,Generic Virtual Service Access and Integration Layer,-,Structured Data,Integration,Structured Data Access,Structured Data,Relational,XML,Semi-structured,Transformation,Registry,Job Submission,Data Transport,Resource Usage,Banking,Brokering,Workflow,Authorisation,Servi,ce,Grid,Servicio,Web,Tecnologa,Grid,O,pen,G,rid,S,ervice,I,nfrastructure,O,pen,G,rid,S,ervices,A,rchitecture,OGSA,OGSI,define,specification,implementa,tion,naming, state, lifetime management, etc,discovery, binding etc,integra,Globus,GT3 结构,实现,Grid Service,接口和行为,利用,GT3 core,实现已有的GT功能并扩展新功能：例如：资源管理、数据传输和信息服务,以,GT3 core and GT3 base services,解决数据管理、工作负载管理和诊断等,