Final Report Summary - CODES (Computing On Demand for Electromagnetic Software and applications) The CODES project achieved many objectives including the configuration of a working geographical computational grid, CODES computer grid (CCG) with at least one node for each participant. This grid has been used as a laboratory to test the result of the development activities and for dissemination purposes. CODES grid framework (CGF) is a multiplatform middleware based on existing GC toolkits, but specifically designed and developed to address the problems related to the distribution of computation of heterogeneous EM software tools over a computational grid and the aspects related to the integration (command and control, data exchange, accounting, security) in a geographical distributed, multi-company computing environment. The main CGF features are: - uses Globus Toolkit 4 (GT4), all remote interfaces are web services (GT4 WSRF) support Linux and Windows platforms; - replaces GT4 components unavailable on Windows (WS-GRAM and GridFTP) with CoG gatekeeper and GASS server; - developes abstraction layers and higher-level services on top of GT4 including grid configuration database, workflow manager, scheduler; - grid configuration with immediate propagation of grid configuration changes; - secures and distributes remote logging; - plans grid-aware system configuration and monitoring GUIs (admin services); - full encapsulation of local execution with transparent move of input / output files from local execution context to a global shared file area; - allows interface to existing application frameworks (legacy interface); - simple adaptation layers for program activation (execution agent). The CGF has been based on the following main topics: - divide-et-impera approach - split each computation (procedure) in sub-blocks (operations) that can be executed separately, possibly on different machines - task dispatching procedures and operations exploit task-dispatching: each sub-block can separately solve a portion of the initial problem. Procedure and operations - CGF manages complex computations specified as procedures. - A procedure is a directed, acyclic graph of operations. - A single operation corresponds to a single sub-computation. - CGF procedures have an XML specification. - A procedure describes only the execution logic needed for a computation. - The business logic related to the computation is specified into dedicated data service providing input data conversion, input files creation and output data checks. The SMEs intended to upgrade the software that demonstrated to be the most suitable and promising for the application of GC techniques. The obtained results depended on how the GC can actually fit into the existing software architecture, leading to proof of concepts, pilots, prototypes or possibly finite products that will be tested on the CCG. It is quite obvious that an update of existing software code will be necessary in order to make it ready to use GC as well as newly developed EM methods and techniques for obtaining the best possible accuracy. Further, the performance prediction tools for enabling performance prediction shall be updated. A lot of knowledge in the frame of grid technology has been acquired by all the CODES participants: - several grid middleware have been evaluated in studied in order to find the most suitable to cover with the SMEs requirement; - Globus toolkit and the additional components for the Windows OS support were selected as the best available, as open source and reliable, as SMEs required; - SMEs were trained on grid technology to acquire the necessary skill to set-up a grid based on GT4 service. The enhancement of the EM software performances have been obtained in two different ways: - running parallel threads of executions on distributed platforms (parallelisation); - distributing autonomous tasks (distribution). The main CODES outcome can be summarised in the following points: - detailed analysis of HPC computing requirements for SMEs involved in complex design and engineering activities; - a framework to grid-enabled legacy software tools used in the world of engineering and design based on multiple heterogeneous software tools, the CGF; - a demonstrator of grid design environment working on a geographical grid involving all the project partners; - the task dispatching approach, able to integrate existing legacy software not designed to run on a grid without modifying it; - reusable for any engineering environment based on legacy HPC-demanding software tools; - fully use of open source, reliable, standard grid middleware in the realm of different emerging grid solutions. Related documents Final Report - CODES (Computing On Demand for Electromagnetic Software and applications)
