Middleware technologies for ADAPTive and composable distributed components

Within this project we have developed an efficient, transparent replication algorithm for stateful J2EE application servers. It guarantees that each client request submitted to the server is executed exactly-once despite server failures. Furthermore, the state maintained by the application server and the database state are always consistent. The algorithm is based on a primary/backup mechanism where the state changes on the application server a multicast to the backups before a response is returned to the client. All request execution is performed within a transaction, and a coordination protocol guarantees that the database and all application server replicas have consistent state. If the primary fails before returning a response, the client side communication stub will automatically resend the request to the new primary guaranteeing that the client will receive a response. We used the ADAPT framework for J2EE replication to integrate this algorithm into the open-source J2EE application server JBoss. Extensive performance evaluations (including some with the ECPerf benchmark) have shown that our protocol has very little overhead. At the same time it provides much stronger correctness properties than most of the replication mechanisms of existing application servers.

In application server systems, such as J2EE, replication is an essential strategy for reliability and efficiency. Many J2EE implementations, both commercial and open-source, provide some replication support. However, the range of possible strategies is wide, and the choice of the best one, depending on the expected application profile, remains an open research question. To support research in this area, we introduce a framework for prototyping J2EE replication algorithms. In effect, it divides replication code into two layers: the framework itself, which is common to all replication algorithms, and a specific replication algorithm, which is ''plugged in'' to the framework. The framework simplifies development in two ways. First, it keeps much of the complexity of modifying a J2EE implementation within the framework layer, which is implemented only once. Second, through the API, the replication algorithm sees a highly abstracted view of the components in the server. This frees the designer to concentrate on the important issues that are specific to a replication algorithm, such as communication. Our key contribution is an API, which clearly separates the concerns of the server and the replication algorithm. The server acts like an ordinary, non-replicated J2EE server, except that at key points in processing, it delegates control to the replication algorithm. The replication algorithm sees a highly abstracted view of the components in the server. Cluster management and inter-server communication are the responsibility of the replication algorithm, not the server. We have implemented this framework, by modifying a version of the open-source J2EE server JBoss. Compared to an unmodified server, the framework adds a performance cost of about 22%. Thus, it is quite practical for the initial development and evaluation of replication algorithms. Several replication algorithms have already been implemented in this environment.

Postgres-R is a prototype replication tool integrated into the open-source database system PostgreSQL. The replication tool allows several PostgreSQL database replicas to run on different machines. A transaction can be submitted to any replica where it is executed locally. Before transaction commit, the replication tool transparently multicasts all data changed by the transaction to the other replicas. The replication tool will detect any two concurrent conflicting transactions and abort one of them. It provides the same snapshot isolation level as the original. PostgreSQL system. The system provides excellent scalability since the overhead to propagate and apply data changes is small. Different to most other work in this direction, replication is integrated into the kernel of the database system. This avoids the indirection of a middleware layer and allows the replication tool to access the internal components of the database system such as concurrency control, query execution etc. The work is already well known among PostgreSQL developers. A group of developers employed or sponsored by several companies have started in January 2005 to reimplement the functionalities of Postgres-R within an advanced prototype that can then be tested by a wide audience. Therefore, we have not yet made our latest version of Postgres-R publicly available. But interested parties can contact us and receive the source code.

Middle-R is a middleware for database replication. There are two versions one in C and another in Java. Both versions count with a JDBC driver for interoperability. Currently Middle-R has two back-ends: one for PostgreSQL and another for MySQL. The most general version of Middle-R can work on top of any database providing snapshot isolation. Middle-R enables to replicate a database in a scalable fashion even with workloads with a significant ratio of updates. It implements eager (synchronous) replication and therefore provides full consistency. The key innovation in Middle-R is its scalability and generosity. It exploits a novel concept, 1-copy-snapshot isolation to boost the scalability of database replication. Additionally, it exploits two services from the database (to get and set the updated tuples from a transaction) to provide scalability with update workloads. This is a distinguishing feature from existing products. Middle-R is currently a pre-industrial product and can be converted in a reasonable time in an industrial product for its commercialisation. We are looking for an industrial partner interested in its commercialisation. Middle-R is also available for free use (research, commercialisation) under a BSD license.

The JOpera workflow engine applies workflow technologies to Web service composition. The engine executes workflows by running each of their tasks in the order specified by the workflow specification produced by the visual composition environment. Task execution involves the invocation of external services through the synchronous or asynchronous exchange of messages specified according to the SOAP standard transported over the HTTP protocol. The engine is able to distinguish between ordinary messages and fault messages and thus map these to exceptions in the execution of the workflow itself. The engine can run several workflows concurrently and also manage several different workflow types. The engine can be deployed to run a large number of workflows on a distributed (cluster) environment, in order to provide enhanced scalability and availability. As it has proven to be quite difficult to configure such distributed engine in advance to provide an acceptable level of performance facing a variable and unpredictable workload, the engine includes advanced autonomic capabilities that allow it to autonomously determine its configuration based on the current workload.

The purpose of the ADAPT demonstrator is to show the inter-working of basic and composite service support. The Demonstrator is based around the WS-I Sample Application Specification - a Supply Chain Management System, based on Web services. The application has been constructed as a set of J2EE Entity Beans, which operate on the persistent state held in a Postgres database. These entity beans are operated upon by Web Services, which are invoked by the composite service engine (JOpera). The Demonstrator can be used to show Basic Services being replicated and composite service being executed as processes.

JBora is a middleware layer for reliable group communication. It consists of a (thin) Java layer on top of Spread (http://www.spread.org). It provides a flavour of group communication that greatly simplifies the job of implementing replicated services on a local scale (i.e., within a LAN). Essentially, it ensures that when a failure occurs, failed replicas receive a prefix of the sequence of multicasts received by surviving replicas. Guarantees of this form are essential for implementing replicated services with strong consistency guarantees. JBora also includes a mechanism for disseminating quickly, easily and efficiently load indexes amongst replicas. This mechanism may be used for implementing a variety of load balancing policies.

The JOpera visual service composition environment is an advanced workflow definition tool that has been augmented with innovative execution monitoring and debugging capabilities. A workflow process is composed of a set of service invocations (or tasks) and its structure defines the control and data flow dependencies between them. Following this approach, it becomes possible to give a high-level description over the partial order of invocation of the services and their interactions while maintaining well-defined, executable semantics. With this tool, a process can be modelled with different syntaxes. Developers of the composition may use a visual representation to draw the interactions between the services that should be composed. However, in order to facilitate the development and the interoperability of the various tools (e.g., editors, model checkers, compilers, monitoring tools) composite service definitions are stored behind the scenes using the XML syntax. As previously mentioned, the tool can be used to manage the entire lifecycle of a process. This begins with the service library, where web services can be imported as reusable components. The developer can browse the library, select a set of basic services and drag and drop them into a process. Then, he or she may begin to define their interactions by drawing the data flow graph. This operation is partially automatic, since the Adapt visual composition editor can bind parameters with matching names. Manual intervention is only required to resolve ambiguities and connect parameters that could not be automatically matched. To get an overview over the order of execution of the tasks and add additional constraints, the user may view and edit the control flow graph. The visual editor automatically keeps the two graphs synchronized and provides advanced model checking facilities that can be used to validate the composition. At runtime, the visual representation is augmented with colour-coded information representing the current state of the execution of the tasks and developers can click on parameters to inspect their current value. The tool also features capabilities to administer the set of deployed processes and control their overall execution. If failures occur, it highlights them so that the debugging facilities can be applied to interactively determine the cause of the problem.

The absence of a single standard for transaction coordination and a non-proprietary royalty free standard for the industry continues to be a roadblock towards the effective exploitation of open source software in this area including the ADAPT software. Arjuna has been driving standards adoption by co-authoring the Web Services Composite Application Framework (WS-CAF) specifications. Madrid has participated in the standards process and implemented an advanced transaction system based on the ideas in WS-CAF. Subsequently, IBM and Microsoft invited Arjuna to be an author to their own specifications - Web Services Atomic Transaction (WS-AtomicTransaction) The benefit to the industry cannot be underplayed. As soon as these specifications are in a standards arena, there will be significant pressure from customers and analysts for some kind of amalgamation with WS-CAF. This achievement is a very good success indicator to demonstrate how Arjuna's standards participation has influenced the future shape of technical specifications and helped to provide a single royalty free specification for the industry. Arjuna Technologies Ltd, has sold its distributed transaction monitor and web services technologies to JBoss Inc., who will open source the technology for the JBoss Enterprise Middleware Suite (JEMS tm) in the first quarter of 2006. This open source transaction engine, with a 20-year-old pedigree, includes Arjuna's Web Services Transaction implementation, one of the few that has demonstrated interoperability with other industry leaders such as Microsoft and IBM. Arjuna's Web Services Transaction benefited from the support of the ADAPT project by its participation in open standards.

This software is a specific replication algorithm for another project result ("A toolkit for J2EE replication"). It allows replicating volatile Java objects accessed as a web service. The replication algorithm provides strong consistency guarantees, in particular, it the same consistency guarantees about the order of execution of operation requests as its non-replicated implementation. Such guarantees are preserved in spite of server replica failure, network failure (either between server replicas or between a client and a server replica) and irrespective of when the failure occurs. Moreover, the service guarantees that in case a client sends a ¿update¿ request multiple times, there is no risk that the request be executed multiple times. No hypothesis about the timing retransmission policy of clients is made, e.g., the very same request might even arrive at different server replicas simultaneously. All these features make the proposed framework particularly suitable for interaction between remote programs, a scenario that is gaining increasing importance.

There are several industry led efforts aimed at specifying composition languages for web services, however all of these language taken a centralised view of composition and subsequent execution. For example the use of shared variables makes it very difficult to coordinate the execution in a distributed manner. Also, many of these languages specify complicated control flow mechanisms, making it difficult to analyse such compositions. ADAPT composition language has been developed with both of these drawbacks in mind, it contains elements to allow distributed composition and the simple data flow sequencing model has been mapped to pi-calculus to allow analysis of compositions. Current tools support for verification of pi-calculus are in their infancy. Most do not support the complete language and require a complex and error prone input syntax. It is also possible to map our language onto Promella and then use the SPIN model checker.

Creating new services by combining a number of existing ones is becoming an attractive way of developing value added web services. This pattern is not new but it does pose some new challenges, which have yet to be addressed by current technologies and tools for web service composition. The first step in this direction is to provide more semantic information about each web service in order to be able to reason about a composition, which has been created manually. One aspect of using a web service correctly is invoking the operations in the order in which the provider intended. However, the WSDL description of a web service does not specify any ordering information for the operations, which are exposed by the service. The ADAPT project has developed a simple language for capturing the order in which the operations of a web service should be invoked to achieve a goal. The language also allows specification of composite web services as a business process; the language permits orchestration of the process using workflow management systems in either a centralized or distributed, peer-to-peer fashion. The language has precise semantics expressed in pi-calculus. This enables a service composer to verify that the composition is well formed: for example that it does not contain any deadlocks or livelocks, which would cause the composition to not terminate under certain conditions.

JASS is a Java implementation of two standards for advanced transaction processing. One standard is the J2EE Activity Service JSR 095 from JCP (Java Community Process). The activity service provides a core service for handling and propagating transactional contexts as JTA does for ACID transactions. It enables the implementation of advanced transactional models on top of this core service. In JASS the Open Nested Transactions model has been implemented. It enables long running transactions composed of short ACID transactions. The open nested transaction guarantees atomicity at global level and isolation at the component ACID transaction level. In this way resources are not retained during the long duration of the transaction. The other standard is OASIS Web Service Composite Application Framework (WS-CAF). WS-CAF consists of a stack of three specifications: WS-Context, WS-Coordination and WS-Transaction. WS-Contex provides an interoperable framework for sharing and propagating context among business partners. A context can be many different things form a shopping cart to a transactional context. WS-Coordination extends WS-Context with the notion of coordinator and deals with guarantees over notification messages. This is important for instance for coordinating a two-phase commit protocol. WS-Coordination also supports the notion of cascading coordinators. Finally, WS-Transactions builds upon WS-Context and WS-Coordination to provide an infrastructure for transactional coordination protocols. It currently specifies three protocol families: two-phase commit for ACID transactions, Long-Running Activities, and Business Processes. JASS currently implements long-running activities that are a more general model than open nested transactions. The current status of JASS is a pre-industrial product that with some little additional work can be used in a commercial product.

Deliverables

Share this page

Download