From Java CoG Kit

The Java CoG Kit Workflow Framework

The Java CoG Kit Workflow Framework Contact: Gregor von Laszewski, gregor@mcs.anl.gov, http://www.cogkit.org Scientist, Argonne National Laboratory and Fellow, University of Chicago

An up to date version of this document is available at:

http://wiki.cogkit.org/index.php/NSF_Workshop_Java_CoG_Kit_Workflow_Framework

The Java CoG Kit provides a number of workflow related features that allow us to easily use Grid and non Grid infrastructure. These are primarily of interest to scientific user communities Our goal is to design a set of tools that allow the creation, deployment, and instantiation of dynamically adapting workflows based on conditions defined by the users and the available infrastructure. To achieve this goal we have defined a number of elementary tools and concepts that help us in making progress to design such an environment. One of the concepts that we introduced with the Java CoG Kit is the availability of a well accepted Grid patterns exposed through an abstraction layer that allows abstracting much of the details of the underlying Grid infrastructure. Such an abstraction layer can for example be useful to protect the users from upgrades to the backend Grid infrastructure in case the Workflows run for month and not just minutes or hours. While using supercomputing centers and large scale Grid infrastructures such as the TeraGrid, with its extensive software stack, we must be aware that future scientific applications should be immune to changes in the software stack during its lifetime. In addition we need the ability to formulate workflows with a simple extensible workflow language that allows the creation of scalable workflow engines. The language must be simple as avariety of specialized workflow engines that focus on particular performance goals set by the user community must be available. We also believe it must be able to specify more than just DAGs. Although we see standardization efforts such as BPEL, we must realize that the implementation of the full standard is challenging and unnecessary for many applications. We also observe that some of the BPEL workflow engines may not be as scalable as some applications may desire. We must analyze which features we need to find a balance between feature, performance, and completeness. Besides the availability of a workflow language we must be able to extend the language through a set of core libraries that target the dynamic nature of the workflows. This includes fault tolerance, check pointing, but also dynamically created forms allowing us to control the workflow in an interactive mode. Furthermore, we have designed a workflow service that allows the scheduling of system tasks with a performance improvement of one magnitude over what a typical Grid execution service is capable of. This much increased performance, allows integration of finer grained scientific applications and will provide the ability to delegate workflows in a future release.

References

A number of publications can be found at the following Web site: