Scientific experiment management

From Java CoG Kit

1 Proposal
- 1.1 References
- 1.2 Related References
2 Progress
- 2.1 October 2, 2005
- 2.2 September 23, 2005

Proposal

The Java CoG Kit Experiment Manager is a project inspired to simplify user interaction with grid environments. The experiment manager allows scientists to easily manage a large number of tasks to be conducted as part of an experiment. It includes features typically found in queueing systems, shells with history, and process monitoring programs such as the UNIX ps command. Specifically, it provides automated checkpointing, or saving the state, of an experiment, transparent output management, experiment metadata management, application status reporting, persistent experiment sessions, and scalable experiment status updating.

I propose that as a senior project, I improve and expand on the experiment management framework. Its current implementation makes use of a client-server architecture that exposes a potential security hole that is currently “secured” by allowing both the client and server to run on the same machine, where the port of communication between client and server is externally inaccessible. To lift the restriction that the experiment client and server operate in a secure Intranet, I will secure communications at the transport level using the Java CoG Kit’s secure grid sockets. Another alternative to securing client-server communication is to use the Globus Toolkit 4’s secure grid services. This entails exposing the experiment manager’s command line interface as web services to enable the use of GT4’s Web Services Resource Framework (WS-RF). With both security alternatives implemented, it would be beneficial to discern the advantages and disadvantages of each approach.

A goal of the experiment manager when it was conceived includes the management of workflow submissions. Workflow support enables users to conduct complex experiments, adapting the experiment manager for more general use than simple experiments that require repeated execution of tasks while varying specific parameters.

Implicit tasks to be performed while providing the aforementioned enhancements include maintaining the system that was developed. Maintenance involves resolving any outstanding bugs listed, improving memory consumption, and exposing the system to more test cases.

[edit]