Karajan:The role of schedulers

From Java CoG Kit

Schedulers in Karajan also play the role of brokers and allow the use of abstract tasks, which are not bound to specific services or protocols. They also allow management of both local and remote resources. Whenever an execute, transfer or a file operation task from the Task Library is executed without a service contact being specified, and if a scheduler is defined (using scheduler) the job, transfer or file operation are sent to the scheduler for execution. The scheduler then selects a resource and a protocol (provider) and sends the task to the selected resource using the chosen provider.

A scheduler has no knowledge of the high level details of the workflow, or the dependencies between different tasks. It assumes that if a task has been queued on the scheduler, it can be executed immediately, and that there is no particular order in which tasks in the queue must be executed.

A scheduler in Karajan must implement the Scheduler interface.

In general Karajan requires the scheduler to be able to do the following:

• Be able to resolve unspecified services and providers in a task to concrete services/providers based on a set of available resources
• Submit a task after resolving the services and providers
• Keep track of resource usage
• Be able to reserve services for bindings that occur outside of the scheduler and have a mechanism for releasing previously reserved services

The last requirement is needed in order to allow the groupping of more than one task on the same resource. It is often necessary, even with abstract tasks, to ensure that two or more such tasks are sent to the same resource. The resource reservation requirement ensures that such a mechanism is possible. However, for efficiency reasons, it is recommended that a late-binding approach be used with service reservation. The scheduler should therefore reserve "virtual" resources. A virtual resource will then only be bound to a concrete resource by the scheduler itself. This recommandation is based on the fact that the workflow system may request the allocation of multiple resources concurrently. If the scheduler is unable to allocate a resource (virtual or not), the threads requesting the services will be blocked until resources become available. The decision of which threads will be able to continue if a resource becomes available will either be arbitrary or the policy will have to be handled outside of the scheduler. In the event the scheduler can allocate virtual resources, tasks, even though no concrete resources are available, can still be sent to the scheduler, pushing the policy of selecting the apropriate tasks when resources become available to the scheduler. Since such a policy appears to belong to the scheduling domain, the latter case (virtual resources) is preferable from both a consistency and clarity of implementation perspective.