Whitepaper: Java CoG Kit Broker Abstractions
From Java CoG Kit
Kaizar Amin and Gregor von Laszewski
Contents |
Status
This is a whitepaper and is considerd to be a proposal. It is not implemented yet.
Introduction
The Java CoG Kit broker module (cog-broker) provides a high-level infrastructure and abstractions that allows a client to conveniently manage its Grid resources and tasks. The Java CoG Kit abstractions module (cog-abstractions) provides a basic set of APIs to submit bound-tasks (A task is bound if it encapsulates a remote service to facilitate its execution) and execute them with the given Grid services. On the other hand, the brokering framework offers a higher level of abstractions that provides a mechanism to dynamically execute unbound-tasks (a task is unbound if it does not encapsulate a remote service). The cog-broker framework provides a dynamic binding (using a mapping algorithm) for the submitted unbound-tasks with the most appropriate available service. Internally, the broker framework utilizes the functionality offered by the cog-abstractions framework.
Design
Figure 1 describes the Java CoG Kit broker framework design. The broker accepts tasks from the Grid client placing them in the BrokerQueue.
Figure 1: Java CoG Kit Broker Design
The broker then invokes the dispatcher to service the queue and retrieve the next available task. If the extracted task is bound, the broker delegates it to the cog-abstraction framework for remote execution. On the other hand if the task is unbound, the broker hands the task to its matchmaker component. The Matchmaker component interacts with the service manager to extract a set of all services available to the client. Based on its mapping policy, the matchmaker binds the unbound task to a service most appropriate for its execution. Once the task is successfully bound to some service, the broker delegates it to the cog-abstractions framework for further execution.
The rest of this document outlines the important functionality and key interfaces for each of the components building the broker framework.
Broker
The Broker forms the basic component of cog-brokering. It encapsulates all other components offering the client with a consolidated functionality. It allows the client submit an unbound tasks and manage its execution. It also allows the client to manage the set of available services. Further, the broker allows the client to control the maximum concurrency during execution. The concurrency of a broker implies the maximum number of active tasks that can be executed by the broker at any given time.
public interface Broker extends Runnable {
public boolean start();
public boolean stop();
public void submit(Task task);
public boolean remove(Identity identity);
public void submit(Collection tasks);
public boolean suspend(Identity identity)
throws InvalidSecurityContextException,
TaskSubmissionException;
public boolean resume(Identity identity)
throws InvalidSecurityContextException,
TaskSubmissionException;
public boolean cancel(Identity identity)
throws InvalidSecurityContextException,
TaskSubmissionException;
public boolean setQueue(BrokerQueue brokerQueue);
public BrokerQueue getQueue();
public boolean setServiceManager(ServiceManager serviceManager);
public ServiceManager getServiceManager();
public boolean setDispatcher(Dispatcher dispatcher);
public Dispatcher getDispatcher();
public boolean setMatchMaker(MatchMaker matchMaker);
public MatchMaker getMatchMaker();
public void addService(Service service);
public void removeService(ServiceContact serviceContact);
public Service getService(ServiceContact serviceContact);
public boolean containsService(ServiceContact serviceContact);
public Collection getAllServices();
public Collection getServices(ClassAd classAd);
public Collection getServices(String provider, int type);
public Collection getServices(int type);
public Collection getAllTasks();
public Collection getTasks(Priority priority);
public Collection getTasks(int status);
public void setConcurrency(int concurrency);
public int getConcurrency();
public void setAttribute(String name, Object value);
public Object getAttribute(String name);
public Enumeration getAllAttributes();
}
Task Extensions
In cog-broker, we augment the definition of an abstract task (as defined in cog-abstractions) to include a Priority and ClassAd. The priority signifies the importance of a task relative to other tasks handled by the broker. For example, a task with higher priority implies it needs to executed before a task with lower priority.
public interface Priority {
public static final int TOP_PRIORITY = 100;
public static final int LOW_PRIORITY = 0;
public void setValue(int value);
public int getValue();
public void setMaximumPriority(int maximumPriority);
public int getMaximumPriority();
public void setMinimumPriority(int minimumPriority);
public int getMinimumPriority();
}
A task may also have an associated ClassAd (classified advertisement). ClassAd is a functional language that can be used to describe the attributes and requirements of a Grid entity (such as a task). It is developed and distributed as a part of the Condor project. The Java CoG Kit ClassAd encapsulates the functionality offered by the Condor ClassAd and further extends it with several convenience functions.
public interface ClassAd {
public void setAd(File classAdFile) throws FileNotFoundException;
public void setAd(RecordExpr recordExpr);
public RecordExpr getAd();
public String toXML();
}
BrokerQueue
The BrokerQueue is a simple extension of the java.util.List class. It provides the necessary interface allowing the broker to aggregate the set of tasks handled by it.
public interface BrokerQueue extends List {
}
Dispatcher
The Dispatcher is responsible for servicing the broker queue. The dispatcher primarily implements the getNext() method which returns the next ``most appropriate task to be executed from the broker queue. Different implementations of the dispatcher can enforce various dispatching policies. For example, the FCFSDispatcher (first come first served dispatcher) will return the first task in the broker queue that has not yet been dispatched for execution. Likewise, the PriorityDispatcher, defined by the user, can select tasks based on their user-defined priority levels.
public interface Dispatcher {
public Task getNext();
public boolean setQueue(BrokerQueue brokerQueue);
public BrokerQueue getQueue();
}
Service
Every Grid Task is bound to a set of remote Services that support the actual execution of the task. The service interface is a local representation of the remote Grid service. Every service has a provider attribute that specifies the technology provider supported by that service. It also has a service contact and a security context specific to that provider. Further, a service can be described by its classified advertisements (ClassAd). The service ClassAd allows a client to describe specific attributes provided by that service. During dynamic task binding, the broker binds an unbound task to a service whose ClassAds mutually match each other.
public interface Service {
public void setIdentity(Identity identity);
public Identity getIdentity();
public void setName(String name);
public String getName();
public void setProvider(String provider);
public String getProvider();
public void setType(int type);
public int getType();
public void setClassAd(ClassAd classAd);
public ClassAd getClassAd();
public void setServiceContact(ServiceContact serviceContact);
public ServiceContact getServiceContact();
public void setSecurityContext(SecurityContext securityContext);
public SecurityContext getSecurityContext();
public void setAttribute(String name, Object value);
public Object getAttribute(String name);
public Enumeration getAllAttributes();
}
ServiceManager
A ServiceManager provides the functionality required to efficiently maintain a set of services available to the client.
public interface ServiceManager {
public void addService(Service service);
public Service removeService(ServiceContact serviceContact);
public Service getService(ServiceContact serviceContact);
public boolean containsService(ServiceContact serviceContact);
public Collection getAllServices();
public Collection getServices(ClassAd classAd);
public Collection getServices(String provider, int type);
public Collection getServices(int type);
public void setAttribute(String name, Object value);
public Object getAttribute(String name);
public Enumeration getAllAttributes();
}
MatchMaker
The MatchMaker component of the broker is responsible for providing a task-to-service mapping for all unbound-tasks submitted to the broker. The matchmaker interacts with the service manager to get access to the set of available services to the client. Different implementations of the MatchMaker will enforce different matching policies. For example, a RandomMatchMaker selects any random service for a particular service type. In other words, it will select any job execution service for an execution task. A ClassAdMatchMaker will match a give task to a service such that their ClassAds are mutually satisfies by each other.
public interface MatchMaker {
public Service match(Task task);
public boolean setServiceManager(ServiceManager serviceManager);
public ServiceManager getServiceManager();
public void setAttribute(String name, Object value);
public Object getAttribute(String name);
public Enumeration getAllAttributes();
}
