Java CoG Kit Ad Hoc Grid Framework User Guide
From Java CoG Kit
Kaizar Amin and Gregor von Laszewski
![[1]](/images/task_1.png "High priority task"){kind=small}
A requirements section is needed with explicit links to the CVS on which version of the CoG Kit and Globus Toolkit are to be used. (Amin)
Contents |
Introduction
The Java CoG Kit Ad hoc Grid framework provides elements required for an ad hoc spontaneous Grid-collaboration between multiple parties. The key features of this framework are as follows:
- On-the-fly Grid collaborations. Individuals with no prior relationship have the ability to collaborate with each other in a flexible and secure fashion without requiring pre-established administrative infrastructure.
- Multiple group creation and membership Individuals can create a scope of their collaborations by defining ad hoc Grid groups and interacting within the domain of those groups. They can also search for existing groups and join them based on their interests and objectives.
- Service provisioning in different technologies. Service providers can offer various Grid services such as execution service, file storage service, or workflow service. Further, our framework imposes no restriction on the development technology of the service. Services in various technologies such as GT2, GT4, WSRF, JXTA, etc. can be seamlessly integrated into the framework.
- Distributed service publishing and discovery. Each of the provisioned service is represented by a service advertisement. The service providers have the capability to create their service advertisements with meta-service information such as its functional offerings, QoS offerings, and other service-specific information. The ad hoc Grid framework allows the providers to publish these advertisements within the member groups using the distributed hash table (DHT) technology.
- Abstraction based service invocation. The ad hoc Grid framework adopts an abstraction-based service invocation scheme that systematically decouples the client-side task specification from the server-side service technology. Using the Java CoG Kit abstractions framework, the service consumer can provide a task specification independent of the implementation technology of the remote service. Based on the service selected for execution, the abstractions framework translates the abstract task specification into technology-specific constructs, thereby allowing the client to access a wide range of technologies.
- Support for ClassAds-based service-task matching. In contrast to direct task submissions (where the client explicitly specifies the remote service to be invoked), tasks can also be submitted in the brokering mode. In the brokering mode of task submission, the ad hoc Grid broker extracts the ClassAds definition from the task and available services and performs a matching between them. This scheme dynamically selects the best service for the task based on its functional and quality requirements (ClassAds) rather than the user (client) judgment.
- Support for service reservation and service agreements. The ad hoc Grid framework offers strong QoS support by allowing service consumers to make immediate and advance service reservations. Based on their quality requirements, the service consumers can reserve exclusive access to offered services. The service provider generates a service-level agreement (SLA) for successful reservations and hands them to the clients. The clients can use these agreements to submit their tasks for the duration of the reservation.
- Policy based authorization framework. Service providers can autonomously protect their service from rogue and malicious elements in the Grid using XACML-based authorization policies. Unless, the policy grants access to the consumer for a particular service, the consumer cannot invoke any operations on that service.
A detailed understanding of the theoretical elements of this framework is available in PDF
Installation
The CoG ad hoc framework is a part of the Java CoG Kit project (http://www.cogkit.org). However, due to its experimental nature, it has not yet been completely integrated with the rest of the Java CoG Kit. Thus, the installation of the ad hoc Grid framework is done in four steps:
Step-1
Install the Java CoG Kit from the sources. Instructions for downloading the source version of the Java CoG Kit are available in the Installation guide (http://www.cogkit.org/release/4_1_3/manual/install.pdf).
We will refer to the main cog directory as $COG_PATH. A listings on $COG_PATH should look similar to this:
$> ls $COG_PATH BUGS.txt README.txt build.xml man qualitycontrol CHANGES.txt VERSION dist mbuild.xml tools LICENSE.txt bin etc modules webstart Makefile build lib pmd webstart.properties
Step-2
Once the Java CoG Kit is downloaded, the next step is to download the ad hoc Grid framework. The source code for the ad hoc Grid framework is available on the surceforge project page for Java CoG Kit: http://sourceforge.net/projects/cogkit/
This project's SourceForge.net CVS repository can be checked out through anonymous (pserver) CVS with the following instruction set. When prompted for a password for anonymous, simply press the Enter key.
$> cvs -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/cogkit login $> Password prompt: Press Enter $> cvs -z3 -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/cogkit co –P src
We refer to the directory where the src module from sourceforge is installed as $TEMP.
Step-3
In this step, we will integrate the ad hoc Grid framework with the rest of the Java CoG Kit.
$> mv $TEMP/src/cog/modules/adhoc $COG_PATH/modules/adhoc
NOTE: In the future, this step will be automated via a shell script
Step-4
This step assumes that you have a current version of the Apache Ant software (http://ant.apache.org/). We now move into the adhoc directory of the Java CoG kit and compile the ad hoc Grid framework by doing an “ant dist”
$> cd $COG_PATH/modules/adhoc $> ant dist
Configuration
After the installation is successful, you are required to setup your Grid environment with the help of the Java CoG Kit Setup utility.
$> cd $COG_PATH/modules/adhoc/dist/adhoc-<version>/bin $>./cog-setup
This command will show the setup screen as shown in Figure 1. It will guide you through a series of steps to create and customize your Grid environment.
Fig 1: The Java CoG Kit Setup Utility
Once the generic-Grid environment is setup, we next move to setup the ad hoc Grid environment. Invoke the command to start the ad hoc Grid desktop.
$> cd $COG_PATH/modules/adhoc/dist/adhoc-<version>/bin $>./cog-adhoc
When the cog-adhoc command is invoked, the JXTA Configurator screen is presented as shown in Figure 2. The JXTA configurator appears only for the first invocation of the ad hoc Grid desktop.
Figure 2: JXTA Configurator (Basic Screen)
For the “Basic” screen you need to enter your publicly visible username. For the “Advanced” screen please use the default settings (unless you know what they mean).
Figure 3: JXTA Configurator (Advanced Screen)
For the “Rendezvous/Relays” screen, download the list of available rendezvous and relay nodes (see Figure 4). If you are working behind firewall or NAT please select the “Use a Relay” checkbox.
Figure 4: JXTA Configurator (Rendezvous/Relays Screen)
For the “Security” screen enter a username/password combination to secure the contents of your ad hoc Grid setting on the local machine.
Figure 5: JXTA Configurator (Security Screen)
Once you click the “OK” button in the JXTA configurator, all your settings are stored in the .jxta directory in your current working directory. If you remove this directory, the JXTA configurator will be invoked again the next time you use the ad hoc Grid desktop.
Group Management
Once you have configured the ad hoc Grid environment, you will see the ad hoc Grid desktop as shown in Figure 6.
Figure 6: CoG Ad Hoc Grid Desktop
Before you can collaborate in an ad hoc Grid, you need to be a member of at least one ad hoc Grid group. You can have simultaneous memberships to as many groups as you like. For the purpose of this guide, we will demonstrate how to create, search, and join an ad hoc Grid group by the name of “Java CoG Kit”.
Figure 7 shows the screen to create a new group. The group creation screen can be invoked from the ad hoc Grid desktop via the “Groups→Create” menu option. As shown in the figure, the user needs to provide with the group name and description. Optionally, if the group is a closed group, the user can also supply a password required to access the group. Creating a new group doesn’t necessarily provide membership to that group. The group creator has to explicitly join the group via “AutoJoin” checkbox on the “Create Group” screen or by invoking the “Groups→Join” menu option. Note: The password facility is not yet implemented.
Figure 7: Group creation screen
Once the group has been created, other ad hoc Grid users can discover the group using the “Groups->Search” menu option (as shown in Figure 8).
Figure 8: Group search screen
Figure 9 shows the interface screen for joining an existing group via the “Groups→Join” menu option. The “GroupID” for the group to be joined is obtained from the “Group Search”. If the group is protected with a password, the appropriate password is required.
Figure 9: Group join screen
Group Communication
Once a peer joins an ad hoc Grid group, it will see a screen as shown in Figure 10. The presence management pane shows the list of other online peers that are members of this group. The figure shows two members of the group, the user “amin” and the user “gregor”. The chat pane allows broadcast-style communication between all the members of the group.
Figure 10: Group Communication
Service Provisioning
Service providers can conveniently contribute Grid services from the “Grid Service” tab. Note that at this time, only execution services are supported. However, the desktop can easily be extended to support different service types. As shown in Figure 11, the service-provisioning screen requires a user-friendly service name, a functional class (only execution supported), service contact, and service technology. The service technology, allows the client-side abstractions framework to translate the abstract task specification into technology specific details. For JXTA technology, the service contact is generated dynamically and does not need to be specified. However, for other technologies such as GT2, GT4, etc. the provider needs to mention a service contact that hosts the remote service. The provider can additionally specify the keywords and description relevant to the contributed service. The service provider can optionally include a qualitative description for the service by defining a ClassAds for it. All the above-mentioned attributes for the service are combined into a service advertisement. The service provider may publish the service advertisement for others to discover or save it into the file system via the “Publish” and “Save” buttons respectively. Using the “Open” button, the provider may also load a pre-stored advertisement into the panel. The “Results” panel shows a log of all the executions performed on that service. Note that the provider using the “New” button at the bottom of the service provision screen can contribute multiple services.
Figure 11: Service provisioning screen
QoS Provisioning
Services can optionally support QoS provisioning. In the current implementation this scheme is only available with execution services offered with the JXTA technology. Further, reservation support is only enabled for the Linux operating system. In order to enable service reservation support the provider needs to start the reservation web service.
$> cd $COG_PATH/modules/adhoc $> ant -f build-axis.xml server
In another shell
$> ant -f build-axis.xml deploy-reservation
The reservation support is now enabled. The reservation service manages task execution by giving the highest priority to the task associated with a reservation. It does so by manipulating “nice” levels of non-root Posix threads and then “renicing” the priority levels to the original value after the task is completed. This enables a lightweight CPU management solution without having to install bulky systems like DSRT.
Service Discovery
Services that are published by the providers can be discovered by the consumers using the “Information Service” panel. Figure 12 shows the interface for service discovery. Note that element-based searching is not yet implemented.
Figure 12: Service discovery screen
Service Invocation
Figure 13 shows the user interface for remote service invocation using the “Execution Manager” tab.
Figure 13: Service invocation screen
Services can be invoked in two modes:
- Direct mode
- Broker mode
In the direct mode, the consumer provides all the information specific to the remote service including the service contact and service technology (published in the service advertisement). Note that the service contact for the JXTA technology is the PeerID published in the service advertisement. Optionally, the consumer may decide to reserve a service for a specific duration before making invocations to it. Figure 14 shows a service reservation interface using the “Reserve” button.
Figure 14: Service reservation screen
In the brokering mode, the consumer can specify the task requirements in terms of its ClassAds. It does not specify the service contact and the service technology. The ad hoc Grid broker dynamically maps the task-ClassAds to a valid service-ClassAds and submits the task to the selected service. Since the service selection is done dynamically, service reservations are not supported in the brokering mode of service invocation.
Authorization Policy
The CoG ad hoc Grid framework facilitates autonomous authorization management of contributed services. Every provider is individually responsible for securing its contributed services. The provider can specify complex authorization policies. By default, access to all services are blocked unless explicitly granted permissions in the authorization policies. The ad hoc Grid framework expects an XACML “policy.xml” in the $HOME/.globus directory. A sample policy.xml file is available in $COG_PATH/modules/adhoc/policies directory. This policy only permits the “/bin/date” command to be executed. You can modify it and use it appropriately. You can modify an existing policy file using the “Policy→Update” menu of the ad hoc Grid desktop (see Figure 15).
Figure 15: Policy update screen
