From Java CoG Kit

Gregor von Laszewski, Mike Hategan, and Deepti Kodeboyina

• Version: 4.1.5, Wiki Revision: 7299

Last update: August 27, 2006

Contents 1 About this Document 2 Quick Decision Guide 3 Introduction 3.1 Concepts 3.2 Bundles 4 Decision Guide 4.1 SVN, Source, or binary? 4.2 Full or partial? 5 Install Bundles 5.1 Complete Binary Install 5.2 Partial Binary Install 5.3 Source Distribution Install 5.4 Development Distribution 5.5 Source Repository Navigation Table 6 Documentation 6.1 Downloading the API Documentation 7 Configuration 7.1 Configuration of the Java CoG Kit 7.2 Debugging 7.2.1 Log4j 7.3 cog-info 8 References 9 Appendix 10 Binary Distributions 10.1 Full Set of Standard Modules 10.1.1 Separate modules 10.2 Source Distribution 10.3 Documentation Distribution (API only) 11 Bugs 12 Administrative Contact

About this Document

This document includes basic information about the different Java CoG Kit distributions in order to help you chose the one that is most suitable to your needs.

Registration

Please be a team player and support us indirectly by registering with us or reporting your use of the Java CoG Kit. Although this software is free, we still need to justify to our funders the usefulness of the projects. If you want to help us with our efforts please take a few seconds to complete this information. We do not use this information for other purposes. If you have special needs or concerns please contact gregor@mcs.anl.gov. The registration form can filled out in a variety of formats. The online form can be found at http://www.cogkit.org/register .

Format

We have augmented the document with comments/tasks at places where we found issues. Our intend is to address these issues in a future release. A task will appear as follows

Text describing the task (wiki account name the task is assigned to)

In case you have a wiki account, you can add new tasks to the Documentation as documented in our Wiki Help.

Quick Decision Guide

• Use the binary release bundles as the fastest installation method for release packages
• Use the source release bundles if you want to see the sources of a release and/or compile a release yourself
• Use the newest SVN version if you are a developer, a student, or need to have a feature that is not supported by the release version

Introduction

Originally, we distributed the Java CoG Kit as a single package called cog-jglobus. However, the distribution has grown and in order to prevent that a package of large size need to be downloaded and installed on each client or server, we decided to divide the contents of the Java CoG Kit into separate modules and build a system that allows incremental deployment. Each module of the Java CoG Kit distribution is preceded by the keyword cog. Please note that the . between version numbers is replaced by an _. The base for the package names are

    cog-<version>-<module>.[bin|src].[tar.gz|zip]

where module is the name of the module. The resulting tar.gz and zip file is refered to as a package.

Concepts

To avoid packages of large size, we have bundled a number of correlated components into separate packages. The names of the packages correspond to the naming scheme mentioned earlier. To illustrate which components of the Java CoG Kit are associated with which packages we like that you spend a moment inspecting Figures 1 and 2.

The Java CoG Kit layered architecture provides a framework for developing Grid clients and servers. Hence, you will be able to access classes and methods not only including a convenient API focusing on elementary Grid functionality, but also the ability to formulate workflows through XML specifications, command line tools, and graphical user interfaces. The Java CoG Kit layers are conceptually organized as follows:

• Providers: A Java CoG Kit provider presents a way of interfacing convenient Grid abstractions pioneered by the Java CoG Kit with existing Grid middleware or distributed systems software. We have developed prototype providers for GT2, various versions of GT4, GT3, SSH, your local machine, and Condor. While using the Java CoG Kit abstractions, the user need not to worry about the underlying Grid middleware changes (see also Basic Grid abstractions). As providers are dynamically loaded at runtime, the provider choice can be made by late binding.

• Basic abstractions: The Java CoG Kit basic abstraction layer is a convenient subset of Grid functionality that may be used by the Grid developer to hide different versions of Grid middleware. These abstractions include file transfer, file access functions, job submission, and authentication. This subset is chosen to fulfill the needs of many, but not all Grid users. As the abstractions contain only a subset, we encourage you to help us in identifying future useful subsets that meet your needs.

• Data and task management abstractions: Besides the basic abstractions, we have developed a number of more advanced abstractions that support more sophisticated Grid paradigms. The abstractions that we provide and will enhance in future versions of the Java CoG Kit deal with data and task management issues. As these paradigms are not yet part of the Globus Toolkit, they provide a significant enhancement to the available Grid tools and services. We believe that our more sophisticated abstractions will have to be included into future Grid middleware in order to increase the functionality of Grids, thus making them easier to use. Not only are many of these abstractions available as a convenient API, but they can also be accessed as simple XML descriptions. Our abstractions are based on Java interfaces making it possible to extend their functionality or provide alternative implementations.

• Grid faces: Based on the observation that visual components follow a particular pattern, we also have introduced an abstraction layer for visual components. To illustrate this concept, consider the common visual pattern of a Grid directory browser. Although the implementation of the directory browser between a stand alone Java Application and a portlet is different, many graphical features between the two are identical. Thus, each has a window in which the tree is displayed or a field in which the URL is entered. At this time, we have included in our release a Java CoG Kit desktop that includes a number of such abstractions available through a single integrated interface. Additionally, we have started to develop Grid portlets that are also distributed with the Open Grid Computing Environment (OGCE) at http://www.ogce.org

Although Figure 1 and 2 contain some additional components, they are not yet part of the Java CoG Kit. However, in future versions we anticipate you include for example also providers contributed by the community, such as the Unicore provider.

Bundles

Pre-packaged downloads of the Java CoG Kit are available on the release pages for each version. A list of versions is available on the Versions page.

Or you could go directly to the latest version's release page: 4.1.5/Java CoG Kit Release Page

Decision Guide

Now that you have an overview about what is available, you may ask which version is the right version for you?

We provide the following guide that will make your choice easier. Please note that we have two dimensions in order to assist you with the install decision. You need to decide if you would like an install from SVN, a source, or a binary install. You need also to decide if you like a full or a partial install.

SVN, Source, or binary?

• Do you want to develop code with the Java CoG Kit?
  • Use the SVN or the source distribution.

• Do you want to use just the Graphical components?
  • Use the binary install.

Full or partial?

• Do you want to make your life easy and do not have any space and time consideration for the download?
  • Use the full distribution otherwise use the partial install.

• Do you have only one version of Grid middleware that you are concerned about?
  • Use the partial install or the full install if space and download speed is not of consideration

• Are you using GT2?
  • Use the distribution ending in main

• Are you suing GT3.x?
  • We suggest you switch to GT4 or use GT2.
  • Neverthelees, if you insist on using GT3 we recommend the following. First, choose carefully which version of GT3 you want to use as there are some differences between them as explained earlier. Then download main, common bundles, and the provider followed by the appropriate version number.

• Are you using GT4?
  • GT4.0 has some important features missing as part of GridFTP and other services. Hence we do not recommend that you use GT4.0 but that you upgrade to GT4.0.1. For this reason, we do not support GT.4.0, but only GT4.0.1.

• Are you integrating Grid resources that have SSH installed on them?
  • Please add the SSH provider. Make sure you also have main.

• Are you integrating UNICORE?
  • We have not yet integrated the UNICORE provider that was contributed to us by the community.

• Are you integrating gsiterm?
  • We have not yet integrated the gsiterm bundle.

• Are you integrating the CoG Kit certificate authority?
  • We have not yet integrated the CoG Kit Grid certificate authority.

• Are you integrating the CoG Kit matlab interfaces?
  • The matlab interfaces are only available via SVN. You can check them out as discussed in Section 8 S:installit LDAP browser?

The Java CoG Kit LDAP browser is distributed through the following link: [http://www-unix.mcs.anl.gov/~gawor/ldap/]. Please note, that this component has a different license from the rest of the Globus Toolkit.

Once you have decided which version is most appropriate for you we recommend that you continue reading at the appropriate section. In case you have not yet made up your mind read the following sections carefully or consult your decision question while using our ticketing system at [http://www.cogkit.org/bugzilla]. Alternatively, you can send e-mal to [gregor@mcs.anl.gov] with the subject line Question: CoG Kit, question, where question is a short summary of the question formulated in the e-mail body.

Install Bundles

Complete Binary Install

After registering and downloading the complete package [tar.gz] [zip], unzip or untar in a directory of your choosing. Proceed by setting up the CoG Kit with the CoG Kit setup component.

Summary

1. register your use of the Java CoG Kit.
2. download the complete package [tar.gz][zip]
3. unpack the package
4. change to the package directory
5. start the setup with ./bin/cog-setup

Partial Binary Install

A partial binary install is almost as easy to conduct. First, you have to be clear about which packages you need. You will always need the main package [tar.gz][zip]. However, you should only download and install providers that you need. After you have downloaded the needed packages you unpack them and start the setup as explained earlier. The list of packages currently available is as follows:

Main package	[tar.gz][zip]	This package is required for all of CoG. It includes currently also the GT 2 provider
SSH provider	[tar.gz][zip]
WebDAV provider	[tar.gz][zip]
Local provider	[tar.gz][zip]

Summary

1. register your use of the Java CoG Kit.
2. download the main package [tar.gz][zip]
3. download other desired providers and packages
4. mkdir cogkit
5. place the packages in ./cogkit
6. cd cogkit
7. unpack the packages
8. cd cog
9. start the setup with ./bin/cog-setup

Source Distribution Install

After downloading the Java CoG Kit >source package [tar.gz][zip], unpack it in a directory, then run

    ant dist

in the cog directory and execute the setup.

The jglobus sources are also available as [tar.gz] and [zip] files.

Summary

1. register your use of the Java CoG Kit.
2. mkdir cogkit
3. mv cog-4_1_5-src.tar.gz cogkit
4. cd cogkit
5. tar -xzvf cog-4_1_5-src.tar.gz
6. cd cog
7. ant dist
8. cd dist/cog-4_1_5/bin
9. ./cog-info
10. ./cog-setup

Development Distribution

In case you need the most up to date version of the Java CoG Kit, you can check this version out from the SVN. To do so you should follow these simple steps:

• register your use of the Java CoG Kit (if you have not already done so).

• svn co https://cogkit.svn.sourceforge.net/svnroot/cogkit/trunk/current/

This will download the code in typical svn manner under

• ./current/src/cog containing the CoG Kit source code (except for JGlobus which is only present as a compiled jar file)
• ./current/src/matlab containing a simple matlab wrapper for GT2

To compile just the cog subdirectory you can do the following:

• 1. After you have downloaded the code
  • cd ./current/src/cog
  • ant dist
• 2. Test if things work
  • cd dist/*/bin
  • ./cog-info
• 2. After the sucessfull compilation you need to configure it as documented in Section #Configuration of the Java CoG Kit.

If you also like to download the sources for jglobus you ned to download it separately. The portion of jglobus that is distributed also as part of the Globus toolkit can be found at

• cvs -d:pserver:anonymous@cvs.globus.org:/home/dsl/cog/CVS checkout jglobus

Source Repository Navigation Table

To help the developers to inspect the code in the CVS/SVN directly, we refer to following table for hyperlinks to the appropriate contents.

Component	API	Changes	CVS	SVN	Bugzilla
cog-jglobus	[html]	[html]	[html]		[html]
Java CoG Kit 4	[html]	[html]		[html]	[html]
matlab	[html]	[html]		[html]	[html]

The link to the old but outdated CVS CoG Kit repositry is at [here].

Documentation

Downloading the API Documentation

In case you like to download the API documentation we have provided an additional package for you:

• API documentation [tar.gz][zip]

However, we recommend that you use the appropriate version's online API documentation at

• Java CoG Kit Java Doc Index

Configuration

Configuration of the Java CoG Kit

Note: If you like to just use ssh services, you can skip this step.

Once you have downloaded the Java CoG Kit, you can proceed to set up the Java CoG Kit. The setup step is conducted with a simple wizard that guides you through the configuration process. In case you like to use a Grid using GSI security, you should obtain a certificate that allows you to get access to this Grid. We recommend that you place the user certificate (usually usercert.pem) and the corresponding private key (typically userkey.pem) in the .globus directory in your home directory. The root certificates for the certificate authorities should be placed in .globus/certificates. We assume that you have started a terminal window and have changed to the directory the contains the release 4_1_5.

Now you can start the setup routine by launching it from your command line as follows. For Linux and Unix say

    ./bin/cog-setup

and for Windows

    cd bin
    cog-setup.bat

Naturally, you can make this even easier if you include the location of the cog_4_1_5/bin directory into your PATH environment variable. Additionally, you should set the variable COG_HOME to the directory that contains your release. Next, call the command cog-setup directly without worrying about having to cd to the bin directory.

You will then be walked through a number of easy to follow steps.

Debugging

Log4j

Debugging is enabled through Log4j. In the dist/etc directory you find a file called log4j.properties in which you can set the logging properties apropiately. A simple way to log a particular package is to enable debugging in the following way. Assume the package is called

org.globus.cog.abstraction.impl.execution.ssh

Than the line

log4j.logger.org.globus.cog.abstraction.impl.execution.ssh=DEBUG

in log4j.properties will enable debugging on that package.

Then set the root category to DEBUG. If it's a specific package then: log4j.logger.package.name=DEBUG

cog-info

cog-info can be used to obtain some elementary information about your machine, os, and software environment. We do recommend that at time of filing a bug you include portions of the information that you think is relevant. Please remember that we devenetly need the version of Globus, cog, Java and ant.

References

1. For citing the Java CoG kit see [las02javacog] and [las01cogconcurency]
2. Java CoG Kit Registration http://www.cogkit.org/register
3. Additional publications about the Java CoG Kit can be found as part of the vita of Gregor von Laszewski at http://www.mcs.anl.gov/~gregor
4. The previous location was at cvs -d:pserver:anonymous@cvs.cogkit.org:/cvs/cogkit checkout src

Appendix

Binary Distributions

NOTE: The GT 3.0.2 and 3.2.0 providers are no longer by default distributed with the binary release. However, for convenience we have provided links from earlier releases. We recommend that you upgrade at least to GT3.2.1.

Full Set of Standard Modules

Modules	4.1.4	4.1.3
Full set of standard modules (includes all modules from the next table)	[tar.gz][zip]	[tar.gz][zip]

A complete list ov available modules in this distribution can be seen at http://wiki.cogkit.org/index.php/Java_CoG_Kit_Java_Doc_Index

Separate modules

Separate Modules	4.1.4	4.1.3
Main package (includes GT2 providers)	[tar.gz][zip]	[tar.gz][zip]
Common GT 3.x.x package (required for all GT 3.x.x providers)	[tar.gz][zip]	[tar.gz][zip]
GT 3.2.1 provider check this.	[tar.gz][zip]	[tar.gz][zip]
GT 4.0.1	see next line	[tar.gz][zip]
GT 4.0.2 and GT4.0.1 provider	[tar.gz][zip]	upgrade to 4.1.4
Condor provider	[tar.gz][zip]	[tar.gz][zip]
SSH provider	[tar.gz][zip]	[tar.gz][zip]
WebDAV provider	[tar.gz][zip]	[tar.gz][zip]
Local provider	[tar.gz][zip]	[tar.gz][zip]
GT2FT provider fault tolerant GT2 provider	[tar.gz][zip]	upgrade to 4.1.4

Deprecated:

• GT 3.0.2 provider (from 4.1.3 release)[tar.gz][zip]
• GT 3.2.0 provider (from 4.1.3 release) [tar.gz][zip

Source Distribution

Modules	4.1.4	4.1.3
Source Distribution	[CVS] [tar.gz][zip]	[CVS][tar.gz][zip]

Documentation Distribution (API only)

• Complete API [HTML] [tar.gz][zip]

Bugs

We use Bugzilla for tracking bugs and for enhancement suggestions. It is located in the bugzilla.globus.org, but you may find it easier to use one of the following quick links:

• Submit new bug report
• current bugs
• Make a simple query
• Make an advanced query
• Create a new account

Administrative Contact

The Java CoG Kit project has been initiated and is managed by Gregor von Laszewski. To contact him, please use the information below.

   Gregor von Laszewski
   Argonne National Laboratory
   Mathematics and Computer Science Division
   9700 South Cass Avenue
   Argonne, IL 60439
   Phone: (630) 252 0472
   Fax: (630) 252 1997
   gregor@mcs.anl.gov

The Java CoG Kit is a team effort and we like to acknowledge the contributions of all that helped. A detailed list of the sponsors and helpers can be found in our Acknowledgement.If we have not listed your name and you have contributed to the CoG Kit, please let us know.