FAQ

From Java CoG Kit
Revision as of 18:48, 25 February 2008 by Mike (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Contents

Java CoG Kit

Globus Nomenclature

Sometimes, it may not be that clear what GRAM and other abbreviations exactly refer to due to the different versions of Globus. We will use the following nomenclature. The Globus Toolkit contains an execution service called 'Grid Resources Allocation Manager' or simply GRAM. However GRAM has been implemented in different ways depending on the version of GT we are talking about. We have

  • gram-s-wsrf
  • gram-s-ogsi
  • gram-s-classic (The GT2 GRAM)

Often, we also have to put the GT version behind the name, as some of the services as the protocols between versions has changed. To indicate if we speak about the protocol we use

  • gram-p-wsrf
  • gram-p-ogsi
  • gram-p-classic (The GT2 GRAM)

Now you may ask which version of GT includes which version of GRAM:

  • gt2 contains only gram-s-classic
  • gt3 contains gram-s-classic and gram-s-ogsi
  • gt4 contains gram-s-classic and gram-s-wsrf

Documentation

Where can I find some Documentation to jGlobus?

http://www.cogkit.org/release/current/api/jglobus/index.html

Where can I find some documentation to the Java CoG Kit?

  • In Gregor's Web page you find some papers .
  • On the Java CoG Kit page you find some pointers to Guides and examples.
  • The FAQ has some more information included at different places.

Which version of the Java CoG Kit should I use?

You should use Java CoG kit version 4.1.5.

What is the difference between jglobus and Java CoG Kit?

<tasks> [ ] TBD (Gregor) </tasks>

Where can I find scholarly references?

The most up to date list of scholarly references can be found as part of Gregors resume at http://www.mcs.anl.gov/~gregor.

For organizing references in Word and LaTeX We use a tool called jabref.

Our bibliographies can be found in the CVS at papers/bib

In there you find a number of actively maintained bib files that all start with vonLaszewski-. In addition you find workflow.bib, www.bib, and grid.bib that include a large number of useful references. If you think we should add your reference here, please do not hazitate to contact us.

Installation

Where to install the Java CoG Kit?

Should I install Java CoG kit in all the machines e.g. including certificate authority issuer and all the client machines? or just on the client machines? or the server?

The answer to this question is: wherever you need to access functionality of the Java CoG Kit. Most often this is just the client side. However the Java CoG Kit can also be used to develop Grid services. If this is the case you also need it on the server side.

The CA issuer should be installed on the computers where Java CoG is running.

On which OS does Java CoG Kit run?

As the minimum requirement is Java JRE/JDK 1.4.2 it will run on Linux, Mac OSX, Windows XP, Solaris 5, and any other OS on which Java 1.4.2 is supported.

Which version of the Globus Toolkit do I need?

The Java CoG Kit is unique in the set of Grid tools as it especially does NOT target an explicit version of Grid middleware. It has a unique concepts of providers which allows the porting of Grid tasks to a variety of Grid middleware.

At present we have developed providers for GT2, GT3.2, GT4 (under development). Community members have also helped to develop a provider for SSH and UNICORE. However we have not yet had the time and resources to distribute the UNICORE provider. Other community members experimented with native providers for PBS.

We are looking actively for community members that develop such providers.

Can I run the Java CoG Kit as user ‘’globus’’?

see FAQ on personal gatekeeper

I like to use ssh on my Windows machines, can I use CoG to talk to them?

You can find more information on setting up ssh on your windows machines at http://pigtail.net/LRP/printsrv/cygwin-sshd.html. With this installed you can use our ssh abstractions and the workflow to talk to them directly. This allows you for example to use scp and ssh to copy and execute remote programs. Naturally full delegation is not supported in this mode. However, if you use CoG as client you can use it to communicate with Globus services and also make use of full delegation if the server supports it. Additional information about ssh servers on Windows can be found at http://www.jfitz.com/tips/ssh_for_windows.html.

I like to test the workflow but do not want to install any Grid software yet. How do I do this?

I assume you have Java and ant, and wget properly installed on your system.

Now follow these steps, where > indicates a shell prompt.

  1. If you have not done so far go to http://www.cogkit.org/register
  2. > cvs -d:pserver:anonymous@cvs.cogkit.org:/cvs/cogkit checkout src/cog
  3. > cd cogkit
  4. > cd src/cog
  5. > ant dist
  6. > cd dist/cog-4_1_4/bin
  7. > wget http://svn.sourceforge.net/viewcvs.cgi/*checkout*/cogkit/trunk/examples/karajan/echo.kxml (If you do not have wget, you can use your favourite browser and store the file in the bin dir.)
  8. > mv echo.kxml.echo.xml
  9. > ./cog-workflow echo.xml

the ouput will be

Hello world!

Please note that xml like workflows must be stored in a file with the ending .xml

Configuration

What is the use of the cog-setup component?

cog-setup is a graphical tool that allows you to conveniently setup your client-side Grid environment. It allows you to auto-generate the $HOME/.globus/cog.properties with all the appropriate entries that are internally used by the Java CoG Kit.

Setting up with Eclipse

<tasks> [ ] The FAQ on eclipse has to be retested and improved if necessary. If someone from the community can help that would be great (Deepti) </tasks>
  • 1) First do a cvs checkout of src/cog in the directory cogkit
  • 2) You will now see the directories, among many others, cogkit/src/cog
  • 3) Start up eclipse and select a workspace outside of the checkout ie) workspace. Close the welcome screen.
  • 4) Go to File->New->Project Select "Java Project". Click "Next >".
  • 5) Create a project
    • 5a) Enter in a Project name: (ie "cog") we will refer to the project name as "cog" from this point on.
    • 5b) Select "Create project at external location" and click "Browse..."
    • 5c) Navigate to cogkit/src/cog and click "OK"
    • 5d) Click "Finish" (this will take a while).
    • 5e) If asked to change to Java Perspective click "Yes
  • 6) On the left you will now see "Package Explorer" with numerous modules listed. Some modules that were imported may not belong (Eclipse does its best guess to what you wanted to import) and may have red X's next to them. Ignore them for now.
  • 7) Specify an output directory.
    • a) Each module outputs the class files to a directory. It is a good idea to ensure this is a separate directory.
    • b) Right Click the project name "cog" and Select Properties
    • c) Go to "Java Build Path"
    • d) ensure that "Allow output folders for source folders" is checked
    • e) enter the value cog/bin to default output folder
    • f) Click OK
    • g) Confirm this setup
      • 1) Click Window->"Show View"->Navigator
      • 2) On the left see if you can see the tree structure cog/bin
      • 3) browse around. These are the files that are included on your classpath

Configure log4j

<tasks> [ ] A new section needs to be created on the bugzilla page in which one explains how to obtain a good submission to the bugzilla. Also we need to explicitly tell others in a separate page how to debug and set log4j for a particular package and method. this should be done with explicit instructions such as cd to .... edit the file ... you can download the example log4j file from her place it in .... the run the example in the following way. links to this page must be done from the faq and the bugzilla page (Deepti) </tasks>

a) Go to Window->Show view->Navigator and browse to cog/etc and create a file called "cog-log4j.properties".

b) A starting value of the contents of this file can be seen below. You may modify this file as you see fit. For more information on log4j see their manual at: http://logging.apache.org/log4j/docs/manual.html


 #####
 # For more information visit: 
 # http://logging.apache.org/log4j/docs/manual.html
 
 ###
 # Root category
 ###
 log4j.rootCategory=WARN, STDOUT
 
 ###
 # STDOUT appends to the console
 ###
 log4j.appender.STDOUT=org.apache.log4j.ConsoleAppender
 log4j.appender.STDOUT.layout=org.apache.log4j.PatternLayout
 log4j.appender.STDOUT.layout.ConversionPattern=%-5p [%c] %x - %m%n
 
 # set anything in the package org.globus.cog.abstraction to debug
 log4j.category.org.globus.cog.abstraction=DEBUG

c) If you have experience with log4j, you may be asking: "Why did we name the file cog-log4j.properties?" This is because many jar files may have log4j.properties file (the default configuration file) inside the jar files. That means when the classpath is searched for the configuration file we do not know which one is being used (there are numerous configuration files on the classpath). To get around this we will specify the configuration file to be cog-log4j.properites.

d) specifying cog-log4j.properties as the configuration file

1) Expand "modules/abstraction-examples/src" and then expand the package org.globus.cog.abstraction.examples.excecution

2) Right click JobSubmission and select Run->"Run..." Select "Java Application" and click "New"

3) Select the "JRE" tab that appears and then Select "Alternate JRE" and click "Installed JREs"

4) click on the JRE that has a check box next to it and then click "Edit..."

5) Go to the "Default VM Arguments:" and type in: "-Dlog4j.configuration=cog-log4j.properties"

6) click OK, OK, Apply, and Close

Adding resources to the classpath.

If you are using the new Maven distribution, please see this page.

a) We still need to add resource files to the classpath. This is because log4j and the cogkit use the classloader to find resources. We need two files to add to the classpath "cog-log4j.properties" and "cog-provider.properties"

b) Add cog-log4j.properties

1) Right click cog in the Package Explorer and go to Properties and select "Java Build Path" and "Source" tab.

2) Click "Add Folder" and navigate to cog/etc and select it click "OK", "OK"

c) Add cog-provider.properties

1) Right click cog in the Package Explorer and go to Properties and select "Java Build Path" and "Source" tab.

2) Click "Add Folder" and navigate to cog/modules/provider-gt2/resource and select it click "OK", "OK"

Run JobSubmission

a) Return to the Package Explorer

b) Expand "modules/abstraction-examples/src" and then expand the package org.globus.cog.abstraction.examples.excecution

c) Right click it and go to Run->"Run..."

d) Select "JobSubmission"

e) Go to the Arguments tab

f) Specify the following as your arguments: "-s hot.anl.gov -p gt2 -e /bin/ls -args /home -r"

1) NOTE: hot.anl.gov will be your servername

g) Click Apply

h) Click Run (if you get there are errors in the project continue to launch? Click "OK")

What is the cog.properties file?

Do I need to pay attention to firewalls and VPN?

Yes, every time you reconnect to the VPN the cog.properties file has to be updated with the IP address for the VPN connection. This can be done by running cog-setup again. If you have a firewall running on you machine this may cause problems. GRAM notifies you when the job is done by connecting to your machine. In order to ensure that you obtain the redirected output you could choose to do one of the following:

1. Disable firewall.
2. Instruct firewall to allow connections from machines within your sub-net alone.
3. Reserve a range of ports for this stuff. Then instruct CoG about those: 
   tcp.port.range=start,end (in cog.properties)

Are there other properties files?

Why is the CoG setup distributed with CoG 4 not used in jglobus?

jglobus is part of the Java CoG kit. The setup component in Java CoG Kit 4 should be used to configure your properties file. You should download the complete Java CoG Kit and not only work with jglobus.

How can I update my IP address automatically?

We recommend the cog-setup program to change the ip address. Experts may look at the following two scripts:

Assume you are using VPN to connect to your remote emachine, the following script on a linux machine can set it automatically for you. (If you have one that works with windows let us know). Please inspect it carefully before you execute it. Call it with cog-vpn-connect.


cog-vpn-connect:

#!/bin/bash
#
vpnupdateip &
sleep 1
vpnclient connect <profile>

cog-vpn-updateip:

#! /bin/bash
LIP=`vpnclient stat|grep "Client address:"`
while [ -z "$LIP" ]; do
       sleep 2
       LIP=`vpnclient stat|grep "Client address:"`
done
IP=${LIP:16}
IFS=$'\n'
echo "Setting IP to ${IP}"
OUT=~/.globus/cog.properties.new
for LINE in `cat ~/.globus/cog.properties`; do
       if [ `echo $LINE|grep "ip="` ]; then
            echo "ip=$IP" >>$OUT
else
       echo $LINE >>$OUT
       fi
done
rm -f ~/.globus/cog.properties
mv $OUT ~/.globus/cog.properties#

How do I get all the jar's in cog lib into the CLASSPATH var without having to type each of then in?

You do not have to set any classpath variable if you use the provided shell scripts. You only need to set tha COG_INSTALL_PATH variable and put $COG_INSTALL_PATH/bin in your classpath. If for some reason ou like to inclue the Jar files anyways, you cam do so with something like

  • CLASSPATH=$CLASSPATH:$(ls $COG_INSTALL_PATH/lib/*.jar | paste -s -d: - )

Security

I am working for DOE; how do I apply for a certificate?

Please note this CA is only for those who have a direct affiliation with DOE. Your account sponsor will know if you are eligible. [1]

I have gotten accounts on TeraGrid resources. Why am I unable to submit jobs to the Globus ToolKit?

You need to me added to the grid-map file to be able to run jobs. Login to the required Teragrid site and run gx-map. gx-map -long-help will give you the list of options. The command below will add you to the grid-map.

tg-jblog@th1:~> gx-map -add -dn "/DC=org/DC=doegrids/OU=People/CN=Joe Bloggs 391618" 
-username tg-jblog -email jblogg@mcs.anl.gov

If you haven't yet been added to the grid-map file you may get an error like this:

Job failed: org.globus.gram.GramException: Authentication with the remote server failed

I am working with the TeraGrid; where do I get the certificate from?

Please go to [2]

I like to build my own Grid. How can I generate certificates

I have a list of trusted CA certificates. How do I instruct the Java CoG Kit to use them while authenticating remote services

You can specify this property and several other properties in the $HOME/.globus/cog.properties file. For specifying a list of trusted CA certs you can use the "cacert" property. In this property you can specify a comma separated list of CA certs. For e.g:

cacert=/home/coguser/cacert/abc.0,/etc/security/xyz.0

Although you can manually create a valid cog.properties file, you are recommended to use the cog-setup tool distributed with the Java COG Kit v 4_0_a1 to establish your cog.properties file.

Additionally, you can specify the CA cert list programatically as follows:

CoGProperties.getDefault().setCaCertLocations(String list);

What is the search strategy for looking for the certificates?

First it looks at the X509_CERT_DIR system property. This gets propagated from the environment variable with the same name if one of the JGlobus launcher scripts is used. You may have to do it manually otherwise.

Second, it looks at the "cacert" property in ~/.globus/cog.properties.

If the above fail, it tries ~/.globus/certificates.

If that doesn't work either, it looks at /etc/grid-security/certificates.

And finally, if none of the above work, it tries $GLOBUS_LOCATION/share/certificates (assuming the GLOBUS_LOCATION system property is set to $GLOBUS_LOCATION).

I get a "Bad certificate" error when trying to submit a job to jglobus.lcrc.anl.gov. Why?

You need a legacy Globus proxy. Use the -old command line argument to grid-proxy-init.

I get GRAM Job submission failed because authentication failed: Expected target subject name="/CN=host/hostname", Target returned subject name="/O=Grid/O=Globus/CN=hostname.domain.edu". Why?

You will get this error when running the client and server, both, on the same machine OR when your Grid is deployed on a local LAN (without access to the Internet or DNS server).

This is because the expected target subject name has just the unqualified hostname but the target returned subject name has the fully qualified domain name (e.g. expected is "hostname" but returned is "hostname.domain.edu").

This is usually becuase the client looks up the target host's IP address in /etc/hosts and only gets the simple hostname back.

The solution is to edit the /etc/hosts file so that it returns the fully qualified domain name. To do this find the line in /etc/hosts that has the target host listed and make sure it looks like:

xx.xx.xx.xx hostname.domain.edu hostname

Where "xx.xx.xx.xx" should be the numeric IP address of the host and hostname.domain.edu should be replaced with the actual hostname in question. The trick is to make sure the full name (hostname.domain.edu) is listed before the nickname (hostname).

While developing Grid Portals with Java COG Kit, I get a ClassCastException for BouncyCastle classes. Why?

This problem is due to library conflicts between Tomcat and the Java CoG Kit. The error you may see is similar to the folowing:

Exception : java.lang.ClassCastException
java.lang.ClassCastException 
    at org.globus.gsi.bc.BouncyCastleCertProcessingFactory.createProxyCertificate(BouncyCastleCertProcessingFactory.java:602)
    at org.globus.gsi.bc.BouncyCastleCertProcessingFactory.createCredential(BouncyCastleCertProcessingFactory.java:278)
    at org.globus.gsi.bc.BouncyCastleCertProcessingFactory.createCredential(BouncyCastleCertProcessingFactory.java:226)

To resolve this issue, You have to remove all cryptix*.jar, jce-jdk*.jar, and puretls.jar the web applications WEB-INF/lib directory - in case of Tomcat from ./webapps/<your web app>/WEB-INF/lib and move them to a web application container shared library directory (for Tomcat this would be shared/lib).

I am confused with the three different formats of the proxy certificate used in the Globus Toolkit. Which one should I use?

The Globus Toolkit contains three different formats of the proxy certificate

  • 1. Globus "legacy" proxy available prior to GT v2.*
  • 2. Globus proxy (non rfc-compliant) available in GT v2.* and later releases
  • 3. RFC-compliant proxy available in GT v4.* http://www.faqs.org/rfcs/rfc3820.html

Pre-GT2.* servers require the Globus legact proxy. Servers in GT2.* and beyond work only with the non-rfc-compliant proxy. Servers in GTv4 work with both, rfc-compliant and non-compliant proxies.

The Different types of proxy certificates can be generated as follows:

$> grid-proxy-init
  1) with -old - generates 'globus legacy proxy' (since gt2)
  2) with -rfc - generates 'rfc complaint proxy' (since gt4)
  3) without any switch - generates 'pre-rfc proxy' (since gt3)

How can I avoid that Tomcat bindings are strongly bound to gridmap authorization?

A support for https connector for Tomcat 4 and 5 is now added. The code can be found at:

cvs -d :pserver:anonymous@cvs.cogkit.org:/cvs/cogkit co src/jglobus-fx/gsi

The https connectors do not perform any authorization on socket level.

See: http://bugzilla.globus.org/bugzilla/show_bug.cgi?id=1733

Programming

Nothing seems to work. I can not even get the simple examples working!

The Java CoG Kit uses abstractions above the Grid middleware. Hence it can only be as capable as the underlying software. Many times we get questions that are related to an issue in the installation of specific middleware or bugs of the middleware. We try our best to identify these issues and forward them to the appropiate support group. Please remember that not all bugs you see are bugs from the Java CoG Kit. Hence, it is of utmost importance that you file proper bug reports through our bugzilla system. In case you have questions with respect to the Java CoG Kit feel free to post your queries to the java@globus.org mailing list. Please DO NOT cross post these queries to discuss@globus.org and developer-discuss@globus.org

I use Mac OS X and now there are .DS_store files everywhere. How do I get rid of these?

To recursively remove these files from the current working directory type the following from the command line:

find . -name .DS_Store -exec rm {} \;

How do I load resources from the filesystem and where should I put these resources?

MPI

How can I submit an MPI job through the Java CoG Kit?

Examples are available in the MPI Guide

How can I submit an MPICH-G job through the Java CoG Kit?

Examples are available in the MPI Guide

Abstractions

What is a provider?

In order to provide cross-platform compatibility between various Grid implementations like GT v2.4.3, GT v3.0.2, GT v3.2.*, GT v3.9.*, and Condor, the Java CoG Kit abstractions offer a suite of generic Grid constructs that are agnostic to implementation-level details. These abstractions provide generic Grid functionality to execute remote jobs, transfer files, and perform file operations irrespective of the Grid implementation technology used. The Java CoG Kit abstraction model follows a "Task-Provider" pattern where the abstract and generic tasks specified by Grid programmers are translated into implementation-specific entities by a technology provider. Thus, to translate the generic task to be executed on a GT v4 Grid resource we need a GT4 provider.

Current release of the Java CoG Kit (v4_0_a1) contains GT v2.4.3, GT v3.0.2, GT v3.2.0, GT v3.2.1, SSH, and local providers for job submission tasks; and GridFTP, FTP, SSH, WebDAV, and local providers for file transfer and file operation tasks.

Does the Java CoG Kit have a provider for GT 2?

At present we have a provider for GT2. You can find the code and examples in the CVS

The Globus Toolkit v2 components are distributed as part of GT3.2.1 (referred to as pre-ws components).

Does the Java CoG Kit have a provider for GT 3?

At present we have a provider for GT3.0.2, 3.2.0, and 3.2.1. You can find the code and examples in the CVS at GT3.0.2, GT3.2.0, and GT3.2.1

Please note that the versions 3.0.2 and 3.2.0 are no longer supported by the Globus project. We strongly recommend that you use GT3.2.1. Hence, we will only devote minimal support for the 3.0.2 and 3.2.0 providers.

Does the Java CoG Kit have a provider for GT 4?

At present we have a prototype provider for GT3.9.5. To use it, please use as provider "GT4.0.0" and not "GT3.9.5". As soon as GT4.0.1 will be released and no major changes are expected we will upgrade to GT4. At that point we wil no longer support GT3.9.5. You can find the code and examples in the CVS

What is the approximate date the GT4 provider in Java CoG kit be released?

To support GT4 the following bugs need to be fixed first. We hope this is scheduled for GT4.0.1

3504 3507

Does The Java CoG Kit have a provider for Condor?

At present we have a prototype provider for Condor. We would like to hear feedback from you on the use of this provider. CVS. The Condor provider is not distributed in the Java CoG Kit 4_0_a1 release. It is only available in the CVS.


Can I use the Java CoG Kit task model to invoke simple web services?

We have a developed a prototype provider (referred as the WS provider) for invoking simple Web services using the Java CoG Kit task model. The current implementation allows invocation of remote methods with only String parameters. Source code demonstrating its usage is available in the CVS in the file ServiceInvocation.java

Furthermore, you can use the cog-ws-invoke command as follows:

bash-2.05b$ ./cog-ws-invoke -h
Usage:
 cog-ws-invoke <options>
   (-service | -s) <ws-location>
     Remote Web service location
   (-method | -m) <name>
     Method name
    [(-arguments | -args) <string>]
     Arguments. If more than one, use quotes
   [(-verbose | -v)]
     If enabled, display information about what is being done
   [(-help | -h)]
     Display usage 

The example given below invokes the "isUserOnline" method on the given Web service with the argument "cog-user". The method returns "true" if the yahoo user "cog-user" is online.

./cog-ws-invoke  -s http://demo.wsabi.org/axis/services/YahooUserPingService 
                 -m isUserOnLine 
                 -args "kaizamin"
DEBUG [org.globus.cog.abstraction.examples.invocation.ServiceInvocation]  - Task Identity: 
urn:cog-1130774719893
DEBUG [org.globus.cog.abstraction.examples.invocation.ServiceInvocation]  - Status changed to Submitted
DEBUG [org.globus.cog.abstraction.examples.invocation.ServiceInvocation]  - Status changed to Active
DEBUG [org.globus.cog.abstraction.impl.invocation.ws.InvocationThread] - Reading WSDL document from  
'http://demo.wsabi.org/axis/services/YahooUserPingService?wsdl'
DEBUG [org.globus.cog.abstraction.impl.invocation.ws.InvocationThread] - Executing operation 
isUserOnLine
DEBUG [org.globus.cog.abstraction.examples.invocation.ServiceInvocation]  - Status changed to Completed
Job completed
true

I like to switch the Grid middleware, what do I have to do?

Here is the big advantage of the Java CoG Kit. This is actually quite easy. All you have to do is change your provider version number and most of the features will be available to you without change.

Hence, if you like to switch between GT version 2 to GT version 4, all you have to do is change the provider variable from "GT2" to "GT4" and provide appropriate contact strings for the services.

How can I write my own provider?

How can I use ssh as part of my Grid?

The Java CoG Kit abstractions have an SSH provider. This allows ssh-enabled remote job executions and scp-enabled file transfers. The SSH provider supports two modes of security: PublicKey authentication and Password authentication.

Can I test the Java CoG Kit abstractions without installing any Grid software?

Yes. The Java CoG Kit offers a local provider that enables job execution, file transfer, and file operations on your local machine. The local provider does not require installation of any Grid software. It also does not require any additional security configurations.

I do not get any notification messages or status updates for my submitted tasks. What is wrong?

There could be two reasons for this.

Firstly, a firewall protecting your domain might be blocking all incoming requests, thereby rejecting ALL notification and status update messages from the server. Please check with your system administrator to verify this is not the case.

Secondly, the IP configurations might not be set correctly in the cog.properties file. The IP setting in the ~/.globus/cog.properties file MUST reflect the IP address of your client machine. If you are using a VPN client, then the IP address must reflect the IP of the vpn client. Additionally, for GT3* providers you need to explicitly set the IP of the vpn client interface: ifconfig cipsec0(or your vpn client interface) <IP address of the vpn client>

To verify that the problems you are getting are due to the lack of notification updates you can always test your programs with the local provider. The local provider does not involve any external domains.


How can I execute a remote job and redirect the output to my local machine?

The following code snippet may give you an indication how to do this.

// Create a JobSpecification
JobSpecification spec = new JobSpecificationImpl();
// Add all the required parameters like executable etc. 
spec.setExecutable(/bin/executable);
// Redirect the output to the local machine
spec.setRedirected(true);
// On the local machine, the if output needs to be redirected to the 
// cog-abstractions-output.txt file, use the spec.setStdOutput() method. 
//  If this method is not used (or if null passed as file name), the 
//  output on local machine is not redirected to any file. However, it is 
//  available via the task.getStdOutput() method.
spec.setStdOutput("cog-abstractions-output.txt");
// set the specification for the task
this.task.setSpecification(spec);

The Task and JobSpecification interfaces both have get- and setStdOutput methods. Which ones should I use?

The setStdOutput(fileName) method in the JobSpecification interface is used to instruct the remote Grid service that the output of the given task is to be redirected to the fileName. If JobSpecification.setRedirected(true), then the fileName is redirected to the local machine. And if the JobSpecification.setRedirected(false), then the fileName is a file on the remote machine.

If JobSpecification.setRedirected(true) then the output of the corresponding task is always redirected to the local client machine. Now if the JobSpecification.setStdOutput(fileName) method is not invoked or if JobSpecification.setStdOutput(null), then the redirected output is available via the Task.getStdOutput() method.


How can I get notified when the stdout of the task changes?

One way to get notified if the stdout associated with the task (Task.getStdoutput()) is updated is via the task.addStatusListener() method. When the status changes to COMPLETED, you can be assured that the task.getStdoutput() method will return the final output.

However, this mechanism might not be practical in all scenarios. Consider the following: A task takes 3 days to COMPLETE, however the first set of output is produced within the first 10 minutes. It is not practical to wait till the end of 3 days to get all the output at once.

To subscribe to the task output events, you can register yourself as the output listener (OutputListener) of the task with the task.addOutputListener() method. The OutputListener interface contains the outputChanged() method which will be invoked every time the output of the task changes.

How do I specify attributes that are not available in the JobSpecification but supported by the remote execution service?

The JobSpecification represents the most generic job specification parameters supported by ALL Grid execution services. However, there are several attributes that are specific to some Grid implementations. For example, job attributes such as "count", "maxMemory", "maxCpuTime", etc are supported by GT2 execution service but not necessarily supported by an SSH, Condor, or Unicore services.

However, application programmers can add these implementation-specific attributes to the JobSpecification using the JobSpecification.setAttribute() method.

spec.setAttribute("count",count);
spec.setAttribute("maxMemory",maxMemory);
spec.setAttribute("maxCpuTime",cpuTime);

// The handling of environment variables will be supported 
// in the JobSpecification interface in our next release.
spec.setAttribute("environment","(env1 value1) (env2 value2)");

The Java CoG Kit abstraction framework simply translates these attributes into native GT2 RSL and submits it to the remote execution service.

What attributes can be specified in this fashion is entirely dependent on the remote execution service. The Java CoG Kit abstraction framework does not define nor imposes any restrictions on the name and value of these attributes. So the programmer is free to specify the following:

spec.setAttribute("foo",bar);

However, if the remote service does not support the "foo" attribute, it will ignore it or generate an error.


What is the difference between the GT2 provider and the GT2FT provider?

The GT2 provider allows you to submit jobs to the Globus Toolkit v2.4.3/pre-ws/pre-wsrf enabled resources. Once a job is submitted to the remote resource, it is expected that the client machine maintains its network connection to the Globus resource. The GT2 provider offers no support to reconnect the client to an existing job on the remote resource. While this mechanism is satisfactory for short jobs, it poses a considerable overhead (in terms of connection maintenance) for long jobs.

Therefore, the Java CoG Kit abstractions module also offers a GT2FT (GT2 Fault Tolerant) provider, that extends the GT2 provider to support fault tolerance in long jobs.

The GT2FT provider offers all the functionality of the GT2 provider. Additionally, it allows the client to checkpoint the jobs submitted to the remote resource, disconnect from the remote resource, and at a later time reconnect to the existing job by submitting the saved checkpoint.

How can I use the fault tolerant features of the GT2FT provider?

Lets assume you want to execute a long job to a Globus v2.4.3 resource spanning a period of 2 months. The GT2FT provider supports such a use case in 3 different ways:

1. Submit the job
2. Checkpoint the submitted job
3. Reconnect to the submitted job  

To demonstrate this functionality, we use the cog-job-submit command-line launcher.

bash-2.05b$ cog-job-submit -help
Usage:
 cog-job-submission <options> 
   [(-name | -n) <taskName>]
     Task name 
   (-service-contact | -s) <host>
     Service contact 
   [(-provider | -p) <provider>]
     Provider; available providers: [gt2ft, gsiftp, file, gt4.0.0, 
     gt3.0.2, ssh, gridftp, local, http, gt3.2.1, gt2, gt3.2.0, ftp, 
     webdav] 
   (-executable | -e) <file>
     Executable 
   [(-arguments | -args) <string>]
     Arguments. If more than one, use quotes 
   [(-environment | -env) <string>]
     Environment variables for the remote execution environment, 
     specified as "name=value[,name=value]" 
   [(-directory | -d) <string>]
     Target directory 
   [(-batch | -b)]
     If present, the job is run in batch mode 
   [(-redirected | -r)]
     If present, the arguments to -stdout and -stderr refer to local 
     files 
   [-stdout <file>]
     Indicates a file where the standard output of the job should be 
     redirected 
   [-stderr <file>]
     Indicates a file where the standard error of the job should be 
     redirected 
   [(-attributes | -a) <string>]
     Additional task specification attributes. Attributes can be 
     specified as "name=value[,name=value]" 
   [(-checkpoint | -c) <fileName>]
     Checkpoint file name. The task will be checkpointed to this file 
     once submitted 
   [(-verbose | -v)]
     If enabled, display information about what is being done 
   [(-help | -h)]
     Display usage


First, we submit a job that takes 2 months (/home/user/simulation) to a remote resource and checkpoint it once it is submitted. This command will submit the Grid task to the GT2FT provider to be executed on hot.anl.gov. It will also checkpoint the task to the file checkpoint.xml

bash-2.05b$ ./cog-job-submit 
             -p gt2ft 
             -s hot.anl.gov 
             -e /home/user/simulation 
             -args " -i 20 -s 2" 
             -c checkpoint.xml
DEBUG [org.globus.cog.abstraction.examples.execution.JobSubmission]  - 
  Task Identity:urn:cog-1116756045868
DEBUG [org.globus.cog.abstraction.impl.execution.gt2ft.JobSubmissionTaskHandler]  - 
  RSL: &(executable=/home/user/simulation)(arguments=-i 20 -s 2)
DEBUG [org.globus.cog.abstraction.examples.execution.JobSubmission]  - 
  Status changed to Submitted
Task checkpointed to file: checkpoint.xml

Once the task is checkpointed, the client can disconnect its network connection with the remote resource. Assuming the task has been executing for a month after its initial submission, the user wants to reconnect to its submitted job.

bash-2.05b$ ./cog-checkpoint-submit -c checkpoint.xml
DEBUG [org.globus.cog.abstraction.impl.execution.gt2ft.JobSubmissionTaskHandler]  - 
  Task binding successful
DEBUG [org.globus.cog.abstraction.impl.execution.gt2ft.JobSubmissionTaskHandler]  - 
  Task identity:urn:cog-1116756045868
DEBUG [org.globus.cog.abstraction.impl.execution.gt2ft.JobSubmissionTaskHandler]  - 
  Previous status = Submitted
DEBUG [org.globus.cog.abstraction.examples.xml.XML2Task]  - 
 Status changed to Completed 

The task checkpoint in checkpoint.xml is as follows:

bash-2.05b$ cat checkpoint.xml
<?xml version="1.0"?>
<task>
   <identity>1116756507318</identity>
   <name>myTask</name>
   <type>Job Submission</type>
   <serviceList>
       <service>
           <identity>1116756507319</identity>
           <provider>gt2ft</provider>
           <type>Job Submission</type>
           <serviceContact>hot.anl.gov</serviceContact>
       </service>
   </serviceList>
   <specification>
       <JobSpecification>
           <executable>/home/user/simulation</executable>
           <arguments>-i 20 -s 2</arguments>
           <batchJob>false</batchJob>
           <redirected>false</redirected>
           <localExecutable>false</localExecutable>
       </JobSpecification>
   </specification>
   <attributeList>
       <attribute name="globusid" value="https://hot.anl.gov:50001/28882/1116756633/"/>
   </attributeList>
   <status>Submitted</status>
   <submittedTime>2005-06-01T05:08:32.496</submittedTime>
</task>

How can I check the status of my checkpointed job?

Assume you have submitted a job using the GT2FT provider and checkpointed it to a file called checkpoint.xml FAQ. You can check the status of this job at any time using the cog-checkpoint-status command.

bash-2.05b$ ./cog-checkpoint-status -c checkpoint.xml 
DEBUG [org.globus.cog.abstraction.impl.execution.gt2ft.JobSubmissionTaskHandler]  - 
  Task binding successful
DEBUG [org.globus.cog.abstraction.impl.execution.gt2ft.JobSubmissionTaskHandler]  - 
  Task identity:urn:cog-1117806344876
DEBUG [org.globus.cog.abstraction.impl.execution.gt2ft.JobSubmissionTaskHandler]  - 
  Previous status = Submitted
Current Status: Active

How can I submit jobs to the TeraGrid?

Jobs can be submitted to the TeraGrid using the cog-job-submit command. The submission process is similar to any other remote resource with one exception. All submissions to the TeraGrid must be accompanied by the TeraGrid account number. The example below submits the /home/user/simulation executable to the PBS jobmanager on the TeraGrid using the GT2 provider.

bash-2.05b$ ./cog-job-submit
-p gt2
-s tg-grid1.uc.teragrid.org/jobmanager-pbs
-e /home/user/simulation
-args " -s 2 -i 20"
-attributes "count=2, project=<your tera grid account number>"
-stdout "cog-out"
-stderr "cog-error"
DEBUG [org.globus.cog.abstraction.examples.execution.JobSubmission]  - 
  Task Identity: urn:cog-1117352441775
DEBUG [org.globus.cog.abstraction.impl.execution.gt2ft.JobSubmissionTaskHandler]  - 
  RSL: &(executable=/home/user/simulation)(arguments=-s 2 -i 
  20)(stdout=tera-out)(stderr=tera-error)(count=2)(project=<account number>)
DEBUG [org.globus.cog.abstraction.examples.execution.JobSubmission]  - 
  Status changed to Submitted  
DEBUG [org.globus.cog.abstraction.examples.xml.XML2Task]  - 
  Status changed to Active
DEBUG [org.globus.cog.abstraction.examples.xml.XML2Task]  - 
  Status changed to Completed
bash-2.05b$

How can I submit fault tolerant jobs to the TeraGrid?

Fault tolerant jobs can be submitted to the TeraGrid using the GT2FT provider and the checkpoint option.

bash-2.05b$ ./cog-job-submit
-p gt2ft
-s tg-grid1.uc.teragrid.org/jobmanager-pbs
-e /home/kamin/goWorld
-args " -s 2 -i 20"
-attributes "count=2, project=*****"
-stdout "terra-out"
-stderr "terra-error"
-c checkpoint.xml
DEBUG [org.globus.cog.abstraction.examples.execution.JobSubmission]  - 
  Task Identity: urn:cog-1117352441775
DEBUG [org.globus.cog.abstraction.impl.execution.gt2ft.JobSubmissionTaskHandler]  - 
  RSL: &(executable=/home/user/simulation)(arguments=-s 2 -i 
  20)(stdout=terra-out)(stderr=terra-error)(count=2)(project=*****)
DEBUG [org.globus.cog.abstraction.examples.execution.JobSubmission]  - 
  Status changed to Submitted
Task checkpointed to file: checkpoint.xml

One the task is checkpointed, the client can disconnect from the remote resource. At a later time, the client can reconnect to the submitted job using the saved checkpoint.

bash-2.05b$ ./cog-checkpoint-submit -c checkpoint.xml
DEBUG [org.globus.cog.abstraction.impl.execution.gt2ft.JobSubmissionTaskHandler]  - 
  Task binding successful
DEBUG [org.globus.cog.abstraction.impl.execution.gt2ft.JobSubmissionTaskHandler]  - 
  Task identity:urn:cog-1117352441775
DEBUG [org.globus.cog.abstraction.impl.execution.gt2ft.JobSubmissionTaskHandler]  - 
  Previous status = Submitted
DEBUG [org.globus.cog.abstraction.examples.xml.XML2Task]  - Status changed to Active
DEBUG [org.globus.cog.abstraction.examples.xml.XML2Task]  - Status changed to Completed
bash-2.05b$

The checkpoint is saved in XML format as follows:

bash-2.05b$ cat checkpoint.xml
<?xml version="1.0"?>
<task>
   <identity>1117352441775</identity>
   <name>myTask</name>
   <type>Job Submission</type>
   <serviceList>
       <service>
           <identity>1117352441776</identity>
           <provider>gt2ft</provider>
           <type>Job Submission</type>     
           <serviceContact>
             tg-grid1.uc.teragrid.org/jobmanager-pbs
           </serviceContact>
       </service>
   </serviceList>
   <specification>
       <JobSpecification>
           <executable>/home/user/simulation</executable>
           <arguments>-s 2 -i 20</arguments>
           <stdOutput>terra-out</stdOutput>
           <stdError>terra-error</stdError>
           <batchJob>false</batchJob>
           <redirected>false</redirected>
           <localExecutable>false</localExecutable>
           <attributeList>
               <attribute name="count" value="2"/>
               <attribute name="project" value="ACCOUNT_NUMBER"/>
           </attributeList>
       </JobSpecification>
   </specification>
   <attributeList>
       <attribute name="globusid" value="https://tg-grid1.uc.teragrid.org:53166/2259$
   </attributeList>
   <status>Submitted</status>
   <submittedTime>2005-05-29T02:40:52.103</submittedTime>
</task>

How do I prepare an execution flow with the abstractions framework?

Lets assume a simple execution flow:

1. Transfer a local Java file to the remote machine
2. Compile the the Java file on the remote machine
3. Run the compiled class file on the remote machine, while redirecting the output to the local machine.

Such an execution flow can be conveniently modeled as a task graph. An example code for this execution flow is available in the CVS. A command line launcher for this example is available at $COG_HOME/src/cog/dist/cog-4*/bin/examples under the name cog-compile-execute.

bash-2.05b$ ./cog-compile-execute -h
Usage:
 cog-compile-execute <options> 
   (-src | -s) <filename>
     Name of the Java source file  
   (-javaHome | -j) <path>
     Path of the Java environment on the remote machine 
   [(-classpath | -cp) <path>]
     Path of the Java classpath on the remote machine 
   (-execution-provider | -ep) <provider>
     Execution provider; available providers: [gt2ft, gsiftp, file, 
     gt4.0.0, gt3.0.2, ssh, gt4ft, gridftp, local, gsiftp-old, http, 
     gt3.2.1, gt2, gt3.2.0, gridftp-old, ftp, webdav] 
   (-transfer-provider | -tp) <provider>
     Transfer provider; available providers: [gt2ft, gsiftp, file, 
     gt4.0.0, gt3.0.2, ssh, gt4ft, gridftp, local, gsiftp-old, http, 
     gt3.2.1, gt2, gt3.2.0, gridftp-old, ftp, webdav] 
   (-service-contact | -sc) <host>
     Service contact for the remote host 
   [(-verbose | -v)]
     If enabled, display information about what is being done 
   [(-help | -h)]
     Display usage 

Assume the sample Java source file is as follows:

public class HelloWorld {
   public static void main(String[] args) {
       System.out.println("Hello Cog Kit World");
   }
}

Further, let the resource hot.anl.gov host a GridFTP and a GRAM execution service. Also let the path to JAVA_HOME on hot.anl.gov be /home/path_to_java/

bash-2.05b$ cog-compile-execute 
-s /home/cog/HelloWorld.java 
-j /home/path_to_java/ 
-ep gt2 
-tp gridftp  
-sc hot.anl.gov 

DEBUG [org.globus.cog.abstraction.examples.taskgraph.RemoteCompileExecute]  - 
  Status of Transfer_Task changed to: Submitted
DEBUG [org.globus.cog.abstraction.examples.taskgraph.RemoteCompileExecute]  - 
  Status of Transfer_Task changed to: Active
DEBUG [org.globus.cog.abstraction.examples.taskgraph.RemoteCompileExecute]  - 
  Status of Transfer_Task changed to: Completed

DEBUG [org.globus.cog.abstraction.examples.taskgraph.RemoteCompileExecute]  - 
   Status of Compile_Task changed to: Submitted
DEBUG [org.globus.cog.abstraction.examples.taskgraph.RemoteCompileExecute]  - 
  Status of Compile_Task changed to: Active
DEBUG [org.globus.cog.abstraction.examples.taskgraph.RemoteCompileExecute]  - 
  Status of Compile_Task changed to: Completed

DEBUG [org.globus.cog.abstraction.examples.taskgraph.RemoteCompileExecute]  - 
  Status of Execution_Task changed to: Submitted
DEBUG [org.globus.cog.abstraction.examples.taskgraph.RemoteCompileExecute]  - 
  Status of Execution_Task changed to: Active
DEBUG [org.globus.cog.abstraction.examples.taskgraph.RemoteCompileExecute]  - 
  Status of Execution_Task changed to: Completed

Output = Hello Cog Kit World


When I submit a task as a batch task, I get the status notification as COMPLETED even though the remote job has not completed its execution. Why?

Semantically, a batch job (in Globus terminology) is one where the client does not wait actively for the status notifications from the server. Hence, from a client's perspective, it is considered to be COMPLETED once its submission is successful. To wait for the remote task to complete and actively receive status notifications, please submit the task as a non-batch task.

I get a java.lang.LinkageError or other class loader errors when using the GT4 provider. Why?

You are probably adding the provider jars to your classpath before starting the application. That interferes with the CoG class loading mechanism for providers. You should only add the jars from cog/lib to the classpath, and not jars from any sub-directory of cog/lib or any other directory in CoG.

Versions after CoG 4.1.3 should display a clear error message when provider jars are added to the class path.

I'm getting "The GT4.0.0 provider does not support redirection" error message. Why?

The GT4 provider does no support redirection at this time. Redirection is enabled with the "-r" switch to cog-job-submit or the "redirect=true" to execute in Karajan.

How do I integrate the CoG Jars into my own project?

  1. The jar files in lib should be placed in the lib directory of your project.
  2. The jar files in lib-<provider-name> must NOT be placed in the CLASSPATH
  3. The location of the lib-<provider-name> relative to the CoG jar files in the lib directory (in particular the cog-provider-clref-<provider-name>) must not change.

Note about Tomcat: The CoG libraries (provided the above conditions are met) should be placed in the shared directory.

Workflow

Which concepts exist in Java CoG Kit to conduct Grid workflows?

We have a number of concepts:

  • Workflows defined on an API level based on the Java CoG Kit abstractions
  • Workflows defined as XML documents
  • Workflows implicitly defined as part of the Grid shell

Why do you give us so many choices?

No one choice can be perfect for everything. Some are better suited for a particular problem for which the others are not.

Which system is the right one to use?

How do you increase the debug level while using Karajan?

You can get detailed Java exceptions of faults that cause the whole workflow to fail in the following way:

Add the following line to your

cog/dist/cog-4_0_a1/etc/log4j.properties
log4j.logger.org.globus.cog.karajan.workflow.ExecutionContext=INFO

Now re-run your application. If it fails, a detailed exception will be printed on the console.

How can I run a simple Java program on a remote machine in the Grid?

Please check the example in the CVS

Karajan does not work from Tomcat. Why?

Note: this applies to Karajan 0.31

Karajan has a thread management system which limits the amount of threads that can be created. When used from Tomcat, the maximum number of threads is already reached, so Karajan cannot create any new threeads.

The solution is to increase the maximum number of threads that the manager can create. This is done using the TMMaxThreads system property. You can do the following before starting Tomcat:

 Bash: export JAVA_OPTS="-DTMMaxThreads=256"
 Windows: set JAVA_OPTS="-DTMMaxThreads=256"

Grid Shell

Grid Desktop

File Transfer

Which version number of the GridFTP serveres is corelated with which version of GT?

gridftp server        gt
1.12 or lower         2.4.3 or lower
1.17                  3.2.0, 3.2.1
2.0                   4.0.0
2.1                   4.0.1


MLSD is supported since version 3.2 supports.

We recommend to use the 2.1 server as 2.0 does not support the list function.

What are the biggest changes between GT2 and GT4 File transfer client libraries from jglobus?

There are some changes in the semantics between the wu based gridftp server used in GT2 server or our new one used in GT4.

LIST command change: One important cahnge is related to the LIST command. The wu gridftp server could accept all the switches like the ls command does, e.g. -d *. But the new one does not. The RFC specification only talks about a single argument to the LIST command, and not about any switches, which makes the new sever more specification compliant.

Ordering of commands: Another change is that some implied ordering to the commands exist that relate to setting up the parameters of the data channel. For example, you can do MODE S, TYPE I, PORT, RETR but you cannot do PORT, MODE S, TYPE I, RETR. Doing MODE, TYPE commands after issuing the PORT command has no effect on the server (the server will assume the MODE, TYPE settings before the PORT command). If you do that on the client, basically the client & server get out of synchronization base don the internal ordering that is implied by the server.

list() or mlsd()?

In case you use jglobus you can get a list of directories for GridFTP servers of version 1.1.7 (GT3.2) while using the list() function. If you use GridFTP 2.0 and newer, just use mlsd() with 2.0.

Now you may ask why have you not implemented the list function to use mlsd internally? Yes we could do that, but at this time we have not implemented as it is easy enough for a user to do and because the GridFTP client librarie is intended to work akainst a protocol. As the list supports the protocol LIST and mlsd supports MLSD we recommend that you use a hihger level library function form the Java CoG Kit. You can help us by letting us know your requiremnets so we start developing such an abstraction.

I am having difficulties with my GridFTP servers, can I upgared to the GT4.0 server only?

Although this question is not realy related to the Java CoG Kit, we have seen in the past compatibility issues with various versions of the GridFTP servers. Hence at this time we encourage those that are in the condition to do so to upgarde to the GT4.0 version of the GridFTP server on a test system and evaluate its feasibility to replace older GridFTP servers. To do so you can conduct the following steps:


1. download the all-source-installer from www.globus.org/toolkit and untar it

cd gt"version"-all-source-installer
export GLOBUS_LOCATION=/path/where/you/want/to/install
./configure --prefix=$GLOBUS_LOCATION

2. if you are on a single processor machine than

make gridftp postinstall

if you are on a multi-processor machine and want a parallel build

make gpt globus_core globus_core-thr // can't build parallel
make gridftp -j#  //# is number of procs
make postinstall  //can't build parallel

3. source the environment

source $GLOBUS_LOCATION/etc/globus-user-env.sh

The source will be in "download-directory"/gt"version"-all-source-installer/source-trees/gridftp

The server executable will be in $GLOBUS_LOCATION/sbin

The globus-url-copy executable will be in $GLOBUS_LOCATION/bin

You do end up with the source code tree for the entire toolkit. You will build GridFTP tools and some useful security utilities like SimpleCA. If you literally want just the server, do

make gridftp-server 

instead. They can use existing security setup. Just copy the xinetd config and give it the correct $GL and path and then any command line options they want.

Help us improving this documentation if you find issues with it.

JGlobus

Extra / in RSL

The following RSL

&(rsl_substitution=(GLOBUSRUN_GASS_URL https://yourhost:yourport))(executable=$(GLOBUSRUN_GASS_URL)/home/gregor/COMMAND)(stdout=$(GLOBUSRUN_GASS_URL)/dev/stdout-1005091586356)(stderr=$(GLOB USRUN_GASS_URL)/dev/stderr-1005091586356)

needs an extra / in front of /home. so it shoud read //home instead.

Can I test a simple Grid program without installing Globus?

The Java CoG Kit JGlobus library contains a personal gatekeeper and personal GASS server that are written in 100% Java. The Personal Gatekeeper is not production quality, but we have gotten good feedback from the climatology community that used this service even on their supercomputers to develop Grid applications.

There is information about this in the old Java CoG Kit manual, as well as on a page described by a community member at Personal Grid.

See also the ssh provider

Can I use the Karajan workflow to interact with personal gatekeepers?

Yes, you can, but we have not provided an example. If you like to develop a tutorial on how to do that, please forward it to us.

What are the limitations of the personal gatekeeper?

The gatekeeper is run with your own credentials and should not be run under root. We recommend to use it for testing purposes and switch to a Grid execution service provided by other Grid middleware. Additionally, the protocol used in the personal gatekeeper is only portable to Globus Toolkit protocol 2.0. However it shoudl still be able to use it with the classical Grid services in GT3 and GT4.

Why is the term personal gatekeeper used?

Some in the Globus community use the term "gatekeeper" incorrectly. Often they refer to it but actually maen a "personal execution service". The gatekeeper is just one of the components of the personal execution service that governs the authentication to it. To not confuse the user we use the same terminology as used in the Globus mailinglists. However, the correct terminology should be personal execution service.

How can I enable sockets to use GSI

We do have an example in the jglobus source tree at src/org/globus/gsi/gssapi/net/example

Why do I get a "org.globus.gram.GramException: The connection to the server failed (check host and port)" when I try to poll the status of a GramJob?

Once the GramJob finishes, the job manager process (in GT v2.*) dies and the client won't be able to connect anymore to check on the status. Thus any calls to

Gram.jobStatus()

will return the following error:

org.globus.gram.GramException: The connection to the server failed (check host and port) 
 [Root error message: Connection refused].  
 Root exception is java.net.ConnectException: Connection refused

Unfortunately, this is how the GT v2.* job manager behaves. It must be handled as a special case by the client application. One way to do this is to assume that a "connection refused exception" indicates that the job is either completed or failed and handle it as "DONE".

Note: This problem has been rectified in GT v4.* job managers. On completion of the job, they return the status as "Completed" instead of throwing an error.

Output redirection through the Gass server is not working. What is wrong?

The most likeley problem here is the way you are specifying the URL of the local file. For example, the complete URL of the local file is:

<gass-url>/<absolute-path-of-local-filename>

Thus, if the Gass URL is https://192.168.0.100 and if the absolute path of local file is /tmp/test, then the complete URL of local file is https://192.168.0.100//tmp/test

The most common mistake is to specify the URL as https://192.168.0.100/tmp/test (without the "//")

How to create a remote gass server process on a Windows PC?

see the tip doicumented at: http://bugzilla.globus.org/globus/show_bug.cgi?id=882

Community

How can I participate?

contact gregor@mcs.anl.gov with a mail that has the subject prefix

"COGKIT PARTICIPATION REQUEST: "

and describe what you like to do.

I like to write a undergrad, graduate, or PhD thesis in Grid computing, Is it possible to have a committee member from ANL?

Contact gregor@mcs.anl.gov with an E-mail with the headline

"COGKIT THESIS REQUEST"

and describe what you like to do or want to find out.

  • Engagement of your advisor: It is best to engage your advisor at your home school in this discussion.
  • Resume: In order to identify a good thesis, it would be necessary to prepare a resume with detailed information about your computer science and other science abilities. We are not relay interested if you worked at McDonalds, but if you for example have provided software outside of school to a project please let us know. The resume can be multiple pages long and should include a list of references in case you have them available. For an example you may want to take a look at Gregors resume PDF.
  • School requirements: You should also provide us with a link or an e-mail description about what requirements your thesis need to fulfill. This will include formalisms set by the school, format and deadlines.
  • Thesis work must be conducted in English or German.
  • In case your thesis must be handed in in any other language, it will be necessary to write a paper in English about the contents of the thesis which is to be submitted to a conference. The paper will be coauthored by Gregor and your advisor.
  • Grade expectations: In case your school gives grades, the grade expectations will be matched with those from your advisor. In general those that write a thesis AND get their thesis paper accepted in a conference or Journal will usually get an A or A+. In case it is a workshop it will be an A-. Those that do not write a paper need to make sure that the thesis is real impressive to get an A+, this is actually not that difficult. PhD students are required to write multiple papers. Precondition to the passing of the PhD thesis is that multiple papers are submitted for review or have been accepted by peer reviewed conferences and journals.
  • Bibliography: The bibliography is supposed to be maintained in bibtex. xemacs or JabRef will be helpful tools. In case you need to use other bibliography tools, please consult with Gregor.

I am a researcher and like to visit ANL in the summer. How can I do this?

  • Funding opportunity: Find two or three undergraduates and apply as an NSF FaST team which pays for the stay. Please contact Gregor.
  • You have money: In case you have funding and you can directly contact gregor@mcs.anl.gov

I am a foreign student and like to spend the summer at ANL. How can I do this?

You need to provide your own funding for housing, living cost, and flight. It will take at least six month to straighten out the visa procedures.

I would like to develop code that I want to contribute to the CoG Kit. How can I do this?

The best way would be if you use the Java CoG Kit module concept. You can develop your own module. This way it will be easy to integrate. We recommend that you discuss your plans of what you like to do with gregor@mcs.anl.gov.

I use cygwin and like to contribute to the community CVS. but I do lose my file permissions

Cygwin can use Unix-like file permissions. To enable it one has to add "ntea" to the CYGWIN variable in the cygwin.bat file in the cygwin root directory, which is usually in C:\cygwin.

set CYGWIN=tty ntea
in C:\cygwin\cygwin.bat. 

You should also restrict access to your cygwin directory under windows to yourself and your system administrators only.

Web pages

Why is the Wiki so slow?

We have not yet enabled page caching.

The Globus Toolkit comes out with a new style, why is yours different

We do not have the time in looking into designing a style for our Wiki that would look like the new Globus Web page. We are looking for someone that can help us achieving this.


Answering questions on the mailinglist

How do I respond to a question that is already answered by an FAQ?

Simply go to the table view and right click on the question to open a new window in the browser. Past and copy the link into the e-mail and respond to the original poster.


I found a question on the mailinglist that is already answered by an FAQ. How

How do I add a new question?

Simply go to the appropriate section and add the question. make sure you use the appropriate number of equal signs to have the indentation correct. As it is possible to overwrite other peoples input, we strongly suggest that you use the edit feature on a section and not the whole page.

In case you need to add a new question simply edit the question directly above the question you like to add and add your question in that editor window. The wiki will reformat the page appropriately after you have added your question. We recommend that you use a spellchecker. If you see mistakes in other questions and answers, please help us correcting them.

Why are you not using FAQ-O-MATIC or a similar tool?

We are using this wiki, as it provides the least amount of overhead to us. As we like to focus on Grid computing we do not have the manpower to install other more convenient FAQ management tools.


TO BE INTEGRATED

Configuration

Programming

Where can I find some example programs ?

http://www.cogkit.org/viewcvs/viewcvs.cgi/src/cog/modules/abstraction-examples/src/org/globus/cog/abstraction/examples/


Personal tools
Collaboration and Jobs