Roadmap

From Java CoG Kit

Jump to: navigation, search

Contents

[edit]

Immediate Roadmap

If an individual has completed a task in a Phase, you can move to the next phase or help others to complete the phase.

[edit]

Immediate Goals

  • Workflow GUI - Deepti
  • Branko meeting for CMCS - Deepti
  • Java2Karajan - Chris
  • Java2Karajan (JNLP) - Chris
  • QStat PBS & qsub (jnlp) - Chris and Matt
  • QStat Cobalt and qsub (jnlp) - Chris and Matt 5.1 Karajan Element wrappers for Qstat and qsub based on (4) and (5) using (2) - Chris and Matt
  • Use the element wrappers to produce wrappers for task abstractions. - Chris and Matt
  • Fault Tolerance example (in a workflow)
  • Update on Jay Alameda (Also try to follow up with him twice next week) - Mike
  • Follow up with Al Rossi on the LEAD project
  • Update on VDL2
  • Update on the water project - running the water program through Karajan workflow on Teragrid - Matt (IMPORTANT) - contact Mike

- Demonstrate the water code through a karajan workflow on TG to Gregor

  • Get the Directory browser working - Matt
  • Get the Qstat for cobalt, LSF, PBS working
  • Improve the documentation on the previous items
  • set up the gridftp tests - Deepti
[edit]

Phase 1

The person at the end is ultimatly responsible that this task gets done. However, help from others can be useful and if determined that a responsibility should be switched, please update the assignments. The tasks are assigned with priorities.

[1] Krajan component Panel - Deepti and Mike (Deepti)
[2] Make sure the CMCS service is running, develop cron script that sends mail to mailinglist if cmcs service is down. Have a test every day. (Deepti)
[1] Save and load panel for Options in cobalt queue (Chris)
[2] JNLP of cobalt queue (Chris)
[1] Working demo of drirectory browser that does not crash and is usable (Matt)
[1] Karajan improvements to support Deepti (Mike)
[2] Karajan ussage of MPI pi program on Teragrid - Mike or Deepti (Mike)
[2] Karajan usage of a /bin/date to be executed on n machines of the TG - Mike or Deepti (Mike)
[2] Develop a tutorial that demonstrates how to execute a /bin/date and an mpi program. Include in the tutorial how the beginner can recognize that the program was executed on a TG backend node. and not on the frontend node. (Mike)
[3] Develop a wiki documentation tutorial as part of the krajan workflow that shows how to do the above two examples on TG (Deepti)
[3] Adding a mechanism to export karajan image workflows to a png from a commandline option without popping up the GUI.
[1] setting up the gridftp karajan test on teragrid (Deepti)

New Items to be addressed

[1] Matt Bones comment on connections (Mike)

Testing the Bug Template

[ ] Bug 201

[edit]

Phase 2

[1] Move the CVS to SVN - Gregor and Mike - trunk/4.1.5 (Mike)
[1] Update the documentation on the Web to now use the SVN instead of the CVS (Deepti)
[1] Update the globdev documentation (Gregor)
[1] Start on the Water project - Gregor and Mike (Mike)
[1] Demonstarte to Gregor how the water project code works with Karajan. mike has tarfile and Gregor wnats to know what to do with it and how to start it from karajan. (Mike)
[1] Create a svn placeholder for the water project trunk/water (Mike)
[2] Tha cog-workflow-gui needs amechanism to indicate the opened workflow. I suggest the title.

[edit]

Phase 3

[ ] Develop cogscript (Deepti)

[edit]

Roadmap Template

  • Description:
    • A description of the goal written in a way that can be user stood by end-users.
  • Benefits:
    • An explanation why this is important, making sure to include benefits to the users.
  • Interested users:
    • A list of interested users
  • Status:
    • One of: proposed, active, complete, or deferred.


This page will serve as a collection to determine a Roadmap for the upcoming month. The information here is just a start and will surely change within the next two weeks.

[edit]

Bugs

Assigned User Issues: Summaryoutstanding bugsmajor bugsaminhategangregordkodeboy
Assigned Developer Issues: Summaryoutstanding bugsmajor bugsaminhategangregordkodeboyrwinch now hategan
[edit]

Bugs

[ ] merge the two areas that link to bugs
[ ] some of the links are broken or query the wrong thing this needs to be fixed
Bugzilla List

By Priority: Blockers Critical Major Normal Minor Trivial Enhancement

By Owner: Amin Gregor Hategan Deepti Winch

Summary: By Owner

[edit]

Applications

[ ] Water application for SC, demonstarte on TG
[ ] Karajan on TG
[ ] Starting MPI programs with Karajan on TG
[ ] Compiling MPI programs on Karajan with TG
[ ] CMCS GUI
[ ] CMCS Service for multiple users
[ ] CMCS new data
[ ] OGCE move CVS to SVN

[edit]

Abstractions

Transition Kaizar's responsibilities to a member of the team (if not otherwise specified it will be Mike)

[ ] gt2-ft provider (Deepti)
[ ] file transfer/operation providers (gt2,webdav) (Deepti)
[ ] abstractions (Mike)
[ ] condor provider (Mike)
[ ] gt2 provider (Mike)
[ ] ws provider (Mike)
[ ] mpi provider (Mike)
[ ] brokering framework (Mike)
[ ] ad hoc Grid framweork (TBD)
[ ] Bug 4424
[ ] EDGEE Provider, Never started (Kaizar)
[ ] TUPELO Provider ,Mike and Community, Joe Futrelle Tupelo

[edit]

Abstraction: Broker Framework

[edit]

Queue Management Service/GUI

This is most likely superseeded by GridShell

[ ] Develop queue service (Deepti)
[ ] Develop swing component for queue (Deepti)

[edit]

Workflow

[edit]

Misc: new

[ ] SAVE/LOAD: add fuctionality to save and load workflows with coordinate layout.
[ ] FILE: close should be renamed to Quit. close may mean something different such
[ ] In the edit mode we should have a feature to edit the elements in the text mode, which displays the *.xml or *.k elements. On saving the changes should reflect in the *.xml or .k files too. This will be great while debugging long workflows rather having to go to the *.xml or .k files each time we fix something and load it back again.
[ ] Going to the list view and clicking on a row causes the following statements to appear on the command prompt everytime. I guess these can go org.globus.cog.gui.grapheditor.targets.swing.util.NodeComponentWrapper[,0,78,158
[ ] We need a debug window for Karajan. no text should go directly to the commandline. There should be at least an option. and by default the utput goes to a window.
[ ] Help -> About should also display the current version.
[ ] HELP->ABOUT: we should have in the about more text such as
[ ] HELP->ABOUT: we should have in the about more text such as
[ ] gvl: COMMAND LINE ARGUMENTS: the command line arguments should be similar between cog-workflow
[ ] LIST VIEW: When in list view there should be a way to display
[ ] ADDING NEW NODES: Under apple mac clicking a node does not
[ ] EXPORT: low priority, a PDF export would be cool.
[ ] TOOLBAR: Icons for the different views, List, Tree, Graph
[ ] TOOLBAR: the layout type should be visible in the toolbar as
[ ] TOOLBAR: On Mac. do graph view, than do tree view, this results in a bug where the. toolbar is wrongly rendered. the same happens to be when you switch between graph and list view.

[edit]

Karajan Workflow Service

[x] Provide a first GSI based workflow service
[ ] Provide Java WebStart for workflow service
[ ] Provide registry for workflow services
[ ] Provide interface to add users to Workflow Service, e.g. mapfile
[ ] Provide implementation based on Web services
[ ] Provide implementation based on Grid services
[ ] make sure long running jobs can be performed
[ ] A note on long running jobs: The architecture allows it, but a more clear set of requirements is needed in order to ensure that issues are addressed properly.

[edit]

Karajan Engine

[ ] integrate the <java> tag
[ ] integrate a <python> tag that starts the jython interpreter.
[ ] document how to integrate a differnt jython interpreter through the workflow repository (Deepti)

[edit]

Karajan Language

[x] No Changes ;-)

[edit]

Karajan GUI

[ ] enable editing
[ ] enable storing
[ ] enable storing with coordinates
[ ] enable loading with coordinates
[ ] enable integration of components through the workflow repository

Roadmap:WorkflowGUI

[edit]

Karajan and BPEL

[ ] Document how Karajan is a subset of BPEL
[ ] provide similar language constructs where possible so that similarities is more obvious. This can be done through bepl-subset.xml which needs to be defied.

[edit]

Karajan and OPAL

[ ] Demonstarte how Karajan and OPAL from Sriram Krishnan can play together

[edit]

Workflow Repository

[ ] Test out Remote repository management.
[ ] Discuss with Mike advantages(if any) of using the repository with the workflow service
[ ] Develop and demonstrate the integration of signed components
[ ] Add authorization to the repository for access and modification. Possibly though ACLs formulated in XACML.
[ ] JSR170
[ ] using svn as a code repository

[edit]

Fault tolerant workflows

[ ] Karajan workflows must be able to run for month at a time. Checkpinting and restarting must be possible.
[ ] we need a service that could do this, but in case the service goes down we need some method to make this robust.
[ ] Mike and Deepti need to read the proposal (link ? )
[x] set up the wiki for the water project. Its is at http://www.cogkit.org/water (Gregor)


[edit]

Misc


[ ] Mike: Identify strategy for including <java> </java> to execute arbitrary java programs in Karajan

  • Strategy identified. It's either using the javac classes and producing a temp file, or with the hacked inline compiler, pending license clarification. More detail in the intro

[ ] Mike: Identify strategy for including <python> </python> to execute python programs via jython

  • That should be easier. (possibly next release). More detail in the intro
[edit]

Workflow Broker repository

[ ] Kaizar: develop the simple broker repository details more details

[edit]

Automatic Quality Control

Goal of this set of tasks is it to deploy automatic software engenerring tools to increase code quality

[ ] set up pmd and findbugs with cronjobs (Kaizar, assigned more than a year ago)
[ ] set up a nightly compile (Kaizar, assigned more than ayear ago)
[ ] Job Submission scalebility test
[ ] develop a test that does condor submissions many many thousends of time
[ ] develop a test that does GT2 submissions many many thousends of time
[ ] develop a test that does GT4 submissions many many thousends of time

[edit]

Documentation

  • Manuals/Guides responsibilities
      • Overall coordination (Gregor)
      • Install (Deepti)
      • MPI (Mike formerly Kaizar)
      • Karajan (Mike, Gregor)
      • Workflow repository (Deepti)
      • Coding standard (Deepti)
      • License and stuff (Gregor)
      • Grapheditor into workflow examples
      • Repository
      • Cert management
      • jGlobus (TBD)
      • Long running jobs (what works, what does not, Kaizar)
      • Document the getenv script to print the environment (Deepti)
      • Document hw to submit a bug in the bugzilla page
        • Includes pointers to an FAQ on: a) how to modify log4j b) what is cog-info c) how should i report the bug.
      • Integrate http://www.cogkit.org/bugzilla/show_bug.cgi?id=103 into other wiki location. possibly projects. not sure if its a guide. Have we tested this?
    • Add to the MPI guide an example how to start a MPICHG2 job from karajan (Kaizar,Mike)
    • Intro to Java CoG Kit with programming examples of the abstractions (Gregor and Mike)
    • Intro to Karajan with programming with examples (Gregor & Mike)
    • Intro to the Workflow Repository with examples (Gregor & Deepti)
    • Note: We may use the monte carlo pi application as base example
  • Tutorials
    • Extensive Karajan Tutorial (Mike and Gregor)
  • Papers
    • GUSS journal paper (Gregor)
    • Portal Paper and Java CoG (Gregor, with Mike)
    • Workflow Paper (Gregor)
    • EDGE Paper (Gregor & others once work is completed)
    • REU Paper (Gregor & Others)
    • Roadmap Bookchapters
  • School Projects
    • 2 hour Tutorial on using Karajan (SDSU)
    • Tutorial on doing montecarlo with various technologies including Karajan
    • Al Ali Thesis link
[edit]

Filetransfer

[ ] Integrate RFT

[edit]

Possible student Projects

[edit]

Misc

The following pages need to be worked on

[edit]

Integrate talsk on Web page

  • PPT Java CoG Kit Overview, Gregor von Laszewski, GlobusWORLD, February 7-11, 2005, Boston, Massachusetts
  • PPT Java CoG Kit Workflow, Gregor von Laszewski, GlobusWORLD, February 7-11, 2005, Boston, Massachusetts



[edit]

Readmes and Changes by Module

all filenames need to be normalized to

README.txt CHANGES.txt TODO.txt

We need to add a collumn for the guides apropiate for each component

Ok Assigned to Component Readme Changes To do Status in %
? ? jglobus [1] [2] [3] [4] ?
? Deepti matlab [5] N/A N/A ?
? Kaizar MPI N/A N/A [6] ?
? Kaizar tools [7] [8] [9] [10] ?


Ok Assigned to Component Readme Changes Bugs Status in %
? Gregor main [11] [12] [13] ?
? Kaizar abstraction N/A [14] N/A ?
? Kaizar abstraction-common N/A [15] N/A ?
? Kaizar abstraction-examples N/A [16] N/A ?
? Mike all N/A N/A N/A ?
? Mike certmanagement [17] N/A N/A ?
? Mike certrequest N/A N/A N/A ?
? Mike grapheditor [18] [19] [20] ?
? Kaizar gridant N/A [21] N/A ?
? Deepti gridfaces N/A [22] N/A ?
? Robert/Mike gridshell N/A N/A N/A ?
? Mike jglobus [23] [24] N/A ?
? Mike karajan N/A [25] [26] ?
? Jonathan/Mike monitor [27] [28] N/A ?
? Deepti portlets [29] [30] N/A ?
? Kaizar provider-condor N/A [31] N/A ?
? Kaizar provider-gt2 N/A [32] N/A ?
? Kaizar provider-gt2ft N/A [33] N/A ?
? Kaizar provider-gt3-common N/A [34] N/A ?
? Kaizar provider-gt3_0_2 N/A [35] N/A ?
? Kaizar provider-gt3_2_0 N/A [36] N/A ?
? Kaizar provider-gt3_2_1 N/A [37] N/A ?
? Kaizar provider-gt4_0_0 N/A [38] N/A ?
? Kaizar provider-local N/A [39] N/A ?
? Kaizar provider-ssh N/A [40] N/A ?
? Kaizar provider-webdav N/A [41] N/A ?
? Mike resources N/A N/A N/A ?
? Mike setup N/A N/A N/A ?
? Mike template N/A N/A N/A ?
? Deepti/Mike testing [42] [43] N/A ?
? Mike util N/A N/A N/A ?
[edit]

Release documentation

Manuals and Release Notes:

[ ] integrate the MpichG2 documentation in the MPI Guide
[ ] improve the karajan documentation
[ ] place special attention to TG documentation

[edit]

Testing

[ ] make link to your production test shere (Mike)
[ ] document the automatic cron scripts for nightly builds (Kaizar)
[ ] document the karajan production tests (Deepti)

[edit]

Performance Measurements

[ ] use karajan to do submission and filetransfer production tests (Deepti)

[edit]

Release Checklist

[ ] Job submisison on TG, fork ... < machine name> ?
[ ] Job submission on TG, batch ... <machine name>
[ ] FAQ#How_can_I_submit_jobs_to_the_TeraGrid.3F Abstractions and TG

[edit]

Uncritical Projects

Retrieved from "http://wiki.cogkit.org/index.php/Roadmap"
Personal tools
Collaboration and Jobs