People: Chris Grubbs

From Java CoG Kit

Tasks: http://wiki.cogkit.org/index.php/Special:Tasks/owner=Chris

Code - parse cobalt qstat output (Chris)
Code - qstat: implement table model (Chris)
Code - qstat: xml/html output (Chris)
Code - implement qsub for various other formats (Chris)
Code - implement qsub for cobalt and pbs (Chris)
Code - improve qsub implementations (Chris)
refresh button for qstat table (Chris)
kill/query buttons for qstat table (Chris)
options window - authentication, refresh rate (Chris)
save as xml/save as html menu options (Chris)
tooltip and help placeholders (Chris)
job specification - create/fetch/suspend/resume (Chris)
parse qstat output for pbs systems (Chris)
implement live refresh, auto-refresh (Chris)
develop guis for reslist and resalloc (Chris)
simple jnlp app (Chris)
in qstat: click column header to sort (Chris)
clean up qstat program's appearance, resizing behavior, etc (Chris)
add status bar to qstat (Chris)
add "test authentication" button to options window (Chris)
develop GUI for Cobalt qsub (Chris)
add view html/view xml options to qstat (Chris)
add search bar to specify which users' jobs to display in qstat (Chris)
fix bug: column widths reset on refresh (Chris)

Contents 1 Experience Report 2 Books Purchased 3 Summaries 3.1 Week 10 Summary 3.2 Week 9 Summary 3.3 Week 8 Summary 3.4 Week 7 Summary 3.5 Week 6 Summary 3.6 Week 5 Summary 3.7 Week 4 Summary 3.8 Week 3 Summary 3.9 Week 2 Summary 3.10 Week 1 Summary

Experience Report

This summer I worked at Argonne National Laboratory, funded by an REU through DePaul University. I worked on an open-source software project called the Java CoG Kit, which is a set of tools for grid-based applications. The CoG Kit allows both application developers and end-users to take advantage of the immense computing power of parallel computing systems. My specific work had to do with the creation of a grid monitoring component, which allows users to observe the status of a grid system. Users are also able to submit jobs to the queue using my component. In creating this component, I also created a standalone library to monitor and submit jobs to BGL, one of the cluster systems here at Argonne. I integrated this library with Karajan, a workflow component in the CoG Kit, to provide further workflow functionality.

Throughout the summer, I learned a great deal about working on a relatively large open-source project, observing proper software engineering principles, and utilizing the various computing resources provided by the laboratory. Because this was my first large-scale software project, I found it very challenging at first. For one thing, it forced me to significantly improve my Java coding skills, as well as learn some specific Java technologies like Swing. Also, I had little experience with Linux, so I had to improve my Linux skills as well.

Teamwork was very important in my work this summer. In my undergraduate career at Loyola, I had some experience coding with others, but none with an organized repository setup. During the REU, I acquired valuable skills in working on a team and coordinating coding projects through version control systems, as well as through face-to-face meetings. Because few software projects are created on an individual basis, this will be of great help in my future graduate work, as well as in any industry or doctoral work I choose to pursue.

Because I will soon be beginning my master's degree in computer science, and because I eventually plan to get a Ph.D, I value the research experience of working at a prestigious national laboratory. Working at Argonne gave me access to some of the most powerful and cutting-edge computing systems in the nation. The work being done at Argonne with grid systems is very exciting, and I hope to integrate what I have learned here into my future research work.

This has been a highly educational and productive summer, and I am grateful to have had the opportunity to participate. I believe that, down the road, participating in the REU at Argonne will have been one of my most valuable educational experiences in computer science.

Books Purchased

1. The Universal History of Computing, by Georges Ifrah
2. The Code Book: The Science of Secrecy from Ancient Egypt to Quantum Cryptography, by Simon Singh
3. Tools for Thought: The History and Future of Mind-Expanding Computing, by Howard Rheingold
4. The World is Flat: A Brief History of the Twenty-first Century, by Thomas Friedman
5. The Cathedral and the Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary, by Eric S Raymond
6. The Making of the West, by Lynn Hunt et al

These six books provide a historical perspective on the discipline of computing. Ifrah's book, The Universal History of Computing, accounts for the development of computers, beginning with the origins of written numbers and mathematics, through the development of symbolic logic and theory of algorithms, into modern computing. Central figures are Charles Babbage, Alan Turing, John von Neumann, and many others. Singh's text, The Code Book, focuses specifically on cryptography, providing a different angle on the history of computing. Modern computing technology is heavily indebted to the work done at Bletchley Park during World War 2, which was motivated primarily by cryptographic concerns. Rheingold's book Tools for Thought, on the other hand, offers a forward-looking perspective on computing. Drawing from the work of the computing visionaries mentioned above (Turing, etc), Rheingold demonstrates how such visionary work is being carried out today, and how it will shape the field of computing.

The second three books on the list provide a historical and societal context for computing. Friedman's book The World Is Flat, although some of its conclusions are suspect, is nevertheless an important work in accounting for computing's relationship to globalization, the primary facet of twenty-first century life. Similarly, Raymond's text The Cathedral and the Bazaar is a seminal work outlining the birth and development of the Free and Open Source Software movement, a movement with broad consequences in areas from software engineering to law to politics. The sixth book provides a historical reference for the history of computing, providing information on the larger context in which the work of Babbage, Hollerith, Turing, and others was done.

I purchased these six books in an effort to broaden my understanding of the field of computing; while I am very interested in writing code and learning new technologies, I am also very interested in grounding computing within a historical context. Computers did not merely appear. Their development is one of mankind's greatest intellectual achievements, and I don't believe it is possible to understand the current world situation without understanding how computers have contributed to it. I feel that learning more about the history of computing will make me both a better computer scientist and a better world citizen.

Summaries

Week 10 Summary

This week I modified the inefficient login>execute>logout ssh implementation. Now, we stay connected, opening a new session channel each time we want to execute a command. This is much faster. Also, for PBS submission, the user can now submit a PBS script on their local filesystem. I tweaked the UI on the PBS submit window as well.

I did some major refactoring in an attempt to bring the code some coherence and a better object-oriented structure. The two qstat modules now implement a common Qstat interface, as do the two qsub modules. Numerous bugs occurred as a result of this refactoring, most of which I have fixed. I also improved the profile implementation, which was very bad before.

As for the Karajan integration, I modified the workflow scripts and tested parallel submission, which seems to work at this time. I have started writing a research paper on my work on integrating Karajan with Cobalt this summer, and have made significant progress.

Week 9 Summary

This week I prepared a new JNLP release. For this release, I removed the monitoring column, since it was not doing anything. I found a neat little library to launch the user's default browser from a hyperlink in the about window, so I implemented that.

I also did some significant refactoring in preparation for the Cobalt submission methods' integration with Karajan. I implemented a passphrase masker, so that when executing a Cobalt workflow, the passphrase will not have to be given in cleartext. I created a CobaltSubmitter class as well as a Karajan wrapper, and created some example workflow scripts that implement my submission functions.

For PBS submissions, the user now has the option to use a PBS script already on the server, instead of having to generate a new one.

Week 8 Summary

This week I improved the Java2Karajan parser, putting the classloader in a separate class, as well as adding command line argument functionality.

I also worked more on my qstat component. The options dialog now appears at startup. I also found that uploading PBS scripts to the server for execution was conflicting with my auto refresh. So I pause the refresh whenever the submit window is displayed to prevent such conflicts. I also fixed some bugs that appeared when trying to switch modes as the program is refreshing. I also added some tooltips to the submit windows so that the fields make more sense.

Matt and I also did some work on organizing Bibtex citations for projects and papers regarding the CoG Kit.

Week 7 Summary

This week I implemented live qstat refresh for PBS systems such as TeraGrid. This involved writing a parser for the slightly different PBS qstat output and adjusting the table accordingly. The new qstat mode introduced some bugs when switching between Cobalt and PBS, most of which I fixed. One that remains is that if the user attempts to log into a PBS system with Cobalt mode on (or vice versa) the program breaks. I also implemented job submission for Cobalt systems. Using a simple dialog box, the user can specify the desired amount of nodes, the allotted completion time, and the path to the remote executable, as well as some other optional arguments.

The other component I worked on this week was the Java2Karajan script generator. This program loads a Java class and parses its constructors and methods into a Karajan XML script which allows access to that class's functionality. Matt and I pair-programmed the parser using Gobby, a collaborative programming tool. After we successfully implemented the parser, I wrote a GUI frontend for it, through which the user can load whatever class they desire and save the generated XML output.

Week 6 Summary

This week I fixed several bugs which resulted from my new TableModel implementation, as well as some other general bugs. I finished a basic job monitoring GUI, which places a column of checkboxes next to each job. If the column is unchecked, the row turns gray, indicating that the job is no longer being monitored. (However, actual job monitoring is not being done at this time.) Finished jobs now stay in the table instead of disappearing, and turn blue to indicate their finished state. These finished jobs stay at the bottom of the table. Queued jobs are yellow and running ones are green. Upon making some further changes for stability and usability, I made a version of my program available through Java Web Start. The link can be accessed here: http://wiki.cogkit.org/index.php/Java_CoG_Kit_Qstat

Week 5 Summary

This week I did a good amount of refactoring and bug fixing on my qstat program. I improved its layout and resizing behavior, as well as my implementation of profile loading/saving. Instead of executing login upon pressing "OK", I added a "Test Authentication" button which, if successful, prompts the user to choose if they would like to begin monitoring that server. This way, the "OK" button does not take forever to execute, instead making instant changes on things such as refresh rate and viewable columns. I also used a better TableModel implementation, and began work on implementing individual job monitoring.

Week 4 Summary

This week I tinkered with a few different methods of issuing qstat commands via SSH. For the moment, I have settled on a basic login>execute>logout model. I implemented live refreshing, which happens either when the user presses a button or automatically according to the specified refresh rate. I also provided save/load profile functionality. On startup the program automatically attempts to load the profile file "q.profile" from ~/.globus/cog. I also fixed a bug in the code that read standard output from the remote host. On another topic, I started a basic JNLP implementation. It can be found at http://www-unix.mcs.anl.gov/~cgrubbs/coggui.jnlp but it does not do much at the moment.

Week 3 Summary

This week I further developed my qstat program. I added XML and HTML output options, generating HTML with an XSLT transform on the XML document. I added login and refresh options to my options window, and wrote a test SSH handler so I could test my parser on a real qstat output. I also created a separate class for the table model, instead of creating it within the frame class, which is a much better implementation. I also added support for PBS-format qstat.

Week 2 Summary

This week I created a graphical table model for qstat output from Cobalt. The user can specify which columns they would like to see, and I added a method to set certain columns as always viewable (i.e. the Job ID field, which should always be viewable.) I added a right click popup menu to kill a selected job (the job is not actually killed yet.) I also began implementing the qsub command for both Cobalt and PBS systems.

Week 1 Summary

In addition to dealing with various administrative tasks, I began to implement a parser for Cobalt-format qstat output, with the goal of creating a graphical table model.