MavenRepository
From Java CoG Kit
Contents |
Introduction
This page is meant as an introduction for CoG Kit programmers who will be writing code and adding resources and scripts to the Java CoG Kit, but who will not have to concern themselves with modifying the build system in any way. Those interested in doing more should consult the list below:
Other Maven Pages:
- MavenThirdPartyDevelopers provides information for third party developers wishing to use Java CoG Kit code within their own Maven Project.
- MavenTutorial provides information for those looking to learn more about Maven internals and/or modify the build system in some way.
The source code repository that this page discuess is located at https://svn.sourceforge.net/svnroot/cogkit/trunk/five.
Build System Goals & Overview
Our Maven based build system not only simplifies the development process but also simplifies the deployment process. Maven and Eclipse are integrated so that the developer is free to work within the friendly confines of the Eclipse IDE, and can issue a few simple commands on the command line to deploy a finished product.
For deployment purposes we create what Maven calls an assembly. While an assembly is nothing more than a collection of files, we have given our assemblies a bit more structure. Assemblies, for our purposes, contain the module's jar file, the jar files on which the module depends, and any scripts, READMEs or other files that are to be bundled with the assembly. The module's jar file, whose creation is also automated by Maven, contains all the class path resources and compiled classes of the module. With the proper scripts (i.e. double clickable launch scripts to start the appropriate programs) and resources, this assembly (which is archived in various formats) can be placed on a website for distribution.
Likewise, the assembly also serves as a starting point for other distributions. Since all of a modules dependencies are gathered together in the assembly, it is easy to create a webstart. Future add-ons to the build system could use the assemblies as a starting point to create a double clickable One-Jar or Mac OS .dmg file.
Eclipse Integration
Using the subclipse and m2eclipse plugins, we have integrated the repository with eclipse. It is worth noting that the multi-module support in the m2eclipse plugin is a bit primitive. However, the plugin is still in active development and support should improve over time.
Checking out the repository from Eclipse
- Download and install Eclipse from eclipse.org.
- Download and install Maven from maven.apache.org.
- Download and install the Subclipse plugin for Eclipse to enable subversion support (additional installation information here):
- GoTo Help -> Software Updates -> Find and Install
- Select 'Search for new features to install'
- Click 'Next'
- Click 'New Remote Site'
- Enter the name 'Subclipse' and the URL http://subclipse.tigris.org/update_1.0.x
- Click 'Ok'
- Confirm there is a check mark next to 'Subclipse' and ONLY 'Subclipse'
- Click 'Finish'
- A new screen will pop up. Place a check mark next to 'Subclipse'
- Click 'Next'
- Accept the terms of the agreement and click 'Next'
- Click 'Finish'
- The Subclipse plugin will now download. This will take some time.
- Another window, 'Feature Verification' will pop up.
- Click 'Install All'
- Click 'Yes' on the dialog asking to restart the workbench.
- Download and install the Maven 2.x plugin for Eclipse (additonal installation information here):
- Go to Help -> Software Updates -> Find and Install
- Select 'Search for new features to install'
- Click 'Next'
- Click 'New Remote Site'
- Enter the name 'Maven' and the URL http://m2eclipse.codehaus.org/
- Click 'Ok'
- Confirm there is a check mark next to 'Maven' and ONLY 'Maven'
- Click 'Finish'
- A new screen will pop up. Place a check mark next to 'Maven'
- Click 'Next'
- Accept the terms of the agreement and click 'Next'
- Click 'Finish'
- The Maven plugin will now download. This will take some time.
- Another window, 'Feature Verification' will pop up.
- Click 'Install All'
- Click 'Yes' on the dialog asking to restart the workbench.
- Configure the Maven plugin:
- Go to Eclipse -> Preferences
- Select 'Maven 2'
- Enter or browse for your local repository folder. By default this will be in /Users/YOUR_USER_NAME/.m2/repository
- Click 'Ok'
- Download the repository from sourceforge
- Go to File -> New -> Other -> SVN -> Checkout Projects from SVN
- Create new location https://svn.sourceforge.net/svnroot/cogkit/trunk
- Click on it and proceed
- Select the
fivedirectory - Click Finish
- Watch the download
- When the download is complete the workspace will build. This may take some time. If it is your first time running Maven this process will take even longer as some dependencies will have to be downloaded in the background, too.
Running/Debugging
The idea with this Maven/Subversion/Eclipse integration is that everything is business as usual inside of Eclipse. You run and debug programs within Eclipse as you would for any typical Eclipse project.
Using Maven from the command line
For those who do not like using an IDE, the build system works just fine through the command line.
Also, some tasks such as the creation of binary distributions or the deployment of artifacts to the Java CoG Kit Maven Repository may only be accomplished through the command line.
Quick Start
For the impatient, we have a crash course example of using the Maven/SVN code base from the command line. Please see the sections that follow for more information.
Note, this quickstart guide is intended for Unix/Linux users but will mostly likely work for Windows users who have installed cygwin.
#Code Checkout svn co https://svn.sourceforge.net/svnroot/cogkit/trunk/five org-cogkit cd org-cogkit #Compile and install all modules in local repository: mvn install # incase you get errors please repeat calling mvn install # sometimes the download may get interrupted. #Create a distribution for a particular module, in this case the ui module: cd ui mvn assembly:assembly
![[ ]](/images/task.png "Regular task"){kind=small}
the next does not work I do not see the *.dir (Matt)
cd target/
cd ui-5.0-SNAPSHOT-dist.dir/
ls
lib/
run_class.py
#Simple script to setup the class path and run an arbitrary class...should change later:
python run_class.py org.cogkit.gridface2.gridfacerunner.GraphicalShell
The directory and class names change if you'd like to compile and run a different class from a different module, however, the process is the same.
The target directory will contain a directory called ui-5.0-SNAPSHOT-dist.dir. Running mvn assembly:assembly in any module will create directories like this (of the form MODULE-NAME-VERSION-dist.dir). These directories contain all the dependent jars (in lib), all of the resources and all binary scripts. The .zip and .tar.gz files of the same name are merely archives of this directory.
The directory or the archives of the directory are binary distributions of the module. Usually the binary scripts are executables that start certain java classes. We also include a default python script run_class.py. Issuing: python run_class.py my.class.name will start the specified class if it is found in the lib/ directory.
Checking out the code from the command line
To checkout the code from the subversion repository, issue the following command:
svn co https://svn.sourceforge.net/svnroot/cogkit/trunk/five org-cogkit
Creating an assembly
Described in the Introduction section, an assembly is a bundle of all the compiled class files, their dependencies and their support scripts that is then suitable for distribution to the end user (note: some post processing on this bundle may be necessary to create the jnlps or system specific icons). Maven automates the creation of these assemblies for each particular module. For example, to create a distribution for the cogkit-core module:
cd cogkit-core mvn package assembly:assembly
the next does not work I do not see the *.dir (Matt)
this does not work as there are no descriptors for the karajan viewer (Matt).
You are left with the directory structure:
org-cogkit/cogkit-core/target/
classes/
cogkit-core-5.0-SNAPSHOT-dist.dir/
run_class.py <- Script, see 'Adding Script' section
lib/
cogkit-core-5.0-SNAPSHOT.jar <- contains module's
resources and
compiled class
files
jdom-b9.jar <- jar this module needs
...
...
cogkit-core-5.0-SNAPSHOT-dist.tar.gz
cogkit-core-5.0-SNAPSHOT-dist.zip
cogkit-core-5.0-SNAPSHOT.jar
exported-pom.xml
surefire-reports/
test-classes/
The files and directories starting with cogkit-core-5.0-SNAPSHOT-dist contain the distribution. The zip and tar.gz files are simply archives of the directory.
Note: Due to a bug in the Maven assembly plugin, you must issue the mvn package assembly:assembly command by hand in each module you wish to deploy. Issuing the command at the root will cause an error. As a temporary fix it would be possible to create a 'fake' module called cogkit-all which served as a sort of stub.
Publishing Artifacts to the Maven Repository--Maven Deploy
The maven deploy mechanism allows us to publish the code to our own maven repository server at http://www.cogkit.org/maven/repository. In order to use the deploy mechanism you must have write access on this server.
To all the modules:
mvn deploy
To release only one particular module, change to that module's directory and issue the mvn release command from there.
This allows others to use the code by specifying the cogkit as a dependency in their own pom.xml (add section just for this).
Configuring your settings.xml -- Different Username
If you would like to use the deploy mechanism and are working on a machine where your username is different from your username on cvs.cogkit.org, it is necessary to add the following to your settings.xml. Your settings.xml is found in the .m2 directory. On unix this is stored in ~/.m2 by default. Please check the maven documentation for more details.
<settings>
<servers>
<server>
<id>cvs.cogkit.org</id>
<username>YOUR_USER_NAME</username>
</server>
</servers>
</settings>
Adding Items to the Repository
This section describes how to add simple things like java source files, resources (images, etc) and scripts to existing modules. If you would like to perform more complex tasks, such as adding a whole new module to the build system, please consult the MavenTutorial.
Adding Code to a Module
To add code to a module, place your code in the directory module-name/src/main/java
For example, to add the source file MyTest.java for the class org.cogkit.MyTest to the cogkit-core module:
org-cogkit/
cogkit-core/
src/
main/
java/
org/cogkit/MyTest.java
Adding Resources to a Module
To add a resource to a specific module, place the resouce in the directory module-name/src/main/resources
For example to add the picture test.jpg to the ui module, use the following directory structure:
org-cogkit/
ui/
src/
main/
resources/
...
test.jpg
...
Adding Dependencies to a Module
To add a dependency to a module, you need to edit the dependencies section of that moundles pom.xml found in org-cogkit/MODULE-NAME/pom.xml
For instance to add bouncycast (group: bouncycastle, artifact: bcprov-jdk15, version: 124) to the UI pom.xml:
org-cogkit/ui/pom.xml:
<dependencies>
...
<dependency>
<groupId>bouncycastle</groupId>
<artifactId>bcprov-jdk15</artifactId>
<version>124</version>
</dependency>
...
</dependencies>
To find the group, artifact and version number for a particular dependency you may wish to consult the MVN Repository Search.
Global Dependencies
Occasionally it is necessary to add a common dependency to all the modules in the project. These, however, are minimal and should only be used in cases where every module actually requires the dependency as well as the same version of that dependency. The process is the same as was just described, however the changes are made to org-cogkit/pom.xml file.
Adding Scripts/READMEs to a Module
Support scripts, usually shell scripts that setup a classpath to start a class, and READMEs or other docs meant to be copied to the root of the distribution created by mvn assembly:assembly are stored in MODULE-NAME/src/main/resources
For example to add my_script.sh and README.txt to the cogkit-core module:
org-cogkit/
cogkit-core/
src/
main/
resources/
my_script.sh
README.txt
Global Scripts/READMEs
Some scripts or generic README files are common to all modules. These can be placed in the org-cogkit/bin directory and are copied over to all modules when an assembly is created.
Summary of Modules
The new Maven repository is subdivided into the modules listed below. These modules may be developed simultaneously and compiled independently, but there are some inter-dependencies. For instance, the cogkit-core module is dependent on the util module. This interdependency implicitly specifies a compilation order that Maven takes care of automatically. However, we must avoid circular dependencies as Maven cannot handle them.
Each module contains its own pom.xml file. Within this file, the module's third party dependencies as well as its dependencies on other modules within the CoG Kit are specified. It is important that all dependencies are specified in this pom.xml file. If two modules depend on a certain jar, this jar must be specified in BOTH modules. The dependency information contained within a module's pom.xml is used to construct bundles suitable for release (i.e. JNLP or standalone bundles that contain all required jar files). While there is a parent pom.xml file that specifies global dependencies for the entire project, this should ONLY be used when the dependencies are truly universal.
util
The util module contains utility code that may be used by any other Java CoG Kitmodules. Its functionality should not be tied to any CoG Kit or 'grid' logic as this module is independent of the cogkit-core module. Ideally, we should strive to keep this module as small as possible.
Internal Dependencies:
- none
cogkit-core
The cogkit-core module is the heart of the Java CoG Kit, containing all the logic that interacts with grids and Globus. Most importantly, the cogkit-core module contains the abstraction package. A developer interested in using the Java CoG Kit to create his or her own program leveraging Grid technologies would most likely be interested in this module.
Internal Dependencies:
- util
karajan
This is the module that contains the backend for the Karjan scripting language.
Internal Dependencies:
- cogkit-core
- util
karajan-viewer
This module contains the GUI to the Karajan scripting language. Though it is a user interface component, the karajan viewer is kept in it's own module to keep the larger ui module independent of Karajan.
Internal Dependencies:
- karajan
- cogkit-core
- util
qstat
Qstat is the GUI for various queueing systems on cluster computers. Though it is a user interface component, qstat is kept seperate as it is independent of the cogkit core and does not require the Grid to do work.
Internal Dependencies:
- util
ui
The ui module contains most of the user interface components. Of particular interest are the gridface components (directory browser, grid shell, image viewer) and the Grid Desktop.
Internal Dependencies:
- util
- cogkit-core
