From Java CoG Kit

Mike Hategan and Gregor von Laszewski (gregor@mcs.anl.gov)

Administrative Notes:

Editing: The maintenance of this document is rather simple. We do NOT recommend that you edit this document in a single edit. instead, we do require the edit as part of a subsection. This will prevent the chance that data will get lost.

Versioning: It is extremely easy to adapt the Document to a new version. All that needs to be done is creating a new page with the appropriate version number and pasting and copying the contents of the current document into it. Than you just have to replace the view places where the actual version occurs. This is best done with an editor such as emacs that has easy query replace capabilities.

1 About this Document
2 The Karajan Language
3 Modularisation
4 Kernel Library
- 4.1 Kernel Constants
- 4.2 Kernel Elements
5 System Library
6 Task Library
- 6.1 Task Elements
7 Java Library
- 7.1 Java Elements
8 HTML Library
- 8.1 HTML Elements
9 Forms Library
- 9.1 Concepts
- 9.2 Form Elements
10 Restart Library
- 10.1 rlog:restartLog
- 10.2 rlog:logged
11 Service
12 Embeding Karajan into Java
13 Notes
14 References

About this Document

This page contains the Reference manual. Although it contains all of the features, we do recommend that you start with the examples document at http://wiki.cogkit.org/index.php/Java_CoG_Kit_Workflow_Guide

Command Line

Usage:

 cog-workflow <options> file <arguments>

Options

(-execute | -e) <string>: Execute the script given as argument

-showstats: Show various execution statistics at the end of the execution

-debug: Enable debugging. This will enable a number of internal tests at the expense of speed. You should not use this since it is useful only for catching subtle consistency issues with the interpreter.

-monitor: Shows a resource monitor

-dumpstate: If specified, in case of a fatal error, the interpreter will dump the state in a file

-intermediate: Saves intermediate XML code resulting from the translation of .k files

(-help | -h): Display usage information

Experiemental Options

The following options have been added and are considered to be experiemental. They are still in the debugging stage.

-debugger: Starts the internal graphical debugger

-cache: Enables cache persistence

File

The file represents the name of the script to run, in either CoG Kit XML or CoG Kit K syntax. A CoG Kit karajan file structure is rather simple and similar to what you would are used to form other programming languages. A CoG Kit Karajan file starts with a series of imports, followed by a block of statements. It is important to note that we have two ways to express CoG Kit karajan syntax. Both are explained in more detail in the following sections.

All Karajan files usually start with the import of the most generally used libraries

in k syntax:

import("cogkit.xml")
print("hello world")

in xml syntax:

<import file="cogkit.xml">
<print message="Hello World">

Arguments

Arguments to the script can be specified after the file name. They are distinct from the interpreter options, and are passed to the script as a constant, in the form of a list named "cmdline:arguments". More information about commandlines can be found here

The Karajan Language

Karajan supports two syntax modes: a syntax called k and an equivalent form called XML. There is no difference on the semantic level between the two forms.

The Karajan Syntax

The following conventions are used:

(xy) is used to group x and y
[x] indicates that x is optional
x+ denotes at least one occurrence of x
x* denotes zero or more occurrences of x
x|y means either x or y
'x' is to be interpreted as the literal x
ε represents the empty production

Elements

The Karajan semantics revolve around the notion of elements. An element is relatively similar to a function in that it has a name, can accept arguments, and may return values. The general syntax for an element is:

element ::= identifier '(' [arguments] ')'

Example:

print(1)
false()

Identifiers

An identifier can consist of alpha-numeric characters and certain symbols, but no whitespace. Symbols that cannot be used in an identifier are symbols that have other syntactic functions, such as brackets (all of them), commas, double quotes, and operators (’+’, ’-’, ’*’, ’/’, ’%’, ’^’, ’=’, ’<’, ’>’, ’&’, ’|’). Identifiers are case insensitive.

identifier ::= (Letter | Digit | '!' | '@' | '#' | '$' | '%' |
                 | '_' | ':' | ';' | ''' | '.' | '?' | '\' | '`' | '~')+

Example:

var, i, v123, @, a$, big_list, grid:task, file.list

Identifiers cannot begin with a digit.

Arguments

The arguments can either be other elements or values, such as numeric values or strings. Elements can be separated by commas or the new line character (or both):

arguments ::= argument [separator arguments]) | ε
separator ::= ',' | Newline

Example:

list(true(), false())
list(
  true()
  false()
)
list(true(),
  false())

Arguments come in two flavors. Named arguments and unnamed arguments:

argument ::= named_argument | unnamed_argument

Named Arguments

Named arguments provide a way of explicitly binding arguments to formal parameters:

named_argument ::= identifier '=' unnamed_argument

Example:

print(true(), nl = false())

Unnamed Arguments

Unnamed arguments can be either immediate values, elements or expressions. Immediate values can be numeric literals, string literals, variables, or quoted lists:

unnamed_argument ::= numeric_literal | string_literal | variable | quoted_list |
                     | element | expression

numeric_literal ::= ['+'|'-'] digit+ ['.' digit+]

digit ::= '0'... '9'

string_literal ::= '"' any_characters_but_double_quotes '"'

variable ::= identifier

Example:

list(1, 2.3, -4.56,
  +7.890, "A string", list("Another string value in a nested list", "*2"))

Expressions

Expressions consist of unnamed arguments to which operators are applied. Parantheses can be used to override the default precedence of operators in expressions.

expression ::= unnamed_argument operator unnamed_argument |
               | '(' expression ')'

Examples of expressions:

set(a, 1+2*3-4)
set(b, subtraction(sum(1, product(2, 3)), 4))
print(a, " = ", b)

<set name="a">
  <subtraction>
    <sum>
      <product>
        <number>2</number>
        <number>3</number>
      </product>
    </sum>
    <number>4</number>
  </subtraction>
</set>
<print message="a = {a}"/>

Operators

The native Karajan syntax supports basic arithmetic and logic operators:

operator ::= '*' | '/' | '%' | '+' | '-' | '<=' | '>=' | '<' | '>' |
             | '==' | '!=' | '&' | '|'

The following lists enumerates operators in the order of precedence, starting with the highest precedence. While the XML syntax does not support the use of operators, each operator has an equivalent element which can be used in both syntaxes (shown in parentheses).

Multiplicative operators
- * (product) Multiplication
- / (quotient) Division
- % (remainder) Remainder

Additive operators
- + (sum) Addition
- - (subtraction) Subtraction

High priority relational operators
- <= (lessOrEqual) Less or equal
- >= (greaterOrEqual) Greater or equal
- < (lessThan) Strictly less
- > (greaterThan) Strictly greater

Low priority relational operators
- == (equals) Equals
- != (No equivalent, but not(equals(...)) can be used) Does not equal

Multiplicative logic operators
- & (and) Logical AND

Additive logic operators
- | (or) Logical OR

Quoted Lists

A quoted list is a special element that produces a list of identifiers. What is specific about a quoted list is that if its arguments are variables, the variables will not be evaluated. Instead their identifiers will be added to the list. Quoted lists are convenience syntax for expressing a list of formal arguments:

quoted_list ::= '[' arguments ']'

However, quoted lists are not limited to expressing list of arguments. They can also be used to express lists of values. The only thing to remember is that variable evaluation will not take place for immediate arguments of a quoted list.

Example:

list("A quoted list follows", [a, b, c])

Programs

A Karajan program is a list of arguments:

program ::= arguments

There exists an implicit root element that sits at the top of the element tree, and implements certain system functions.

Comments

And finally, Karajan uses C-style comments. Single-line comments begin with two forward slashes and end at the following new line character, while multi-line comments are delimited by ’/*’ and ’*/’:

//This is a comment
print("This is not a comment")
/*This is
  also a
  comment
 */

The XML Syntax

Karajan also supports XML as its syntax. In the XML syntax, each XML element corresponds to a Karajan element. Arguments can be expressed either through XML attributes or nested elements.

Particularities of Using XML

One of the particular aspects of using XML with Karajan is that when using XML attributes for arguments, it is impossible to make a syntactic distinction between a numeric value and its string representation. In general, Karajan will try to use the context to figure out which one is desired, but there are instances when it is impossible to do so. Therefore, when using the XML syntax, the following elements can be used for the purpose of differentiating between numeric and string values: number and string

<list>
  <number>1</number>
  <string>1</string>
</list>

The equivalent Karajan construct would be:

list(1, "1")

Karajan can load and interpret arbitrary XML files, provided that definitions exist for the XML elements present in the file, but XML mixed content is not handled properly. The unfortunate aspect is that it is impossible to handle XML mixed content in a generic way. For example, it cannot be known whether whitespace between two XML elements is to be interpreted as content or not, without knowledge of the implementation of an element. Since Karajan is a dynamic language, the implementation of an element is not known statically, at the time the parsing takes place. Therefore, the following rule was adopted: An element will consider textual content content if and only if no nested elements exist. If nested elements exist, textual content will be ignored.

If processed, textual content will be mapped as a string argument. A consequence of the above rule is that textual content and multiple arguments are mutually exclusive.

Lastly, a well-formed XML document must always have a root element. While in the native Karajan syntax, the root element is implicit, in XML the project or karajan elements can be used as root elements.

Parameters and Return Values

An element can accept any number of arguments and can generate any number of return values, and that includes an infinite number of arguments and/or return values (at least in theory).

Parameters and Arguments

Arguments are divided into two major types: single value arguments and channels. As their name implies, single value arguments can have only one value. By contrast, channels can be used for any number of values.

Single Value Arguments

Single value arguments can be specified using the named argument form. For example, the print element has a message argument. Thus passing a string as the message argument to print element can be done in the following way:

print(message = "Some string")

Or in XML:

<print message = "Some string"/>


<print>
  <argument name = "message" value = "Some string"/>
</print>

Single value arguments can be further divided into mandatory and optional arguments.

Channels

Channels can be used to pass multiple arguments to an element. Each channel has a name, except for the default channel. The default channel is similar to the notion of variable arguments in C. Passing arguments on the default channel is done implicitly when arguments are not passed as single value arguments:

k:

list(1, "value")

xml:

<list>
  <number>1</number>
  <string>value</string>
</list>

Argument Mapping

It is not always convenient to use the named argument form to pass arguments to an element. Elements in Karajan will automatically map arguments received on the default channel to single value arguments. The mapping is done dynamically, in the order arguments are received. Suppose there is an element foo that takes three arguments, namely one, two and three. The following would then be equivalent:

foo(one = 1, two = 2, three = 3)
foo(one = 1, two = 2, 3)
foo(one = 1, 2, 3)
foo(1, 2, 3)
foo(1, 2, three = 3)
...

<foo one = "1" two = "2" three = "3"/>
<foo one = "1" two = "2">
  <number>3</number>
</foo>
<foo one = "1">
  <number>2</number>
  <number>3</number>
</foo>
<foo>
  <number>1</number>
  <number>2</number>
  <number>3</number>
</foo>
<foo>
  <number>1</number>
  <number>2</number>
  <argument name="three" value="3"/>
</foo>
...

Optional Arguments

The unfortunate side-effect of using automatic mapping of default channel arguments to single value arguments is that elements that would accept both single value arguments and arguments on the default channel (variable arguments) cannot avoid mapping of variable arguments to certain single value arguments unless different semantics are introduced: optional arguments. Optional arguments do not need to be specified. However, if specified, the named form must always be used. An example is the print element, which has an optional argument named nl . It can be set to false to indicate that no new-line character should be appended at the end of the message argument:

print(message = "Message", nl = false())

or

print("Message", nl = false())

The following however, is not valid:

print("Message", false())

Return values

Return values are a mirror image of the arguments concept. Whatever can be accepted as an argument by an element can also be returned by another. Thus, it is possible to define a single element that returns all arguments to any given element. The following example defines an element that returns both a message and the named form of the nl argument, suitable for the print element:

//The following defines an element foo() which takes no
//arguments and returns "Message" and nl = false()
element(foo, []
  "Message", nl = false()
)

print(foo())

<element name="foo" arguments="">
  <string>Message</string>
  <argument name="nl">
    <false/>
  </argument>
</element>

<print>
  <foo/>
</print>

Argument Evaluation Order

There is no imposed order for evaluating arguments. The order is controlled by each element. Most elements, by default, evaluate their arguments in sequential order. However, it is very easy to override the default order by using elements that use a different execution order. For example, the parallel element evaluates all of its arguments in parallel, returning all the resulting values. Evaluating the arguments to an element in parallel then becomes as easy as surrounding them with a parallel element:

list(
  parallel(
    "Value1"
    "Value2"
  )
)

<list>
  <parallel>
    <string>Value1</string>
    <string>Value2</string>
  </parallel>
</list>

Furthermore, the way in which an element processes the arguments is also left to each element. For example, an element can choose to start executing after the evaluation of all arguments has been completed, or process arguments as they arrive. In other words, and in the most general case, arguments are both generated and processed asynchronously.

A concrete example is the print element, which simply returns the message argument on the stdout channel. When Karajan starts execution, an implicit root element is created that receives arguments on the stdout channel, and prints them to the console. Since the processing is done asynchronously, the appearance of print doing the actual work when executed is achieved. The advantage of such a mechanism is that, provided that an element does not produce any side-effects, its execution becomes equivalent to the totality of values returned (both single values, and channels).

Variables and Scope

We will not insult the reader’s intelligence by explaining what variables are. There is no explicit declaration of variables in Karajan. A variable is defined when it is assigned the first time.

The scope of a variable extends to the element that it was defined in, and is pseudo-lexical. By pseudo-lexical it is meant that internally, the scoping is dynamic, but provisions are made to make it impossible to access variables outside the lexical scope. Therefore, Karajan does not support closures.

On a lexical level, it is possible to read the value of a variable defined in a parent element, but setting the value of the same variable will create a new scope. In other words, Karajan uses deep access and shallow binding. The following example should make things clearer:

...
set(v, 1)   // v is '1' on stack frame n
list(      // A new stack frame is created: n+1
  v                    // $v$ refers to the variable on frame n
  set(v, 2) // A new binding is made for v on frame n+1
                       // the new binding shadows the one from frame n
  v                    // $v$ now refers to the binding on frame n+1
)                      // The returned list contains the values 1 and 2
print(v)    //v refers again to the binding on frame n
                       //Therefore the printed value will be 1
...

...
<set name="v" value="1"/>  
<list> 
  <variable>v</variable> 
  <set name="v" value="2"/> 
  <variable>v</variable>    
</list> 
<print>
  <variable>v</variable> 
</print>
...

In the above example, it would be impossible for the definition of list to access variable v, since the body of the definition of list does not fall within the lexical scope of the definition of v.

The reason for this kind of scoping is to reduce the ambiguity that could be introduced by not knowing the order in which child elements are executed. Please note that such ambiguity is not completely eliminated. The order in which the arguments to list are evaluated does matter, and can change the resulting list. However, the scope of the ambiguity is the same as the scope of the ambiguity in the order of evaluation of the elements. If set, list, and print are evaluated in sequence, no change in the way list evaluates its arguments can change the outcome of the execution of print(v). 1

Global Variables

Global variables are provided for conveniently defining settings that have a global scope. Please note that in the future, global variables will be single-assignment, this approaching more the notion of constants.

global(foo, "Foo")
element(boo, []
  print(foo)
)
boo()

Variable Expansion

Karajan offers convenient variable expansion constructs. All pairs of curly brackets inside strings are replaced by the value of the variable with the name of the identifier inside the brackets. If no such variable exists, the element trying to access the string will fail. If the ’{’ literal is needed inside a string, it must be used twice. There is no need to escape the closing curly bracket, since it cannot be part of an identifier. If a closing bracket is part of a variable expansion expression, it will mark its end. If not, it will be interpreted as the closing curly bracket literal:

set(a, 1)
print("A is {a}")
print("An opening curly bracket: {{")
print("A closing curly bracket: }")

<set name="a" value="1"/>
<print message="A is {a}"/>
<print message="An opening curly bracket: {{"/>
<print message="A closing curly bracket: }"/>

Futures

Futures are a mechanism of binding a variable to the results of a future computation. Until the value of the computation to which the future is bound to, the future exists in an unbound state. Any attempt to use the value of an unbound future will cause the execution of the thread that tried to access the future to block until the future becomes bound.

In Karajan there are two types of futures:

single value futures: are used to hold a single value of a future computation. They are defined using the future element.

future iterators: can be used to hold multiple values. However, not all the values need to be generated before the future iterator can be used. Iterating over a future iterator will cause the iteration to use as many values as are available, then block waiting for more values to be added to the future iterator. Future iterators are defined using futureIterator.

Modularisation

Source Files

As mentioned in Section ??, Karajan understands two syntaxes: the native Karajan syntax and the XML syntax. The distinction between them is made using the file extension. A file with the “.k” extension will be parsed using the native parser, while a file with the “.xml” syntax will be parsed using an XML parser. ^[1]

Libraries

Libraries are collections of elements grouped by the functionality they provide. A library is defined in a source file. Its functionality can be reused in other source files by using the import (equivalent to include) element:

import("sys.k")

It is possible to include XML libraries from native Karajan files. It is also possible to include native Karajan libraries from XML Karajan files. Consequently, the following are valid:

import("task.xml")

<import file="task.k"/>

Namespaces

Namespaces provide a way of distinguishing between elements with conflicting names in different libraries. Suppose a library “a” defines an element named foo, and a library “b” also defines an element named foo. Also, suppose that both libraries are included in a certain file. Namespaces make it possible to access both instances of the foo definition, without ambiguity, by prefixing the name with the namespace prefix in which the element was defined: a:foo and b:foo. Any reference to foo without a prefix will result in an error. Nonetheless, if only one of the libraries is used, the use of foo without a prefix will be allowed. Namespaces are defined using namespace.

Kernel Library

The Karajan kernel contains a minimum set of elements that are required in order to get the rest of the system running. All kernel elements are automatically available in any program.

Kernel Constants

kernel:true

true: Represents the true boolean value

kernel:false

false: Represents the false boolean value

kernel:cmdline:arguments

cmdline:arguments

Holds a list with the command line arguments (if any). Arguments to the CoG Kit karajan script can be specified after the file name. They are distinct from the interpreter options, and are passed to the script as a constant, in the form of a list named "cmdline:arguments".

kernel:user.home

user.home: Contains the path to the user home directory

kernel:user.name

user.name: Contains the current user's name

Kernel Elements

kernel:project

kernel:project(stdout)

Aliases: karajan

The root element of a Karajan program. Accepts arguments on the stdout channel and prints them immediately on the console.

kernel:import

kernel:import(file, ...)

Aliases: include

Executes the file specified by the file argument. All arguments received on the default channel are considered to be element definitions (created with export), which import binds to the parent environment, such that after import completes execution, the definitions will be available for use.

foo(
  import("file.k")
) //scope of foo() ends

Import uses the library search path specified in etc/karajan.properties, which defaults to searching the current directory first, then the Java class path. If no file with the given name is found in the library search path, import will fail.

kernel:export

kernel:export(name, value)

Returns the pair (name, value). The value argument should be a lambda, which can be bound by import.

export(foo, element([x], print(x)))

It is also possible for export to be used without arguments, but with a set of immediately enclosed definitions. In this case it would take all the definitions and export them. This makes it easier to have old code somewhat cleanly converted:

export(
  element(x, [], print("x"))
  element(y, [], print("y"))
)

kernel:define

kernel:define(name, value)

Binds a lambda, specified by the value argument to the given name.

kernel:namespace

kernel:namespace(prefix)

Allows the specification of a namespace prefix. Any elements defined in the scope of namespace will automatically have the prefix indicated by the prefix argument, unless another namespace is nested.

kernel:elementdef

kernel:elementdef(type, classname)

Used by the current implementation to map element names to Java implementation classes.

kernel:named

kernel:named(value, *name)

Used internally by the named argument form, but can be used by the user equally well. The following are equivalent:

print("Test", nl = false())
print("Test", kernel:named(name = nl, false())

However, the fact that the *name argument is optional makes it impossible to completely avoid the named form.

If the *name argument is not present, it simply returns the value of the value argument on the default channel.

kernel:number

kernel:number(value)

Can be used with the XML syntax to represent a number. There is no distinction between integral numbers and floating point numbers in Karajan.

kernel:string

kernel:string(value)

Can be used with the XML syntax to represent a string value.

kernel:variable

kernel:variable(name)

Used to represent the value of a variable.

Example:

<set name="n" value="5"/>
<list>
  <number>10</number>
  <string>10</string>
  <variable>n</variable>
</list>

will return the list [10, “10”, 5].

kernel:quotedlist

kernel:quotedlist(...)

Used internally to represent a quoted list. The following two lines of code will produce the same result:

[a, b, c]
quotedlist(a, b, c)

kernel:cache

kernel:cache()

Caches the evaluation of the arguments. Upon subsequent executions of cache, the arguments will not be re-evaluated. Instead, the cached values will be returned. Cache does not make any checks for the invariance of the evaluation of the arguments.

System Library

Files: sys.k, sys.xml

The system library contains general purpose elements that help implement the most common tasks in Karajan.

Flow Control Elements

sys:sequential

sys:sequential()

Executes all arguments in sequence.

sys:parallel

sys:parallel()

Executes all arguments in parallel.

sys:unsynchronized

sys:unsynchronized()

Asynchronously executes all arguments in sequence. Does not return any value. If return values from asynchronous computations is required, use either future or futureIterator

sys:choice

sys:choice()

Executes arguments in succession. Terminates when one of the arguments completes successfully. If an argument fails, the next argument will have access to the following variables:

element: Contains a reference to the element that caused the initial failure.
error: A textual message detailing the error that occurred
trace: A textual representation of the Karajan stack trace.
exception: Available if a Java exception caused the failure.

If no argument completes successfully choice will fail with the last failure encountered.

Choice exhibits a transactional behavior when it comes to return values. All single values and all channels are buffered until one argument completes successfully. If that happens then choice returns the buffered values. If an argument fails, all the buffered values produced by that argument are discarded.

sys:catch

sys:catch(match)

Catch will match the error variable against the regular expression in match. If successful, it will execute the rest of its arguments, otherwise it will fail with the last failure encountered. Catch can be used with choice to selectively handle specific failures:

choice(
  ...
  catch(".*File not found.*"
    print("File not found")
  )
  catch(".*Connection refused.*"
    print("Connection refused")
  )
)

sys:guard

sys:guard()

Guard expects two sub-elements. It will execute the first, then the second, even if the first one fails. In other words, the second sub-element will always be executed. If the first element failed, after executing the second element, guard will also fail. If the second element fails, guard will fail with the same error, regardless of whether the first element failed or not. Guard can be used to implement clean-up actions, in a fashion similar to try/finally from C++, Java or Python:

set(myhost, "sunny.mcs.anl.gov")
transfer(srcfile="a", desthost=myhost)
guard(
  sequential(
    execute(executable="/usr/bin/hammer", arguments="a", host=myhost,    
      provider="gt2")
  )   
  sequential(
    //always clean up
    file:remove(name="a", host=myhost, provider="gridftp")
  )
)

sys:race

sys:race()

Aliases: parallelChoice

Can be used to race a number of arguments. Race will execute all its arguments in parallel, buffer their return values, and wait for the first one that completes. It will then return all the values that the winner generated. If an any argument fails before any other argument completes, then race will fail.

Race is similar in behavior to the discriminator workflow pattern.

sys:for

sys:for(name, in)

Can be used to iterate sequentially across a range of values. The name argument is an identifier that indicates the name of the variable that will be set to the successive values of the in argument, which Karajan will try to convert to an iterator before beginning the iteration process.

After evaluating name and in, for will proceed and evaluate the rest of the arguments repeatedly, while setting the variable indicated by the name argument to each value produced by the iterator (in).

Example:

equals(
  list(
    for(i, range(1, 5), i)
  )
  list(1, 2, 3, 4, 5)
)

will return true

sys:parallelFor

sys:parallelFor(name, in)

Will behave in a similar way to for with the exception that iterations will occur in parallel. Each iteration will occur in a separate scope. Therefore variables set in one of the iterations will not be visible in the others (which is also the case with for). Each scope will have the variable indicated by the name argument set to one of the values obtained from the in argument.

sys:while

sys:while(condition)

Will repeatedly execute its arguments in sequence until a value of false is received on the condition channel. A convenience element that returns a boolean argument on the condition channel is condition (or ?). While will check for a condition every time an argument completes. It is therefore possible to exit the loop after the termination of any of the arguments.

Example:

list(
  while(
    1, 2, 3, ?(false)
  )
)

list(
  while(
    1, ?(false), 2, 3
  )
)

list(
  while(
    ?(false), 1, 2, 3
  )
)

list(
  while(
    sequential(
      ?(false)
      0 
      /* zero will make it to the list
       * because while will only do the check
       * after sequential() completes
       */
    )
    1, 2, 3
  )
)

will return the following lists:

[1,2,3]
[1]
[]
[0]

Or, in XML:

<list>
  <while>
    <number>1</number>
    <number>2</number>
    <number>3</number>
    <condition>
      <false/>
    </condition>
  </while>
</list>

<list>
  <while>
    <number>1</number>
    <condition>
      <false/>
    </condition>
    <number>2</number>
    <number>3</number>
  </while>
</list>

<list>
  <while>
    <condition>
      <false/>
    </condition>
    <number>1</number>
    <number>2</number>
    <number>3</number>
  </while>
</list>

<list>
  <while>
     <sequential>
       <condition>
         <false/>
       </condition>
       <number>0</number>
     </sequential>
     <number>1</number>
     <number>2</number>
     <number>3</number>
   </while>
</list>

sys:condition

sys:condition(value)

Aliases: ?

Evaluates the value argument and returns its value on the condition channel.

sys:break

sys:break()

Can be used to break out of a while loop. By contrast with using the condition channel, break will immediately exit the loop, no matter how deep the nesting level. It does that by generating a failure, which is intercepted by the enclosing while.

sys:continue

sys:continue()

Can be used to skip the evaluation of the remaining arguments in an iteration in a while loop and jump to the next iteration. Similar to break, continue achieves its purpose by generating a failure which is intercepted by while.

sys:if

sys:if()

Executes its elements using the following scheme:

start with k = 0
evaluate element 2 * k;
if element 2 * k is the last element, then return its return values and complete
if no value is element by argument 2 * k, fail
if value returned by element 2*k is true then evaluates element 2*k+1 returning its return values, and completes
if value returned by element 2 * k is false then continue with k = k + 1

Informally:

if(
  <condition> <then>
  [<condition2> <then2>
  [<condition3> <then3>
  ...]]
  [<else>]
)

<condition>, <then>, and <else> can be any elements. However, each <condition>, <then>, and <else> must be only one element (possibly with multiple child elements). For convenience and clarity then and else can be used.

sys:then

sys:then()

Aliases: else

Then and else are the same as sequential, but can be used to make constructs using if more intuitive:

if(
  a == 1 
    then(print("a is 1"))
  a == 2
    then(print("a is 2"))
  else(print("a is not 1 nor 2"))
)

<if>
  <equals>
    <number>1</number>
    <variable>a</variable>
  </equals>
  <then>
    <print message = "a is 1"/>
  </then>
  <equals>
    <number>2</number>
    <variable>a</variable>
  </equals>
  <then>
    <print message = "a is 2"/>
  </then>
  <else>
    <print message = "a is not 1 nor 2"/>
  </else>
</if>

sys:exclusive

sys:exclusive()

Defines a mutual exclusion block. It guarantees that at any give time, within a given execution, only one instance of this (lexically) exclusive element is executing.

Elements Dealing with Variables and Arguments

sys:set

sys:set(names, ...)

Sets a variable or more to a value (or more). Set tries to interpret the first argument as an identifier or a list of identifiers. If the first argument is an identifier, it is treated as being a list with one identifier. Set expects the number of arguments on the default channel to be the same as the number of identifiers. It is important that a quoted list be used to specify the list of identifiers in order to avoid evaluating the variables that the identifiers represent:

set(a, 1)
set([a], 1)
set([a, b, c], 1, 2, 3)

Differences in XML:

The XML variant of the set element uses a slightly different set of arguments. If a single variable is assigned, the name argument can be used. If multiple variables are assigned, the names argument must be used. As opposed to the native syntax, the names argument can be a string of comma separated identifiers which will be tokenized by set

<set name="a" value="1"/>
<set names="a, b, c">
  <number>1</number>
  <number>2</number>
  <number>3</number>
</set>

sys:default

sys:default(name, value)

Assigns a value to a variable if no binding of that variable can be accessed within the current scope. If an accessible variable with the indicated name is already defined, then the assignment does not take place:

default(a, 1)
//a is assigned the value 1
set(b, 2)
default(b, 3)
//b is not assigned the value of 3 because
//b is already accessible within the current scope

sys:maybe

sys:maybe()

Evaluates its arguments. If the evaluation completes successfully, it returns all the arguments. If the evaluation fails at any point, maybe completes without returning anything. In particular, maybe can be used in extending existing elements with optional arguments:

element(one, [a, b, optional(c, d)]
  print("a = {a}", "b = {b}", maybe("c = {c}"), maybe("d = {d}"))
)

element(two, [a, b, optional(c, d)]
  one(a = a, b = b, maybe(c = c), maybe(d = d))
)

sys:global

sys:global(name, value)

Sets a global variable. A global variable is a variable that has a global scope, and thus can be accessed from anywhere within the program. While the use of global variables is discouraged, it can prove useful to create constant-like definitions.

Differences in XML:

The XML variant of global uses the same arguments as the XML variant of the set element.

sys:...()

Aliases: vargs

Can be used inside an element definition to return all arguments received on the default channel.

Differences in XML:

In the XML syntax the vargs alias must be used, because “...” is not a valid XML element name.

channel:to

channel:to(name, ...)

Returns all arguments received on the default channel on the specified channel.

channel:from

channel:from(name, <varies>)

Returns all arguments received on the specified channel on the default channel.

channel:close

channel:close(name)

Closes a channel. All iterations that are active on that channel and waiting for values will complete.

channel:fork

channel:fork(name, count)

Splits a channel into a number of identical channels and returns these channels. All values written to the initial channel will be available in all the forked channels. Reading from the original channel will cause inconsistencies and should not be done. When the original channel is closed, all the forked channels will also be closed.

sys:isDefined

sys:isDefined(name)

Determines whether a variable is accessible within the current scope. Returns true if it is; false otherwise.

sys:quoted

sys:quoted(name)

Returns an identifier without evaluating the variable it might point to.

sys:discard

sys:discard(...)

Sometimes only the side-effect of an element is needed, while ignoring the return values of the element. Discard will evaluate its arguments, but avoid returning anything on the default channel.

sys:future

sys:future(...)

Evaluates the arguments asynchronously. Returns a future representing the first return value generated by the arguments. All other arguments received on the default channel are ignored.

sys:futureIterator

sys:futureIterator(...)

Evaluates its arguments asynchronously. Returns a future iterator representing all values received on the default channel. The particular aspect of a future iterator is that it can only be iterated over once. Every time a value is used from a future iterator, that value is removed. If access to the iterator values is needed more than once, a list can safely be created with the values. However, the process of creating the list will force synchronization with the thread that produced the values since the iterator is only closed when that thread completes.

sys:each

sys:each(items)

Returns all elements in items as separate values. It is roughly equivalent to the following:

for(i, items, i)

Element Definition Elements

sys:element

sys:element(name, arguments)

Allows the definition of an element. Evaluates the name and arguments arguments. It expects the name argument to be an identifier, and arguments to be a list of identifiers. The scope of the definition is the same as the scope of a variable that could be defined instead of the element. The arguments are a list of mandatory arguments. Optional arguments can also be specified using the optional element. If the element accepts arguments on the default channel, the ... identifier can be used in the argument list. Other channels can be specified using the channel element. The rest of the arguments are not evaluated when the definition takes place, but will be evaluated whenever the element is invoked.

The following example defines an element foo, which takes no arguments, and prints ’foo’ on the console:

element(foo, []
  print("foo")
)

foo()

In the following example, foo takes two arguments and prints them both on the screen:

element(foo, [one, two]
  print(one)
  print(two)
)

foo(1, 2)

Arguments on the default channel can be accessed using the ... identifier:

element(foo, [one, ...]
  print(one)
  for(i, ...
    print(i)
  )
)

foo("one", 1, 2, 3, 4)

Other channels can be used in a similar way:

element(foo, [one, ..., channel(channelOne)]
  print(one)
  for(i, ..., print(i))
  for(i, channelOne, print(i))
)

foo("one", 1, 2, 3, 4, to(channelOne, 5, 6, 7, 8))

Optional arguments can be assigned a default value using default:

element(foo, [one, optional(two)]
  default(two, 2)
  print(one)
  print(two)
)

foo("one")
foo("one", two = "two")

Element can also be used to define anonymous elements. If the first argument evaluates to a list of identifiers instead of an identifier, element considers that an anonymous element was instead desired, defines the element, and returns the definition, which can later be used through executeElement:

set(foo,
  element([]
    print("Foo")
  )
)

executeElement(foo)

Each definition of an element keeps a reference to the environment that was used at the time of the definition. When evaluated, elements in the body of the definition will be resolved by first searching in the local scope (eventually for elements defined by the execution of the body of this element) and, if not found, in the environment that was used at the time of the definition. In the following example, the result will be the printing of the string "a":

element(foo, []
  element(a, [], print("a"))
  //return an anonymous element
  element([]
    a() //this is the a() defined above
  )
)

set(b, foo())
element(a, [], print("b"))
executeElement(b)

This behaviour is particularly important when import and export are used.

Differences in XML:

The arguments list is a string of comma separated identifiers.

The XML version of element uses a different set of arguments. If an element accepts arguments on the default channel, the vargs="true" attribute must be used. The arguments received on the default channel will then be available in the body of the definition through the vargs identifier.

Optional arguments are indicated using the optargs attribute. The value must be a comma separated list of identifiers.

In a similar way, the channels are specified using a comma separated list of identifiers and the channels attribute.

<element name = "foo" arguments = "one" vargs = "true" channels = "channelOne">
  <print message = "{one}"/>
  <for name = "i" in = "{vargs}">
    <print message = "{i}"/>
  </for>
  <for name = "i" in = "{channelOne}">
    <print message = "{i}"/>
  </for>
</element>

<foo one = "one">
  <number>1</number>
  <number>2</number>
  <number>3</number>
  <number>4</number>
  <to name="channelOne">
    <number>5</number>
    <number>6</number>
    <number>7</number>
    <number>8</number>
  </to>
</foo>

<element name ="foo" arguments = "one" optargs = "two">
  <default name = "two" value = "2"/>
  <print message = "{one}"/>
  <print message = "{two}"/>
</element>
 
<foo one = "one"/>
<foo one = "one" two = "two"/>

sys:parallelElement

sys:parallelElement(name, arguments)

Like element, parallelElement also defines an element. However, elements defined using parallelElement will evaluate their arguments in parallel with their bodies. All single value arguments will automatically be futures, and all channels will automatically be future iterators. ParallelElement can be used to define elements that process their arguments asynchronously.

parallelElement(consumer, [...]
  for(i, ..., print("Received {i}")
)

element(producer, []
  for(i, range(0, 100)
    i
    print("Sent {i}")
    wait(delay = 100)
  )
)

consumer(producer())

Differences in XML:

The differences for the XML syntax from element also apply to parallelElement

<parallelElement name = "consumer" vargs = "true">
  <for name = "i" in = "{vargs}">
    <print message = "Received {i}"/>
  </for>
</parallelElement>

<element name = "producer">
  <for name = "i">
    <range from = "0" to = "100"/>
    <variable>i</variable>
    <print message = "Sent {i}"/>
    <wait delay = "100"/>
  </for>
</element>

<consumer>
  <producer/>
</consumer>

sys:channel

sys:channel(...)

Used to specify channel arguments for an element. Quotes all arguments, such that identifiers are not evaluated. Returns values that can be interpreted by element and parallelElement as representing channels.

sys:optional

sys:optional(...)

Allows the specification of optional arguments for element definitions. Quotes all arguments and returns values that can be interpreted by element and parallelElement as representing optional arguments.

sys:self

sys:self()

Self can be used to build recursive anonymous elements:

set(f 
  element([x]
    if(x == 0 1 x*self(x - 1))
  )
)

print(executeElement(f, 6))

Service Interaction Elements

sys:remote

sys:remote(host)

Uses the Karajan:Service that runs on the specified host to evaluate the rest of the arguments. The element supports forwarding of the variable environment to the remote service ^[1], and returning values from the service. Therefore the following code will work as expected:

set(a, 1)
set(b
  remote("https://somehost:1984", a+1)
)
print(b) //will print 2

A consequence is that if print is used remotely, the actual output is going to be automatically and transparently forwarded (since it is merely a return value on a particular channel) to the execution instance that initiated the remote execution.

It is also possible to use this feature recursively, such that a remote code snippet in turn delegates part of the execution to other services.

Please note that the code inside remote might reference entities that are sensitive to the actual location of the execution. In particular, the meaning of “localhost” if used with execute or transfer will not refer to the machine where the execution originated.

Element resolution is performed separately when a script is submitted to a service. In other words, import-ed files on the client side will be re-imported on the service side, if and only if they are system libraries. If imports refer to user-defined libraries not part of the system libraries, the code defining the libraries will be forwarded to the service, thus enabling the use of client-side user-defined elements on the remote side. Similarly, in-line user defined elements are also available for use on the remote side:

element(foo, [], print("Foo"))
remote("https://somehost:1984"
  foo()
)

List Manipulation Elements

list:list

list:list(*items, ...)

Constructs a list from values received on the default channel.

Alternatively the *items argument can be used to specify a a string with comma separated list of items. This may be particularly convenient with the XML syntax.

list:append

list:append(list, *items, ...)

Appends all values received on the default channel to the list indicated by the list argument. Does not return anything.

Alternatively, the *items argument could be used with a string of comma separated items.

list:prepend

list:prepend(list, ...)

Works like append with the exception that values are added to the beginning of the list. The order of the values in the list will be the reverse of the order in which they are received by prepend:

set(l, list(4, 5, 6))
prepend(l, 1, 2, 3)
print(l)

will print [3, 2, 1, 4, 5, 6]

list:join

list:join(...)

Concatenates all lists received on the default channel and returns the resulting list.

list:size

list:size(list)

Returns the size of the list indicated by the list argument.

list:first

list:first(list)

Returns the first element in a list.

list:last

list:last(list)

Returns the last element in a list.

list:butFirst

list:butFirst(list)

Returns a list composed of all but the first element in the specified list.

list:butLast

list:butLast(list)

Returns a list containing all elements but the last from the specified list.

list:isEmpty

list:isEmpty(list)

Tests whether a list is empty. Returns true if the list is empty, and false otherwise.

Map Elements

map:map

map:map(...)

Returns a map with the entries received on the default channel. See entry.

map:entry

map:entry(key, value)

Allows the specification of an entry that can be used with map to construct a map.

map:put

map:put(map, ...)

Adds the entries received on the default channel to the specified map. Existing entries with the same key are replaced. Does not return any value.

map:delete

map:delete(map, key)

Deletes the entry with the specified key from a map. Does not return a value.

map:get

map:get(map, key)

Returns the value corresponding to the specified key from a map.

map:size

map:size(map)

Returns the size of the map (the number of entries in the map).

map:contains

map:contains(map, key)

Tests whether the map contains an entry with the specified key.

Logic Elements

Logic elements do not at this time use shortcut evaluation. They always evaluate all their arguments.

sys:and

sys:and(...)

Aliases: &

Returns the boolean and value of the arguments received on the default channel.

sys:or

sys:or(...)

Aliases: |

Returns the boolean or value of the arguments received on the default channel.

sys:not

sys:not(value)

Returns the boolean negation of the value in the value argument.

sys:equals

sys:equals(value1, value2)

Aliases: ==

Tests for the equality of two values. Makes a deep comparison of the arguments.

sys:true

sys:true()

Returns true.

sys:false

sys:false()

Returns false.

Numeric Elements

math:sum

math:sum(...)

Aliases: +

Returns the sum of all the values received on the default channel.

math:product

math:product(...)

Aliases: *

Returns the product of all the values received on the default channel.

math:subtraction

math:subtraction(from, value)

Aliases: -

Returns the difference between the value specified by the from argument and the value indicated by the value argument.

math:quotient

math:quotient(divisor, dividend)

Aliases: /

Divides divisor by dividend and returns the resulting value.

math:remainder

math:remainder(divisor , dividend)

Aliases: %

Returns the remainder of the division of divisor and dividend

math:square

math:square(value)

Returns the square of a number.

math:sqrt

math:sqrt(value)

Returns the square root of a number.

math:equalsNumeric

math:equalsNumeric(value1, value2)

Makes a numeric comparison of the values specified by the two arguments. A numeric comparison will try to convert string values to numbers before the comparison is performed. Like equals, equalsNumeric performs a deep comparison.

equalsNumeric(1, "1") //true
equalsNumeric("2", "2.0") //true
equals("2", 2) //false
equalsNumeric([1, 2, "3"], ["1", "2", 3]) //true

<equalsNumeric value1 = "1">
  <number

Java CoG Kit Karajan Workflow Reference Manual 4.1.4

From Java CoG Kit

Contents

About this Document

Command Line

Options

Experiemental Options

File

Arguments

The Karajan Language

The Karajan Syntax

Elements

Identifiers

Arguments

Named Arguments

Unnamed Arguments

Expressions

Operators

Quoted Lists

Programs

Comments

The XML Syntax

Particularities of Using XML

Parameters and Return Values

Parameters and Arguments

Single Value Arguments

Channels

Argument Mapping

Optional Arguments

Return values

Argument Evaluation Order

Variables and Scope

Global Variables

Variable Expansion

Futures

Modularisation

Source Files

Libraries

Namespaces

Kernel Library

Kernel Constants

kernel:true

kernel:false

kernel:cmdline:arguments

kernel:user.home

kernel:user.name

Kernel Elements

kernel:project

kernel:import

kernel:export

kernel:define

kernel:namespace

kernel:elementdef

kernel:named

kernel:number

kernel:string

kernel:variable

kernel:quotedlist

kernel:cache

System Library

Flow Control Elements

sys:sequential

sys:parallel

sys:unsynchronized

sys:choice

sys:catch

sys:guard

sys:race

sys:for

sys:parallelFor

sys:while

sys:condition

sys:break

sys:continue

sys:if

sys:then

sys:exclusive

Elements Dealing with Variables and Arguments

sys:set

sys:default