Condor
From Chemical Informatics and Cyberinfrastructure Collaboratory
Installing Condor
Notes by Marlon.
First, there are some excellent notes here: http://docs.optena.com/display/CONDOR/A+Simple+Linux+Installation I successfully followed these instructions, more or less, to get Condor installed on gf1.ucs.indiana.edu (master node) and gf3.ucs.indiana.edu (worker node). I made an account /home/condor, but I installed everything under /usr/local/condor. Although gf1 and gf3 share a file system, I decided not to take advantage of this for generality's sake.
One quick note: don't forget to set the CONDOR_CONFIG variables before starting the daemons.
export CONDOR_CONFIG=/usr/local/condor/etc/condor_config
- You will also need to set this before you use condor as a regular user.
Start the master up with
[shell@gf1> /usr/local/condor/sbin/condor_master
Note on both gf1 and gf3 there is an older condor installation included in the NMI release under /home/globus, which you don't want to use, so give the full path. If you set GLOBUS_LOCATION, you will get this older version in your path.
To start the worker condor on gf3, you first need to run condor_init and then condor_master.
[shell@gf3> /usr/local/condor/sbin/condor_init [shell@gf3> /usr/local/condor/sbin/condor_master
BirdBath
BirdBath is Condor's SOAP server project stuff. This requires a bit of additional configuration. Thanks to Matthew Farrellee for help with this section.
First, make sure you have the 6.7 series. BB is included in the release but not enabled. To enable it, do this
- Create the file /usr/local/condor/web
- Grab the Schedd and Collector WSDL files from http://www.cs.wisc.edu/condor/birdbath/. I just used wget.
- Edit the condor_config file to add the following lines
[condor@gridfarm001 condor]$ tail etc/condor_config #These are needed by birdbath WEB_ROOT_DIR=$(RELEASE_DIR)/web ENABLE_SOAP = TRUE ALLOW_SOAP = *.ucs.indiana.edu ENABLE_WEB_SERVER = TRUE SCHEDD_ARGS = -p 345093
ALLOW_SOAP is used to provide some access control on the SOAP server. The last, SCHEDD_ARGS, is optional but useful because it lets you specify the schedular port.
Sample Java Code with BirdBath
You can download examples from the BirdBath web site, but you will need Java 1.5 to use the jars. You will also need to get a copy of the Axis jars and put them in your classpath. Finally, you will need to get your hands on the client stubs generated from the WSDL. Or you can make them yourself in the usual way.
You can also do it the harder way (nothing is easy), as shown below. First, generate client stubs from the BirdBath WSDL files. Compile them and put them in a jar in your classpath. I'm actually going to skip this step and use stubs generated by the TACC condor portlet in the OGCE project.
Create a new file called SubmitExample2.java and edit it. Start with imports.
//These are my stubs. import edu.tacc.gridport.web.services.condor.*; import edu.tacc.gridport.web.services.condor.stubs.*; //These are from axis. import java.rmi.*; import javax.xml.rpc.*; //These are standard java import java.io.*; import java.net.*;
The first two sets of imports must correspond to jars in your classpath.
Below is some junky code that illustrates the basic concepts. It shows how to run the "ls" command.
public class SubmitExample2 {
public static void main(String[] arguments) throws Exception {
String jobHandle = null;
String scheddLocationStr = null;
String condorCollectorUrl=arguments[0];
URL collectorLocation = new URL(condorCollectorUrl);
CondorCollectorLocator collectorLocator = new CondorCollectorLocator();
CondorCollectorPortType collector = collectorLocator.getcondorCollector(collectorLocation);
ClassAdStructArray casArray = collector.queryScheddAds("HasSOAPInterface=?=TRUE");
ClassAdStruct[] casArr = casArray.getItem();
int i = 0;
for(i=0; i<casArr.length; i++ ) {
ClassAdStructAttr[] casAttrArr = casArr[i].getItem();
for (int j=0; j<casAttrArr.length; j++) {
ClassAdStructAttr casAttr = casAttrArr[j];
if(casAttr.getName().equals("ScheddIpAddr")) {
scheddLocationStr=casAttr.getValue();
}
}
}
String tmpStr="http://" + scheddLocationStr.substring(1, scheddLocationStr.length()-1);
URL scheddLocation = new URL(tmpStr);
CondorScheddLocator scheddLocator = new CondorScheddLocator();
CondorScheddPortType schedd = scheddLocator.getcondorSchedd(scheddLocation);
CondorJob condorJob = new CondorJob();
OK, the above mysterious code allows us to get access to the condor scheduler by first talking to the collector. We are ready to set up the job.
String jobName = "junk";
String executable = "/bin/ls";
String cmd_args = "/tmp";
String input = null;
//Note the UniverseType was grokked from the WDSL
UniverseType universeType = UniverseType.VANILLA;
String requirementString="OpSys==\"LINUX\"";
The UniverseType is a string constant that is set in the schedd.wsdl file. If you have some familiarity with WSDL, XML schema, and their java bindings, you may have been able to guess this (or you could just weedwack through the generated client stubs if you have the source code).
The requirementString is a list of Condor requirements in usual condor syntax.
String[] filesToSend = null;
if (input == null) {
filesToSend = new String[1];
filesToSend[0] = executable;
} else {
filesToSend = new String[2];
filesToSend[0] = executable;
filesToSend[1] = input;
}
The code above specifies that you will actually move the executable to the remote file. This will actually be placed in a subdirectory of $condor_home/spool. If you have an input file, this will also be uploaded.
//This shows how to capture standard output...I think.
ClassAdStructAttr[] otherAttr=new ClassAdStructAttr[2];
otherAttr[0]=new ClassAdStructAttr("Out",ClassAdAttrType.value3,"junkerrelli.out");
otherAttr[1]=new ClassAdStructAttr("Err",ClassAdAttrType.value3,"junkerrelli.err");
The above code shows how to specify standard output and error. This is somewhat confusing since these are usually specified to be "output" and "error" in Condor examples, but they are actually translated to "Out" and "Err". You have to go to $condor_home/spool/job_queue.log to get the real story. Also, your stdout and stderr will be left in your spool subdirectory. They will not be moved to some more reasonable place (like your $HOME). Specifying complete paths (like /tmp/junk.out) will not change this.
Finally, we can submit the job. This must run as someone, but this name appears to be only a superficial handle and not connected to any actual accounts, at least if you run as non-root.
String condorRunsAsName="condor";
//String condorRunsAsName="kjdorjeljr"; //This would also work.
condorJob = SOAPScheddApiHelper.submitJobHelper(schedd, null, -1, -1,
condorRunsAsName,
jobName,
universeType,
executable,
cmd_args,
requirementString,
otherAttr,
filesToSend);
jobHandle = condorJob.getJobHandle();
}
}
