Job submission and management with NorduGrid ARC

Prerequisites

To job submission to our cluster occurs via Nordugrid ARC middleware. You need:

• Grid user certificate

This is best obtained from CERN: CERN CA. Being registered with CERN HR and having a CERN computer account are the pre-requisites. Users with FNAL accounts can obtain a certificate from the relevant FERMILAB CA. From those not eligible to either solution above, there is the possibility of requesting a certificate via the Swiss CA

In order to carry on the following steps, your user certificate and private key must be installed in the .globus subdirectory of your home directory on the client machine. You can find instructions on how to do that in here. While this focuses on ATLAS, the steps to extract and install the certificate are the independent from the specific experiment.

• Have the certificate registered with a Virtual Organisation (VO), e.g. for ATLAS, browse to the [https://lcg-voms2.cern.ch:8443/voms/atlas/user/home.action ATLAS VOMS service). The prerequisite for this step is having installed your personal certificate in your browser

• ARC client

Users operating from the UIs (recommended) can setup the client from CVMFS:

# export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase
# source ${ATLAS_LOCAL_ROOT_BASE}/user/atlasLocalSetup.sh
# lsetup emi

Otherwise installing a local client on any linux machine is possible by following the instructions for your specific operating system: Install ARC client

arcinfo

First, verify that the user interface is installed and you can see the submission endpoints

# arcinfo --version
# arcinfo -c ce01.lhep.unibe.ch
# arcinfo -c ce02.lhep.unibe.ch
# arcinfo -c ce03.lhep.unibe.ch

arcproxy / voms-proxy-init/info

Generates proxy with your credentials:

# arcproxy --voms <yourVO>  (e.g. atlas)
# voms-proxy-int --voms <yourVO)

View the generated proxy information:

# arcproxy -I
# voms-proxy-info

Job Submission

Check to see if you can submit a trivial job test.xrsl:

# vi test.xrsl
&
(executable = /usr/bin/env)
(jobname = "test")
(stdout=test.log)
(* join stdout and stderr *)
(join=yes)
(gmlog=log)
(wallTime="100")
(memory=1000)
(queue="CLUSTER")

# arcsub -S org.ogf.glue.emies.activitycreation -c ce01.lhep.unibe.ch -o joblist.xml test.xrsl -d DEBUG
...
Job submitted with jobid: <job ID>

We have 3 submission endpoints: ce01, ce02, ce03

Additional resources on job submission and job description:

arc6 submit job

xrls job specification reference manual

arcstat (Cluster job status)

Prints job status and some additional information from cluster, such as jobid, name, and status:

# arcstat -c ce01.lhep.unibe.ch
# arcstat <job id>

arckill (One job, a list of jobs in filename or all jobs)

# arckill <job id>
# arckill filename
# arckill -a

arcget

When the status of the task is FINISHED, you can download the results of one specific task:

# arcget <job id>
Results stored at: 4fQLDmY3BxjnmmR0Xox1SiGmABFKDmABFKDmvxHKDmABFKDmiPhU9m
Jobs processed: 1, successfully retrieved: 1, successfully cleaned: 1

or results of all completed tasks:

# arcget -a

or all tasks in the list in the joblist.xml file:

# arcget -i joblist.xml