Job submission and management with NorduGrid ARC - Revision history

Lhep: /* arcinfo */

2026-05-04T09:51:14Z

arcinfo

← Older revision		Revision as of 11:51, 4 May 2026
Line 23:		Line 23:
	# arcinfo --version		# arcinfo --version
	# arcinfo -c ce00.lhep.unibe.ch:'''8443'''		# arcinfo -c ce00.lhep.unibe.ch:'''8443'''
	# arcinfo -c ~~ce00~~.lhep.unibe.ch:'''443'''		# arcinfo -c ce02.lhep.unibe.ch:'''443'''
	# arcinfo -c ce03.lhep.unibe.ch:'''443'''		# arcinfo -c ce03.lhep.unibe.ch:'''443'''

Lhep: /* Job submission */

2026-05-04T06:15:28Z

Job submission

← Older revision		Revision as of 08:15, 4 May 2026
Line 65:		Line 65:
	Additional resources on job submission and job description:		Additional resources on job submission and job description:

	[https://www.nordugrid.org/arc/~~arc6~~/users/submit_job.html~~?highlight=job%20description arc6~~ submit job]		[https://www.nordugrid.org/arc/arc7/users/submit_job.html arc7 submit job]

	[http://www.nordugrid.org/documents/xrsl.pdf xrls job specification reference manual]		[http://www.nordugrid.org/documents/xrsl.pdf xrls job specification reference manual]

Lhep: /* arcinfo */

2026-05-04T06:12:54Z

arcinfo

← Older revision		Revision as of 08:12, 4 May 2026
Line 22:		Line 22:

	# arcinfo --version		# arcinfo --version
	# arcinfo -c ce00.lhep.unibe.ch:8443		# arcinfo -c ce00.lhep.unibe.ch:'''8443'''
	# arcinfo -c ce00.lhep.unibe.ch:443		# arcinfo -c ce00.lhep.unibe.ch:'''443'''
	# arcinfo -c ce03.lhep.unibe.ch:443		# arcinfo -c ce03.lhep.unibe.ch:'''443'''

	=== arcproxy / voms-proxy-init/info ===		=== arcproxy / voms-proxy-init/info ===

Lhep: /* Job submission */

2026-05-04T06:12:30Z

Job submission

← Older revision		Revision as of 08:12, 4 May 2026
Line 58:		Line 58:
	Job submitted with jobid: <job ID>		Job submitted with jobid: <job ID>

			# arcsub -d info -T arcrest --computing-element ce02.lhep.unibe.ch:'''443''' -o joblist.xml test.xrsl
	# arcsub -d info -T arcrest --computing-element ce03.lhep.unibe.ch:'''443''' -o joblist.xml test.xrsl		# arcsub -d info -T arcrest --computing-element ce03.lhep.unibe.ch:'''443''' -o joblist.xml test.xrsl

Lhep: /* arcinfo */

2026-05-04T06:12:04Z

arcinfo

← Older revision		Revision as of 08:12, 4 May 2026
Line 19:		Line 19:

	=== arcinfo ===		=== arcinfo ===
	First, verify that the user interface is installed and you can see the submission endpoints		First, verify that the user interface is installed and you can see the submission endpoints. We have 3 CEs:

	# arcinfo --version		# arcinfo --version
	# arcinfo -c ce00.lhep.unibe.ch		# arcinfo -c ce00.lhep.unibe.ch:8443
	# arcinfo -c ce03.lhep.unibe.ch		# arcinfo -c ce00.lhep.unibe.ch:443
			# arcinfo -c ce03.lhep.unibe.ch:443

	=== arcproxy / voms-proxy-init/info ===		=== arcproxy / voms-proxy-init/info ===

Lhep at 10:30, 28 July 2025

2025-07-28T10:30:14Z

← Older revision		Revision as of 12:30, 28 July 2025
Line 22:		Line 22:

	# arcinfo --version		# arcinfo --version
	# arcinfo -c ~~ce01.lhep.unibe.ch~~		# arcinfo -c ce00.lhep.unibe.ch
	~~# arcinfo -c ce02~~.lhep.unibe.ch
	# arcinfo -c ce03.lhep.unibe.ch		# arcinfo -c ce03.lhep.unibe.ch

	=== arcproxy / voms-proxy-init/info ===		=== arcproxy / voms-proxy-init/info ===
	Generates proxy with your credentials:		Generates proxy with your credentials with one of these two commands:

	# arcproxy --voms <yourVO> (e.g. atlas)		# arcproxy --voms <yourVO> (e.g. atlas)
	# voms-proxy-int --voms <yourVO)		# voms-proxy-int --voms <yourVO)

	View the generated proxy information:		View the generated proxy information with one of these two commands

	# arcproxy -I		# arcproxy -I
Line 54:		Line 53:
	(queue="CLUSTER")		(queue="CLUSTER")

	# arcsub -d info -T ~~emies~~ --computing-element ~~ce01~~.lhep.unibe.ch:'''8443''' -o joblist.xml test.xrsl ~~-d info~~		# arcsub -d info -T arcrest --computing-element ce00.lhep.unibe.ch:'''8443''' -o joblist.xml test.xrsl
	...		...
	Job submitted with jobid: <job ID>		Job submitted with jobid: <job ID>

	arcsub -d info -T ~~emies --computing-element ce02.lhep.unibe.ch:'''443''' -o joblist.xml test.xrsl -d info~~		# arcsub -d info -T arcrest --computing-element ce03.lhep.unibe.ch:'''443''' -o joblist.xml test.xrsl
	~~arcsub -d info -T emies~~ --computing-element ce03.lhep.unibe.ch:'''443''' -o joblist.xml test.xrsl ~~-d info~~

	In the job description file, one can also add a list of input files to upload to the cluster at job submission and a list of output files to retrieve once execution is FINISHED.		In the job description file, one can also add a list of input files to upload to the cluster at job submission and a list of output files to retrieve once execution is FINISHED.
Line 72:		Line 70:
	Lists the job directory on the cluster		Lists the job directory on the cluster

	#arcls <job id>		# arcls <job id>

	=== arcstat ===		=== arcstat ===
	Prints job status and some additional information from cluster, such as jobid, name, and status:		Prints job status and some additional information from cluster, such as jobid, name, and status:

	# arcstat -c ~~ce01~~.lhep.unibe.ch		# arcstat -c ce00.lhep.unibe.ch
	# arcstat <job id>		# arcstat <job id>

Line 93:		Line 91:

	=== arcget ===		=== arcget ===
	When the status of the task is FINISHED, you can download the results of one specific task or multiple tasks in directories named after the job id's. In the example above, only the stdout/stderr file test.log will be downloaded as there are no further output files specified:		When the status of the task is FINISHED, you can download the results of one specific task or multiple tasks. These and up in directories created by the download client and named after the job id's. In the example above, only the stdout/stderr file test.log will be downloaded as there are no further output files specified:

	# arcget <job id>		# arcget <job id>

Lhep: /* Advanced xrls using DTR and singularity */

2022-07-15T09:52:24Z

Advanced xrls using DTR and singularity

← Older revision		Revision as of 11:52, 15 July 2022
Line 125:		Line 125:
	(outputfiles=("/" ""))		(outputfiles=("/" ""))

	This specific job is a clone Panda of JobID 5333700233 to run on 8 cores within singularity		This specific job is a clone of Panda JobID 5333700233 to run on 8 cores within singularity

	NOTE: in this example the input files are uploaded from the local user directory, meaning you have to download them with rucio first. You can find the URI of the files to download with:		NOTE: in this example the input files are uploaded from the local user directory, meaning you have to download them with rucio first. You can find the URI of the files to download with:

Lhep: /* Advanced xrls using DTR and singularity */

2022-07-15T09:51:12Z

Advanced xrls using DTR and singularity

← Older revision		Revision as of 11:51, 15 July 2022
Line 111:		Line 111:
	In this example we will use the DTR (Data TRansfer) feature of ARC to specify input files to copy to the cluster and retrieve the outputs. In addition, the job is setup to run in a container which is the default way for ATLAS.		In this example we will use the DTR (Data TRansfer) feature of ARC to specify input files to copy to the cluster and retrieve the outputs. In addition, the job is setup to run in a container which is the default way for ATLAS.

	# cat arc_test-lhep-~~16core~~.xrls		# cat arc_test-lhep-8core.xrls
	&(executable=5333700233-~~16core~~.sh)		&(executable=5333700233-8core.sh)
	(inputfiles=(container_script-~~16core~~.sh container_script-~~16core~~.sh)(my_release_testsetup.sh my_release_testsetup.sh)(HITS.25720417._000948.pool.root.1 HITS.25720417._000948.pool.root.1)(RDO.26819764._019417.pool.root.1 RDO.26819764._019417.pool.root.1)(RDO.26819764._016665.pool.root.1 RDO.26819764._016665.pool.root.1)(RDO.26811885._009174.pool.root.1 RDO.26811885._009174.pool.root.1)(RDO.26819766._027499.pool.root.1 RDO.26819766._027499.pool.root.1)(RDO.26811885._004931.pool.root.1 RDO.26811885._004931.pool.root.1))		(inputfiles=(container_script-8core.sh container_script-8core.sh)(my_release_testsetup.sh my_release_testsetup.sh)(HITS.25720417._000948.pool.root.1 HITS.25720417._000948.pool.root.1)(RDO.26819764._019417.pool.root.1 RDO.26819764._019417.pool.root.1)(RDO.26819764._016665.pool.root.1 RDO.26819764._016665.pool.root.1)(RDO.26811885._009174.pool.root.1 RDO.26811885._009174.pool.root.1)(RDO.26819766._027499.pool.root.1 RDO.26819766._027499.pool.root.1)(RDO.26811885._004931.pool.root.1 RDO.26811885._004931.pool.root.1))
	(arguments="")		(arguments="")
	(gmlog="gridlog")		(gmlog="gridlog")

Lhep: /* Prerequisites */

2022-07-15T09:46:50Z

Prerequisites

← Older revision		Revision as of 11:46, 15 July 2022
Line 7:		Line 7:
	In order to carry out the following steps, your user certificate and private key must be installed in the .globus subdirectory of your home directory on the client machine. You can find instructions on how to do that in [https://atlassoftwaredocs.web.cern.ch/ASWTutorial/basicSetup/grid_vo/#using-the-grid-certificate here]. While this focuses on ATLAS, the steps to extract and install the certificate are independent from the specific experiment.		In order to carry out the following steps, your user certificate and private key must be installed in the .globus subdirectory of your home directory on the client machine. You can find instructions on how to do that in [https://atlassoftwaredocs.web.cern.ch/ASWTutorial/basicSetup/grid_vo/#using-the-grid-certificate here]. While this focuses on ATLAS, the steps to extract and install the certificate are independent from the specific experiment.

	* Have the certificate registered with a Virtual Organisation (VO), e.g. for ATLAS, browse to the [https://lcg-voms2.cern.ch:8443/voms/atlas/user/home.action ATLAS VOMS service]. The prerequisite for this step is having installed your personal certificate in your browser		Have the certificate registered with a Virtual Organisation (VO), e.g. for ATLAS, browse to the [https://lcg-voms2.cern.ch:8443/voms/atlas/user/home.action ATLAS VOMS service]. The prerequisite for this step is having installed your personal certificate in your browser

	* '''ARC client'''		* '''ARC client'''

Lhep: /* Prerequisites */

2022-07-15T09:45:34Z

Prerequisites

← Older revision		Revision as of 11:45, 15 July 2022
Line 5:		Line 5:
	This is best obtained from CERN: [https://ca.cern.ch/ca/ CERN CA]. Being registered with CERN HR and having a CERN computer account are the pre-requisites. Users with FNAL accounts can obtain a certificate from the relevant FERMILAB CA. From those not eligible to either solution above, there is the possibility of requesting a certificate via the [https://www.scits.unibe.ch/services/escience_certificates/ Swiss CA]		This is best obtained from CERN: [https://ca.cern.ch/ca/ CERN CA]. Being registered with CERN HR and having a CERN computer account are the pre-requisites. Users with FNAL accounts can obtain a certificate from the relevant FERMILAB CA. From those not eligible to either solution above, there is the possibility of requesting a certificate via the [https://www.scits.unibe.ch/services/escience_certificates/ Swiss CA]

	In order to carry out the following steps, your user certificate and private key must be installed in the .globus subdirectory of your home directory on the client machine. You can find instructions on how to do that in [https://atlassoftwaredocs.web.cern.ch/ASWTutorial/basicSetup/grid_vo/#using-the-grid-certificate here]. While this focuses on ATLAS, the steps to extract and install the certificate are ~~the~~ independent from the specific experiment.		In order to carry out the following steps, your user certificate and private key must be installed in the .globus subdirectory of your home directory on the client machine. You can find instructions on how to do that in [https://atlassoftwaredocs.web.cern.ch/ASWTutorial/basicSetup/grid_vo/#using-the-grid-certificate here]. While this focuses on ATLAS, the steps to extract and install the certificate are independent from the specific experiment.

	* Have the certificate registered with a Virtual Organisation (VO), e.g. for ATLAS, browse to the [https://lcg-voms2.cern.ch:8443/voms/atlas/user/home.action ATLAS VOMS service]. The prerequisite for this step is having installed your personal certificate in your browser		* Have the certificate registered with a Virtual Organisation (VO), e.g. for ATLAS, browse to the [https://lcg-voms2.cern.ch:8443/voms/atlas/user/home.action ATLAS VOMS service]. The prerequisite for this step is having installed your personal certificate in your browser