How to use the clusters: Difference between revisions
No edit summary |
No edit summary |
||
| (8 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
* '''The AEC clusters can only be accessed via Grid tools. There is no local login on to the clusters for direct batch submission.''' <br/>The cluster endpoints are: '''<code>ce01.lhep.unibe.ch</code>, <code>ce02.lhep.unibe.ch</code>''' | * '''The AEC HTC clusters can only be accessed via Grid tools (ARC middleware). There is no local login on to the clusters for direct batch submission.''' <br/>The cluster endpoints are: '''<code>ce01.lhep.unibe.ch</code>, <code>ce02.lhep.unibe.ch</code>''' | ||
* '''Prerequisite: you need a valid user eScience certificate''' in order to access the Grid. For all Swiss institutes, the certificate issuing and yearly renewal is handled by the Science IT Support unit at the University of Bern. Follow the procedure detailed [http://www.scits.unibe.ch/services/escience_certificates/certificate_signing_request here]. The procedure creates a file ''userkey.pem''. This is your private key, keep it safe. | * '''Prerequisite: you need a valid user eScience certificate''' in order to access the clusters via the Grid. <br/>The certificate represent your digital identity in the eScience world. For all Swiss institutes, the certificate issuing and yearly renewal is handled by the Science IT Support unit at the University of Bern. Follow the procedure detailed [http://www.scits.unibe.ch/services/escience_certificates/certificate_signing_request here]. The procedure creates a file ''userkey.pem''. This is your private key, keep it safe. | ||
* '''You will receive the signed certificate by email from SEE-GRID CA ithin 4 working days'''. The certificate itself is an attachment to the email: ''<serial nr>.pem''. Copy it to your home directory and change the name to ''usercert.pem''. Put both files in the following directory (create it if it isn't there already) and set the correct permissions: | * '''You will receive the signed certificate by email from SEE-GRID CA ithin 4 working days'''. <br/>The certificate itself is an attachment to the email: ''<serial nr>.pem''. Copy it to your home directory and change the name to ''usercert.pem''. Put both files in the following directory (create it if it isn't there already) and set the correct permissions: | ||
mkdir $HOME/.globus | mkdir $HOME/.globus | ||
| Line 17: | Line 17: | ||
fs setacl -dir $HOME/.globus -acl system:anyuser l | fs setacl -dir $HOME/.globus -acl system:anyuser l | ||
The user's Grid certificate/key pair (''usercert.pem'' and ''userkey.pem'') can be copied to any other machine to access the Grid simply by copying the ''$HOME/.globus'' directory. The security measures described above have to be repeated. | The user's Grid certificate/key pair (''usercert.pem'' and ''userkey.pem'') can be copied to any other machine to access the Grid simply by copying the ''$HOME/.globus'' directory. In alternative, it can be restored from a backup .p12 bundle (see below ''"restore a backup of your certificate"''). The security measures described above have to be repeated. | ||
| Line 33: | Line 33: | ||
* '''Enroll in the appropriate Virtual Organisation (VO) for your experiment''', by visiting their VOMS (Virtual Organization Management Service) server. On the AEC clusters, the following experiments are currently supported | * '''Enroll in the appropriate Virtual Organisation (VO) for your experiment''', by visiting their VOMS (Virtual Organization Management Service) server. On the AEC clusters, the following experiments are currently supported: | ||
atlas - [https://lcg-voms2.cern.ch:8443/voms/atlas/register/start.action VOMS server] | atlas - [https://lcg-voms2.cern.ch:8443/voms/atlas/register/start.action VOMS server] | ||
| Line 42: | Line 42: | ||
* '''Grid ARC client setup''' | |||
'''Our clusters run the ARC (Advanced Resource Connector), middleware''' from Nordugrid.<br/> The general documentation can be found [http://www.nordugrid.org/arc/about-arc.html here]. | |||
'''ARC client installation and user manual reference can be found [http://www.nordugrid.org/manuals.html#client here]''' <br/>This gives you the ARC tools like <code>arcsub</code>, <code>arcls</code>, <code>arcstat</code>, <code>arcget</code> etc. With those you can submit and manage your jobs. | |||
'''NOTE: I strongly advise to use ARC clients from CVMFS ( [https://cernvm.cern.ch/portal/filesystem CernVM File System] ) as opposed to a local installation''', since you are very likely to be needing CVMFS access anyway on your client machine.<br/> The ATLAS User Interface servers are setup with CVMFS. | |||
Make sure you have a file | Make sure you have a file ''$HOME/.voms/vomses'' including one or more of the following lines (create it with your editor): | ||
" | "atlas" "lcg-voms.cern.ch" "15001" "/DC=ch/DC=cern/OU=computers/CN=lcg-voms.cern.ch" "atlas" | ||
"fermilab" "voms.fnal.gov" "15001" "/DC=org/DC=doegrids/OU=Services/CN=http/voms.fnal.gov" "fermilab" | |||
"t2k.org" "voms.gridpp.ac.uk" "15003" "/C=UK/O=eScience/OU=Manchester/L=HEP/CN=voms.gridpp.ac.uk t2k.org" "t2k.org" | |||
'''NOTE: under review from here on ...''' | |||
To set up ARC (standalone) choose the needed configuration from [http://download.nordugrid.org/standalone.html] and | To set up ARC (standalone) choose the needed configuration from [http://download.nordugrid.org/standalone.html] and | ||
Latest revision as of 17:22, 29 May 2017
- The AEC HTC clusters can only be accessed via Grid tools (ARC middleware). There is no local login on to the clusters for direct batch submission.
The cluster endpoints are:ce01.lhep.unibe.ch,ce02.lhep.unibe.ch
- Prerequisite: you need a valid user eScience certificate in order to access the clusters via the Grid.
The certificate represent your digital identity in the eScience world. For all Swiss institutes, the certificate issuing and yearly renewal is handled by the Science IT Support unit at the University of Bern. Follow the procedure detailed here. The procedure creates a file userkey.pem. This is your private key, keep it safe.
- You will receive the signed certificate by email from SEE-GRID CA ithin 4 working days.
The certificate itself is an attachment to the email: <serial nr>.pem. Copy it to your home directory and change the name to usercert.pem. Put both files in the following directory (create it if it isn't there already) and set the correct permissions:
mkdir $HOME/.globus
chmod go-rx $HOME/.globus
chmod 400 $HOME/.globus/userkey.pem
chmod 600 $HOME/.globus/usercert.pem
If the $HOME/.globus directory holding the certificate resides in an afs home-directory (e.g. lxplus), the directory has to be further secured using afs-tools in addition to set the normal unix file access permissions.
fs setacl -dir $HOME/.globus -acl system:anyuser l
The user's Grid certificate/key pair (usercert.pem and userkey.pem) can be copied to any other machine to access the Grid simply by copying the $HOME/.globus directory. In alternative, it can be restored from a backup .p12 bundle (see below "restore a backup of your certificate"). The security measures described above have to be repeated.
- Load the certificate on to your browser. This will allow you to access any web-based resources accessible only to grid users. Run the following command:
openssl pkcs12 -export -in usercert.pem -inkey userkey.pem -out mycert.p12
The command creates the bundle mycert.p12 which can be imported into your browser (ans also mail client, OS keychain, etc.). Procedure for importing the .p12 bundle differ from browser to browser. E.g. with Firefox: Preferences->Advanced->Certificates->View certificate->Import. Safari makes use of the Mac OS keychain.
- Optional: restore a backup of your certificate. You can recreate the certificate/key pair from the .p12 bundle. Export the bundle from your browser and run the following commands:
openssl pkcs12 -in mycert.p12 -clcerts -nokeys -out $HOME/.globus/usercert.pem
openssl pkcs12 -in mycert.p12 -nocerts -out $HOME/.globus/userkey.pem
- Enroll in the appropriate Virtual Organisation (VO) for your experiment, by visiting their VOMS (Virtual Organization Management Service) server. On the AEC clusters, the following experiments are currently supported:
atlas - VOMS server
fermilab - VOMS server
t2k.org - VOMS server
- Grid ARC client setup
Our clusters run the ARC (Advanced Resource Connector), middleware from Nordugrid.
The general documentation can be found here.
ARC client installation and user manual reference can be found here
This gives you the ARC tools like arcsub, arcls, arcstat, arcget etc. With those you can submit and manage your jobs.
NOTE: I strongly advise to use ARC clients from CVMFS ( CernVM File System ) as opposed to a local installation, since you are very likely to be needing CVMFS access anyway on your client machine.
The ATLAS User Interface servers are setup with CVMFS.
Make sure you have a file $HOME/.voms/vomses including one or more of the following lines (create it with your editor):
"atlas" "lcg-voms.cern.ch" "15001" "/DC=ch/DC=cern/OU=computers/CN=lcg-voms.cern.ch" "atlas"
"fermilab" "voms.fnal.gov" "15001" "/DC=org/DC=doegrids/OU=Services/CN=http/voms.fnal.gov" "fermilab"
"t2k.org" "voms.gridpp.ac.uk" "15003" "/C=UK/O=eScience/OU=Manchester/L=HEP/CN=voms.gridpp.ac.uk t2k.org" "t2k.org"
NOTE: under review from here on ...
To set up ARC (standalone) choose the needed configuration from [1] and
download it. Then, for linux, extract and source:
# tar -xvf nordugrid-arc-standalone-<your version>.tgz
# cd nordugrid-arc-standalone-<your-version>
# . ./setup.sh
The ARC environment should be set, you can make a proxy (default valid 12 hours)
# arcproxy --voms aec
Type your password:
To test it (other clusters at UNIBE are ce01.lhep.unibe.ch and nordugrid.unibe.ch) :
#arctest -c ce01.lhep.unibe.ch -J 1
To check your job go to: [2]. Now describe a real job in the xrsl language, submit and retrive it:
# arcsub -c ce.lhep.unibe.ch myjob.xrsl (returns a job identifier gsiftp://.... if everything is ok)
# arcget gsiftp://..
Here an xrsl file example (documentation here : http://www.nordugrid.org/documents/xrsl.pdf):
$&(executable=myjobscript.sh)
(inputfiles=
(myjob.exe myjob.exe)
)
(* comments within stars *)
(outputfiles=("/" ""))
(jobname=MyVeryFirstJob)