-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathNAF.html
244 lines (224 loc) · 15.9 KB
/
NAF.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
<meta charset="utf-8" /><meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>NAF specifics — grid-control documentation</title>
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
<!--[if lt IE 9]>
<script src="_static/js/html5shiv.min.js"></script>
<![endif]-->
<script src="_static/jquery.js?v=5d32c60e"></script>
<script src="_static/_sphinx_javascript_frameworks_compat.js?v=2cd50e6c"></script>
<script src="_static/documentation_options.js?v=5929fcd5"></script>
<script src="_static/doctools.js?v=9a2dae69"></script>
<script src="_static/sphinx_highlight.js?v=dc90522c"></script>
<script src="_static/js/theme.js"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="Tips & tricks" href="tips_tricks.html" />
<link rel="prev" title="Getting started" href="getting_started.html" />
</head>
<body class="wy-body-for-nav">
<div class="wy-grid-for-nav">
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
<div class="wy-side-scroll">
<div class="wy-side-nav-search" >
<a href="index.html" class="icon icon-home">
grid-control
</a>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="search.html" method="get">
<input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
<p class="caption" role="heading"><span class="caption-text">Contents</span></p>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="installation.html">Installation</a><ul>
<li class="toctree-l2"><a class="reference internal" href="installation.html#requirements">Requirements</a></li>
<li class="toctree-l2"><a class="reference internal" href="installation.html#installation-from-pypi-pip">Installation from PyPI (<code class="docutils literal notranslate"><span class="pre">pip</span></code>)</a></li>
<li class="toctree-l2"><a class="reference internal" href="installation.html#clone-from-git">Clone from git</a><ul>
<li class="toctree-l3"><a class="reference internal" href="installation.html#switch-to-a-different-branch">Switch to a different branch</a></li>
<li class="toctree-l3"><a class="reference internal" href="installation.html#pull-remote-changes">Pull remote changes</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="getting_started.html">Getting started</a><ul>
<li class="toctree-l2"><a class="reference internal" href="getting_started.html#first-configuration-run-jobs-on-the-local-machine">First configuration (run jobs on the local machine)</a></li>
<li class="toctree-l2"><a class="reference internal" href="getting_started.html#further-examples">Further examples</a></li>
</ul>
</li>
<li class="toctree-l1 current"><a class="current reference internal" href="#">NAF specifics</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#central-installation">Central installation</a></li>
<li class="toctree-l2"><a class="reference internal" href="#backend">Backend</a></li>
<li class="toctree-l2"><a class="reference internal" href="#submitting-to-centos7-nodes">Submitting to CentOS7 nodes</a></li>
<li class="toctree-l2"><a class="reference internal" href="#custom-entries-in-condor-submit-file">Custom entries in condor submit file</a></li>
<li class="toctree-l2"><a class="reference internal" href="#singularity-virtualization">Singularity (Virtualization)</a></li>
<li class="toctree-l2"><a class="reference internal" href="#use-a-voms-proxy">Use a VOMS proxy</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="tips_tricks.html">Tips & tricks</a><ul>
<li class="toctree-l2"><a class="reference internal" href="tips_tricks.html#gnu-screen">GNU Screen</a></li>
<li class="toctree-l2"><a class="reference internal" href="tips_tricks.html#global-configuration">Global configuration</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="error_codes.html">Error codes</a></li>
</ul>
</div>
</div>
</nav>
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="index.html">grid-control</a>
</nav>
<div class="wy-nav-content">
<div class="rst-content">
<div role="navigation" aria-label="Page navigation">
<ul class="wy-breadcrumbs">
<li><a href="index.html" class="icon icon-home" aria-label="Home"></a></li>
<li class="breadcrumb-item active">NAF specifics</li>
<li class="wy-breadcrumbs-aside">
<a href="_sources/NAF.rst.txt" rel="nofollow"> View page source</a>
</li>
</ul>
<hr/>
</div>
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
<section id="naf-specifics">
<h1>NAF specifics<a class="headerlink" href="#naf-specifics" title="Link to this heading"></a></h1>
<p>If you are a user of the National Analysis Facility (<a class="reference external" href="https://naf.desy.de">NAF</a>) and want to submit to the
<a class="reference external" href="http://bird.desy.de">BIRD</a> batch farm, this page provides some specific instructions and features for you.</p>
<nav class="contents local" id="contents">
<ul class="simple">
<li><p><a class="reference internal" href="#central-installation" id="id1">Central installation</a></p></li>
<li><p><a class="reference internal" href="#backend" id="id2">Backend</a></p></li>
<li><p><a class="reference internal" href="#submitting-to-centos7-nodes" id="id3">Submitting to CentOS7 nodes</a></p></li>
<li><p><a class="reference internal" href="#custom-entries-in-condor-submit-file" id="id4">Custom entries in condor submit file</a></p></li>
<li><p><a class="reference internal" href="#singularity-virtualization" id="id5">Singularity (Virtualization)</a></p></li>
<li><p><a class="reference internal" href="#use-a-voms-proxy" id="id6">Use a VOMS proxy</a></p></li>
</ul>
</nav>
<section id="central-installation">
<span id="naf-central-inst"></span><h2>Central installation<a class="headerlink" href="#central-installation" title="Link to this heading"></a></h2>
<p>You don’t need to install grid-control on your own, if you’re a NAF user. We provide a central installation (actually different versions) using the usual environment module system:</p>
<div class="highlight-shell-session notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>module<span class="w"> </span>avail<span class="w"> </span>grid-control
<span class="go">--------------- /afs/desy.de/group/cms/modulefiles/ ---------------</span>
<span class="go">grid-control/current(default) grid-control/testing</span>
<span class="go">grid-control/previous</span>
<span class="gp">$ </span>module<span class="w"> </span>load<span class="w"> </span>grid-control/current
</pre></div>
</div>
<p>The default version is <code class="docutils literal notranslate"><span class="pre">current</span></code>, which is the general recommendation. Whenever <code class="docutils literal notranslate"><span class="pre">current</span></code> is updated, the last version will be made available as <code class="docutils literal notranslate"><span class="pre">previous</span></code> for the case that you rely on that specific version for some time still (e.g. for existing tasks).
If you need a newer version containing recent changes, you can use the <code class="docutils literal notranslate"><span class="pre">testing</span></code> version.</p>
<p>After you loaded the module (you might want to do this in your <code class="file docutils literal notranslate"><span class="pre">.[bash|zsh|*]rc</span></code> file), you can use the <code class="docutils literal notranslate"><span class="pre">grid-control</span></code> executable.</p>
</section>
<section id="backend">
<h2>Backend<a class="headerlink" href="#backend" title="Link to this heading"></a></h2>
<p>On the NAF, the HTCondor batch system is used. This is detected automatically, if you specify that a local backend should be used:</p>
<div class="highlight-ini notranslate"><div class="highlight"><pre><span></span><span class="k">[global]</span>
<span class="na">backend</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">local</span>
</pre></div>
</div>
</section>
<section id="submitting-to-centos7-nodes">
<h2>Submitting to CentOS7 nodes<a class="headerlink" href="#submitting-to-centos7-nodes" title="Link to this heading"></a></h2>
<p>At the moment (end of 2019), the batch system default is to submit to EL6 (= Scientific Linux 6, SL6) nodes only
if you submit from a EL6 workgroup server. This behaviour can be altered in grid-control using <code class="docutils literal notranslate"><span class="pre">user</span> <span class="pre">requirements</span></code>.</p>
<p>To submit to EL7 (= CentOS7) worker nodes only, use</p>
<div class="highlight-ini notranslate"><div class="highlight"><pre><span></span><span class="k">[condor]</span>
<span class="na">user requirements</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">( OpSysAndVer == "CentOS7" )</span>
</pre></div>
</div>
<p>To submit to EL6 worker nodes only, use</p>
<div class="highlight-ini notranslate"><div class="highlight"><pre><span></span><span class="k">[condor]</span>
<span class="na">user requirements</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">( OpSysAndVer == "SL6" )</span>
</pre></div>
</div>
<p>To allow the jobs to run on EL6 <em>and</em> EL7 nodes, use</p>
<div class="highlight-ini notranslate"><div class="highlight"><pre><span></span><span class="k">[condor]</span>
<span class="na">user requirements</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">( OpSysAndVer == "CentOS7" || OpSysAndVer == "SL6" )</span>
</pre></div>
</div>
</section>
<section id="custom-entries-in-condor-submit-file">
<h2>Custom entries in condor submit file<a class="headerlink" href="#custom-entries-in-condor-submit-file" title="Link to this heading"></a></h2>
<p>You can add custom entries to the submission file used for condor:</p>
<div class="highlight-ini notranslate"><div class="highlight"><pre><span></span><span class="k">[condor]</span>
<span class="na">jdldata</span><span class="w"> </span><span class="o">=</span>
<span class="w"> </span><span class="na">+MyProject</span><span class="o">=</span><span class="s">"xyz"</span>
<span class="w"> </span><span class="na">+MyCustomAttribute</span><span class="o">=</span><span class="s">12</span>
</pre></div>
</div>
<p>Make sure that you don’t use whitespaces in the attributes, because they are interpreted as list separators!</p>
</section>
<section id="singularity-virtualization">
<h2>Singularity (Virtualization)<a class="headerlink" href="#singularity-virtualization" title="Link to this heading"></a></h2>
<p>To run EL6 jobs on EL7 worker nodes in a
<a class="reference external" href="https://confluence.desy.de/display/IS/Containers">container environment</a>,
<a class="reference external" href="https://confluence.desy.de/display/IS/Singularity">singularity</a>
can be used
<a class="reference external" href="https://confluence.desy.de/display/IS/Singularity+support+in+BIRD">within the batch system</a>.</p>
<p>If you already have a working image, you can use that.
Otherwise, store an image to DUST (not afs!), e.g. with</p>
<div class="highlight-shell-session notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>mkdir<span class="w"> </span>/nfs/dust/cms/user/<span class="si">${</span><span class="nv">USER</span><span class="si">}</span>/singularity
<span class="gp">$ </span><span class="nv">SINGULARITY_CACHEDIR</span><span class="o">=</span><span class="s2">"/nfs/dust/cms/user/</span><span class="si">${</span><span class="nv">USER</span><span class="si">}</span><span class="s2">/singularity"</span><span class="w"> </span>singularity<span class="w"> </span>pull<span class="w"> </span>/nfs/dust/cms/user/<span class="si">${</span><span class="nv">USER</span><span class="si">}</span>/singularity/slc6_latest.sif<span class="w"> </span>docker://cmssw/slc6:latest
<span class="go">INFO: Converting OCI blobs to SIF format</span>
<span class="go">INFO: Starting build...</span>
<span class="go">Getting image source signatures</span>
<span class="go">[... working ...]</span>
</pre></div>
</div>
<p>This will take a few seconds to minutes, but you’ll only have to do it once.</p>
<p>In the grid-control config, set the corresponding attributes:</p>
<div class="highlight-ini notranslate"><div class="highlight"><pre><span></span><span class="k">[condor]</span>
<span class="na">user requirements</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">( OpSysAndVer == "CentOS7" )</span>
<span class="na">jdl data</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">+MySingularityImage="/nfs/dust/cms/user/<user>/singularity/slc6_latest.sif"</span>
</pre></div>
</div>
</section>
<section id="use-a-voms-proxy">
<h2>Use a VOMS proxy<a class="headerlink" href="#use-a-voms-proxy" title="Link to this heading"></a></h2>
<p>If your jobs need a VOMS proxy token, the easiest way is to place it on a shared file system, i.e. AFS.
For this, set the environment variable <code class="docutils literal notranslate"><span class="pre">X509_USER_PROXY</span></code>, e.g. in your <code class="file docutils literal notranslate"><span class="pre">.[bash|zsh|*]rc</span></code> file:</p>
<div class="highlight-shell-session notranslate"><div class="highlight"><pre><span></span><span class="go">export X509_USER_PROXY=/afs/desy.de/user/<U>/<USER>/private/x509_voms</span>
</pre></div>
</div>
<p>After this is set, create your new proxy (<code class="docutils literal notranslate"><span class="pre">voms-proxy-init</span></code>).</p>
<p>In the grid-control config, add</p>
<div class="highlight-ini notranslate"><div class="highlight"><pre><span></span><span class="k">[backend]</span>
<span class="na">proxy</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">VomsProxy</span>
</pre></div>
</div>
</section>
</section>
</div>
</div>
<footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
<a href="getting_started.html" class="btn btn-neutral float-left" title="Getting started" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
<a href="tips_tricks.html" class="btn btn-neutral float-right" title="Tips & tricks" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
</div>
<hr/>
<div role="contentinfo">
<p>© Copyright 2019, Johannes Lange.</p>
</div>
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
</footer>
</div>
</div>
</section>
</div>
<script>
jQuery(function () {
SphinxRtdTheme.Navigation.enable(false);
});
</script>
</body>
</html>