NetCDF  4.4.0-rc2
auth.md
1 Authorization Support in the netCDF-C Libraries {#oc_auth}
2 ==================================================
3 
4 \brief It is possible to support a number of authorization schemes
5 in the netCDF-C library.
6 
7 With one exception, authorization in the netCDF-C library is
8 delegated to the oc2 code, which in turn delegates it to the
9 libcurl library. The exception is that the location of the rc
10 file can be specified by setting the environment variable *NCRCFILE*.
11 Note that the value of this environment variable should be the
12 absolute path of the rc file, not the path to its containing directory.
13 
14 Following is the authorization documentation.
15 <center>
16 
17 OC Authorization Support {#oc_auth_support}
18 ========================
19 
20 Author: Dennis Heimbigner<br>
21 dmh at ucar dot edu
22 
23 Draft: 11/21/2014<br>
24 Last Revised: 12/23/2014<br>
25 OC Version 2.1
26 </center>
27 
28 ## Table of Contents {#auth_toc}
29 
30 1. [Introduction](#Introduction)
31 2. [URL-Based Authentication](#URL-AUTH)
32 3. [RC File Authentication](#DODSRC)
33 4. [Redirection-Based Authentication](#REDIR)
34 5. [URL Constrained RC File Entries](#URLCONS)
35 6. [Client-Side Certificates](#CLIENTCERTS)
36 7. [Appendix A. All RC-File Keys](#allkeys)
37 8. [Appendix B. ESG Access in Detail](#ESGDETAIL)
38 
39 ## Introduction {#Introduction}
40 
41 OC can support user authorization using those provided by the curl
42 library. This includes basic password authentication as well as
43 certificate-based authorization.
44 
45 With some exceptions (e.g. see the section on [redirection](#REDIR)) The
46 libcurl authorization mechanisms can be accessed in two ways
47 
48 1. Inserting the username and password into the url, or
49 2. Accessing information from a so-called *rc* file named either
50  *.daprc* or *.dodsrc*
51 
52 ## URL-Based Authentication {#URL-AUTH}
53 
54 For simple password based authentication, it is possible to directly
55 insert the username and the password into a url in this form.
56 
57  http://username:password@host/...
58 
59 This username and password will be used if the server asks for
60 authentication. Note that only simple password authentication is
61 supported in this format. Specifically note that [redirection](#REDIR)
62 based authorization will not work with this.
63 
64 ## RC File Authentication {#DODSRC}
65 
66 The oc library supports an *rc* file mechanism to allow the passing of a
67 number of parameters to liboc and libcurl.
68 
69 The file must be called one of the following names: ".daprc", ".dodsrc"
70 If both .daprc and .dodsrc exist, then the .daprc file will take
71 precedence.
72 
73 Searching for the rc file first looks in the current directory and then
74 in the home directory (as defined by the HOME environment variable). It
75 is also possible to specify a direct path using the *-R* option to
76 ocprint or using the *oc\_set\_rcfile* procedure (see oc.h). Note that
77 for these latter cases, the path must be to the file itself, not to the
78 containing directory.
79 
80 The rc file format is a series of lines of the general form:
81 
82  [<host:port>]<key>=<value>
83 
84 where the bracket-enclosed host:port is optional and will be discussed
85 subsequently.
86 
87 The currently defined set of authorization-related keys are as follows.
88 The second column is the affected curl\_easy\_setopt option(s).
89 
90 Key
91 
92 curl\_easy\_setopt Option
93 
94 HTTP.COOKIEJAR
95 
96 CURLOPT\_COOKIEJAR, CURLOPT\_COOKIEFILE
97 
98 HTTP.PROXY\_SERVER
99 
100 CURLOPT\_PROXY, CURLOPT\_PROXYPORT, CURLOPT\_PROXYUSERPWD
101 
102 HTTP.SSL.CERTIFICATE
103 
104 CURLOPT\_SSLCERT
105 
106 HTTP.SSL.KEY
107 
108 CURLOPT\_SSLKEY
109 
110 HTTP.SSL.KEYPASSWORD
111 
112 CURLOPT\_KEYPASSWORD
113 
114 HTTP.SSL.CAINFO
115 
116 CURLOPT\_SSLCAINFO
117 
118 HTTP.SSL.CAPATH
119 
120 CURLOPT\_SSLCAPATH
121 
122 HTTP.SSL.VERIFYPEER
123 
124 CURLOPT\_SSL\_VERIFYPEER
125 
126 HTTP.CREDENTIALS.USERPASSWORD
127 
128 CURLOPT\_USERPASSWORD
129 
130 ### Password Authentication
131 
132 The key HTTP.CREDENTIALS.USERPASSWORD can be used to set the simple
133 password authentication. This is an alternative to setting it in the
134 url. The value must be of the form "username:password".
135 
136 ### Cookie Jar
137 
138 The HTTP.COOKIEJAR key specifies the name of file from which to read
139 cookies (CURLOPT\_COOKIEJAR) and also the file into which to store
140 cookies (CURLOPT\_COOKIEFILE). The same value is used for both CURLOPT
141 values. It defaults to in-memory storage.
142 
143 ### Certificate Authentication
144 
145 HTTP.SSL.CERTIFICATE specifies a file path for a file containing a PEM
146 cerficate. This is typically used for client-side authentication.
147 
148 HTTP.SSL.KEY is essentially the same as HTTP.SSL.CERTIFICATE and should
149 usually have the same value.
150 
151 HTTP.SSL.KEYPASSWORD specifies the password for accessing the
152 HTTP.SSL.KEY/HTTP.SSL.CERTIFICATE file.
153 
154 HTTP.SSL.CAPATH specifies the path to a directory containing trusted
155 certificates for validating server sertificates.
156 
157 HTTP.SSL.VALIDATE is a boolean (1/0) value that if true (1) specifies
158 that the client should verify the server's presented certificate.
159 
160 HTTP.PROXY\_SERVER specified the url for accessing the proxy:
161 (e.g.http://\[username:password@\]host\[:port\])
162 
163 ## Redirection-Based Authentication {#REDIR}
164 
165 
166 Some sites provide authentication by using a third party site to to the
167 authentication. One example is
168 [URS](https://uat.urs.earthdata.nasa.gov), the EOSDIS User Registration
169 System.
170 
171 The process is usually as follows.
172 
173 1. The client contacts the server of interest (SOI), the actual
174  data provider.
175 2. The SOI sends a redirect to the client to connect to the URS system.
176 3. The client authenticates with URS.
177 4. URS sends a redirect (with authorization information) to send the
178  client back to the SOI to actually obtain the data.
179 
180 In order for this to work with libcurl, the client will usually need to
181 provide a .netrc file so that the redirection will work correctly. The
182 format of this .netrc file will contain content that typically look like
183 this.
184 
185  machine uat.urs.earthdata.nasa.gov login xxxxxx password yyyyyy
186 
187 where the machine is the one to which the client is redirected for
188 authorization, and the login and password are those needed to
189 authenticate.
190 
191 The .netrc file can be specified in two ways.
192 
193 1. Specify the netrc file to liboc using the procedure in oc.h:
194 
195  oc_set_netrc(OClink* link, const char* file)
196 
197  (This is equivalent to the -N flag to ocprint).
198 
199 2. Put the following line in your .daprc/.dodsrc file.
200 
201  HTTP.NETRC=<path to netrc file>
202 
203 One final note. In using this, it is probable that you will need to
204 specify a cookie jar (HTTP.COOKIEJAR) so that the redirect site can pass
205 back authorization information.
206 
207 ## URL Constrained RC File Entries {#URLCONS}
208 
209 Each line of the rc file can begin with a host+port enclosed in square
210 brackets. The form is "host:port". If the port is not specified then the
211 form is just "host". The reason that more of the url is not used is that
212 libcurl's authorization grain is not any finer than host level.
213 
214 Examples.
215 
216  [remotetest.unidata.ucar.edu]HTTP.VERBOSE=1
217  or
218  [fake.ucar.edu:9090]HTTP.VERBOSE=0
219 
220 If the url request from, say, the *oc\_open* method has a host+port
221 matchine one of the prefixes in the rc file, then the corresponding
222 entry will be used, otherwise ignored.
223 
224 For example, the URL
225 
226  http://remotetest.unidata.ucar.edu/thredds/dodsC/testdata/testData.nc
227 
228 will have HTTP.VERBOSE set to 1.
229 
230 Similarly,
231 
232  http://fake.ucar.edu:9090/dts/test.01
233 
234 will have HTTP.VERBOSE set to 0.
235 
236 ## Client-Side Certificates {#CLIENTCERTS}
237 
238 Some systems, notably ESG (Earth System Grid), requires the use of
239 client-side certificates, as well as being [re-direction based](#REDIR).
240 This requires setting the following entries:
241 
242 - HTTP.COOKIEJAR — a file path for storing cookies
243  across re-direction.
244 - HTTP.NETRC — the path to the netrc file.
245 - HTTP.SSL.CERTIFICATE — the file path for the client side
246  certificate file.
247 - HTTP.SSL.KEY — this should have the same value
248  as HTTP.SSL.CERTIFICATE.
249 - HTTP.SSL.CAPATH — the path to a "certificates" directory.
250 - HTTP.SSL.VALIDATE — force validation of the server certificate.
251 
252 Note that the first two are to support re-direction based
253 authentication.
254 
255 ## Appendix A. All RC-File Keys {#allkeys}
256 
257 For completeness, this is the list of all rc-file keys.
258 
259 Key
260 
261 curl\_easy\_setopt Option
262 
263 HTTP.DEFLATE
264 
265 CUROPT\_DEFLATE\
266 with value "deflate,gzip"
267 
268 HTTP.VERBOSE
269 
270 CUROPT\_VERBOSE
271 
272 HTTP.TIMEOUT
273 
274 CUROPT\_TIMEOUT
275 
276 HTTP.USERAGENT
277 
278 CUROPT\_USERAGENT
279 
280 HTTP.COOKIEJAR
281 
282 CUROPT\_COOKIEJAR
283 
284 HTTP.COOKIE\_JAR
285 
286 CUROPT\_COOKIEJAR
287 
288 HTTP.PROXY\_SERVER
289 
290 CURLOPT\_PROXY,\
291 CURLOPT\_PROXYPORT,\
292 CURLOPT\_PROXYUSERPWD
293 
294 HTTP.SSL.CERTIFICATE
295 
296 CUROPT\_SSLCERT
297 
298 HTTP.SSL.KEY
299 
300 CUROPT\_SSLKEY
301 
302 HTTP.SSL.KEYPASSWORD
303 
304 CUROPT\_KEYPASSWORD
305 
306 HTTP.SSL.CAINFO
307 
308 CUROPT\_SSLCAINFO
309 
310 HTTP.SSL.CAPATH
311 
312 CUROPT\_SSLCAPATH
313 
314 HTTP.SSL.VERIFYPEER
315 
316 CUROPT\_SSL\_VERIFYPEER
317 
318 HTTP.CREDENTIALS.USERPASSWORD
319 
320 CUROPT\_USERPASSWORD
321 
322 HTTP.NETRC
323 
324 CURLOPT\_NETRC,CURLOPT\_NETRC\_FILE
325 
326 ## Appendix B. ESG Access in Detail {#ESGDETAIL}
327 
328 It is possible to access Earth Systems Grid (ESG) datasets from ESG
329 servers through the OC API using the techniques described in the section
330 on [Client-Side Certificates](#CLIENTCERTS).
331 
332 In order to access ESG datasets, however, it is necessary to register as
333 a user with ESG and to setup your environment so that proper
334 authentication is established between an oc client program and the ESG
335 data server. Specifically, it is necessary to use what is called
336 "client-side keys" to enable this authentication. Normally, when a
337 client accesses a server in a secure fashion (using "https"), the server
338 provides an authentication certificate to the client. With client-side
339 keys, the client must also provide a certificate to the server so that
340 the server can know with whom it is communicating.
341 
342 The oc library uses the *curl* library and it is that underlying library
343 that must be properly configured.
344 
345 ### Terminology
346 
347 The key elements for client-side keys requires the constructions of two
348 "stores" on the client side.
349 
350 - Keystore - a repository to hold the client side key.
351 - Truststore - a repository to hold a chain of certificates that can
352  be used to validate the certificate sent by the server to
353  the client.
354 
355 The server actually has a similar set of stores, but the client need not
356 be concerned with those.
357 
358 ### Initial Steps
359 
360 The first step is to obtain authorization from ESG. Note that this
361 information may evolve over time, and may be out of date. This
362 discussion is in terms of BADC and NCSA. You will need to substitute as
363 necessary.
364 
365 1. Register at http://badc.nerc.ac.uk/register to obtain access to badc
366  and to obtain an openid, which will looks something like:
367 
368  https://ceda.ac.uk/openid/Firstname.Lastname
369 
370 2. Ask BADC for access to whatever datasets are of interest.
371 3. Obtain short term credentials at
372  http://grid.ncsa.illinois.edu/myproxy/MyProxyLogon/ You will need to
373  download and run the MyProxyLogon program. This will create a
374  keyfile in, typically, the directory ".globus". The keyfile will
375  have a name similar to this: "x509up\_u13615" The other elements in
376  ".globus" are certificates to use in validating the certificate your
377  client gets from the server.
378 4. Obtain the program source ImportKey.java from this location:
379  http://www.agentbob.info/agentbob/79-AB.html (read the whole page,
380  it will help you understand the remaining steps).
381 
382 ### Building the KeyStore
383 
384 You will have to modify the keyfile in the previous step and then create
385 a keystore and install the key and a certificate. The commands are
386 these:
387 
388  openssl pkcs8 -topk8 -nocrypt -in x509up_u13615 -inform PEM -out key.der -outform DER
389 
390  openssl x509 -in x509up_u13615 -inform PEM -out cert.der -outform DER
391 
392  java -classpath -Dkeypassword="" -Dkeystore=./ key.der cert.der
393 
394 Note, the file names "key.der" and "cert.der" can be whatever you
395 choose. It is probably best to leave the .der extension, though.
396 
397 ### Building the TrustStore
398 
399 Building the truststore is a bit tricky because as provided, the
400 certificates in ".globus" need some massaging. See the script below for
401 the details. The primary command is this, which is executed for every
402 certificate, c, in globus. It sticks the certificate into the file named
403 "truststore"
404 
405  keytool -trustcacerts -storepass "password" -v -keystore "truststore" -importcert -file "${c}"
406 
407 ### Running the C Client
408 
409 Refer to the section on [Client-Side Certificates](#CLIENTCERTS). The
410 keys specified there must be set in the rc file to support ESG access.
411 
412 - HTTP.COOKIEJAR=\~/.dods\_cookies
413 - HTTP.NETRC=\~/.netrc
414 - HTTP.SSL.CERTIFICATE=\~/esgkeystore
415 - HTTP.SSL.KEY=\~/esgkeystore
416 - HTTP.SSL.CAPATH=\~/.globus
417 - HTTP.SSL.VALIDATE=1
418 
419 Of course, the file paths above are suggestions only; you can modify as
420 needed. The HTTP.SSL.CERTIFICATE and HTTP.SSL.KEY entries should have
421 same value, which is the file path for the certificate produced by
422 MyProxyLogon. The HTTP.SSL.CAPATH entry should be the path to the
423 "certificates" directory produced by MyProxyLogon.
424 
425 As noted, also uses re-direction based authentication. So, when it
426 receives an initial connection from a client, it redirects to a separate
427 authentication server. When that server has authenticated the client, it
428 redirects back to the original url to complete the request.
429 
430 ### Script for creating Stores
431 
432 The following script shows in detail how to actually construct the key
433 and trust stores. It is specific to the format of the globus file as it
434 was when ESG support was first added. It may have changed since then, in
435 which case, you will need to seek some help in fixing this script. It
436 would help if you communicated what you changed to the author so this
437 document can be updated.
438 
439  #!/bin/sh -x
440  KEYSTORE="esgkeystore"
441  TRUSTSTORE="esgtruststore"
442  GLOBUS="globus"
443  TRUSTROOT="certificates"
444  CERT="x509up_u13615"
445  TRUSTROOTPATH="$GLOBUS/$TRUSTROOT"
446  CERTFILE="$GLOBUS/$CERT"
447  PWD="password"
448 
449  D="-Dglobus=$GLOBUS"
450  CCP="bcprov-jdk16-145.jar"
451  CP="./build:${CCP}"
452  JAR="myproxy.jar"
453 
454  # Initialize needed directories
455  rm -fr build
456  mkdir build
457  rm -fr $GLOBUS
458  mkdir $GLOBUS
459  rm -f $KEYSTORE
460  rm -f $TRUSTSTORE
461 
462  # Compile MyProxyCmd and ImportKey
463  javac -d ./build -classpath "$CCP" *.java
464  javac -d ./build ImportKey.java
465 
466  # Execute MyProxyCmd
467  java -cp "$CP myproxy.MyProxyCmd
468 
469  # Build the keystore
470  openssl pkcs8 -topk8 -nocrypt -in $CERTFILE -inform PEM -out key.der -outform DER
471  openssl x509 -in $CERTFILE -inform PEM -out cert.der -outform DER
472  java -Dkeypassword=$PWD -Dkeystore=./${KEYSTORE} -cp ./build ImportKey key.der cert.der
473 
474  # Clean up the certificates in the globus directory
475  for c in ${TRUSTROOTPATH}/*.0 ; do
476  alias=`basename $c .0`
477  sed -e '0,/---/d' <$c >/tmp/${alias}
478  echo "-----BEGIN CERTIFICATE-----" >$c
479  cat /tmp/${alias} >>$c
480  done
481 
482  # Build the truststore
483  for c in ${TRUSTROOTPATH}/*.0 ; do
484  alias=`basename $c .0`
485  echo "adding: $TRUSTROOTPATH/${c}"
486  echo "alias: $alias"
487  yes | keytool -trustcacerts -storepass "$PWD" -v -keystore ./$TRUSTSTORE -alias $alias -importcert -file "${c}"
488  done
489  exit

Return to the Main Unidata NetCDF page.
Generated on Wed Aug 19 2015 17:25:41 for NetCDF. NetCDF is a Unidata library.