Jblob Documentation

Jblob on DKRZ's HPC system

Jblob is already installed on levante and available as a module, just execute module load jblob.

Download

Important: Please read the Terms of use document available here!
Download Jblob version 4.1

Installation instructions

Prerequisites

Jblob needs an installed Java distribution (JRE or JDK) of version 8 or above.
On Linux systems OpenJDK can be installed from the distribution's repository and for Windows systems OpenJDK distributions are available from several providers, for example Microsoft or the Eclipse Foundation.

Installation

  • Download the program archive linked above. This file contains all files neccessary for use with Windows and Linux.
  • Extract the contents of the archive to the desired installation directory.
  • Linux: The script jblob.sh should work without modifications if the Java executable is available from $PATH, otherwise uncomment and edit the line containing JAVA_HOME.
  • Windows: Edit the line containing JAVA_HOME in the jblob.bat batch script.

Changes for use with HTTP-Proxies

This is only necessary if your internet connection requires use of a proxy!
The java options -Dhttp.proxyHost and -Dhttp.proxyPort can be used to enable jblob connections over http-proxies.
For linux systems add the following options to the last line of the jblob script (before the de.dkrz.cera.applications.JblobClient "$@" part)
-Dhttp.proxyHost=<your_proxy_host> -Dhttp.proxyPort=<your_proxy_port>
For windows systems add the following options to the last line of the jblob.bat batch script (before the de.dkrz.cera.applications.JblobClient %* part)
-Dhttp.proxyHost=<your_proxy_host> -Dhttp.proxyPort=<your_proxy_port>

Synopsis


  jblob --dataset | -d dataset_acronym [ options ]

  jblob --cmip5file drs_name [ options ]

  jblob --login

  jblob --info

  jblob --help

  jblob --version

Options

--cdocdo operatorCDO processing during data download. See section "CDO processing" for details.
--dateextAppend timestamp(s) to filename instead of record number(s). Works only for datasets with a "calendrical" temporal structure.
--dirdirectoryThe transferred dataset will be placed in the specified directory.
--filefilenameThe transferred dataset will be given the specified name instead of datasetname_rmin-rmax.
--originfilenameFile to transfer when accessing a DOKU dataset.
--quietQuiet operation. No output except error messages.
--rmaxnumberLast record to transfer. If omitted, the transfer will end with the last available record.
--rminnumberFirst record to transfer. If omitted, the transfer will start with the first available record.
--tmaxdateLast date to transfer. The standard date format is yyyy-mm-ddThh:mm, however unnecessary parts may be omitted (i.e. yyyy-mm is sufficient when working with monthly data).
--tmindateFirst date to transfer. The standard date format is yyyy-mm-ddThh:mm, however unnecessary parts may be omitted (i.e. yyyy-mm is sufficient when working with monthly data).

Supplying username and password

Starting with version 4.1 jblob supports either WDCC or DKRZ credentials and requires a separate login step instead of using a .netrc file. A login token will be stored in the home directory of each user (~/.jblob/config.json). This token is valid for 30 days.
Creating a login token:
jblob --login
Displaying current login information (username and token expire date):
jblob --info

Exit values

0Success
1Unspecified errorPlease check program output for additional messages (i.e. do not use the -quiet option).
3Invalid dataset nameThe requested dataset name is doesn't exist in the database.
4Invalid dataset idThe requested dataset id doesn't exist in the database.
5Insufficient permission to datasetYou do not have the neccessary permission to access the requested dataset. Please contact
  data@nullwdc-climate.de
 for further assistance.
8I/O errorAn I/O error was encountered during the data transfer.
12Temporal structure not supported for tmin/tmax selectionThe selected dataset cannot be downloaded by selecting start and end dates, please use selection by record numbers (--rmax, --rmin) instead.
13Selected start/end record/time outside of datasets coverage.The supplied rmin/rmax/tmin/tmax value(s) are outside of the dataset's coverage.
14Dataset is empty (data_size = 0)Please contact
  data@nullwdc-climate.de
 for further information.

CDO Processing

The following CDO (1.6.9) operators are currently available for use with Jblob:
sellonlatbox--cdo "sellonlatbox,<min_lon>,<max_lon>,<min_lat>,<max_lat>"
selcode--cdo "selcode,<code>"
selgridname--cdo "selgridname,<grid_name>"
CDO processing is usable with these dataset formats: GRIB, NetCDF, IEG
Please note: processing may take some time, please be patient and DO NOT start the same download again if the first one doesn't start downloading immediately!

Examples

Transfer all records of dataset EH4OPYC_SRES_A2_WIND10 to /tmp.
jblob --dataset EH4OPYC_SRES_A2_WIND10 --dir /tmp
Transfer data from Jan. 2001 to Dec. 2010 of dataset EH5_OM_A1B_1_MM_TSURF, reading username and password from command line.
jblob --dataset EH5_OM_A1B_1_MM_TSURF --tmin 2001-01 --tmax 2010-12
Download a single file from a DOKU dataset.
jblob --dataset DKRZ_LTA_564_ds00001 --origin "/doku/bm0564/DKRZ_LTA_564_ds00001/DMS-paper/dms-paper-all-data-for-fig.tar"
Download CMIP5 data by drs_name.
jblob --cmip5file "cmip5/output1/MPI-M/MPI-ESM-MR/amip/day/atmos/day/r1i1p1/v20120330/clt/clt_day_MPI-ESM-MR_amip_r1i1p1_19790101-20081231.nc"
Select a specific region of a dataset for download.
jblob --dataset EH5_OM_A1B_1_MM_TSURF --cdo "sellonlatbox,0,180,-45,45"

Known Issues

Contents of DOKU datasets currently can only be downloaded separately due to large filesizes.