Installation

Installation#

Horizontal regridding needs xESMF (conda)#

Horizontal regridding (regrid_horizontal and the full regrid wrapper) needs xESMF, whose ESMF/esmpy backend has no PyPI wheels and must come from conda-forge. ci/environment-regrid.yml creates the environment and installs xhycom into it in one step:

git clone https://github.com/nansencenter/xhycom.git
cd xhycom
conda env create -f ci/environment-regrid.yml
conda activate hycom-analysis-env

From GitHub#

If you don’t need horizontal regridding:

pip install git+https://github.com/nansencenter/xhycom.git

This includes lazy / Dask-backed loading and vertical regridding (regrid_vertical and depth interpolation), both dask and xgcm are core, pip-installable dependencies.

On Olivia / Betzy (NRIS)#

We can use a dedicated conda environment for HYCOM analysis with xhycom. The repo ships the recipe in ci/environment-regrid.yml (xESMF/ESMF, xgcm, Dask, JupyterLab, and xhycom), named hycom-analysis-env. Olivia and Betzy build and activate this environment differently: Betzy uses conda directly; Olivia uses HPC-container-wrapper, which builds the environment inside a container instead.

Build the environment (once)#

Betzy

Betzy only provides Miniforge3. First redirect the package cache and environments to project space, since ${HOME} quota is limited, this is one-time setup, saved to ~/.condarc:

module load Miniforge3/24.1.2-0
source ${EBROOTMINIFORGE3}/bin/activate
conda config --append pkgs_dirs /cluster/projects/nn2993k/conda/${USER}/package-cache
conda config --append envs_dirs /cluster/projects/nn2993k/conda/${USER}

Then create the environment from the clone, this also pip installs xhycom itself (in editable mode, from the clone) as part of ci/environment-regrid.yml:

cd ${HOME}/xhycom                   # your xhycom clone (holds ci/environment-regrid.yml)
conda env create -f ci/environment-regrid.yml
conda activate hycom-analysis-env

The envs_dirs redirect means the named environment lands in project space automatically, no -p prefix needed.

Olivia

Load the container wrapper (session-only, no need to add these to ~/.bashrc):

export http_proxy=http://10.63.2.48:3128/
export https_proxy=http://10.63.2.48:3128/
module load NRIS/CPU
module load hpc-container-wrapper

Build the environment as a container in project space, from the clone, this also pip installs xhycom itself (in editable mode, from the clone) as part of ci/environment-regrid.yml:

cd ${HOME}/xhycom                   # your xhycom clone (holds ci/environment-regrid.yml)
conda-containerize new --mamba \
    --prefix /cluster/projects/nn2993k/conda/${USER}/hycom-analysis-env \
    ci/environment-regrid.yml

Keep --prefix’s directory around permanently, it holds the container and executables. Since the containerised environment can’t be modified in place, rebuild with the same command (after removing the old prefix) if ci/environment-regrid.yml changes.

Start JupyterLab via Open OnDemand#

For interactive work, including running the example notebooks, start a JupyterLab session through Open OnDemand:

Click JupyterLab, and paste the appropriate snippet below into the app’s Environment setup field before launching, this runs before the Jupyter server starts and makes the hycom-analysis-env environment (python, jupyter, xhycom) the one the session runs against, reusing the same cached regrid weights across sessions.

Betzy

Load the module system first, then activate the environment by name:

module load Miniforge3/24.1.2-0
source ${EBROOTMINIFORGE3}/bin/activate
conda activate hycom-analysis-env
export XHYCOM_CACHE_DIR="/cluster/projects/nn2993k/${USER}/.xhycom-cache-dir"
Olivia

The containerised environment is activated by prepending its bin/ to PATH (no conda activate needed):

export PATH="/cluster/projects/nn2993k/conda/${USER}/hycom-analysis-env/bin:${PATH}"
export XHYCOM_CACHE_DIR="/cluster/projects/nn2993k/${USER}/.xhycom-cache-dir"

For an editable / development install and how to run the test suite, see the Contributor Guide.

Cache directory for regrid weights#

Horizontal regridding builds an xESMF weight matrix that maps the source grid onto the target grid. Generating it is the slow part of a regrid, and it depends only on the grids, not on the field values, so xhycom caches it to disk and reuses it on later calls (see Regridding).

By default the cache lives under $XDG_CACHE_HOME/xhycom/regrid_weights (i.e. ~/.cache/xhycom/regrid_weights). Set the XHYCOM_CACHE_DIR environment variable to put it somewhere else; weight files and the manifest are then written directly into that directory. Point it at shared project or scratch space so the same weights are reused across jobs and by collaborators:

export XHYCOM_CACHE_DIR="/cluster/projects/nn2993k/${USER}/.xhycom-cache-dir"

A leading . keeps it out of the way of a plain ls alongside your other project-space directories, same as ~/.cache.

Always add that export to your ~/.bashrc, even if you also put it in the Open OnDemand Environment setup field (see On Olivia / Betzy (NRIS)) or a batch job script.

Dependencies#

Required#

Package

Purpose

numpy

Array operations and binary I/O

xarray

Dataset construction

cftime

Calendar-aware datetime objects

dask

Lazy / out-of-core loading via the chunks parameter in open_dataset and open_mfdataset

xgcm

Vertical regridding (regrid_vertical) and depth interpolation

xhycom bundles its own HYCOM binary reader, pip installs everything needed for reading, lazy loading, and vertical regridding.

Optional#

Package

Purpose

xesmf (conda-forge only)

Horizontal regridding (regrid_horizontal and the full regrid wrapper). Kept optional since its ESMF/esmpy backend can conflict with other ESMF installs on some platforms (see above).