Quick start

Download sources and build access-om3 executable (optional)

Follow the steps in the README.

Build access-om3 executable (optional)

See here if you want to build access-om3 yourself.

Otherwise you can use a precompiled executable from /g/data/ik11/inputs/access-om3/bin/ which requires membership of the ik11 project - apply at https://my.nci.org.au/mancini/project/ik11 if needed.

Setting up a configuration

There are several configurations available, listed here. You should use the accessom3_exe branch of these.

For example, to run the MOM6-CICE6 configuration, clone and switch to the accessom3_exe branch:

git clone https://github.com/COSIMA/MOM6-CICE6.git
cd COSIMA/MOM6-CICE6
git checkout accessom3_exe

If you've compiled your own executable you'll need to edit the exe entry in config.yaml to point to it.

Running ACCESS-OM3 requires the latest payu. Until this is put into the standard conda environment you'll need to create a custom conda environment with the latest payu: create a ~/.condarc file as described here, then do this:

conda create -n my_payu python
conda activate my_payu
pip install git+https://github.com/payu-org/payu.git

This only needs to be done once.

The executable requires some custom modules, so you need to do this before you can run (only needs to be done once per log in):

conda activate my_payu
module use /g/data/ik11/spack/modules

Running

Now you're ready to run:

payu run

This uses the payu workflow management tool to prepare the run and submit it as a job to the PBS job queue. See the Gadi User Guide to learn more about PBS job management.

Check the status of the job (state 'Q'=waiting in queue, 'R'=running, 'E'=exiting, 'H'=held) with

uqstat -c

To kill the run early, do qdel N, where N is the job number (first column given by uqstat). If you kill the job (or it crashes), a work directory will be left behind after the job has disappeared from uqstat and you'll need to do payu sweep before you can run again.

When your run has finished successfully, payu puts its output in archive/output000. payu also records a log of your experiment in the git history, including the identity of the inputs and executables used (see the files in manifests).

To do another run, just type payu run again. Or to do (say) 10 runs, type payu run -n 10 and they'll automatically be submitted one after the other.

The outputs from each run will be in numbered subdirectories in archive.

Each run creates a restart directory in archive which is used as the initial condition for the next run. These restarts can accumulate and consume disk space, but only the most recent one is needed (unless you plan to restart a new experiment from an intermediate state).

See the payu documentation for more explanation of where run output is stored.

If the run is unsuccessful you can find output in access-om3.err and any output from the run will be in work. You will need to do payu sweep before you will be able to do another payu run.

WARNING restarts are stored on /scratch (so will be deleted after 100 days). If you plan to continue your run in the future you'll need to store the necessary restarts somewhere safe.

Customising your experiment

See the Configurations section.

Before running, commit your changes with an informative message, e.g. git commit -am "initial setup for experiment to test... bla bla"