compile flags for FESOM

[tsemmler05]

Similarly to oifs.env.yaml, I would like to define FESOM_FFLAGS, i.e. specific fortran compile flags for FESOM2 on a specific machine, in my case ecmwf-atos.

To achieve this, I added to fesom-2.0.yaml (under configs/fesom):

+compiletime_environment_changes:

• choose_computer.name:
• ecmwf-atos:

•  FESOM_FFLAGS: '"-r8 -i4 -fp-model precise -no-prec-div -no-prec-sqrt -fimf-use-svml -xHost -ip -init=zero -no-warp-margin -march=core-avx2 -mtune=core-avx2"'
  

However, this change does not lead to any difference in my executable. I also tried to replace FESOM_FFLAGS with FESOM2_FFLAGS, without success.

Do I have to do anything in addition to make sure that this change is picked up?

I am running esm-tools version 6.22.0

Thanks in advance for your advice.

[nwieters]

Hi @tsemmler05,
can you give a path to the changed fesom-2.0.yaml file please? Then I can have a look tomorrow.
Thank you.

[nwieters]

Hi @tsemmler05,
are you running FESOM2 standalone?

[nwieters]

Hi @tsemmler05,

I tried to add these compile flags to FESOM2 in AWICM3.1 on Albedo. This works for me:

Add the following lines in the fesom block in configs/setups/awicm3/awicm3.yaml

compiletime_environment_changes:
    choose_computer.name:
        ecmwf-atos:
            add_export_vars:
                - 'FFLAGS="-r8 -i4 -fp-model precise -no-prec-div -no-prec-sqrt -fimf-use-svml -xHost -ip -init=zero -no-warp-margin -march=core-avx2 -mtune=core-avx2"'

To check if this flags were applied, check if they are in the CMakeChache.txt file in the build folder of fesom.2.0.

Let me know, if this works also for you on ecmwf-atos.

[tsemmler05]

Tried this now but I don't see the flags in CMakeChache.txt in the build folder of fesom.2.0, and because of this the executable is bit-identical with the previous one.

I did see further down in awicm3.yaml that for some machines compiletime_environment_changes are defined for different components. Maybe I have to add the compile flags there?

I am running the coupled system awicm3-frontiers-xios. The path of my setup is here: /home/duts/esm-tools/awicm3-frontiers-xios, and the directory of my esm-tools here: /home/duts/esm-tools

[nwieters]

Yes. Sorry, maybe I was not clear. I added this compiletime_environment_changes (see my answer above) in awicm3.yaml for component fesom.
And then these flags can be found in CMakeChache.txt under the cmake variable CMAKE_Fortran_FLAGS.
But I tried this on Albedo. Maybe I can also try this on Atos.

[tsemmler05]

mmh, maybe you can try it on ecmwf-atos. Many of the flags are missing at /home/duts/esm_tools/awicm3-v3.1/fesom-2.0/build

you can see what I have implemented here: /home/duts/esm_tools/configs/setups/awicm3

[tsemmler05]

Is there a specific place in the fesom subsection of awicm3.yaml where I have to put the compiletime_environment_changes?

[nwieters]

mmh, maybe you can try it on ecmwf-atos. Many of the flags are missing at /home/duts/esm_tools/awicm3-v3.1/fesom-2.0/build

you can see what I have implemented here: /home/duts/esm_tools/configs/setups/awicm3

I thought you want to compile awicm3-frontiers-xios?

I can't look into the build folders nor the awicm3.yaml file, since I don't have permission.

[nwieters]

Hi @tsemmler05 ,

I compiled awicm3-frontiers-xios on ecmwf-atos and I still get these compile flags in the CMakeCache.txt file.

You can have a look in my setup for awicm3.yaml:
/perm/deu5349/esm_tools/configs/setups/awicm3/awicm3.yaml
lines 339-343

After compiling I get the following entries:
FFLAGS
in /perm/deu5349/model_codes/test_compiles/awicm3-frontiers-xios/comp-fesom-2.0-frontiers_script.sh

and

CMAKE_Fortran_FLAGS:STRING=-r8 -i4 -fp-model precise -no-prec-div -no-prec-sqrt -fimf-use-svml -xHost -ip -init=zero -no-warp-margin -march=core-avx2 -mtune=core-avx2

in /perm/deu5349/model_codes/test_compiles/awicm3-frontiers-xios/fesom-2.0/build/CMakeCache.txt
line 112

I used esm_tools branch ecmwf-atos (version 6.22.0).

Please let me know if you have missing permission to read the files.

[tsemmler05]

Getting permission denied for esm_tools and model_codes subdirectories of /perm/deu5349

[nwieters]

Sorry @tsemmler05 ,
you should now have read access to these folders.

[tsemmler05]

Thanks @nwieters , it works for me now! It seems like the location of the compiletime_environment_changes entry is key. Directly after comp_command entry it works, but at the end of fesom block it doesn't.

[nwieters]

Hi @tsemmler05,
great. And thanks for the hint to the reason why it fails in your case. I will try to reproduce it, and maybe add a note to the documentation about where to put this kind of entry.

I will close this issue.