Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add grib2 writing capability to ocnicepost #89

Open
AminIlia-NOAA opened this issue Dec 11, 2024 · 12 comments
Open

Add grib2 writing capability to ocnicepost #89

AminIlia-NOAA opened this issue Dec 11, 2024 · 12 comments

Comments

@AminIlia-NOAA
Copy link
Contributor

What new functionality do you need?

In order to add ocean and ice products to GFS and other global models, there is a need to add grib2 writer to ocenicepost module.

What are the requirements for the new functionality?

The module uses NCEP-G2 library and an logical option has been added to the namelist.

Acceptance Criteria

Three grib2 files will be generated.
The ice model output time unit is daily should be changed to hourly to align with ocean model. (will be changed for HR5).

Suggest a solution (optional)

A code has been written. @aerorahul If you create a new branch I can pull my changed to it. So others also can take a look into it?
In meantime you can look into it on my fork at https://github.com/AminIlia-NOAA/gfs-utils-AI/tree/grib2w

The code is not final and still need work. I already tested the code for a low res ice model results and it's working however it is not efficient. The section 5 of messages need to be modified to increase the speed of writing. If someone can help with section 5 message it would be great.
@JessicaMeixner-NOAA and @WalterKolczynski-NOAA

@JessicaMeixner-NOAA
Copy link
Contributor

@AminIlia-NOAA thanks for creating an issue and sharing your branch!

Perhaps this is my inexperience with what you are trying to do, but I don't understand "The section 5 of messages need to be modified to increase the speed of writing." - For example is this a specific part of your code?

@AminIlia-NOAA
Copy link
Contributor Author

@JessicaMeixner-NOAA Section 5 of grib2 message is dealing with how data is written into the grib2 file (e.g . compressed vs simple packing, scaling and ...). In my code lines 772 to 781 in utils_mod.F90 assign those parameters. We need an algorithm to generate those parameters based on each field data rather than some constant values. Probably wgrib has a algorithm for it.

@JessicaMeixner-NOAA
Copy link
Contributor

@WenMeng-NOAA or @GeorgeGayno-NOAA - any chance you'd have more insight on this or who to ask for advice for @AminIlia-NOAA

@WenMeng-NOAA
Copy link

@AminIlia-NOAA Can you provide us with the instructions of duplicating the issue you are encountering? The UPP might not be a good sample for reference. CC @BoCui-NOAA in this loop, as she has been developing and maintaining the NAEFS package which outputs many variables in GRIB2.

@GeorgeGayno-NOAA
Copy link
Contributor

For G2 library help, you can contact - @edwardhartnett, @AlysonStahl-NOAA or @Hang-Lei-NOAA.

@BoCui-NOAA
Copy link
Contributor

@AminIlia-NOAA The NAEFS is a downstream product of GEFS and uses the GRIB2 message directly from the GEFS data, so there are no specific settings for Section 5 in the NAEFS codes. Sorry there is no experience to share with you.

@GeorgeGayno-NOAA
Copy link
Contributor

@JessicaMeixner-NOAA Section 5 of grib2 message is dealing with how data is written into the grib2 file (e.g . compressed vs simple packing, scaling and ...). In my code lines 772 to 781 in utils_mod.F90 assign those parameters. We need an algorithm to generate those parameters based on each field data rather than some constant values. Probably wgrib has a algorithm for it.

@AminIlia-NOAA - unless the G2 library has changed from when I last used it, you should only need to set the decimal scaling factor - idrtmpl(3) - and the library will compute the rest of the idrtmpl array.

@Hang-Lei-NOAA
Copy link
Contributor

Confirm what George mentioned.
The idrtmpl is a template specified by idrtnum and actually a pointer to an array that holds the data.
In your case, I suggest that you find out a model code (e.g. GEFS) to see how they proceed the grib file, and then do a similar approach.

@BoCui-NOAA
Copy link
Contributor

BoCui-NOAA commented Dec 12, 2024

@JessicaMeixner-NOAA Section 5 of grib2 message is dealing with how data is written into the grib2 file (e.g . compressed vs simple packing, scaling and ...). In my code lines 772 to 781 in utils_mod.F90 assign those parameters. We need an algorithm to generate those parameters based on each field data rather than some constant values. Probably wgrib has a algorithm for it.

@AminIlia-NOAA - unless the G2 library has changed from when I last used it, you should only need to set the decimal scaling factor - idrtmpl(3) - and the library will compute the rest of the idrtmpl array.

@GeorgeGayno-NOAA Agree with you. Here is a sample code using idrtmpl(3), After setting up idetmpl(3), call putgb2 to output the variable.

https://github.com/NOAA-EMC/NAEFS/blob/develop/sorc/gefs_bias.fd/gefs_bias.f90#L306

@GeorgeGayno-NOAA
Copy link
Contributor

Here is how I set up section 5 for gribbing snow data in UFS_UTILS:
https://github.com/GeorgeGayno-NOAA/UFS_UTILS/blob/develop/sorc/emcsfc_snow2mdl.fd/snow2mdl.F90#L930

@WenMeng-NOAA
Copy link

@AminIlia-NOAA The UPP use both g2 and g2tmpl libs to write out GRIB2 data. You might explore g2tmpl for settings of Section 5.

@AminIlia-NOAA
Copy link
Contributor Author

Thanks All, I wish I knew these sooner!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants