Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Version 1.0: North Atlantic Update - Tides, Curved Boundaries, Config, and More!😀🤯🥳 #193

Draft
wants to merge 109 commits into
base: main
Choose a base branch
from

Conversation

manishvenu
Copy link
Collaborator

@manishvenu manishvenu commented Oct 11, 2024

Authors: @ashjbarnes, @manishvenu

Description:

Ashley and I worked together to add a few features to RM6. While we get the changes ready to merge with the main COSIMA branch, please check out this draft with a list of the changes below.

Major Features:

  1. Accepting curved boundary supergrids and bathymetry (Add angled grids to RMOM6 CROCODILE-CESM/regional-mom6#13)
  2. Implement Boundary Tides from OSU's TPXO using GFDL NWA25 Code (Merge various quality-of-life changes & tides boundary functions (From GFDL NWA25) into RMOM6 CROCODILE-CESM/regional-mom6#12)
  3. Import experiment configuration from a light config JSON (Merge various quality-of-life changes & tides boundary functions (From GFDL NWA25) into RMOM6 CROCODILE-CESM/regional-mom6#12)
  4. Major bathymetry fixes for land-sea mask (Various bathymetry improvements / bugfixes CROCODILE-CESM/regional-mom6#17)
  5. Moving experiment specific changes to MOM_override (All OBC, Tides, and other minor changes) and out of MOM_input
  6. New GLORYS downloader (that can account for curved boundaries) (Merge various quality-of-life changes & tides boundary functions (From GFDL NWA25) into RMOM6 CROCODILE-CESM/regional-mom6#12)
  7. Tests: Additional tests (tides. rotation, and regridding), Conftest.py, Add to Docker Image (MOM6 Angle Calculation CROCODILE-CESM/regional-mom6#34)
  8. Changing function names to verb beginnings (initial_condition -> setup_initial_condition) (Merge various quality-of-life changes & tides boundary functions (From GFDL NWA25) into RMOM6 CROCODILE-CESM/regional-mom6#12)
  9. Add properties (i.e. more functional object variables) and dataset plotting (Merge various quality-of-life changes & tides boundary functions (From GFDL NWA25) into RMOM6 CROCODILE-CESM/regional-mom6#12)
  10. Add Rotation: Add MOM6 angle calculation at t points (MOM6 doesn't use an input hgrid's angle_dx field. It calculate the angle internally at t-points) & two angle approximation methods for the boundary q/u/v points (MOM6 Angle Calculation CROCODILE-CESM/regional-mom6#34)
  11. Add Land Mask (Fix tides on angled boundaries, consolidate regridded of velocity and tracers, add proper body tide parameters, add MOM6's angle calculation. CROCODILE-CESM/regional-mom6#33)

Minor Features:

  1. Ability to code without greek letters
  2. Removed unintentional machine-specific code
  3. Increased print statements

Breaking Changes (for backwards compatibility):

  1. Functions changed names, so it's not compatible with older code. All that needs to be updated is the function name itself in previous code (like the demos). (Merge various quality-of-life changes & tides boundary functions (From GFDL NWA25) into RMOM6 CROCODILE-CESM/regional-mom6#12)

Documentation Changes:

  1. Update README and Docs (MOM6 Angle Calculation CROCODILE-CESM/regional-mom6#34, Fix tides on angled boundaries, consolidate regridded of velocity and tracers, add proper body tide parameters, add MOM6's angle calculation. CROCODILE-CESM/regional-mom6#33)

Co-authored-by: @ashjbarnes

this PR closes #184 , #168 , and also #55 in a roundabout way since user now specifies which boundaries they want, and the MOM inputs are adjusted accordingly.

Before merging, need to resolve the following:

Outstanding Questions:

  1. Logging was implemented in the two new modules of RM6 (regridding.py and rotation.py). Do we want to switch over RM6 to use python logging instead of print statements? Currently, we would prefer that but open it up to opinions.
  2. A while back, we all briefly discussed that this should be version 0.8 instead of 1. Any opinions or insight?

ashjbarnes and others added 30 commits August 28, 2024 14:00
Changes to make compatible with CESM
having mask table and layout no longer required
Replace minimum layers with min depth
…nt num

After talking with Ashley, the find_MOM6_orientation will likely be removed
The function names for rectangular and sinmple boundaries were changed because the tides are a kind of boundary function, the old names now give a warning and call the correct function. GFDL had rough horizontal subsetting for the tpxo dataset (probably for efficiency?), implemented in setup_tides_rectangular_boundaries.
Collapsed the *_tidal_dims functions into the encode_tides function in segment, and add citing documentation to the docstrings for now.
@ashjbarnes
Copy link
Collaborator

BTW this PR closes #184 , #168 , and also #55 in a roundabout way since user now specifies which boundaries they want, and the MOM inputs are adjusted accordingly.

@navidcy
Copy link
Contributor

navidcy commented Oct 14, 2024

BTW this PR closes #184 , #168 , and also #55 in a roundabout way since user now specifies which boundaries they want, and the MOM inputs are adjusted accordingly.

the closes #XYZ either needs to be in the first comment of the PR or we need to manually link the issues with the PR

@navidcy
Copy link
Contributor

navidcy commented Oct 14, 2024

Yes it's a huge PR! We did have this broken down into smaller PRs on the CROCODILE fork, which we reviewed and merged one by one over the last couple of months. Unfortunately these PRs might not be useful for us to break up this big one any more, because we did need to do a lot of merging given that @manishvenu and I were working in parallel on several changes.

It's still in draft state for now while we polish it (build on the properties for each input which is WIP, add more comments, remove commented out code, review the demos etc) but thought it could be good to get the ball rolling on the review.

OK, when is ready to review we can talk again? Huge PRs are harder to review though and take longer...

Is there a way you suggest that we break up the PR?

Each of the major features mentioned in the first comment sounds like could be a PR? Just a suggestion..

@navidcy navidcy added the enhancement New feature or request label Oct 14, 2024
@navidcy
Copy link
Contributor

navidcy commented Oct 14, 2024

Ability to code without greek letters

haha, sorry if I introduced this difficulty to devs

… + tests (#30)

* Package type changes for CRR

* Black Formatting

* Change MOM Param

* Change MOM_param to Work

* black

* Revert back to reg

* Add tide boundary flexibility

* Update Config to only vars

* Update Write and Read Config for computer independence

* Update Read/Write Config for Computer Independenc

* Fix Change MOM Param Issues

* Fix bug in __getattr__ and comment out matplotlib dependencies in demo

* Package type changes for CRR

* Black Formatting

* Add Boundaries to Init part 1

* black formatting

* Changing Boundary Def to __init__ pt 2

* Minor Bugs

* Make Direction Dir a one-time calculation because keys() doesn't gaurentee order

* Rearrange handling boundaries in __init__ to a boundary array and segments dict in the boundary functions ocean_state and tides

* Clean

* Improve commenting on setuop_ocean_state_boundaries

* Fix bug in declaring number of OBC segments

* Minor Big in override

* Add boundaries to config
Copy link
Collaborator

@angus-g angus-g left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haven't looked the whole way through (only got to the end of the param file manipulation). Lots of style-related stuff, but the functionality is exciting!

regional_mom6/regional_mom6.py Outdated Show resolved Hide resolved
regional_mom6/regional_mom6.py Outdated Show resolved Hide resolved
regional_mom6/regional_mom6.py Outdated Show resolved Hide resolved
regional_mom6/regional_mom6.py Outdated Show resolved Hide resolved
regional_mom6/regional_mom6.py Outdated Show resolved Hide resolved
regional_mom6/regional_mom6.py Outdated Show resolved Hide resolved
regional_mom6/regional_mom6.py Outdated Show resolved Hide resolved
regional_mom6/regional_mom6.py Outdated Show resolved Hide resolved
regional_mom6/regional_mom6.py Outdated Show resolved Hide resolved
@@ -1754,6 +2504,198 @@ def setup_run_directory(
0,
]
nml.write(self.mom_run_dir / "input.nml", force=True)
return

def change_MOM_parameter(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this MOM parameter file stuff could get lifted out to a separate file?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey Angus,

This would be a pretty big change. Any chance you would mind if we made this an issue instead and punted on it so we can get this PR out the door? I have no disagreements with this comment!

  • Manish

@q1165600785
Copy link

Is this still being verified? I can't wait

@helenmacdonald
Copy link

Thanks for prompting!
I am close to having an example notebook that goes through the new features.

@navidcy – I have something to run by you. The new features we want to demonstrate in an example notebook are a) tides, b) rotated grids and c) user can input own grid

Using the existing Tasman example, we can add a couple of lines to generate the tidal forcing to tick off a. To tick off b and c I am creating an example rotated grid for the user to input. There is only one small difference between a user-inputted grid and a grid generated by this toolbox – so we are suggesting that we include a comment to explain this on the existing notebook rather than creating 2 separate example notebooks.

Will this work for you? When I have it, I will upload the example notebook for everyone to comment.

@q1165600785
Copy link

谢谢你的提示! 我马上就要有一本介绍新功能的示例笔记本了。

@navidcy – 我有事要跟你讲。我们想在示例笔记本中演示的新功能是 a) 潮汐、b) 旋转网格和 c) 用户可以输入自己的网格

使用现有的 Tasman 示例,我们可以添加几行来生成潮汐力以单击 a。要单击 b 和 c,我正在创建一个示例旋转网格供用户输入。用户输入的网格和生成此工具箱网格之间只有一个小差异 - 因此我们在现有笔记本中添加一条注释建议来解释这一点,而不是创建 2 个单独的笔记本笔记本。

这对你有用吗?当我有的时候,我会上传笔记本供大家评论。

Yes, thank you, this should be very useful, I urgently need a tidal boundary to run a model

@ashjbarnes
Copy link
Collaborator

Is this still being verified? I can't wait

Hi, you can use branch of the code as it currently is. It's stable for domains with boundaries parallel to lines of constant lat and lon, but has had issues on curved boundaries so far. That's what is holding up the merge, but if you need tides right now then no reason to wait for this

- correct the first arg of a classmethod (from self to cls).
- introduce optional write_to_file arg in setup_bathymetry.
- introduce optional bathymetry obj arg in tidy_bathymetry.
…filenames. Similarly, add thickesses arg to _make_vgrid.
Minor changes ahead of CrocoDash-visualCaseGen integration
@helenmacdonald
Copy link

Hi @navidcy
ccing @ashjbarnes and @manishvenu
I believe that this pull request is close to being ready. I am working on a notebook that you suggested to include which displays some of the new features:

  1. The ability to bring your own h_grid.nc
  2. The ability to create forcing for curvilinear+rotated and domains, and
  3. The ability to create Tidal forcing.

Note that the new package does not include the ability to create hgrid.nc for curvilinear and rotated domains, you need to BYO.

I have created one notebook that demonstrates BYO domain and tidal forcing, which I am running with an example rotated + curvilinear h_grid.nc. Do you have suggestions for how the rotated/curvilinear h_grid should be delivered for testing the pull request? It is currently on the crocodile docker and we are discussing if this should be imported to the testing suite for the COSIMA regional mom6.

I can also email you (or the pull request reviewer) a copy or put it online somewhere

@navidcy
Copy link
Contributor

navidcy commented Jan 13, 2025

@navidcy – I have something to run by you. The new features we want to demonstrate in an example notebook are a) tides, b) rotated grids and c) user can input own grid

Using the existing Tasman example, we can add a couple of lines to generate the tidal forcing to tick off a. To tick off b and c I am creating an example rotated grid for the user to input. There is only one small difference between a user-inputted grid and a grid generated by this toolbox – so we are suggesting that we include a comment to explain this on the existing notebook rather than creating 2 separate example notebooks.

Will this work for you? When I have it, I will upload the example notebook for everyone to comment.

I think it's great to modify slightly the current example for a. and then include another one for b and c.!

(sorry for the slack; I was on break and slowly catching up!)

@navidcy
Copy link
Contributor

navidcy commented Jan 13, 2025

@helenmacdonald and others, when this PR is ready for review ping me (and @angus-g?)?

@helenmacdonald
Copy link

I think it's great to modify slightly the current example for a. and then include another one for b and c.!

Thanks!
@ashjbarnes and @manishvenu I can include a modified version of the current example in my pull request

@navidcy
Copy link
Contributor

navidcy commented Jan 13, 2025

@helenmacdonald, I'm a bit confused: you have a different PR? isn't this the PR we are referring to?

@helenmacdonald
Copy link

Sorry @navidcy I should have clarified. I am working on an example notebook for Manish to pull into the crocodile-CESM:main to then be added to this pull request. From the perspective of cosima:main there will only be 1 pull request.

@navidcy
Copy link
Contributor

navidcy commented Jan 13, 2025

Gotcha

manishvenu and others added 5 commits January 17, 2025 11:13
… tracers, add proper body tide parameters, add MOM6's angle calculation. (#33)

* Add GFDL Rotation Code and minor tides debugging in setup_run_directory

The GFDL rotation code was commented out🙈

* Black

* Actual Angle Change

* Minor Clean

* Add First Regridding Option

* More tidal changes

* Change default tides to use all constituents in TPXO allowed in MOM6

* Fix Tests

* Start of Angle Calc

* Blacl

* Rand

* Clean up testing  (#37)

* Remove extra directories created

* Black

* Clean up created dirs

* Rand

* Keith Method Step 1, add t-point boundaries to coords()

* Start implementing alternative angles approaches...

* Set up rotational method framework

* Add h, t, and change default mom6_angle_calc to regional

* black

* Fred Angle Calc

* Black

* Black 2

* Keith Method

* Tidal Rotational Methods Completed

* Finish rotational methods for velocity_tracers

* Combine the grid_rotation_calc for fred_pseudo and mom6 general (keiths)

* Move rotation/angle functions to a different file

* Doesn't make sense to use rot method in expt because the regrid methods are in segment right now. Something to discuss and then add

* Clean up the keith method to use the regridding func add_secondary_dimension

* Add angle_calc to docs

* Main Page Link for now...

* Start testing setup

* Change func name for consistency

* Minor Commenting

* Minor Commenting

* Black

* Redo Auto Docs

* Start of testing

* Test until curved grid generation comes in

* Bug

* Test_Rotation Changes

* Adjust Docker Path

* Testing Framework Trial 1

* Docker Docs

* Test Cleaning

* Change name of docker docs

* Adjust docker image to crocodile one

* Step 1: Setup Framework Tidal Regridding Nans/LandMask BugFix

* Finish setting up masking framework

* Clean up process of masking

* Regridding Testing except for the mask functions

* First round testing w/o mask tests

* First Round Testing Completed

* Minor Clean

* MOM6 Angle Calculation (#34)

* Start of Angle Calc

* Blacl

* Rand

* Rand

* Keith Method Step 1, add t-point boundaries to coords()

* Start implementing alternative angles approaches...

* Set up rotational method framework

* Add h, t, and change default mom6_angle_calc to regional

* black

* Fred Angle Calc

* Black

* Black 2

* Keith Method

* Tidal Rotational Methods Completed

* Finish rotational methods for velocity_tracers

* Combine the grid_rotation_calc for fred_pseudo and mom6 general (keiths)

* Move rotation/angle functions to a different file

* Doesn't make sense to use rot method in expt because the regrid methods are in segment right now. Something to discuss and then add

* Clean up the keith method to use the regridding func add_secondary_dimension

* Add angle_calc to docs

* Main Page Link for now...

* Start testing setup

* Change func name for consistency

* Minor Commenting

* Minor Commenting

* Black

* Redo Auto Docs

* Start of testing

* Test until curved grid generation comes in

* Bug

* Test_Rotation Changes

* Adjust Docker Path

* Testing Framework Trial 1

* Docker Docs

* Test Cleaning

* Change name of docker docs

* Adjust docker image to crocodile one

* Step 1: Setup Framework Tidal Regridding Nans/LandMask BugFix

* Finish setting up masking framework

* Revert "Finish setting up masking framework"

This reverts commit 9287030.

* Revert "Step 1: Setup Framework Tidal Regridding Nans/LandMask BugFix"

This reverts commit a2b961a.

* respond to alper comments

* Docs changes

* Seg Fault Issue Pt1

* Disable threading

* Just single threading

* Clean up testing

* Remove Single Threading

* Add back single threading

* Spelling Mistake

* Zero Out Attempt

* Black + Add Mask to Regrid_vt

* Zero Out Corner + 3 Cell

* Update Land Mask Tests

* Fix Bathy NTiles Issue

* Minor Cleaning + Variable Renaming

* minor changes ahead of visualCaseGen integration in CrocoDash:

- correct the first arg of a classmethod (from self to cls).
- introduce optional write_to_file arg in setup_bathymetry.
- introduce optional bathymetry obj arg in tidy_bathymetry.

* Mirro GFDL NWA12-Cobalt a bit more and add diag table date adjustment

* Black

* Add aiohttp and copernicusmarine dependencies

* add hgrid_path and vgrid_path args to __init__ to allow non-standard filenames. Similarly, add thickesses arg to _make_vgrid.

* black reformatting

* Update Initial Condition + Alternate Rotation Function

* Rename get_glorys_rect to get_glorys

* Move RM6 to logging over print statements, but still report regional_mom6 to stdout

* Formatting

* Fix logging bug

* Revert RM6 Logging

* Formatting

* Alper Review Comments Pt1

* Relative Imports

* Revert "Relative Imports"

This reverts commit 15d8d36.

* Check if the ESMF warning is the issue in the tests failure

* Add Rotational Methods function

* Small Adj to Docs'

---------

Co-authored-by: alperaltuntas <[email protected]>
* First Round outside of OBC String making and strptime dates

* Completed Review Comments

* Black

* Minor Bug Catch
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Need to override MOM6 min depth based on value set in setup_bathymetry function
7 participants