Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update JGDAS_ENKF_SFC and JGDAS_ENKF_UPDATE jobs with COMIN/COMOUT instead of COM #3170

Open
wants to merge 22 commits into
base: develop
Choose a base branch
from
Open

Update JGDAS_ENKF_SFC and JGDAS_ENKF_UPDATE jobs with COMIN/COMOUT instead of COM #3170

wants to merge 22 commits into from
+54 −53

Conversation

mingshichen-noaa
Copy link
Contributor

Description

NCO has requested that each COM variable specify whether it is an input or an output. This completes that process for the global JGDAS_ENKF_SFC and JGDAS_ENKF_UPDATE jobs.

Refs #2451

Type of change

  • Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

  • Is this a breaking change (a change in existing functionality)? NO
  • Does this change require a documentation update? NO

How has this been tested?

  • Clone and build on RDHPCS
  • Cycled tests on Hercules
  • Forecast-only tests on Hercules

Checklist

  • Any dependent changes have been merged and published
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • I have made corresponding changes to the documentation if necessar

@KateFriedman-NOAA KateFriedman-NOAA added the CI-Wcoss2-Building **Bot use only** CI testing is cloning/building on WCOSS label Dec 16, 2024
@KateFriedman-NOAA
Copy link
Member

@mingshichen-noaa FYI, please ignore CI test messages in your PR. We are using your PR to test a new CI setup on WCOSS2. We will rerun the CI tests for this PR once reviews have completed. Thanks!

@KateFriedman-NOAA KateFriedman-NOAA added CI-Wcoss2-Running **Bot use only** CI testing on WCOSS for this PR is in-progress and removed CI-Wcoss2-Building **Bot use only** CI testing is cloning/building on WCOSS labels Dec 16, 2024
@KateFriedman-NOAA
Copy link
Member

CI Tests set up to run in /lfs/h2/emc/ptmp/emc.global/PR/PR_3170/RUNTESTS on WCOSS

@mingshichen-noaa
Copy link
Contributor Author

@KateFriedman-NOAA
I stopped CI test on Hercules.

@KateFriedman-NOAA
Copy link
Member

I stopped CI test on Hercules.

@mingshichen-noaa CI tests that you were running under your account? Why did you stop them? Was there an error?

@mingshichen-noaa
Copy link
Contributor Author

I stopped CI test on Hercules.

@mingshichen-noaa CI tests that you were running under your account? Why did you stop them? Was there an error?

@mingshichen-noaa
Copy link
Contributor Author

@KateFriedman-NOAA
After I checked the running status with rocotostat and found all running states with SUCCEEDED in cycle from 202112201800 to 202112210600, I stopped the crontab running.

@mingshichen-noaa
Copy link
Contributor Author

mingshichen-noaa commented Dec 17, 2024 •
edited
Loading

@KateFriedman-NOAA
My running results are located at /work2/noaa/global/mchen/test/jobs/JGDAS_ENKF_SFC/EXPDIR/C96C48_hybatmDA.
Besides, you told me you would test the PR in WCOSS2 and ignored CI test message. I missed your comments.

@KateFriedman-NOAA
Copy link
Member

@mingshichen-noaa Glad to hear your tests on Hercules succeeded. Please leave this PR open until it's merged or the code managers decide to close. Thanks!

@WalterKolczynski-NOAA WalterKolczynski-NOAA removed the CI-Wcoss2-Running **Bot use only** CI testing on WCOSS for this PR is in-progress label Dec 24, 2024
@mingshichen-noaa
Copy link
Contributor Author

@WalterKolczynski-NOAA
How about CI-Wcoss2-Running on PR (#3170)?

WalterKolczynski-NOAA
WalterKolczynski-NOAA requested changes Jan 8, 2025
View reviewed changes
scripts/exgdas_enkf_update.sh Outdated
@@ -412,7 +412,7 @@ $APRUN_ENKF ${DATA}/$(basename $ENKFEXEC) 1>stdout 2>stderr
export err=$?; err_chk

# Cat runtime output files.
cat stdout stderr > "${COM_ATMOS_ANALYSIS_STAT}/${ENKFSTAT}"
cat stdout stderr > "${COMIN_ATMOS_ANALYSIS_STAT}/${ENKFSTAT}"
Copy link
Contributor

@WalterKolczynski-NOAA WalterKolczynski-NOAA Jan 8, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an output. It's possible COM_ATMOS_ANALYSIS is both an IN and an OUT for this job. Please evaluate what the script is doing carefully to determine what are inputs and what are outputs.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I second Walter's comment. It appears that COM_ATMOS_ANALYSIS is both an IN and an OUT for this job. Please examine and if you agree, please define the variable twice, once as an IN and once as an OUT. Then update the script to use the appropriate IN or OUT version of the variable. Thanks!

jobs/JGDAS_ENKF_UPDATE Outdated Show resolved Hide resolved
@mingshichen-noaa
Copy link
Contributor Author

@WalterKolczynski-NOAA @KateFriedman-NOAA
I have modified jobs/JGDAS_ENKF_UPDATE and scripts/exgdas_enkf_update.sh and git add/commit/push the modified files.
I am building the modified PR and then do the CI PR test in Hercules.

@mingshichen-noaa
Copy link
Contributor Author

@WalterKolczynski-NOAA, @KateFriedman-NOAA
After I made modification on jobs/JGDAS_ENKF_UPDATE and scripts/exgdas_enkf_update.sh, I did PR tests in local repository in Hercules. The results show most of tasks were run successfully except task ( gdas_atmanlupp & gfs_fcst_seg0) failed to run.
Besides, above-mentioned CI Unit Tests / Run unit tests on CI system (pull_request) showed failed info:
fatal: remote error: upload-pack: not our ref f0a1649115882e3446c896ebe7efa7e0778d6840
fatal: Fetched in submodule path 'ufs_model.fd', but it did not contain f0a1649115882e3446c896ebe7efa7e0778d6840. Direct fetching of that commit failed.
Error: Process completed with exit code 128.
Could you give me advice to handle the failure? Thank you in advance.

@WalterKolczynski-NOAA
Copy link
Contributor

This PR should not be updating any submodules.

@mingshichen-noaa
Copy link
Contributor Author

@WalterKolczynski-NOAA

@mingshichen-noaa mingshichen-noaa deleted the feature/jgdas_enkf_sfc_update_com_in_out branch January 14, 2025 17:25
@mingshichen-noaa mingshichen-noaa restored the feature/jgdas_enkf_sfc_update_com_in_out branch January 14, 2025 17:26
@mingshichen-noaa
Copy link
Contributor Author

@WalterKolczynski-NOAA
In order to resolve the above-mentioned conflicting files: sorc/gdas.cd sorc/gfs_utils.fd, I remembered you ever told me the following resolved method:
"you should have run git submodule update --init --recursive to update the submodules locally. Now, you need to check out the correct versions from develop and commit them (or rebase interactively to remove the offending commits, but that is a bit advanced). "
I forgot how to get the correct versions of the two directories from develop. Will you be available tomorrow to make google chat?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@WalterKolczynski-NOAA WalterKolczynski-NOAA WalterKolczynski-NOAA requested changes

@KateFriedman-NOAA KateFriedman-NOAA Awaiting requested review from KateFriedman-NOAA

@DavidHuber-NOAA DavidHuber-NOAA Awaiting requested review from DavidHuber-NOAA

Requested changes must be addressed to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@mingshichen-noaa @KateFriedman-NOAA @WalterKolczynski-NOAA @DavidHuber-NOAA