Fix for GPU-enabled restart runs - #1503
Open
abishekg7 wants to merge 1 commit into
Open
Conversation
abishekg7
force-pushed
the
atmosphere/fix_acc_data_movement
branch
from
August 19, 2026 16:50
e2c174a to
9aaf86e
Compare
abishekg7
marked this pull request as ready for review
August 19, 2026 16:50
Contributor
|
@abishekg7 This looks like a fix we should include in the upcoming v8.4.2 release. If you agree, could you set the base branch to |
This commit fixes the OpenACC data movement for the field rho_zz in routine mpas_atm_pre_compute_solve_diagnostics. Previously, an OpenACC create statement was used for this field. This is still correct for the cold initialization case , but produces incorrect results in the case of a restarted run. This has now been changed to a copyin statement. This issue was discovered as a result of restarted real simulations on GPUs crashing. The unphysical values from the dynamical core lead to a segmentation fault occuring in the SW radiation calls. Furthermore, the Jablonowski- Williamson baroclinic wave case also showed discrepancies between the cold initialization and the restarted test case. Both of these issues have been addressed by this commit.
abishekg7
force-pushed
the
atmosphere/fix_acc_data_movement
branch
from
August 19, 2026 19:17
9aaf86e to
b81c51f
Compare
Collaborator
Author
Rebased it to |
mgduda
added
Atmosphere
bug fix
OpenACC
mgduda
self-requested a review
August 19, 2026 20:52
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR corrects the OpenACC data movement for the field
rho_zzin routinempas_atm_pre_compute_solve_diagnostics. Previously, an OpenACCcreatestatement was used for this field. This is still correct for the cold initialization case, but produces incorrect results in the case of a restarted run. This is now changed to acopyinstatement.If the following condition is met
.not. config_do_restart .or. (config_do_restart .and. config_do_DAcycling)then the call toatm_init_coupled_diagnosticsinatm_mpas_init_blockcomputes the first time level ofrho_zz. Allowing us to use a create statement inmpas_atm_pre_compute_solve_diagnostics. However, in the case of a restart run with no DA cycling, the usage of the create statement is incorrect.This issue was discovered as a result of restarted real simulations on GPUs crashing. The unphysical values from the dynamical core lead to a segmentation fault occuring in the SW radiation calls. Furthermore, the Jablonowski-Williamson baroclinic wave case also showed discrepancies between the cold initialization and the restarted test case. Both of these issues have been addressed by this PR.