Something is amiss in IR radiance assimilation #426
Comments
|
Here is another set of diagnostics. These are now calculated as follows: displayed-inc-diff = aave(abs(jedi(temperature),g) - aave(abs(gsi(temperature),g) for three instruments/platforms: GMI-GPM; ATMS-N20 and IASI-Metop-B, respectively: GMI ATMS-N20 IASI-MetopB From my perspective, differences of a few hundredths of a degree is this type of measure should be ok - GMI differs in the thousandths of degrees (perhaps because its increments are considerable smaller in magnitude than those of ATMS); ATMS differences seem a little large (perhaps reasonable down in the troposphere, but not so in the stratosphere esp around 1 hPa); IASI differences seem completely unreasonable, pretty much along the entire profile. Just for completion, let me add:: AIRS-AQUA |
|
Another curious result comes from running both GSI and JEDI, for IASI-MetopB, with and without correlated errors. Now here is what the measure above looks like for the two corresponding GSI experiments (no-Correlations-minus-Correlations): 
and here is a similar results from JEDI: 
In this sense, both code (GSI and JEDI) behave similarly. These obviously don't show how different JEDI's results are from GSI's (see above), which indicate JEDI's to be the odd ball. But when it comes to on/off correlated R, the behavior of JEDI is more what I'd expect to see. Indeed even in the differences above one can see JEDI's differences near the surface are larger than GSI's - this is clearly visible in the T-skin increment differences that qualitatively don't change much from what's been illustrate above already. |
|
I am finding out that this whole IR thing might be due to differences in CRTM coeffs:
CRTM_AerosolCoeff_Load(FAILURE) : Error reading AerosolCoeff file /discover/nobackup/projects/gmao/dadev/rtodling/JEDI/x49/j49rt00/fvInput/gsi/etc/fix_ncep20221018/REL-2.4.0-jcsda/CRTM_Coeffs/Little_Endian/AerosolCoeff.bin
CRTM_Init(FAILURE) : Error loading IRwaterCoeff data from /discover/nobackup/projects/gmao/advda/SwellStaticFiles/je The funny thing is that months ago I had been able to run JEDI pointing to the GSI-CRTM coeffs ... obviously JEDI evolved and this is no longer possible - I know for a fact that one thing the changed in JEDI-CRTM are the Aerosol-related files. Perhaps I can change JEDI to allow it to use the V4 aerosol files ... looking ... |
|
Thanks for Matt, I am not able to run GSI w/ exactly the same version of CRTM as that used in JEDI - pointing to the CRTM coeffs that JEDI uses. Here is figure of the increments for Tskin when using GSI as we normally have it in x0049 (top) and GSI running JEDI's version of CRTM (bottom): 
minor differences over ice that would/could be of importance weren't the issues above way more prominent. And here is the difference in the profile of temperature averaged in the way I showed above for other cases gsi(def) - gsi(jedi(crtm)). 
Some differences in the upper atmosphere, but nothing to be too alarmed as the differences we are finding between GSI and JEDI. |
|
Just to give a general feel for the difference in handling antenna vs brightness temperature for some instruments / platforms in GSI, here is a profile fig (similar to calculations above) comparing two all obs exps w/ GSI when Ta is converted to Tb. 
From this, it would seem the differences seen in the treatment of, say, ATMS-N20 in JEDI vs GSI would not all be explained by the Ta vs Tb story ... nonetheless, I will do two exps in GSI (ta vs tb) when using only ATMS-N20 for completion. it is possible that w/ multiple instruments at play, the signal and differences between Ta vs Tb get mixed and dimmed ... |
|
Thanks, Ricardo. You may be right, that the signal for MW alone would likely be larger since those obs are affected most by the Ta/Tb diff. |
|
I think it could be explained by Ta and Tb difference. In Ricardo’s figure, the difference is mostly shown above 5mb, those are higher levels affected by ATMS channel 15. ATMS channel 15 is not bias corrected, and all of the other channels are bias corrected so attena correction is partially taken care of in those channels but not in channel 15. That’s why we see differences above 5mb.
From: rtodling ***@***.***>
Date: Monday, September 30, 2024 at 2:47 PM
To: GEOS-ESM/swell ***@***.***>
Cc: Zhu, Yanqiu (GSFC-6101) ***@***.***>, Assign ***@***.***>
Subject: [EXTERNAL] [BULK] Re: [GEOS-ESM/swell] Something is amiss in handling IASI (perhaps all hyperspectral IR) (Issue #426)
CAUTION: This email originated from outside of NASA. Please take care when clicking links or opening attachments. Use the "Report Message" button to report suspicious messages to the NASA SOC.
Just to give a general feel for the difference in handling antenna vs brightness temperature for some instruments / platforms in GSI, here is a profile fig (similar to calculations above) comparing two all obs exps w/ GSI when Ta is converted to Tb.
tinc_allobs_tbXta.png (view on web)<https://github.com/user-attachments/assets/2b9acff1-9ebb-439d-9c4f-440f6c302d84>
From this, it would seem the differences seen in the treatment of, say, ATMS-N20 in JEDI vs GSI would not all be explained by the Ta vs Tb story ... nonetheless, I will do two exps in GSI (ta vs tb) when using only ATMS-N20 for completion.
—
Reply to this email directly, view it on GitHub<#426 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ARUMEBX25MF2QJFRPVRBKLTZZGMDZAVCNFSM6AAAAABOG6HI4CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOBTHEZDCMRSGM>.
You are receiving this because you were assigned.Message ID: ***@***.***>
|
@gmao-yzhu I understand the diffs I see I due to Ta vs Tb ... |
|
Here is the case when only ATMS-N20 is used. It seems the differences are still of the order of when all other observations are being assimilated. 
|
|
Just want to check – did you use bufr_ncep_atms_ta_remap.yaml or bufr_ncep_atms_remap.yaml?
From: rtodling ***@***.***>
Date: Wednesday, October 2, 2024 at 12:41 PM
To: GEOS-ESM/swell ***@***.***>
Cc: Zhu, Yanqiu (GSFC-6101) ***@***.***>, Mention ***@***.***>
Subject: [EXTERNAL] [BULK] Re: [GEOS-ESM/swell] Something is amiss in handling IASI (perhaps all hyperspectral IR) (Issue #426)
CAUTION: This email originated from outside of NASA. Please take care when clicking links or opening attachments. Use the "Report Message" button to report suspicious messages to the NASA SOC.
Here is the case when only ATMS-N20 is used. It seems the differences are still of the order of when all other observations are being assimilated.
Screenshot.2024-10-02.at.12.38.47.PM.png (view on web)<https://github.com/user-attachments/assets/0c945213-ec33-4d15-9d64-d2722a2f1016>
—
Reply to this email directly, view it on GitHub<#426 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ARUMEBSDXQEP4I2EUNJMNILZZQO3XAVCNFSM6AAAAABOG6HI4CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOBZGEZDMOBYGM>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
@gmao-yzhu Not quite sure I understand your question. The tests I have done are all GSI tests; there are no yaml's involved in these! Also: can you check your replies to these exchanges ... something odd is happening where it seems that every reply from you have some email thing attached to it that makes the msgs nearly unreadable. Maybe you're replying directly from your emails - that come from giihub and what ends up on github is odd looking. The best way to reply to msgs from GitHub is to get to GitHub and reply from within the system. |
|
I am getting a better picture of what the true issue behind these bad T-skin increments might be. I think I have nailed it to the https://github.com/JCSDA-internal/ufo/blob/2149dd956ecdf4c8b1924c0d0682a34eb8b1d94d/src/ufo/filters/obsfunctions/ObsErrorFactorSurfJacobianRad.cc#L4 After trying multiple things (including w/ other ObsFunctions), not only did I seem to have nailed the issue to this, as I believe the problem to be in entries in this line: and the following line. As a hack test I multiplied the term quantity in line 180 by factor in the dozens, the increments of T skin start looking more like what I get in GSI. Obviously, the effect of this multiplication is to make the output value of this routine large - which makes varinv smaller since these will multiply varinv (or the inverse obs error). Alternatively, instead of wiring a multiplying factor I can inflate the obserr_demisf and obserr_dtempf that are the coefficients in the eq in lines 178-179. Inflating the latter one(corresponding to the sensitivity to T-skin) by a factor of 10 results in much more reasonable increment again - basically the same as wiring numbers. Here is a original comparison for T skin increment for AIRS (GSI on top; JEDI at the bottom): 
and here is the similar fig when the JEDI yaml is changed as above (factor of 10 applied to T skin sensitivity): 
The above is an exercise in debugging - this is not to say we should reset the values of the constants above to new numbers. This is simply to show that the issue in T skin increment is associated w/ this particular ObsFunction. What I believe is that there a bias correction term missing from this filter. If I look at the corresponding filter / obs-function for MW I see those have a bias correction term, triggers like this: but adding this trigger seems to cause an error ... it seems the code starts wanting hydrometeor information QC - not sure why. |
|
I understand now why the bias term knob about would not work - it would not be consistent w/ GSI to have bias terms in the IR in the same in MW. But still don't see what else could be wrong. |
|
When adjusting the sensitivity coefficients (for area types) my magnifying them by certain factor this is what the comparison of the vertical increments of jedi vs gsi look like - compare w/ similar plot above. 
notice there is a small difference in the x-axis as compared to the earlier figure. This adjustment does make things look better ... and it could benefit from closer attention - but that's the wrong thing to do. just for the record these are the values used: |
|
BTW: I am not find in JEDI the "massaging" of the jacobian wrt ozone for wave numbers between 996 and 1170. That is, GSI (in setuprad) zeroes out these jacobians to remove the ozone influence on Tb from this channels. These sensitivities are associated w/ the stratosphere and lower troposphere. We should consider adding this to JED at some point ... however, just for the record, I show here a case when GSI's zeroing out of these Jac is removed - that is, the sensitivities are present. The black covers compares JEDI w/ the modified GSI and the green curve compares the modified GSI w/ the original GSI. The differences resulting from the wiping out of the jacobinans are much smaller than the differences we see between any GSI variation and JEDI. These results are for AIRS. 
And for completion I should say that I am referring to this chunk of code in GSI: |
|
Adding to the discussion, I want to revisit the impression given above that results for GMI are considerably better than for other instruments. I don't think this is the case. The thing w/ GMI is that its increments are small to begin with so the differences are "differences" look small. Here is a zoomed version of the GMI vertical differences show about (black), and on top of that adding the JEDI (green) and GSI (yellow) increments - show as global_ave(abs(tinc)) 
BTW: it is always the case that JEDI's "increment" is alway larger than GSI's ... |
rtodling
changed the title
|
It’s not just radiance assimilation that results in large increments in JEDI. The assimilation of observations from rawinsondes behaves similarly when comparing increments from the two systems. Therefore, this is not related to VarBC at all.
|
|
Some time has passed since I last updated this and showed any results here. I want to show the group - and remind myself - where things stand on this T-skin issue for the hyperspectral IR. I am now using a version of JEDI from 19 Nov 2024 - this is combined with a updated version of SWELL (still at works) that updates to the Oct 2024 variable change sprint (issue #457). Here is a figure comparing GSI vs JEDI Tskin from IASI on Metop-B. 
It seems to me what we are dealing w/ here is a magnitude issue; the shape of the increment is certain correct; and we get JEDI increment over all surfaces just as we do from GSI. |
|
Just for the sake of argument I am show here the same picture as above, but now scaling the Tskin increment from JEDI by 0.18 - ie multiplying by a fact to reduce the increment on Tskin: 
This is just to make the argument that all this is is some sort of scaling in JEDI. Something tells me this is associated w/ the nearSSTRetCheck term ... and I know I "played" with this term before ... but I want to do it again. |
|
I agree Ricardo, other than scaling it would be hard to choose between the two...so that's a real clue and perhaps a simpler issue to resolve than if something more insidious was wrong in the UFO, etc. I realize you scaled this arbitrarily just to make the point, but I'm still curious how you arrived at the value 0.18 to get such a good match. Did you scale by something like the ratio of the global RMS values between the GSI and (unscaled) JEDI increment? |
The 0.18 is a trial value ... I eye balled 0.5 - but it wasn't quite there ... then, I reduced and so and so ... But I was just remembering to have seen larger differences between GSI and JEDI (in this context) before, so I checked a little more and my GSI run goes down to 100 iterations; whereas the JEDI run goes only to 30 (because I forgot this was the default I had set in my test env). So I just re-run setting the max iteration number in JEDI to 100 ... but unfortunately JEDI crashed w/ a memory fault! This is becoming a real problem in JEDI ... anyway, I will see what I can do ...to run hundred iterations (or so). |
|
Sounds good. Btw, I had a long tag-up with Tom yesterday and I mentioned that we really have to focus on making JEDI more efficient, in terms of memory and speed, now that we're trying to run at scale. He completely agrees, and this was originally a priority of the AOP this year. Unfortunately, NOAA pulled the money designated to pay for this as part of the GDAS testbed app, which has now been removed from NOAA's plans for the year. So we'll all need to figure out a way to get JCSDA covered to do this work. |
|
I managed to run successfully now ... but JEDI - as usual - converges earlier than GSI (that rarely ever converges when handling radiance. JEDI goes up to 58 iterations .. and so, the increment doesn't change in any significant way - no point in showing (visually identical). So the factor 0.18 remains reasonable - again the factor is just for argument sake. |
|
I am starting to suspect the problem is in this function: CloudDetectMinResidualIR |
|
Since the overall structures are quite similar, the data selection from QC is likely consistent between GSI and JEDI. Therefore, it is unlikely that there is an issue with CloudDetectMinResidualIR, as it is a QC filter and does not directly affect the magnitude of the increments. |
|
Clearly something is amiss in ObsErrorFactorSurfJacobianRad.cc I can multiple this line: by a factor and make the increment look rather rather acceptable (not shown to avoid confusion). Since that's the case, it means I can "play" with the demisd and dtempf ... indeed, I seemed to have gotten nearly this far last time I poked at this - I am just reading what I've discussed above a number of weeks ago - though I don't think I was very explicit in what I wrote. Here I am making the following change in the corresponding filter (SurfJacobian): From: Which to me indicates that somewhere in UFO the emissivities for IR are returns incorrectly from CRTM. I recall having this discussion w/ Ben J before ... but we haven't followed up on this. I need to go back to the discussion I had: I am also keeping this discussion in two places - impossible to do it. So please follow it here: |
|
here is the result of the increment from the change above: 
|
|
Closing in. Nice. |
rtodling
changed the title
|
The adjusted values I proposed above turn out to be a little too large for CrIS-FSR - for this, I believe a factor of 5 from the original values should suffices. Here is a figure of difference of increment w/ GSI for two cases in JEDI: top using default values (which are in principle common to GSI), and bottom, is diff w/ JEDI using factor of 5 readjusted values. 
|
|
Ricardo, the new factor clearly provides very close values for the increments for JEDI and GSI, which is great. But do I understand correctly that you view this as a "cosmetic" fix and that there's still an actual problem with how the emissivity values are communicated between UFO and CRTM? Since I don't have access to JCSDA-internal, I'm not clear about Ben's and your thinking on this. Thanks. |
As I introduce T-skin as a control variable, I am re-doing some of the exercise I've done before of looking at each of the main instruments increments and how they compare w/ GSI's. IASI (and CrIS-FSR) has always been in the suspicious list. I want to have a record of this here so we can work together to address whatever issue affects hyperspectral IR.
The temperature increment for microwave instruments such as ATMS (clear sky) and GMI (all sky) are quite consistent between GSI and JEDI. The corresponding T-skin increments are also quite consistent - with still a puzzle over Antartica (but not a show stopper - see e.g., https://github.com/JCSDA-internal/fv3-jedi/pull/1257.
For IASI, the story is quite alarmingly different. For starts, GSI does not fully converge its minimization when given up to 100 iteration to crank; but so be it. JEDI, on the other hand, seems to converge in about 40 something iterations, which is suspicious in itself. But the real issue is how different the increments from both systems look (perhaps not a surprise given the convergence differences - chicken and egg).
Interestingly enough, the increments on specific humidity between the two systems are quite reasonable - here is an example at 850 hPa - GSI (top) JEDI (bottom)
Unfortunately, this is what temperature looks like at the same level:
notice T increments are multiplied by a factor of 2 for plotting enhancement purposes; still -diff in increments are larger than increments themselves.
Now here is what T-skin increments looks like:
It looks as though the weight given to whatever induces the sensitivity of Tb to T-skin is overweighted.
The text was updated successfully, but these errors were encountered: