SSMI/S problems in 3D-FGAT

[rtodling]

I am finding that running 3DFGAT w/ SSMI/S only produces negative Jo's:

Quadratic cost function: JoJc( 0) = 2904.536994156287
Quadratic cost function: JoJc( 1) = 1761.685897988107
Quadratic cost function: JoJc( 2) = 41.58146053894939
Quadratic cost function: JoJc( 3) = -2188.836274347395
Quadratic cost function: JoJc( 4) = -4574.266068595534
Quadratic cost function: JoJc( 5) = -7173.171878308596
Quadratic cost function: JoJc( 6) = -8398.747569179868
Quadratic cost function: JoJc( 7) = -9260.910107256032
Quadratic cost function: JoJc( 8) = -9691.212019756096
Quadratic cost function: JoJc( 9) = -9915.784237898803
Quadratic cost function: JoJc(10) = -10072.29704553444

When all observing systems are assimilated, this goes unnoticed due to the larger Jo's from having many other instruments contribute to Jo.

[rtodling]

Just to illustrate the issue - increments from both GSI and JEDI due to only assimilation of SSMI/S-F17

[rtodling]

Now, if perform a static var bc the Jo's don't go negative:

Quadratic cost function: JoJc( 0) = 2456.840727561951
Quadratic cost function: JoJc( 1) = 1305.393860583277
Quadratic cost function: JoJc( 2) = 948.8254693765491
Quadratic cost function: JoJc( 3) = 781.4938529589878
Quadratic cost function: JoJc( 4) = 699.2939196665482
Quadratic cost function: JoJc( 5) = 619.8866151395243
Quadratic cost function: JoJc( 6) = 581.5810146327685
Quadratic cost function: JoJc( 7) = 566.2892365090478
Quadratic cost function: JoJc( 8) = 559.2420980111779
Quadratic cost function: JoJc( 9) = 556.936276645966
Quadratic cost function: JoJc(10) = 555.6681761983374
Quadratic cost function: JoJc(11) = 555.2053321348783
Quadratic cost function: JoJc(12) = 555.0491884160099
Quadratic cost function: JoJc(13) = 554.9953654891689
Quadratic cost function: JoJc(14) = 554.975554031243

[rtodling]

To me the test of static var bc suggests the problem to be in the VarBC settings.

I tested a couple of knobs in the VarBC ... if I remove this chunk:

      - ch19h: 12
        ch19v: 13
        ch22v: 14
        ch37h: 15
        ch37v: 16
        ch91h: 18
        ch91v: 17
        name: cloudWaterContent
        sensor: SSMIS

The JoJc becomes:

Quadratic cost function: JoJc( 0) = 3338.186435793299
Quadratic cost function: JoJc( 1) = 2185.0967698639
Quadratic cost function: JoJc( 2) = 1976.28106477083
Quadratic cost function: JoJc( 3) = 1818.144958847054
Quadratic cost function: JoJc( 4) = 1728.53091065418
Quadratic cost function: JoJc( 5) = 1658.193195549391
Quadratic cost function: JoJc( 6) = 1613.79882799066
Quadratic cost function: JoJc( 7) = 1575.749755278724
Quadratic cost function: JoJc( 8) = 1554.116824812512
Quadratic cost function: JoJc( 9) = 1536.775890743126
Quadratic cost function: JoJc(10) = 1521.959693561407
Quadratic cost function: JoJc(11) = 1511.483239276236
Quadratic cost function: JoJc(12) = 1506.108712173916
Quadratic cost function: JoJc(13) = 1502.340138675458
Quadratic cost function: JoJc(14) = 1499.853823594972
Quadratic cost function: JoJc(15) = 1498.292098157722
Quadratic cost function: JoJc(16) = 1497.347788911004
Quadratic cost function: JoJc(17) = 1496.898634591321
Quadratic cost function: JoJc(18) = 1496.574786869804
Quadratic cost function: JoJc(19) = 1496.336648404486
Quadratic cost function: JoJc(20) = 1496.197148284333
Quadratic cost function: JoJc(21) = 1496.114394586408
Quadratic cost function: JoJc(22) = 1496.066643854569
Quadratic cost function: JoJc(23) = 1496.042179345805
Quadratic cost function: JoJc(24) = 1496.035393026689
Quadratic cost function: JoJc(25) = 1496.044307629198
Quadratic cost function: JoJc(26) = 1496.062576961458
Quadratic cost function: JoJc(27) = 1496.085593682322
Quadratic cost function: JoJc(28) = 1496.119354887723
Quadratic cost function: JoJc(29) = 1496.151743411756
Quadratic cost function: JoJc(30) = 1496.177147788303

reasonable and the increment comparison becomes:

I am putting this fix in a PR, but I would like to see an explanation from @gmao-yzhu and @gmao-yzhu who probably understand better what these knobs are supposed to do and under what conditions.

[gmao-yzhu]

SSMI/S in GEOS-FP, I don't think its VarBC predictors have a cloud term, no matter it is for clear-sky or all-sky. SSMI/S should have an ascend/descend node term in VarBC.

These several lines look like to calculate an empirical cloud water formula either for QC or observation error. This is the original setting from Jianjun?

Yanqiu

To me the test of static var bc suggests the problem to be in the VarBC settings.

I tested a couple of knobs in the VarBC ... if I remove this chunk:

      - ch19h: 12
        ch19v: 13
        ch22v: 14
        ch37h: 15
        ch37v: 16
        ch91h: 18
        ch91v: 17
        name: cloudWaterContent
        sensor: SSMIS

The JoJc becomes:

Quadratic cost function: JoJc( 0) = 3338.186435793299 Quadratic cost function: JoJc( 1) = 2185.0967698639 Quadratic cost function: JoJc( 2) = 1976.28106477083 Quadratic cost function: JoJc( 3) = 1818.144958847054 Quadratic cost function: JoJc( 4) = 1728.53091065418 Quadratic cost function: JoJc( 5) = 1658.193195549391 Quadratic cost function: JoJc( 6) = 1613.79882799066 Quadratic cost function: JoJc( 7) = 1575.749755278724 Quadratic cost function: JoJc( 8) = 1554.116824812512 Quadratic cost function: JoJc( 9) = 1536.775890743126 Quadratic cost function: JoJc(10) = 1521.959693561407 Quadratic cost function: JoJc(11) = 1511.483239276236 Quadratic cost function: JoJc(12) = 1506.108712173916 Quadratic cost function: JoJc(13) = 1502.340138675458 Quadratic cost function: JoJc(14) = 1499.853823594972 Quadratic cost function: JoJc(15) = 1498.292098157722 Quadratic cost function: JoJc(16) = 1497.347788911004 Quadratic cost function: JoJc(17) = 1496.898634591321 Quadratic cost function: JoJc(18) = 1496.574786869804 Quadratic cost function: JoJc(19) = 1496.336648404486 Quadratic cost function: JoJc(20) = 1496.197148284333 Quadratic cost function: JoJc(21) = 1496.114394586408 Quadratic cost function: JoJc(22) = 1496.066643854569 Quadratic cost function: JoJc(23) = 1496.042179345805 Quadratic cost function: JoJc(24) = 1496.035393026689 Quadratic cost function: JoJc(25) = 1496.044307629198 Quadratic cost function: JoJc(26) = 1496.062576961458 Quadratic cost function: JoJc(27) = 1496.085593682322 Quadratic cost function: JoJc(28) = 1496.119354887723 Quadratic cost function: JoJc(29) = 1496.151743411756 Quadratic cost function: JoJc(30) = 1496.177147788303

reasonable and the increment comparison becomes:

I am putting this fix in a PR, but I would like to see an explanation from @gmao-yzhu and @gmao-yzhu who probably understand better what these knobs are supposed to do and under what conditions.

[gmao-yzhu]

@rtodling I agree with you, let's remove these lines, I don't think they belong to VarBC.