DIA-NN 1..8.1 vs 1.9.2 performance benchmark

[daichengxin]

PXD026600 was selected as benchmark datasets (narrow window and wide window). We compared peptides and proteins identified at q-value 0.01. The peptide number has increased a little. The number of protein quantified essentially the same. It is likely that large-scale data will show the difference better, according to DIA-NN document

[ypriverol]

@daichengxin thanks a lot for this great benchmark and been so quick. Im wondering the following:

• for the new version of DIANN are we using all the parameters in the right way?
• Are we losing some features/ids because of the parallelisation?

Im tagging @vdemichev in case he can help to find out if these are the expected results.

[vdemichev]

Seems fine I guess

[daichengxin]

We further tested the large-scale PXD030304 datasets using quantms with DIA-NN 1.8.1 and DIA-NN 1.9.2, and comparing with original paper results.

Identified #PSM

1.8.1	1.9.2	original results
252015828	285336801	204148465

Identified PSM at Global.Q.value < 0.01:

quantified peptides:

quantified proteins:

The quantms with DIA-NN 1.8.1 results seems to be ok and intersects well with the original results as well at peptide level. The weird thing is that 1.9.2 overlap is poor at PSM and peptide level although it identifies to more psm and peptides. Is it a change in the scan of the input spectra or something else?

[vdemichev]

Any warnings printed by 1.9.2?

[ypriverol]

@daichengxin we shouldn't compute PSMs only features, please remove the PSMs concept because PSMs are difficult to figure out in the original results.

I would be interested to understand the differences in protein IDs between original results and DIANN 1.9.2.