Load user-defined cpd aliase from XCA (compounds_auto.csv)

[phraenquex]

Need a mechanism to upload compound aliases. Probably a CSV, indexed by smiles string.

Need a way to see aliases in LHS. Probably tooltip.

Need a way to find by alias. Likely, search button should search through all aliases

Need a way to switch which alias is displayed in LHS. E.g. "switch alias" modal from top of Hit Navigator

[mwinokan]

My conversations with Jenke about including ASAP IDs in the download makes this ticket relevant again

[mwinokan]

In the legacy/v1 implementation, aliases could be defined in the metadata.csv.

The ASAP ID's are in SoakDB so maybe we don't need to place the onus for those on the curators, can they just be extracted from SoakDB.

We still need to think about how the aliases are shown in the frontend (see #1322)

[mwinokan]

@phraenquex says the simplest solution may be for XCA to put the compound aliases (crystal_name, zinc_id, asap_id) into a new CSV file in extra_files. This will mean that the ASAP ID's will be in the download.

After XCA the file exists in extra_files and the curator can add/change the aliases before upload

The f/e work is separate, see #1412.

@tdudgeon to get the ASAP ID's from SoakDB (@Waztom to provide example) and create the csv in the extra_files output

[mwinokan]

@tdudgeon was unable to find the ASAP ID's. @Waztom please help Tim with this.

@tdudgeon are you able to proceed with generating the CSV using the existing zinc/compound IDs while you wait for the ASAP ID's from SoakDb? Can the XCA generated sites be included in the CSV too?

@phraenquex please provide @tdudgeon with the required properties

[phraenquex]

Columns to include:

• observation (long code, not the one from the loader)
• dataset
• compound ID
• compound alias 1
• observation (SC)
• canonical site (short code)
• canonical site (long code)
• conformation site (SC)
• conformation site (LC)
• crystalform site (SC)
• crystalform site (LC)
• assembly (SC)
• assembly (LC)
• crystalform (SC)
• crysatlform (LC)

XCA generates columns 1-4; Loader will append the remaining columns.
For upload 1, leave column 4 blank.
For upload 2+, carry over any alias columns from previous uploads.

[kaliif]

@phraenquex SC vs. LC for conformation site, crystalform site, assembly, and crystalform - there's currently no separate short and long code for these. Do you mean the short name generated for a tag?

[phraenquex]

I meant the short name, the thing that goes before the dash.

Come to think of it, just put in a single column for each of those - it's anyway just for info.

[mwinokan]

@tdudgeon has implemented the following:

• XCA generated compounds_auto.csv
• The user can specify their own identifiers by copying to compounds_manual.csv

@kaliif the columns 5 and onwards from Frank's spec are no longer needed in the loader (as they are covered by the metadata.csv)

[mwinokan]

@kaliif please verify that any extra columns added to the summary csv will be loaded by the target loader, and also included in the metadata.csv generated for the download.

[mwinokan]

@kaliif is still working on this

[kaliif]

@mwinokan what needs to happen on data load? The way it's implemented now is on download, the extra columns are added to the file.

[phraenquex]

@kaliif yes that will be sufficient.

[mwinokan]

@kaliif says this is in staging but testing has only been local so far.

Likely one of Ryan's targets will be first to test these manual alias files, once they've been uploaded we should test the download as well.

[mwinokan]

Spec has been updated above, and a spin-out ticket for the f/e and API changes #1540.

@tdudgeon adds that we will need to support combi-soaks for the compounds_auto.csv.

@tdudgeon says we could add a ligand_name column to the generated CSV:

xtal	ligand_name	compound_code
CHIKV_MacB-x0270	LIG	Z100643660
CHIKV_MacB-x0270	LG1	Z100643662
CHIKV_MacB-x0270	LG2	Z100643663
CHIKV_MacB-x0281	LIG	Z1041785508
CHIKV_MacB-x0289	LIG	Z104492884

The user can then duplicate this file (compounds_manual.csv) and add columns, e.g.:

xtal	ligand_name	compound_code	enamine_id	ASAP_id	CDD_id
CHIKV_MacB-x0270	LIG	Z100643660	EN-121	ASAP-12129384293	CDD-23123313
CHIKV_MacB-x0270	LG1	Z100643662	EN-122	ASAP-12129384294	CDD-23123314
CHIKV_MacB-x0270	LG2	Z100643663	EN-123	ASAP-12129384295	CDD-23123315
CHIKV_MacB-x0281	LIG	Z1041785508
CHIKV_MacB-x0289	LIG	Z104492884

• For every new column create a new alias type, named after the header of that column, and populate the DB with the associated values
• Also create a special case compound_code_update that supersedes what comes from SoakDB and updates the default compound code alias (@kaliif please shout if it particularly difficult to implement)
• N.B. no changes to the compound_code in the compounds_manual.csv should be permitted, @kaliif throw an error.

[mwinokan]

@tdudgeon has added the ligand_name column to the compounds_auto.csv and all the data is in the metadata.yaml

@tdudgeon says that the compounds_auto.csv is not used by the loader, but it serves a purpose in the final target download and to serve as a template for compounds_manual.csv

@phraenquex clarifies that there is no explicit format specification for the new aliases

[mwinokan]

@kaliif is starting to work on this ticket now

[kaliif]

Also create a special case compound_code_update that supersedes what comes from SoakDB and updates the default compound code alias (@kaliif please shout if it particularly difficult to implement)

@mwinokan I don't understand this point, I'm afraid. I don't know what comes from SoakDB or what needs to be updated.

[mwinokan]

@tdudgeon has added the ligand_name column.

@kaliif Regarding the compound_code_update column:

• Assume that in the compounds_manual.csv the compound_code column is unchanged and corresponds exactly to what is in SoakDB
• If the user provides the compound_code_update column, these values should be used to supersede the default alias.

So a user might provide (for a given dataset/lig name):

• compound_code: from SoakDB and is what is currently shown in the f/e
• compound_code_update: column for user to override the SoakDB value, use as the default alias and served to the f/e
• Any other column please store as a separate alias in the DB, but is currently not used by the f/e

e.g. in this compounds_manual.csv example the compound_code for the first ligand is overridden by the compound_code_update:

xtal	ligand_name	compound_code	compound_code_update	enamine_id	ASAP_id	CDD_id
CHIKV_MacB-x0270	LIG	Z100643660	Z100643661	EN-121	ASAP-12129384293	CDD-23123313
CHIKV_MacB-x0270	LG1	Z100643662		EN-122	ASAP-12129384294	CDD-23123314
CHIKV_MacB-x0270	LG2	Z100643663		EN-123	ASAP-12129384295	CDD-23123315
CHIKV_MacB-x0281	LIG	Z1041785508
CHIKV_MacB-x0289	LIG	Z104492884

@tdudgeon says that the lookup can be done using the xtal and ligand_name columns alone, the compound_code column is there to match against what is in the YAML. And the loader should throw an error if the compound_code column does not match what is expected. Throw a useful error that says exactly what doesn't match and that it should go into the compound_code_update column instead.

[kaliif]

Dev done, needs testing, not merged to staging yet

[phraenquex]

@mwinokan please test once the clusters are back

[mwinokan]

b/e PR692 for staging fix implemented. @mwinokan to test

[mwinokan]

@kaliif As described in #1540 (comment) the upload A71EV2A_xca_staging_20241104_fake_aliases.tar.gz fails on both Matej's stack and staging

[mwinokan]

@kaliif's latest backend has fixed the tarball issue (see comment in 1540).

[mwinokan]

@kaliif says the b/e part is in staging

[mwinokan]

@matej-vavrek is there a b/e difference between your stack and staging currently that affects this ticket?