Releases: xchem/fragalysis-backend
Releases · xchem/fragalysis-backend
Significant bug-fix, feature, and metrics changes
Alkigns production from the latest staging branch
Add support for private key SSH tunnels
- Ports logic back from the V2 production code
Improved SSHTunnel & MySQL handling
- Improved timeouts
- Implementation of a MySQL connection retry
- Adds MySQL read timeout
- Improved logging (now recording, succinctly, MySQL connection issues)
- No more TIMEOUT state, services are either OK or DEGRADED
A replacement for 2023.03.4
- 2023.03.4 was a tag on staging, not production
2024.03.4 (Kalev's metadata fix)
What's Changed
- Some changes to cset_upload.py to allow site observation short codes by @kaliif in #527
- Short code prefix and tooltip read from yaml file (issue 1334) by @kaliif in #528
- New format to download zip (issue 1326) by @kaliif in #530
- cset_upload.py: lhs_pdb renamed to ref_pdb (issue 1327) by @kaliif in #531
- Renamed canon- and conf site tags by @kaliif in #532
- Adds support for key-based SSH connections by @alanbchristie in #534
- build(deps): bump cryptography from 42.0.0 to 42.0.2 by @dependabot in #533
- Updates documentation for 1328 by @alanbchristie in #536
- build(deps): bump django from 3.2.20 to 3.2.24 by @dependabot in #535
- Reverting unnecessary changes re RHS upload by @kaliif in #538
- Download zip; ligand sdfs now in aligned_files not in missing_sdfs by @kaliif in #540
- RHS upload backend side functional by @kaliif in #541
- Increased loglevel to error on unexpected exceptions block by @kaliif in #542
- Discourse service check now checks API key by @alanbchristie in #544
- build(deps): bump cryptography from 42.0.2 to 42.0.4 by @dependabot in #539
- Changes metadata.csv format in download and adds 'upload_name' attribute to tag model (issues 1343 and 1355) by @kaliif in #546
- Updates to api/download_structures endpoint (issue 1326) by @kaliif in #547
- Added parsing directives to DownloadStructuresserializer by @kaliif in #548
- Consecutive numbering of observations under canon site by @kaliif in #549
- Issue 1326, crystallographic_files folders in download now sans suffix by @kaliif in #550
- SiteObservatdion.tag split to tag and tag_prefix (1361) by @kaliif in #551
- Tag names understand prefix in download's metadata.csv by @kaliif in #552
- Return all proteins listed in api/download_structure (issue 1375) by @kaliif in #553
- Fixed 'All structures' option not working in download dialog by @kaliif in #554
- Restore 'upload_name' field in SiteObservationTags to prefix-tag format by @kaliif in #556
- Serve all uploaded files in download by @kaliif in #557
- Allow empty code_prefix and tooltip by @kaliif in #558
- Add force_error_display to connection functions (default False) by @alanbchristie in #559
- Align staging with production by @alanbchristie in #560
- Add DISABLE_RESTRICT_PROPOSALS_TO_MEMBERSHIP by @alanbchristie in #561
- Metadata.csv in download now showing correct tags by @kaliif in #563
- Fixed tag creation process for multiple uploads by @kaliif in #565
Full Changelog: 2024.03.3...2024.03.4
Contributors
kaliif, dependabot, and alanbchristie
Assets 2
v2 b/e release
Latest staging code (#562) * Some changes to cset_upload.py to allow site observation short codes (#527) * stashing * fix: cset_upload.py updated to allow new-style site observation codes NB! this probably still won't work! I suspect the file I was given is broken and I cannot test it further * stashing * stashing * Short code prefix and tooltip to backend Target loader now reads short code prefix and tooltip from meta_aligner.yaml. Tooltip is saved to Experiment model. TODO: make tooltip available via API * Prefix tooltip now serverd by api/site_observation * stashing * Site observation groups for shortcodes now by experiment * feat: download structure fixed TODO: add all the yamls * All yaml files added to download * New format to download zip (issue 1326) (#530) * stashing * stashing * feat: download structure fixed TODO: add all the yamls * All yaml files added to download * cset_upload.py: lhs_pdb renamed to ref_pdb * Renamed canon- and conf site tags * Adds support for key-based SSH connections (#534) * Centralised environment variables (#529) * refactor: Restructured settings.py * docs: Minor tweaks * refactor: Move security and infection config to settings * refactor: b/e & f/e/ tags now in settings (also fixed f/e tag value) * refactor: Move Neo4j config to settings * refactor: More variables into settings * refactor: Moved remaining config * docs: Adds configuration guide as comments * docs: Variable prefix now 'stack_' not 'stack_env_' --------- Co-authored-by: Alan Christie <[email protected]> * feat: Adds support for private keys on SSH tunnel * fix: Fixes key-based logic --------- Co-authored-by: Alan Christie <[email protected]> * build(deps): bump cryptography from 42.0.0 to 42.0.2 (#533) Bumps [cryptography](https://github.com/pyca/cryptography) from 42.0.0 to 42.0.2. - [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst) - [Commits](https://github.com/pyca/cryptography/compare/42.0.0...42.0.2) --- updated-dependencies: - dependency-name: cryptography dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * docs: Updates documentation (#536) Co-authored-by: Alan Christie <[email protected]> * build(deps): bump django from 3.2.20 to 3.2.24 (#535) Bumps [django](https://github.com/django/django) from 3.2.20 to 3.2.24. - [Commits](https://github.com/django/django/compare/3.2.20...3.2.24) --- updated-dependencies: - dependency-name: django dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix: reverting wrong changes * fix: reverting wrong changes (#538) * stashing * add site observation's ligand sdf to aligned_files * fix: custom pdb now downloadable * fix: increased loglevel to error on unexpected exceptions block * fix: Discourse service check now checks API key before creating a service (#544) Co-authored-by: Alan Christie <[email protected]> * build(deps): bump cryptography from 42.0.2 to 42.0.4 (#539) Bumps [cryptography](https://github.com/pyca/cryptography) from 42.0.2 to 42.0.4. - [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst) - [Commits](https://github.com/pyca/cryptography/compare/42.0.2...42.0.4) --- updated-dependencies: - dependency-name: cryptography dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * metadata.csv populated Started working on issue 1355 as well, it's too tightly coupled. Some work remaining re that: - when tag created in UI, make sure upload_name attribute is populated * upload_name automatically pouplated when creating tags in UI Only populated on creation, updates won't touch it * changes to api/download_structures - apo_file, bound_file, sdf_info and smiles_info merged into all_aligned_structures - added pdb_info field NB! download_structures was requred to provide ligand_pdb as well. This wasn't tracked previously, so I added field to SiteObservation model. Meaning there's a migration and on stack deployment data needs to be wiped and reuploaded * don't download neighbourhoods.yaml unless trans_matrix_info is checked * fixed error handling (errors.csv) and not returning combined sdf * fix: Added parsing directives to DownloadStructuresserializer * Consecutive numbering of observations under canon site * SiteObservatdion.tag split to tag and tag_prefix (1361) * fix: crystallographic_files folders in download now sans suffix (#550) * fix: tag names underdand prefix in download's metadata.csv * fix: return all proteins listed in api/download_structures * fix: fixed 'All structures' option not working in download dialog * Migrations for new file fields * Issue 1326 - mol and smiles added to download bundle NB! not prodction/staging ready, still contains a hack for testing because XCA doesn't provide all the attributes. * Target loader should handle empty code_prefix and tooltip 'Should' because haven't tested yet with real data * Column 'Downloaded' to metadata.csv in downloads * fix: restore 'upload_name' in site obvs tags to prefix-tag format * Removed ligand_smiles workaround All necessary files are now tracked by the database and returned in download. * fix: Add force_error_display to connection functions (default False) (#559) Co-authored-by: Alan Christie <[email protected]> * Align production with staging (#555) (#560) * Some changes to cset_upload.py to allow site observation short codes (#527) * stashing * fix: cset_upload.py updated to allow new-style site observation codes NB! this probably still won't work! I suspect the file I was given is broken and I cannot test it further * stashing * stashing * Short code prefix and tooltip to backend Target loader now reads short code prefix and tooltip from meta_aligner.yaml. Tooltip is saved to Experiment model. TODO: make tooltip available via API * Prefix tooltip now serverd by api/site_observation * stashing * Site observation groups for shortcodes now by experiment * feat: download structure fixed TODO: add all the yamls * All yaml files added to download * New format to download zip (issue 1326) (#530) * stashing * stashing * feat: download structure fixed TODO: add all the yamls * All yaml files added to download * cset_upload.py: lhs_pdb renamed to ref_pdb * Renamed canon- and conf site tags * Adds support for key-based SSH connections (#534) * Centralised environment variables (#529) * refactor: Restructured settings.py * docs: Minor tweaks * refactor: Move security and infection config to settings * refactor: b/e & f/e/ tags now in settings (also fixed f/e tag value) * refactor: Move Neo4j config to settings * refactor: More variables into settings * refactor: Moved remaining config * docs: Adds configuration guide as comments * docs: Variable prefix now 'stack_' not 'stack_env_' --------- * feat: Adds support for private keys on SSH tunnel * fix: Fixes key-based logic --------- * build(deps): bump cryptography from 42.0.0 to 42.0.2 (#533) Bumps [cryptography](https://github.com/pyca/cryptography) from 42.0.0 to 42.0.2. - [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst) - [Commits](https://github.com/pyca/cryptography/compare/42.0.0...42.0.2) --- updated-dependencies: - dependency-name: cryptography dependency-type: indirect ... * docs: Updates documentation (#536) * build(deps): bump django from 3.2.20 to 3.2.24 (#535) Bumps [django](https://github.com/django/django) from 3.2.20 to 3.2.24. - [Commits](https://github.com/django/django/compare/3.2.20...3.2.24) --- updated-dependencies: - dependency-name: django dependency-type: direct:production ... * fix: reverting wrong changes * fix: reverting wrong changes (#538) * stashing * add site observation's ligand sdf to aligned_files * fix: custom pdb now downloadable * fix: increased loglevel to error on unexpected exceptions block * fix: Discourse service check now checks API key before creating a service (#544) * build(deps): bump cryptography from 42.0.2 to 42.0.4 (#539) Bumps [cryptography](https://github.com/pyca/cryptography) from 42.0.2 to 42.0.4. - [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst) - [Commits](https://github.com/pyca/cryptography/compare/42.0.2...42.0.4) --- updated-dependencies: - dependency-name: cryptography dependency-type: indirect ... * metadata.csv populated Started working on issue 1355 as well, it's too tightly coupled. Some work remaining re that: - when tag created in UI, make sure upload_name attribute is populated * upload_name automatically pouplated when creating tags in UI Only populated on creation, updates won't touch it * changes to api/download_structures - apo_file, bound_file, sdf_info and smiles_info merged into all_aligned_structures - added pdb_info field NB! download_structures was requred to provide ligand_pdb as well. This wasn't tracked previously, so I added field to SiteObservation model. Meaning there's a migration and on stack deployment data needs to be wiped and reuploaded * don't download neighbourhoods.yaml unless trans_matrix_info is checked * fixed error handling (errors.csv) and not returning combined sdf * fix: Added parsing directives to DownloadStructuresserializer * Consecutive numbering of observations under canon site * SiteObservatdion.tag split to tag and tag_prefix (1361) * fix: crystallographic_files folders in download now sans suffix (#550) * fix: tag names underdand prefix in download's metadata.csv * fix: return all proteins listed in api/download_structures * fix: fixed 'All structures' option not working in download dialog * Migrations for new file fields * Issue 1326 - mol and smiles added to download bundle NB! not prodction/staging ready, still contains a hack for testing because XCA doesn't provide all the attributes. * Target loader should handle empty code_prefix and tooltip 'Should' because haven't tested yet with real data * Column 'Downloaded' to metadata.csv in downloads * fix: restore 'upload_name' in site obvs tags to prefix-tag format * Removed ligand_smiles workaround All necessary files are now tracked by the database and returned in download. * fix: Add force_error_display to connection functions (default False) (#559) --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Kalev Takkis <[email protected]> Co-authored-by: Warren Thompson <[email protected]> Co-authored-by: Alan Christie <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Warren Thompson <[email protected]> * Add DISABLE_RESTRICT_PROPOSALS_TO_MEMBERSHIP (#561) * feat: Add DISABLE_RESTRICT_PROPOSALS_TO_MEMBERSHIP * style: Minor log tweak --------- Co-authored-by: Alan Christie <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Kalev Takkis <[email protected]> Co-authored-by: Warren Thompson <[email protected]> Co-authored-by: Alan Christie <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Warren Thompson <[email protected]>
Candidate v2 (purple) release
Align production with staging (#555) * Some changes to cset_upload.py to allow site observation short codes (#527) * stashing * fix: cset_upload.py updated to allow new-style site observation codes NB! this probably still won't work! I suspect the file I was given is broken and I cannot test it further * stashing * stashing * Short code prefix and tooltip to backend Target loader now reads short code prefix and tooltip from meta_aligner.yaml. Tooltip is saved to Experiment model. TODO: make tooltip available via API * Prefix tooltip now serverd by api/site_observation * stashing * Site observation groups for shortcodes now by experiment * feat: download structure fixed TODO: add all the yamls * All yaml files added to download * New format to download zip (issue 1326) (#530) * stashing * stashing * feat: download structure fixed TODO: add all the yamls * All yaml files added to download * cset_upload.py: lhs_pdb renamed to ref_pdb * Renamed canon- and conf site tags * Adds support for key-based SSH connections (#534) * Centralised environment variables (#529) * refactor: Restructured settings.py * docs: Minor tweaks * refactor: Move security and infection config to settings * refactor: b/e & f/e/ tags now in settings (also fixed f/e tag value) * refactor: Move Neo4j config to settings * refactor: More variables into settings * refactor: Moved remaining config * docs: Adds configuration guide as comments * docs: Variable prefix now 'stack_' not 'stack_env_' --------- Co-authored-by: Alan Christie <[email protected]> * feat: Adds support for private keys on SSH tunnel * fix: Fixes key-based logic --------- Co-authored-by: Alan Christie <[email protected]> * build(deps): bump cryptography from 42.0.0 to 42.0.2 (#533) Bumps [cryptography](https://github.com/pyca/cryptography) from 42.0.0 to 42.0.2. - [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst) - [Commits](https://github.com/pyca/cryptography/compare/42.0.0...42.0.2) --- updated-dependencies: - dependency-name: cryptography dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * docs: Updates documentation (#536) Co-authored-by: Alan Christie <[email protected]> * build(deps): bump django from 3.2.20 to 3.2.24 (#535) Bumps [django](https://github.com/django/django) from 3.2.20 to 3.2.24. - [Commits](https://github.com/django/django/compare/3.2.20...3.2.24) --- updated-dependencies: - dependency-name: django dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix: reverting wrong changes * fix: reverting wrong changes (#538) * stashing * add site observation's ligand sdf to aligned_files * fix: custom pdb now downloadable * fix: increased loglevel to error on unexpected exceptions block * fix: Discourse service check now checks API key before creating a service (#544) Co-authored-by: Alan Christie <[email protected]> * build(deps): bump cryptography from 42.0.2 to 42.0.4 (#539) Bumps [cryptography](https://github.com/pyca/cryptography) from 42.0.2 to 42.0.4. - [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst) - [Commits](https://github.com/pyca/cryptography/compare/42.0.2...42.0.4) --- updated-dependencies: - dependency-name: cryptography dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * metadata.csv populated Started working on issue 1355 as well, it's too tightly coupled. Some work remaining re that: - when tag created in UI, make sure upload_name attribute is populated * upload_name automatically pouplated when creating tags in UI Only populated on creation, updates won't touch it * changes to api/download_structures - apo_file, bound_file, sdf_info and smiles_info merged into all_aligned_structures - added pdb_info field NB! download_structures was requred to provide ligand_pdb as well. This wasn't tracked previously, so I added field to SiteObservation model. Meaning there's a migration and on stack deployment data needs to be wiped and reuploaded * don't download neighbourhoods.yaml unless trans_matrix_info is checked * fixed error handling (errors.csv) and not returning combined sdf * fix: Added parsing directives to DownloadStructuresserializer * Consecutive numbering of observations under canon site * SiteObservatdion.tag split to tag and tag_prefix (1361) * fix: crystallographic_files folders in download now sans suffix (#550) * fix: tag names underdand prefix in download's metadata.csv * fix: return all proteins listed in api/download_structures * fix: fixed 'All structures' option not working in download dialog * Migrations for new file fields * Issue 1326 - mol and smiles added to download bundle NB! not prodction/staging ready, still contains a hack for testing because XCA doesn't provide all the attributes. * Target loader should handle empty code_prefix and tooltip 'Should' because haven't tested yet with real data * Column 'Downloaded' to metadata.csv in downloads * fix: restore 'upload_name' in site obvs tags to prefix-tag format * Removed ligand_smiles workaround All necessary files are now tracked by the database and returned in download. * fix: Add force_error_display to connection functions (default False) (#559) Co-authored-by: Alan Christie <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Kalev Takkis <[email protected]> Co-authored-by: Warren Thompson <[email protected]> Co-authored-by: Alan Christie <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Warren Thompson <[email protected]>
Candidate v2 release
Aligns production with latest staging (#545) * Some changes to cset_upload.py to allow site observation short codes (#527) * stashing * fix: cset_upload.py updated to allow new-style site observation codes NB! this probably still won't work! I suspect the file I was given is broken and I cannot test it further * stashing * stashing * Short code prefix and tooltip to backend Target loader now reads short code prefix and tooltip from meta_aligner.yaml. Tooltip is saved to Experiment model. TODO: make tooltip available via API * Prefix tooltip now serverd by api/site_observation * stashing * Site observation groups for shortcodes now by experiment * feat: download structure fixed TODO: add all the yamls * All yaml files added to download * New format to download zip (issue 1326) (#530) * stashing * stashing * feat: download structure fixed TODO: add all the yamls * All yaml files added to download * cset_upload.py: lhs_pdb renamed to ref_pdb * Renamed canon- and conf site tags * Adds support for key-based SSH connections (#534) * Centralised environment variables (#529) * refactor: Restructured settings.py * docs: Minor tweaks * refactor: Move security and infection config to settings * refactor: b/e & f/e/ tags now in settings (also fixed f/e tag value) * refactor: Move Neo4j config to settings * refactor: More variables into settings * refactor: Moved remaining config * docs: Adds configuration guide as comments * docs: Variable prefix now 'stack_' not 'stack_env_' --------- Co-authored-by: Alan Christie <[email protected]> * feat: Adds support for private keys on SSH tunnel * fix: Fixes key-based logic --------- Co-authored-by: Alan Christie <[email protected]> * build(deps): bump cryptography from 42.0.0 to 42.0.2 (#533) Bumps [cryptography](https://github.com/pyca/cryptography) from 42.0.0 to 42.0.2. - [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst) - [Commits](https://github.com/pyca/cryptography/compare/42.0.0...42.0.2) --- updated-dependencies: - dependency-name: cryptography dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * docs: Updates documentation (#536) Co-authored-by: Alan Christie <[email protected]> * build(deps): bump django from 3.2.20 to 3.2.24 (#535) Bumps [django](https://github.com/django/django) from 3.2.20 to 3.2.24. - [Commits](https://github.com/django/django/compare/3.2.20...3.2.24) --- updated-dependencies: - dependency-name: django dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix: reverting wrong changes * fix: reverting wrong changes (#538) * stashing * add site observation's ligand sdf to aligned_files * fix: custom pdb now downloadable * fix: increased loglevel to error on unexpected exceptions block * fix: Discourse service check now checks API key before creating a service (#544) Co-authored-by: Alan Christie <[email protected]> * build(deps): bump cryptography from 42.0.2 to 42.0.4 (#539) Bumps [cryptography](https://github.com/pyca/cryptography) from 42.0.2 to 42.0.4. - [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst) - [Commits](https://github.com/pyca/cryptography/compare/42.0.2...42.0.4) --- updated-dependencies: - dependency-name: cryptography dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * metadata.csv populated Started working on issue 1355 as well, it's too tightly coupled. Some work remaining re that: - when tag created in UI, make sure upload_name attribute is populated * upload_name automatically pouplated when creating tags in UI Only populated on creation, updates won't touch it * changes to api/download_structures - apo_file, bound_file, sdf_info and smiles_info merged into all_aligned_structures - added pdb_info field NB! download_structures was requred to provide ligand_pdb as well. This wasn't tracked previously, so I added field to SiteObservation model. Meaning there's a migration and on stack deployment data needs to be wiped and reuploaded * don't download neighbourhoods.yaml unless trans_matrix_info is checked * fixed error handling (errors.csv) and not returning combined sdf * fix: Added parsing directives to DownloadStructuresserializer * Consecutive numbering of observations under canon site * SiteObservatdion.tag split to tag and tag_prefix (1361) * fix: crystallographic_files folders in download now sans suffix (#550) --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Kalev Takkis <[email protected]> Co-authored-by: Warren Thompson <[email protected]> Co-authored-by: Alan Christie <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Documentation & Django update
- Documents configuration (1328)
- Bumps Django version (to 3.2.24 from 3.2.20)
Adds initial support for key-based SSH ISPyB access
- Without new playbooks stacks will continue to use the older password-based SSH tunnel access [#1316]
- Centralises configuration (in settings.py) [#1318]