Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Staramr add DM and update tool with database information #6609

Open
wants to merge 46 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
0c6c58f
add StarAMR wrapper and creator list from phac-nml
clsiguret Sep 19, 2024
c49866a
Merge branch 'galaxyproject:main' into staramr_add
clsiguret Sep 19, 2024
48f5d5f
Indentation correction
clsiguret Sep 19, 2024
ee0f205
Merge branch 'staramr_add' of github.com:clsiguret/tools-iuc into sta…
clsiguret Sep 19, 2024
a706e49
Changed macro token
clsiguret Sep 24, 2024
7255928
Parameter arguments and values modified
clsiguret Sep 24, 2024
db4da6a
Functional test 1
clsiguret Sep 25, 2024
2985b2a
Functional test 2
clsiguret Sep 26, 2024
0942dad
Functional test 3
clsiguret Sep 26, 2024
9526638
Functional test 4
clsiguret Sep 26, 2024
4285e38
Functional test 5
clsiguret Sep 26, 2024
2189a22
Functional test 6
clsiguret Sep 26, 2024
4d2c1bc
Functional test 7
clsiguret Sep 26, 2024
dc84ee9
Functional test 8
clsiguret Sep 26, 2024
63c0e72
Functional test 9
clsiguret Sep 26, 2024
6b17bcf
Functional test 10
clsiguret Sep 26, 2024
f7530c8
Changed output label
clsiguret Sep 26, 2024
fa21b9b
Init staramr dta manager
clsiguret Oct 4, 2024
92f636d
Modify pointfinder part to use an argument
clsiguret Nov 14, 2024
18945b0
Change tag for resfinder
clsiguret Nov 15, 2024
f0e5a0f
start reworking
hugolefeuvre Nov 20, 2024
5a06db5
finish DM
hugolefeuvre Nov 20, 2024
a31eee9
add DM part into tool part, add param directory, add db into test, th…
hugolefeuvre Nov 20, 2024
108a3a9
DB staramr git index files
hugolefeuvre Nov 22, 2024
0b5edef
delete index files wrong path
hugolefeuvre Nov 22, 2024
316182d
delete index files
hugolefeuvre Nov 25, 2024
fdb68f8
index files with good path
hugolefeuvre Nov 25, 2024
d63371f
delete repo index files
hugolefeuvre Nov 28, 2024
f40450f
Revert "delete repo index files"
hugolefeuvre Nov 28, 2024
fd2393f
Merge branch 'galaxyproject:main' into staramr_add
hugolefeuvre Nov 28, 2024
9b5d189
remove git submodule
hugolefeuvre Dec 2, 2024
8690cda
Squashed 'tools/staramr/test-data/resfinder_d1e607b_pointfinder_69491…
hugolefeuvre Dec 2, 2024
4cb3a6b
Merge commit '8690cda403a59c74e281a3cf3d7b825754585208' as 'tools/sta…
hugolefeuvre Dec 2, 2024
7121809
Squashed 'tools/staramr/test-data/resfinder_d1e607b_pointfinder_69491…
hugolefeuvre Dec 2, 2024
17cf6a0
Merge commit '7121809013ad7872a50bc27bd72260b0bdcf21a8' as 'tools/sta…
hugolefeuvre Dec 2, 2024
789121f
Squashed 'tools/staramr/test-data/resfinder_d1e607b_pointfinder_69491…
hugolefeuvre Dec 2, 2024
bff83af
Merge commit '789121f2306f28cdcfe21fb3c1eea9d953983eff' as 'tools/sta…
hugolefeuvre Dec 2, 2024
9a4542a
files modified by commit + hope .git will be present
hugolefeuvre Dec 3, 2024
07f7fba
delete reference data, add db build with test values into staramr sea…
hugolefeuvre Dec 3, 2024
a397cb2
Merge branch 'galaxyproject:main' into staramr_add
hugolefeuvre Dec 4, 2024
2d20089
update version suffix
hugolefeuvre Dec 4, 2024
e84dae7
Merge branch 'staramr_add' of https://github.com/clsiguret/tools-iuc …
hugolefeuvre Dec 4, 2024
7ffa4fb
delete other requirements in DM because dont need them here
hugolefeuvre Dec 4, 2024
1cab066
just delete . because create biocontainer with mull, hope it will pas…
hugolefeuvre Dec 4, 2024
c765ca6
new staramr version, delete numpy requirement, delete db at the end o…
hugolefeuvre Dec 18, 2024
39cfeff
Delete .gitmodules
hugolefeuvre Jan 8, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions data_managers/data_manager_build_staramr/.shed.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
name: data_manager_build_staramr
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could update the shed.yml file for the data mananger. Currently it reads as for the tool.

owner: iuc
description: "Scan genome contigs against the ResFinder, PlasmidFinder, and PointFinder antimicrobial resistance databases."
homepage_url: "https://github.com/phac-nml/staramr"
long_description: |
staramr - Scans genome contigs (in FASTA format) against the ResFinder,
PlasmidFinder, and PointFinder databases to search for antimicrobial resistance genomes.
Makes predictions of the drugs these genes give resistance to.
remote_repository_url: "https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_staramr"
type: unrestricted
categories:
- Data Managers
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
<tool id="data_manager_build_staramr" name="Download and build StarAMR database" version="@TOOL_VERSION@+galaxy@VERSION_SUFFIX@" tool_type="manage_data" profile="@PROFILE@">
<description></description>
<macros>
<token name="@TOOL_VERSION@">0.10.0</token>
<token name="@VERSION_SUFFIX@">0</token>
<token name="@PROFILE@">22.05</token>
<xml name="element_assert" token_name="" token_text="">
<element name="@NAME@">
<assert_contents>
<has_text text="@TEXT@"/>
<yield/>
</assert_contents>
</element>
</xml>
</macros>
<requirements>
<requirement type="package" version="@TOOL_VERSION@">staramr</requirement>
</requirements>
<stdio>
<exit_code range=":-1" level="fatal" description="Error: Cannot open file"/>
<exit_code range="1:" level="fatal" description="Error"/>
</stdio>
<command><![CDATA[
Comment on lines +19 to +23
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

equivalent:

Suggested change
<stdio>
<exit_code range=":-1" level="fatal" description="Error: Cannot open file"/>
<exit_code range="1:" level="fatal" description="Error"/>
</stdio>
<command><![CDATA[
<command detect_errors="exit_code"><![CDATA[

#import re
#set $match_resfinder = re.search(r'^resfinder_(?:[^_]*_)?([0-9a-z]*)_[0-9]{4}-[0-9]{2}-[0-9]{2}$', str($resfinder.resfinder_database_select))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could simplify parsing by make the values a comma-separated string (then you can just use split). Of course any other character will do as well.

#set $resfinder_commit = ($match_resfinder).group(1)
#set $match_plasmidfinder = re.search(r'^plasmidfinder_(?:[^_]*_)?([0-9a-z]*)_[0-9]{4}-[0-9]{2}-[0-9]{2}$', str($plasmidfinder.plasmidfinder_database_select))
#set $plasmidfinder_commit = ($match_plasmidfinder).group(1)
#set $match_pointfinder = re.search(r'^pointfinder_(?:[^_]*_)?([0-9a-z]*)_[0-9]{4}-[0-9]{2}-[0-9]{2}$', str($pointfinder.pointfinder_database_select))
#set $pointfinder_commit = ($match_pointfinder).group(1)
mkdir -p '$out_file.extra_files_path' &&
staramr db build
--dir resfinder_${resfinder_commit}_pointfinder_${pointfinder_commit}_plasmidfinder_${plasmidfinder_commit}
--resfinder-commit $resfinder_commit
--pointfinder-commit $pointfinder_commit
--plasmidfinder-commit $plasmidfinder_commit &&
mv resfinder_${resfinder_commit}_pointfinder_${pointfinder_commit}_plasmidfinder_${plasmidfinder_commit} '$out_file.extra_files_path' &&
cp '$dmjson' '$out_file'
]]></command>
<configfiles>
<configfile name="dmjson"><![CDATA[
#from datetime import date
#import re
#set $match_resfinder = re.search(r'^resfinder_(?:[^_]*_)?([0-9a-z]*)_[0-9]{4}-[0-9]{2}-[0-9]{2}$', str($resfinder.resfinder_database_select))
#set $resfinder_commit = ($match_resfinder).group(1)
#set $resfinder_parts = str($resfinder.resfinder_database_select).split('_')
#set $resfinder_tag_commit_date = '_'.join($resfinder_parts[1:])
#set $match_plasmidfinder = re.search(r'^plasmidfinder_(?:[^_]*_)?([0-9a-z]*)_[0-9]{4}-[0-9]{2}-[0-9]{2}$', str($plasmidfinder.plasmidfinder_database_select))
#set $plasmidfinder_commit = ($match_plasmidfinder).group(1)
#set $plasmidfinder_parts = str($plasmidfinder.plasmidfinder_database_select).split('_')
#set $plasmidfinder_tag_commit_date = '_'.join($plasmidfinder_parts[1:])
#set $match_pointfinder = re.search(r'^pointfinder_(?:[^_]*_)?([0-9a-z]*)_[0-9]{4}-[0-9]{2}-[0-9]{2}$', str($pointfinder.pointfinder_database_select))
#set $pointfinder_commit = ($match_pointfinder).group(1)
#set $pointfinder_parts = str($pointfinder.pointfinder_database_select).split('_')
#set $pointfinder_tag_commit_date = '_'.join($pointfinder_parts[1:])
{
"data_tables":{
"staramr_database":[
{
"value": "staramr_downloaded_#echo date.today().strftime('%d%m%Y')#_resfinder_${resfinder_commit}_pointfinder_${pointfinder_commit}_plasmidfinder_${plasmidfinder_commit}",
"name": "starAMR databases with ResFinder: ${resfinder_tag_commit_date}, PointFinder: ${pointfinder_tag_commit_date}, PlasmidFinder: ${plasmidfinder_tag_commit_date}",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this info shown to the user contain the hash? Maybe do not show it?

"path": "resfinder_${resfinder_commit}_pointfinder_${pointfinder_commit}_plasmidfinder_${plasmidfinder_commit}",
"resfinder_commit": "${resfinder_commit}",
"pointfinder_commit": "${pointfinder_commit}",
"plasmidfinder_commit": "${plasmidfinder_commit}"
}
]
}
}]]></configfile>
</configfiles>
<inputs>
<section name="resfinder" title="ResFinder database to build" expanded="true">
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest to remove the sections.

<param name="resfinder_database_select" type="select" label="Database version" help="Choose a database version to download">
<option value="resfinder_2.4.0_d1e607b_2024-08-06">ResFinder tag 2.4.0 (commit d1e607b) 2024-08-06</option>
<option value="resfinder_2.3.1_c8c38c3_2024-03-22">ResFinder tag 2.3.1 (commit c8c38c3) 2024-03-22</option>
<option value="resfinder_2.3.0_16d7fd3_2023-03-08">ResFinder tag 2.3.0 (commit 16d7fd3) 2023-03-08</option>
<option value="resfinder_2.2.1_208efbd_2023-10-27">ResFinder tag 2.2.1 (commit 208efbd) 2023-10-27</option>
<option value="resfinder_2.1.1_1a53e55_2023-05-31">ResFinder tag 2.1.1 (commit 1a53e55) 2023-05-31</option>
<option value="resfinder_2.1.0_c0cab25_2023-03-03">ResFinder tag 2.1.0 (commit c0cab25) 2023-03-03</option>
<option value="resfinder_2.0.2_18e3988_2023-02-08">ResFinder tag 2.0.2 (commit 18e3988) 2023-02-08</option>
<option value="resfinder_2.0.1_8306696_2023-01-16">ResFinder tag 2.0.1 (commit 8306696) 2023-01-16</option>
<option value="resfinder_2.0.0_585ec75_2022-10-13">ResFinder tag 2.0.0 (commit 585ec75) 2022-10-13</option>
</param>
</section>
<section name="pointfinder" title="PointFinder database to build" expanded="true">
<param name="pointfinder_database_select" type="select" label="Database version" help="Choose a database version to download">
<option value="pointfinder_4.1.1_694919f_2024-08-08">PointFinder tag 4.1.1 (commit 694919f) 2024-08-08</option>
<option value="pointfinder_4.1.0_9d55d02_2024-03-08">PointFinder tag 4.1.0 (commit 9d55d02) 2024-03-08</option>
<option value="pointfinder_4.0.1_b6e9e0b_2023-11-02">PointFinder tag 4.0.1 (commit b6e9e0b) 2023-11-02</option>
<option value="pointfinder_3.0.1_cb7806f_2023-05-31">PointFinder tag 3.0.1 (commit cb7806f) 2023-05-31</option>
<option value="pointfinder_3.0.0_3556bad_2023-01-18">PointFinder tag 3.0.0 (commit 3556bad) 2023-01-18</option>
<option value="pointfinder_2.0.1_0b281b0_2023-01-18">PointFinder tag 2.0.1 (commit 0b281b0) 2023-01-18</option>
<option value="pointfinder_2.0.0_afbc82d_2022-10-13">PointFinder tag 2.0.0 (commit afbc82d) 2022-10-13</option>
</param>
</section>
<section name="plasmidfinder" title="PlasmidFinder database to build" expanded="true">
<param name="plasmidfinder_database_select" type="select" label="Database version" help="Choose a database version to download">
<option value="plasmidfinder_3e77502_2024-03-07">PlasmidFinder (commit 3e77502) 2024-03-07</option>
<option value="plasmidfinder_81c11f4_2023-12-04">PlasmidFinder (commit 81c11f4) 2024-12-04</option>
<option value="plasmidfinder_314d85f_2023-03-17">PlasmidFinder (commit 314d85f) 2023-03-17</option>
<option value="plasmidfinder_2.1_1307168_2019-08-28">PlasmidFinder tag 2.1 (commit 1307168) 2019-08-28</option>
</param>
</section>
</inputs>
<outputs>
<data name="out_file" format="data_manager_json" label="${tool.name}"/>
</outputs>
<tests>
<test expect_num_outputs="1">
<section name="resfinder">
<param name="resfinder_database_select" value="resfinder_2.4.0_d1e607b_2024-08-06" />
</section>
<section name="pointfinder">
<param name="pointfinder_database_select" value="pointfinder_4.1.1_694919f_2024-08-08" />
</section>
<section name="plasmidfinder">
<param name="plasmidfiner_database_select" value="plasmidfinder_3e77502_2024-03-07" />
</section>
<output name="out_file">
<assert_contents>
<has_text text='"staramr_database":'/>
<has_text_matching expression='"value": "staramr_downloaded_[0-9]{8}_resfinder_d1e607b_pointfinder_694919f_plasmidfinder_3e77502"'/>
<has_text text='"name": "starAMR databases with ResFinder: 2.4.0_d1e607b_2024-08-06, PointFinder: 4.1.1_694919f_2024-08-08, PlasmidFinder: 3e77502_2024-03-07"'/>
<has_text_matching expression='"path": "resfinder_d1e607b_pointfinder_694919f_plasmidfinder_3e77502"'/>
<has_text text='"resfinder_commit": "d1e607b"'/>
<has_text text='"pointfinder_commit": "694919f"'/>
<has_text text='"plasmidfinder_commit": "3e77502"'/>
</assert_contents>
</output>
</test>
</tests>
<help><![CDATA[
This tool downloads and builds the StarAMR databases.
]]></help>
<citations>
<citation type="doi">10.3390/microorganisms10020292</citation>
</citations>
</tool>
22 changes: 22 additions & 0 deletions data_managers/data_manager_build_staramr/data_manager_conf.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
<?xml version="1.0"?>
<data_managers>
<data_manager tool_file="data_manager/data_manager_build_staramr_download.xml" id="data_manager_build_staramr_download">
<data_table name="staramr_database"> <!-- Defines a Data Table to be modified. -->
<output> <!-- Handle the output of the Data Manager Tool -->
<column name="value"/> <!-- columns that are going to be specified by the Data Manager Tool -->
<column name="name"/> <!-- columns that are going to be specified by the Data Manager Tool -->
<column name="path" output_ref="out_file">
<move type="directory">
<source>${path}</source>
<target base="${GALAXY_DATA_MANAGER_DATA_PATH}">staramr/${path}</target>
</move>
<value_translation>${GALAXY_DATA_MANAGER_DATA_PATH}/staramr/${path}</value_translation>
<value_translation type="function">abspath</value_translation>
</column>
<column name="resfinder_commit"/> <!-- columns that are going to be specified by the Data Manager Tool -->
<column name="pointfinder_commit"/> <!-- columns that are going to be specified by the Data Manager Tool -->
<column name="plasmidfinder_commit"/> <!-- columns that are going to be specified by the Data Manager Tool -->
</output>
</data_table>
</data_manager>
</data_managers>
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#This is a tab separated file describing the location of StarAMR databases
#used for the StarAMR tool
#
#file has this format (white space characters are TAB characters)
#
#The columns are:
#value name path resfinder_commit pointfinder_commit plasmidfinder_commit
#
#For example
#staramr_downloaded_20241004_resfinder_d1e607b_pointfinder_694919f_plasmidfinder_3e77502 ResFinder: tag 4.6.0, commit d1e607b, 2024-08-06 - PointFinder: tag 4.1.1, commit 694919f, 2024-08-08 - PlasmidFinder: commit 3e77502, 2024-03-07 (downloaded 20241004) /path/to/data d1e607b 694919f 3e77502
staramr_downloaded_20112024_resfinder_d1e607b_pointfinder_694919f_plasmidfinder_3e77502 starAMR databases with ResFinder: 2.4.0_d1e607b_2024-08-06, PointFinder: 4.1.1_694919f_2024-08-08, PlasmidFinder: 3e77502_2024-03-07 /tmp/tmp3frcu7r3/galaxy-dev/tool-data/staramr/resfinder_d1e607b_pointfinder_694919f_plasmidfinder_3e77502 d1e607b 694919f 3e77502
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#This is a tab separated file describing the location of StarAMR databases
#used for the StarAMR tool
#
#file has this format (white space characters are TAB characters)
#
#The columns are:
#value name path resfinder_commit pointfinder_commit plasmidfinder_commit
#
#For example
#staramr_downloaded_20241004_resfinder_d1e607b_pointfinder_694919f_plasmidfinder_3e77502 ResFinder: tag 4.6.0, commit d1e607b, 2024-08-06 - PointFinder: tag 4.1.1, commit 694919f, 2024-08-08 - PlasmidFinder: commit 3e77502, 2024-03-07 (downloaded 20241004) /path/to/data d1e607b 694919f 3e77502
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
<tables>
<table name="staramr_database" comment_char="#">
<columns>value, name, path, resfinder_commit, pointfinder_commit, plasmidfinder_commit</columns>
<file path="tool-data/staramr_database.loc"/>
</table>
</tables>
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
<tables>
<table name="staramr_database" comment_char="#">
<columns>value, name, path, resfinder_commit, pointfinder_commit, plasmidfinder_commit</columns>
<file path="${__HERE__}/test-data/staramr_database.loc.test"/>
</table>
</tables>
7 changes: 7 additions & 0 deletions tools/staramr/.shed.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
categories: [Sequence Analysis]
description: Scan genome contigs against the ResFinder, PlasmidFinder, and PointFinder antimicrobial resistance databases.
name: staramr
owner: iuc
long_description: staramr - Scans genome contigs (in FASTA format) against the ResFinder, PlasmidFinder, and PointFinder databases to search for antimicrobial resistance genomes. Makes predictions of the drugs these genes give resistance to.
remote_repository_url: https://github.com/galaxyproject/tools-iuc/tree/master/tools/staramr
homepage_url: https://github.com/phac-nml/staramr
Loading
Loading