0.9.7

• A critical dependency version conflict error during lambda deployment introduced in 0.9.6 is now fixed.
• loosened dependency requirements further
  • urllib==1.24 --> urllib3>=1.24

0.9.6

• dependency requirements loosened
  • boto3==1.9.218 --> boto3>=1.9
  • botocore==1.12.218 --> botocore>=1.12.1
  • request==2.20.0 --> request==2.22.0 (to avoid conflict with urllib3==1.24)

0.9.5

• Minor fix in the metrics report html (e.g. plot lines extending outside of the plot area when there are very few data points)
• now a function that checks whether an instance is idle (i.e. 'not doing anything for the past hour') is added to the core API.

from tibanna.core import API
res = API().is_idle(instance_id=<instance_id>)  # res is either True (is idle) or False (is not idle)

• Benchmark-4dn version 0.5.6

• zebra (cgap)

  • The metrics report html for zebra is now different from the unicorn metrics report html and is developed separately.

• pony/zebra

  • A newly introduced issue of handling user-supplied processed file is fixed

• pony

  • A newly introduced error of lab/award not getting overwritten by wfr_meta and custom_pf_fields now fixed

0.9.3

• minor x coordinate shifting bug fix for resource metrics html (tibanna plot_metrics)
• kill function now uses dynamoDB (no need to specify --sfn if --job-id is specified)
• Newly introduced error of plot_metrics now working with CLI fixed (0.9.2).

0.9.1

• A new functionality of generating a resource metrics report html is now added! This report includes a graph of CPU/Memory/disk space utilization and usage at 1min interval, as well as a table of summary metrics.

  • After each run, an html report gets automatically added to the log_bucket which can be viewed using a Web Browser. However, for this to take effect, the unicorn must be redeployed.
  • The new plot_metrics function of CLI (tibanna plot_metrics -h) allows users to create the resource metrics report before a run it complete.
  • The same function can be used through Python API (API().plot_metrics(job_id=<jobid>, ...))

• A new functionality cost is added to the tibanna CLI/API, to retrieve the cost of a specific run.

  • tibanna cost --job-id=<jobid>
  • It usually takes a day for the cost to be available.
  • the cost can also be added to the resource plot, by

    tibanna cost -j <jobid> --update-tsv
    tibanna plot_metrics -j <jobid> --update-html-only --force-upload
    

• A new dynamoDB-based jobID indexing is enabled! This allows users to search by jobid without specifying step function name and even after the execution expires (e.g. tibanna log, tibanna plot_metrics)

  • To use this feature, the unicorn must be redeployed. Only the runs created after the redeployment would be searchable using this feature. When the jobid index is not available, tibanna automatically switches to the old way of searching.
  • DynamoDB may add to the cost but very minimally (up to $0.01 per month in case of 4DN)

• Benchmark 0.5.5 is used now for 4DN pipelines.

• run_workflow now has --do-not-open-browser option that disables opening the Step function execution on a Web Browser.

• Pony

  • now tibanna_cgap (zebra) is available along with tibanna_4dn (pony) that connects to and communicates with the CGAP portal instead of the 4DN Portal. Their common components have been moved to tibanna_ffcommon.
  • S3_ENCRYPT_KEY is now used instead of SECRET for 4DN/CGAP authentication
  • Multiple QC objects per file are allowed and can be handled automatically by zebra.

0.9.0

• root_ebs_size now supported (default 8) as a config field. (useful for large docker images or multiple docker images, which uses root EBS)

• TIBANNA_AWS_REGION and AWS_ACCOUNT_NUMBER no longer required as environment variables.

• pony

  • update_ffmeta.py and pony_utils.py completely refactored to use FourfrontUpdater class (Awsem and AwsemFile classes are deprecated)

0.8.8

• Fixed installation issue caused by python-lambda-4dn
• Input file can now be a directory for shell and snakemake
  • e.g. "file:///data1/shell/somedir" : "s3://bucketname/dirname"
• Output target can now be a directory for shell and snakemake
  • e.g. "file:///data1/shell/somedir": "dirname"

0.8.7

• ec2 termination policy is added to usergroup to support kill function
• run_workflow verbose option is now passed to dynamodb

pony

• validate_md5_s3_trigger lambda no longer deleted before updating

0.8.6

• A newly introduced issue of not reporting Metric after the run is now fixed.
• With tibanna log, when the log/postrunjson file is not available, it does not raise an error but prints a message.
• Benchmark 0.5.4 is used instead of 0.5.3 for 4DN pipelines.

0.8.5

• A newly introduced bug in the rerun cli now fixed.