Skip to content

Commit

Permalink
Version 2.0.0 (#898)
Browse files Browse the repository at this point in the history
* Update runhistory2epm.py

* pre_commit

* Updated changelog

* Fixed comment

* Updated changelog

* Some small improvements

* Removed trailing space

* Updated changelog

* Restricted pynisher

* Updated changelog

* Link JMLR paper

* Update bibtex entry

Before: arxiv paper, now journal article.

* Fixed citation in docs

* Fixed callback typos

* BOinG for mixed hyperparameter search space

* fix format

* maint

* maint documentation

* maint pre-commit

* fix workflow

* maint

* Add dependabot for workflow files

* Bump actions/setup-python from 2 to 4 (#849)

Bumps [actions/setup-python](https://github.com/actions/setup-python) from 2 to 4.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](actions/setup-python@v2...v4)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump actions/checkout from 2 to 3 (#850)

Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v2...v3)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* maint

* Updated stale bot

* Updated stale options again

* Fixed typing

* Increased version

* rename check_point_in_ss to check_subspace_point

* Make format

* Uncomment test procedure

* docstrings for boing facade

* avoid typing namespace everywhere in epm_configuration_chooser

* fix docstring in rh2epm_boing

* Limit on dask

* Trying to fix tests

* Replaced typing in all files

* docstring for boing subspace

* Changed feature requests to feature

* allow acq_optimizer_local_kwargs to be passed to boing subspace without acq_optimizer_local

* fix pre-commit

* Added any

* Update facade

* Fix typing
* Match arguments from pSMAC with SMAC4AC signature
* Add description of pSMAC to docstring
* Add option to pass facade class to pSMAC

* Add example for pSMAC

* Removed docstring in readme's example

* Corrected text

* move construct_gp_kernels to smac.epm_gpytorch.boing_kernels

* maint

* fix wrongly imported func name in tests

* Add reference for pSMAC #609

* Rename example file

* Added roadmap to readme

* Prettification

* Add result merger

* Reformat with black

* Delete unnecessary import

* Fix typing

* Inject first trajectory entry from file from first rundir

* Add test for ResultMerger

* Fix docstring

* Get runhistory and trajectory from pSMAC run

Via the result merger. Searchers rundirs and
merges found runhistories and calculates a
new trajectory.

* Get trajectory

* Add regex to requirements

* Fixed MO cache problem

* Fix MO tests

* Fixing MO in AbstractRacer

* Accepting ints

* Hacking in the sum cost for MO

* Fix mypi

* Bugfix

* Make mypi happy

* Fix tests

* Make format

* Remove num_obj in MO optimizer

* A lot of restructuring

* Solve importings

* Solve importings

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Fixing, fixing, fixing, ...

* Updated changelog

* Still fixing :)

* fix docstrings

* Change directory of tests

* Fix ambiguous variable

* Del newlines

* fix mypy

* mypy and doc fix

* Removed int type checking

* fix tests

* Always return List of incs

* fix path issue in example

* maint boing facade

* Better docstring

* Return values if no bounds passed

* Removed import

* Return only one incumbent

* Rename n_optimizers -> n_workers

* Update example

* Update docstring

* Remove legacy function

* Update to new signature

* Update changelog.md

* Format black

* Update test

* Please pre-commit

* Merge development

* Fix tests

* Fix typos

* Format black

* Updated text

* Increased version

* Improved readme

* SMAC logo 50%

* SMAC logo 50%

* Removed dev-2.0

* Added doc workflow for latest tag

* Fixing docs latest tag

* Fixing docs latest tag

* First steps new intensifier

* Make it work again

* MOVE random weight initialization to smbo ask method

* ADD enable parego in unittests

* Test issues

* FIX formatting

* Link to old documentation

* MOVE random weight initialization to smbo ask method

* ADD enable parego in unittests

* FIX formatting

* Fix unit tests

* Add annotation

* corrected log prob for tophat prior

* Removed configurations from callback

* Fixed tests

* Fix mypy

* Save docs as latest version too

* Enabled all tests again

* Make version an env

* Added last version to docs

* Global variable fix

* Added v before version number

* New Intensifier Implementation (#888)

- Completely reimplemented the intensifiers (including Successive Halving and Hyperband): All intensifiers support multi-fidelity, multi-objective and multi-threading by nature now.
- Expected behaviour for ask-and-tell interface ensured (also for Successive Halving).
- Continuing a run is now fully supported.
- Added more examples.
- Updated documentation based on new implementation.

* First benchmark steps

* Added a warning

* Set xi to 0.01

* Incorporated walltime in trajectory

* Working version of benchmark

* Changed requirements

* Changed xi back to 0.0 again

* No duplicate budgets

* Fixed bug which updated tracker wrongly

* Bugfix: Delete correct bracket+stage

* Fixed tests

* Added costs to the trajectory

* Changed initial design max ratio to 0.25

* Make format

* Added more details to intensifier

* Finished benchmark

* Added benchmark source files

* Fixed tests because of initial design changes

* Improved runhistory API

* Fixed typo (#894)

* Merge doc fixes

* Doc fixes (#893)

Co-authored-by: timruhkopf <[email protected]>
Co-authored-by: Carolin Benjamins <[email protected]>
Co-authored-by: dengdifan <[email protected]>
Co-authored-by: Alexander Tornede <[email protected]>
Co-authored-by: Sarah Krebs <[email protected]>

* Make format

* Satisfied mypy

* Enabled more tests

* Increased version

* Mypy

* Mypy

* Mypy

* Fixed critical bug when using highest budget for inc selection

* Added SH+HB comparison

* Stuff

* Updated report

* Fixed docstring

* Fixed image path

* Removed vscode folder

* Bugfixes

* Doc fixes

* Updated changelog

* Make isort

* Fix typos

* Removed new line

* Added alpha2 to docs selection

* update description

* Fix 531 correctly create integers in initial design

This PR fixes a bug in which integer hyperparameters would not
be created correctly in the initial design. This went
unnoticed, except in the case where it was part of a condition
and the function deactivate_inactive_hyperparameters was
called.

This PR fixes this bug by mapping the random float that was
sampled by, for example and LHD or Sobol sequence, to the
float representation of an integer by calling _transform and
_inverse_transform of the integer hyperparameter.

* Add contact paragraph to README.

* Increase days until stale issues are closed by stalebot (#904)

* Edit changelog

* Add issue number

* Closes #907

* Clarify origin of configurations (#908)

* Random forest speed up (#903)

Co-authored-by: René Sass <[email protected]>

* Add: workflow to update pre-commit versions (#874)

Co-authored-by: René Sass <[email protected]>

* Add Git-Flow Workflow Specification

* [DOC] acquisition function maximizers

* Add stale reminder email action

* Increase version number of ConfigSpace

* Add guideline for updating copyright notice years to CONTRIBUTING.md

* Update requirement for pyrfr (#920)

* Update pre-commit: isort,flake8 version numbers

* Format black

* Fix mypy

* Ac bench (#929)

* Add Branin AC Task

* Add ac optimization type

* Nicer plotting

* Add ac mode

* Update ac task

* Make new smac env name adjustable

* Increase python version for smac 1.4

* Update benchmark

* Update plotting

* Goodby comments

* Fix path

* Fix 1.4 wrapper

* Update benchmark results

---------

Co-authored-by: Carolin Benjamins <[email protected]>

* [DOC] added documentation on the incumbent update

* Feature/add version to make publish (#931)

* Add version, correct package name and use bash in makefile

* Increase version

* Increase version in makefile, add places with versions to make publish

* Fix typo

* Transform integer feature names to strings (#930)

* Transform int features to str

* Fix pre-commit

* Update version to 2.0.0

* Update version to 2.0.0

* Update CHANGELOG.md

* Add information on deepcave to the readme

* Correct doc version

---------

Co-authored-by: dengdifan <[email protected]>
Co-authored-by: dengdifan <[email protected]>
Co-authored-by: Matthias Feurer <[email protected]>
Co-authored-by: Carolin Benjamins <[email protected]>
Co-authored-by: eddiebergman <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Carolin Benjamins <[email protected]>
Co-authored-by: C. Benjamins <[email protected]>
Co-authored-by: Katharina Eggensperger <[email protected]>
Co-authored-by: Eric Kalosa-Kenyon <[email protected]>
Co-authored-by: Pascal <[email protected]>
Co-authored-by: helegraf <[email protected]>
Co-authored-by: timruhkopf <[email protected]>
Co-authored-by: Alexander Tornede <[email protected]>
Co-authored-by: Sarah Krebs <[email protected]>
Co-authored-by: Alexander Tornede <[email protected]>
Co-authored-by: Alexander Tornede <[email protected]>
Co-authored-by: Sarah Krebs <[email protected]>
Co-authored-by: helegraf <[email protected]>
Co-authored-by: Jeroen Rook <[email protected]>
Co-authored-by: Tim Ruhkopf <[email protected]>
  • Loading branch information
22 people authored Mar 3, 2023
1 parent 0a1dff4 commit 731854e
Show file tree
Hide file tree
Showing 43 changed files with 9,197 additions and 95 deletions.
2 changes: 1 addition & 1 deletion .github/stale.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
daysUntilStale: 60

# Number of days of inactivity before a stale issue is closed
daysUntilClose: 7
daysUntilClose: 14

# Issues with these labels will never be considered stale
exemptLabels:
Expand Down
30 changes: 30 additions & 0 deletions .github/workflows/pre-commit-update.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
name: Pre-commit auto-update

on:
# every Monday at 1pm utc
schedule:
- cron: "0 13 * * 1"
# on demand
workflow_dispatch:

jobs:
auto-update:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2

- uses: actions/setup-python@v2

- uses: browniebroke/pre-commit-autoupdate-action@main

- name: Run updated pre-commit
run: |
pre-commit run --all-files
- uses: peter-evans/create-pull-request@v3
with:
token: ${{ secrets.GITHUB_TOKEN }}
branch: update/pre-commit-hooks
title: Update pre-commit hooks
commit-message: "chore: update pre-commit hooks"
body: Update versions of pre-commit hooks to latest version.
61 changes: 61 additions & 0 deletions .github/workflows/stale_reminder.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
name: Stale-Reminder

on:
schedule:
- cron: '42 14 * * 3'
workflow_dispatch:

jobs:
stale-reminder:
runs-on: ubuntu-latest
steps:
- name: Get cutoff date for soon-to-be-stale issues
id: date
run: |
echo "CUTOFF_DATE=$(date -d '-46 days' '+%Y-%m-%d')" >> $GITHUB_ENV
- name: Collect issues that may become stale
id: stale
uses: lee-dohm/select-matching-issues@v1
with:
format: list
path: "potentially_stale_issues.md"
token: ${{ github.token }}
query: >-
is:open
-label:dependency,documentation,feature,enhancement,bug,test,example
updated:<${{ env.CUTOFF_DATE }}
sort:updated-asc
- name: Collect labelled issues that have not had interaction in a long time (but will not become stale)
id: old
uses: lee-dohm/select-matching-issues@v1
with:
format: list
path: "old_issues.md"
token: ${{ github.token }}
query: >-
is:open
label:dependency,documentation,feature,enhancement,bug,test,example
updated:<${{ env.CUTOFF_DATE }}
sort:updated-asc
- name: Combine issues into mail content
id: combine
run: |
echo $date
echo "## Issues that may become stale in <= 14 days <br />" >> mail.html
echo "$(<potentially_stale_issues.md) <br />" >> mail.html
echo "## Issues that have not had interaction in the last 46 days but will not go stale due to their labels<br />" >> mail.html
echo "$(<old_issues.md) <br />" >> mail.html
- name: Send mail
id: mail
uses: dawidd6/action-send-mail@v3
with:
server_address: ${{secrets.MAIL_SERVER_ADDRESS}}
server_port: ${{secrets.MAIL_SERVER_PORT}}
secure: true
username: ${{secrets.MAIL_USERNAME}}
password: ${{secrets.MAIL_PASSWORD}}
subject: '[Stale Issues] Issues with last interaction before ${{ env.CUTOFF_DATE }}'
to: ${{secrets.MAIL_TARGET}}
from: SMAC3 Stale-Bot <${{secrets.MAIL_ADDRESS}}>
html_body: file://mail.html
convert_markdown: true
6 changes: 3 additions & 3 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
# If things break on update, raise an issue

repos:
- repo: https://github.com/pycqa/isort
rev: 5.10.1
- repo: https://github.com/PyCQA/isort
rev: 5.12.0
hooks:
- id: isort
name: isort smac
Expand Down Expand Up @@ -37,7 +37,7 @@ repos:
exclude: "scripts|tests"

- repo: https://github.com/PyCQA/flake8
rev: 5.0.4
rev: 6.0.0
hooks:
- id: flake8
name: flake8 smac
Expand Down
24 changes: 22 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,33 @@
# 2.0.0

## Improvements
- Clarify origin of configurations (#908).
- Random forest with instances predicts the marginalized costs by using a C++ implementation in `pyrfr`, which is much faster (#903).
- Add version to makefile to install correct test release version

## Bugfixes
- Continue run when setting incumbent selection to highest budget when using Successive Halving (#907).
- If integer features are used, they are automatically converted to strings.

## Workflows
- Added workflow to update pre-commit versions (#874).

## Misc
- Added benchmarking procedure to compare to previous releases.


# 2.0.0b1

- Completely reimplemented the intensifiers (including Successive Halving and Hyperband): All intensifiers support
multi-fidelity, multi-objective and multi-threading by nature now.
- Completely reimplemented the intensifiers (including Successive Halving and Hyperband): All intensifiers support multi-fidelity, multi-objective and multi-threading by nature now.
- Expected behaviour for ask-and-tell interface ensured (also for Successive Halving).
- Continuing a run is now fully supported.
- Added more examples.
- Updated documentation based on new implementation.
- Added benchmark to compare different versions.

## Bugfixes
- Correct handling of integer hyperparameters in the initial design (#531)


# 2.0.0a2

Expand Down
18 changes: 18 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,9 +53,27 @@ If you know how to fix a bug or implement your own feature, follow this small gu
- Pull request must include tests.
- We are using the Numpy Style Python Docstrings. Take a look at this [example](https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_numpy.html#example-numpy).
- The code should follow the PEP8 coding convention.
- Check that the copyright notice of all files that have been significantly changed includes the current year.

We try to react as fast as possible to your pull request.

## Git-Flow Workflow and Branch-Naming

We aim to approximately follow the [Git-Flow Workflow](https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow).
Thus, the following conventions apply:

Branches are named as follows:
- `main` The main branch, containing the latest stable release (tagged)
- `development` The development branch, for merging features and bugfixes in-between releases
- `release/<version>` Release branches for *preparing* a new release (no new features developed here),
deleted after release
- `feature/<feature-name>` Feature branches for implementing new functionality, updating existing functionality
or fixing bugs on the development branch
- `hotfix/<bug-name>` Hotfix branches for fixing bugs in the latest release
- `support/<version>` Support branches for bugfixes in older releases (e.g., `support/v0.12.x`). New 'old' versions are
tagged here.
- `gh-pages` The branch for the GitHub pages

## Reporting Security Issues

If you find security related issues, vulnerabilities or bugs including sensitive information, please do *not* report this to the issue tracker, or elsewhere public. Instead, please let us know via [email protected]
10 changes: 7 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
# These have been configured to only really run short tasks. Longer form tasks
# are usually completed in github actions.

SHELL := /bin/bash

NAME := SMAC3
PACKAGE_NAME := smac
VERSION := 2.0.0

DIR := "${CURDIR}"
SOURCE_DIR := ${PACKAGE_NAME}
Expand Down Expand Up @@ -133,15 +136,16 @@ clean-data:
# Will echo the commands to actually publish to be run to publish to actual PyPi
# This is done to prevent accidental publishing but provide the same conveniences
publish: clean build
read -p "Did you update the version number? Did you add the old version to docs/conf.py?"
read -p "Did you update the version number in Makefile, smac/__init__.py, benchmark/src/wrappers/v20.py? \
Did you add the old version to docs/conf.py? Did you add changes to CHANGELOG.md?"

$(PIP) install twine
$(PYTHON) -m twine upload --repository testpypi ${DIST}/*
@echo
@echo "Test with the following:"
@echo "* Create a new virtual environment to install the uplaoded distribution into"
@echo "* Create a new virtual environment to install the uploaded distribution into"
@echo "* Run the following:"
@echo "--- pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ ${NAME}"
@echo "--- pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ ${PACKAGE_NAME}==${VERSION}"
@echo
@echo "* Run this to make sure it can import correctly, plus whatever else you'd like to test:"
@echo "--- python -c 'import ${PACKAGE_NAME}'"
Expand Down
15 changes: 12 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ Status](https://codecov.io/gh/automl/SMAC3/branch/master/graph/badge.svg)](https

<img src="docs/images/logo.png" style="width: 50%;" />

SMAC is a tool for algorithm configuration to optimize the parameters of arbitrary algorithms, including hyperparameter
optimization of Machine Learning algorithms. The main core consists of Bayesian Optimization in combination with an
aggressive racing mechanism to efficiently decide which of two configurations performs better.
SMAC offers a robust and flexible framework for Bayesian Optimization to support users in determining well-performing
hyperparameter configurations for their (Machine Learning) algorithms, datasets and applications at hand. The main core
consists of Bayesian Optimization in combination with an aggressive racing mechanism to efficiently decide which of two configurations performs better.

SMAC3 is written in Python3 and continuously tested with Python 3.8, 3.9, and 3.10. Its Random
Forest is written in C++. In further texts, SMAC is representatively mentioned for SMAC3.
Expand Down Expand Up @@ -96,6 +96,10 @@ incumbent = smac.optimize()

More examples can be found in the [documentation](https://automl.github.io/SMAC3/main/examples/).

## Visualization via DeepCAVE

With DeepCAVE ([Repo](https://github.com/automl/DeepCAVE), [Paper](https://arxiv.org/abs/2206.03493)) you can visualize your SMAC runs. It is a visualization and analysis tool for AutoML (especially for the sub-problem
hyperparameter optimization) runs.

## License

Expand All @@ -110,6 +114,11 @@ You should have received a copy of the 3-clause BSD license
along with this program (see LICENSE file).
If not, see [here](https://opensource.org/licenses/BSD-3-Clause).

## Contacting us

If you have trouble using SMAC, a concrete question or found a bug, please create an [issue](https://github.com/automl/SMAC3/issues). This is the easiest way to communicate about these things with us.

For all other inquiries, please write an email to smac[at]ai[dot]uni[dash]hannover[dot]de.

## Miscellaneous

Expand Down
Loading

0 comments on commit 731854e

Please sign in to comment.