Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Batch sampling improvement #1154

Open
wants to merge 17 commits into
base: development
Choose a base branch
from
Open

Conversation

dengdifan
Copy link
Contributor

Closing #1152
This is the first step towards solving the batch sampling

TODO:

  • checking if everything works well under batch setting
  • adding unit tests

@benjamc benjamc mentioned this pull request Oct 24, 2024
@benjamc benjamc added this to the v2.3 milestone Nov 27, 2024
@dengdifan dengdifan requested a review from benjamc December 2, 2024 14:50
Copy link
Collaborator

@benjamc benjamc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Update CHANGELOG.md
  • Add example
  • Update docs / somewhere mention this new feature

smac/main/config_selector.py Show resolved Hide resolved
smac/main/config_selector.py Outdated Show resolved Hide resolved
smac/main/config_selector.py Outdated Show resolved Hide resolved
smac/main/config_selector.py Outdated Show resolved Hide resolved
# gaussian process
assert isinstance(self._model, GaussianProcess), 'Sample based estimate strategy only allows ' \
'GP as surrogate model!'
return self._model.sample_functions(X_test=X_running, n_funcs=1)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why cannot we sample from the random forest?

trial: self.runhistory[trial]
for trial in self.runhistory
if self.runhistory[trial].status == StatusType.RUNNING
# and runhistory.data[run].time >= self._algorithm_walltime_limit # type: ignore
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this commented out / why would we need this?
If it should stay there commented, please explain why

trial: self.runhistory[trial]
for trial in self.runhistory
if self.runhistory[trial].status == StatusType.RUNNING
# and runhistory.data[run].time >= self._algorithm_walltime_limit # type: ignore
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here

smac/runhistory/encoder/abstract_encoder.py Outdated Show resolved Hide resolved
@benjamc benjamc linked an issue Dec 4, 2024 that may be closed by this pull request
@benjamc benjamc marked this pull request as ready for review January 13, 2025 08:50
@benjamc
Copy link
Collaborator

benjamc commented Jan 13, 2025

Outstanding TODOs:

  • benchmark a bit to give an estimate (check example 7_0)
  • add to docs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: In Progress
Development

Successfully merging this pull request may close these issues.

Improve batch sampling
2 participants