Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Filter-out local datasets when calling base-rule #805

Merged
merged 3 commits into from
Jan 12, 2024

Conversation

khurram-ghani
Copy link
Collaborator

Related issue(s)/PRs: None

Summary

Filter out local datasets when calling the EGO base rule. This will reduce the burden of filtering on most acquisition rules/functions that we currently have. If in the future we have acquisition rules/functions that need to see the local datasets, we can expand the functionality then.

Also took the opportunity to fix a very minor typing issue in mk_batch_observer.

Fully backwards compatible: yes

PR checklist

  • The quality checks are all passing
  • The bug case / new feature is covered by tests
  • Any new features are well-documented (in docstrings or notebooks)

@khurram-ghani khurram-ghani changed the title Khurram/tr dataset filter Filter-out local datasets when calling base-rule Jan 10, 2024
Copy link
Collaborator

@hstojic hstojic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good I think, but perhaps would be good that @uri-granta takes a look as well

Copy link
Collaborator

@uri-granta uri-granta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Couple of minor comments.

@@ -1282,7 +1282,19 @@ def state_func(
_points.append(rule.acquire(subspace, _models, _datasets))
points = tf.stack(_points, axis=1)
else:
points = self._rule.acquire(acquisition_space, models, datasets)
# Filter out local datasets as this is the EGO rule with normal acquisition
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fact that self._rules implies (EGO + no local models) isn't obvious here. Maybe rename ._rules to
._subspace_rules and change the initialisation (above) to:

        if self._rules is None and not (
            _num_local_models == 0 and isinstance(self._rule, EfficientGlobalOptimization)
        ):

?

Copy link
Collaborator Author

@khurram-ghani khurram-ghani Jan 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure about renaming to _subspace_rules, but am happy to apply De Morgan's law to the condition expression.

Also, I'll change the comment slightly to be not so definitive about being EGO. That is the case for now, but may support more rules in the future.


# Create a BatchTrustRegionBox instance with the mock base_rule.
subspaces = [SingleObjectiveTrustRegionBox(search_space) for _ in range(2)]
rule = BatchTrustRegionBox(subspaces, mock_base_rule) # type: ignore[var-annotated]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(what's the mypy error here?)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error is:
tests/unit/acquisition/test_rule.py:1819: error: Need type annotation for "rule" [var-annotated]

It wants the type of the rule spelled out, as it seems it can't figure that out iteself (via the base_rule). This is a standard error in most instantiations of TR rules as they have a base rule (I think). This can be avoided by using the following (for this instance). However, for unit tests I chose to be not that verbose; we instance the rules in many places.

rule: BatchTrustRegionBox[ProbabilisticModel] = BatchTrustRegionBox(subspaces, mock_base_rule)

@khurram-ghani khurram-ghani merged commit 82f929d into develop Jan 12, 2024
12 checks passed
@khurram-ghani khurram-ghani deleted the khurram/tr_dataset_filter branch January 12, 2024 11:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants