Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add run_options option and support for processing level 1 data to BackendSamplerV2 #13357

Merged
merged 8 commits into from
Nov 5, 2024

Conversation

wshanks
Copy link
Contributor

@wshanks wshanks commented Oct 22, 2024

This change adds support to the BackendSamplerV2 class so that it will accept a run_options dict option and pass its content through to the underlying BackendV2's run() method.

Additionally, to support meas_level=1, the results processing code checks for a meas_level option inside of run_options and handles meas_level=1 data appropriately when it is set, rather than always assuming the returned data is level 2.

Finally, the behavior of BackendSamplerV2 was changed so that it does clear the QuantumCircuit.metadata of its input circuits before passing them on to backend.run like it did previously.

@wshanks wshanks requested review from a team as code owners October 22, 2024 16:55
@qiskit-bot
Copy link
Collaborator

One or more of the following people are relevant to this code:

  • @Qiskit/terra-core
  • @ajavadia
  • @levbishop
  • @t-imamichi

@wshanks
Copy link
Contributor Author

wshanks commented Oct 22, 2024

These changes were needed to get the qiskit-experiments tests to pass in this PR that moves all circuit execution from Backend.run to SamplerV2.run, making use of BackendSamplerV2 as the translation layer. We can add a release note and a test if needed but I wanted to see if these changes would be acceptable or not first. There are a couple other related issues/questions I had:

  • Right now, BackendSamplerV2 accepts an options dict and unpacks it into its Options class. These options are then passed through to Backend.run (except shots that is passed through only if it not overridden). This mechanism means that support for any additional arguments also requires changing the Options class again. Could all the options just be passed through besides shots?

  • Secondary request to the previous one -- I can see why keeping to the Options format is more standard so it might not be acceptable to accept and pass through all options. If so, could we change the __init__ to filter the keys in options and only pass the ones through to Options that have defined fields and warn about the ones that do not? This would make it a little easier to add new options in the future without breaking support for previous versions of Qiskit since an option not in the older Options definition will cause Options(**options) to fail and Options being a non-public class it is not supported to peak at its fields and filter before passing the options (edit: one could create a dummy BackendSamplerV2 and check its options fields and use them to filter the options for the real BackendSamplerV2 without accessing a private class).

  • Caveat to the previous points -- maybe we won't need to add support for any more backend run options.

  • For the qiskit-experiments tests, I also needed to delete this line:

    Is there any way to work around that issue? Can we revisit if the metadata still needs to be blanked out like that or could we pass through some kind of keep_metadata option? The qiskit-experiments test backends use the metadata to determine what results to return for the circuits. We can look into how to refactor the experiments tests to get the metadata a different way if not.

@coveralls
Copy link

coveralls commented Oct 22, 2024

Pull Request Test Coverage Report for Build 11599154700

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 24 of 25 (96.0%) changed or added relevant lines in 2 files are covered.
  • 54 unchanged lines in 5 files lost coverage.
  • Overall coverage increased (+0.07%) to 88.758%

Changes Missing Coverage Covered Lines Changed/Added Lines %
qiskit/primitives/backend_sampler_v2.py 22 23 95.65%
Files with Coverage Reduction New Missed Lines %
crates/qasm2/src/lex.rs 5 92.23%
qiskit/result/mitigation/utils.py 8 89.74%
qiskit/result/mitigation/correlated_readout_mitigator.py 10 89.69%
qiskit/result/mitigation/local_readout_mitigator.py 13 90.15%
crates/accelerate/src/commutation_checker.rs 18 97.09%
Totals Coverage Status
Change from base Build 11597322858: 0.07%
Covered Lines: 75643
Relevant Lines: 85224

💛 - Coveralls

…erV2

This change adds support to the `BackendSamplerV2` class so that it will
pass through the `meas_level`, `meas_return`, and `noise_model` options
passed to it through to the underlying `BackendV2`'s `run()` method. For
the sake of compatibility with backends that might not expect those
options, it does not pass default values for them (like the previously
defined options for the class) if the default `None` value is not
overridden.

Additionally, to support `meas_level=1`, the results processing code
checks the `meas_level` option and handles `meas_level=1` data
appropriately, rather than always assuming the returned data is level 2.
@wshanks
Copy link
Contributor Author

wshanks commented Oct 22, 2024

Qiskit/qiskit-ibm-runtime#1990 tries to make use of this change to translate between the IBM Quantum SamplerV2 options and common BackendV2.run options.

@t-imamichi
Copy link
Member

Is it possible to add some tests using GenericBackendV2 or something? It would be nice to have some tests at least.

@ElePT ElePT added this to the 1.3.0 milestone Oct 24, 2024
@wshanks
Copy link
Contributor Author

wshanks commented Oct 29, 2024

Is it possible to add some tests using GenericBackendV2 or something? It would be nice to have some tests at least.

There is not much support built in for testing with meas_level=1 results. I am investigating what is feasible to add.

run_options is a dict passed on to backend.run as it is for SamplerV2 in
qiskit-aer.
All of the backend primitives use the same helper function for calling
`backend.run` and this function has been clearing metadata because the
estimator primitives can add large objects to the metadata that payload
large for running the circuits on a remote service. In some cases, it is
helpful to have the circuit metadata make it to the `backend.run` call.
Since the concern about large metadata entries should not affect the
sampler case, an option is added here to skip clearing the metadata and
`BackendSamplerV2` is updated to use this option.
@wshanks
Copy link
Contributor Author

wshanks commented Oct 30, 2024

I added a simple test and also addressed the other points I had commented on in the way that works best for my case, but we can revisit them if those changes are a problem.

@raynelfss
Copy link
Contributor

Is this still on track for 1.3? If so, who is interested in reviewing it? @Qiskit/qiskit-primitives

@wshanks
Copy link
Contributor Author

wshanks commented Oct 31, 2024

On my end, I think it is ready for review, so it would be great if it could still make 1.3.

@t-imamichi
Copy link
Member

Thank you for adding tests. It looks good overall. You don't have to add a test with a real device, but I would like to confirm that it works fine with real devices. Have you tested it yet?

@wshanks
Copy link
Contributor Author

wshanks commented Nov 1, 2024

Thank you for adding tests. It looks good overall. You don't have to add a test with a real device, but I would like to confirm that it works fine with real devices. Have you tested it yet?

My main interest is in using BackendSamplerV2 with test backends in qiskit-experiments because we haven't had time to convert all the tests to use test samplers (maybe just using BackendSamplerV2 with test backends is good enough) because support for running with backend.run on a real device is going away very soon. I was still able to do it though. Here is a quick test using the readout discriminator experiment from qiskit-experiments:

image

@t-imamichi
Copy link
Member

Thank you for sharing a result with a real device.

Copy link
Member

@t-imamichi t-imamichi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ElePT ElePT added this pull request to the merge queue Nov 5, 2024
Merged via the queue into Qiskit:main with commit 2efb59c Nov 5, 2024
17 checks passed
@wshanks wshanks added the Changelog: New Feature Include in the "Added" section of the changelog label Nov 5, 2024
@wshanks wshanks deleted the backendsampler-options branch November 5, 2024 16:43
@wshanks wshanks changed the title Add meas_level, meas_return, and noise_model options to BackendSamplerV2 Add run_options option and support for processing level 1 data to BackendSamplerV2 Nov 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Changelog: New Feature Include in the "Added" section of the changelog mod: primitives Related to the Primitives module
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants