Add more efficient bit packing functions for BitArray

[chriseclectic]

Summary

This adds internal utility functions for improved bit-packing, unpacking, and slicing for internal use by BitArray.

Details and comments

These functions give additional functionality for packing and unpacking arbitrary indexes bits in larger arrays by only working with the non-zero bytes, rather than having to pack or unpack the full array as was previous done.

The functions can also be used independently for working with byte and boolean arrays outside of BitArray, since they don't assume the shot-axis formatting required by BitArray. These helper functions might be a good candidate for porting to Rust in the future.

[qiskit-bot]

One or more of the following people are relevant to this code:

• @Qiskit/terra-core
• @ajavadia
• @levbishop
• @t-imamichi

[coveralls]

Pull Request Test Coverage Report for Build 11500126152

Details

• 185 of 199 (92.96%) changed or added relevant lines in 4 files are covered.
• 7 unchanged lines in 4 files lost coverage.
• Overall coverage increased (+0.004%) to 88.689%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
qiskit/primitives/containers/bit_packing.py	164	178	92.13%

Files with Coverage Reduction	New Missed Lines	%
crates/accelerate/src/two_qubit_decompose.rs	1	92.09%
crates/accelerate/src/unitary_synthesis.rs	1	92.24%
qiskit/primitives/containers/bit_array.py	1	95.86%
crates/qasm2/src/lex.rs	4	92.48%

Totals
Change from base Build 11481266566:	0.004%
Covered Lines:	75047
Relevant Lines:	84618

---

💛 - Coveralls

[t-imamichi]

Thank you for the improvement. I tested it with the following script and confirmed the performance improvement for slice_bits.

from timeit import timeit
import numpy as np
from qiskit.primitives.containers import BitArray

def bench_from_bool_array(num_shots: int, num_qubits: int, seed: int):
    rng = np.random.default_rng(seed)
    arr = rng.integers(2, size=(num_shots, num_qubits), dtype=bool)
    print("from_bool_array")
    print(f"{timeit(lambda: BitArray.from_bool_array(arr), number=1)} sec\n")
    ba = BitArray.from_bool_array(arr)
    return ba

def bench_slice_bits(ba: BitArray):
    indices = [1] * 4
    print("slice_bits")
    print(ba.slice_bits(indices).get_counts())
    print(f"{timeit(lambda: ba.slice_bits(indices), number=1)} sec\n")

def bench_concatenate(ba: BitArray):
    print("concatenate")
    print(f"{timeit(lambda: BitArray.concatenate([ba] * 10), number=1)} sec\n")

ba = bench_from_bool_array(100_000, 100_000, 123)
ba = ba.reshape(1, ba.size)
bench_slice_bits(ba)
bench_concatenate(ba)

main branch

from_bool_array
0.8163162080018083 sec

slice_bits
{'0000': 49992, '1111': 50008}
1.7400712500020745 sec

concatenate
1.5998445830045966 sec

this PR

from_bool_array
0.8202344169985736 sec

slice_bits
{'0000': 49992, '1111': 50008}
0.003202332998625934 sec

concatenate
1.6464810420002323 sec

[ihincks]

This PR just came back onto my radar. @t-imamichi thanks for the timings. Based on them, do you think that we should only consider slice_bits?

[t-imamichi]

Thank you for paying attention to this PR, @ihincks
I updated the microbenchmarking script to evaluate more methods affected and found that significant improvement for slice_bits and postselect but a bit slower for expectation_values.
I personally like the speed-up for slice_bits. What do you think?
If you decide to merge this, I will write a reno and get back to you.

from timeit import timeit

import numpy as np

from qiskit.primitives.containers import BitArray

rng = np.random.default_rng(123)


def bench_from_bool_array(num_shots: int, num_qubits: int):
    arr = rng.integers(2, size=(num_shots, num_qubits), dtype=bool)
    print("from_bool_array")
    print(f"{timeit(lambda: BitArray.from_bool_array(arr), number=1)} sec\n")
    ba = BitArray.from_bool_array(arr)
    return ba


def bench_from_counts(ba: BitArray):
    counts = ba.get_counts(np.arange(10_000))
    print("from_counts")
    print(f"{timeit(lambda: BitArray.from_counts(counts), number=1)} sec\n")


def bench_slice_bits(ba: BitArray):
    indices = [1] * 4
    print("slice_bits")
    print(ba.slice_bits(indices).get_counts())
    print(f"{timeit(lambda: ba.slice_bits(indices), number=1)} sec\n")


def bench_concatenate(ba: BitArray):
    print("concatenate")
    print(f"{timeit(lambda: BitArray.concatenate([ba] * 10), number=1)} sec\n")


def bench_postselect(ba: BitArray):
    print("postselect")
    n = 100
    print(f"{timeit(lambda: ba.postselect(np.arange(n), np.ones(n)), number=1)} sec\n")


def bench_expval(ba: BitArray):
    ba = ba.slice_bits(np.arange(2000))
    obs = "Z" * ba.num_bits
    print("expectation_values")
    print(f"{timeit(lambda: ba.expectation_values(obs), number=1)} sec\n")


ba = bench_from_bool_array(100_000, 100_000)
bench_from_counts(ba)
ba = ba.reshape(1, ba.size)
bench_slice_bits(ba)
bench_concatenate(ba)
bench_postselect(ba)
bench_expval(ba)

# main branch

from_bool_array
0.6368412500014529 sec

from_counts
1.1952290839981288 sec

slice_bits
{'0000': 49992, '1111': 50008}
1.3271455420035636 sec

concatenate
1.1935540419945028 sec

postselect
0.4070415000023786 sec

expectation_values
0.7278565000015078 sec

# this PR

from_bool_array
0.6289394590130541 sec

from_counts
1.1899380420072703 sec

slice_bits
{'0000': 49992, '1111': 50008}
0.0023455000045942143 sec

concatenate
1.107531416011625 sec

postselect
0.029799125011777505 sec

expectation_values
0.8104642919934122 sec

[t-imamichi]

I tried to push reno, but I cannot do it. Anyways, I wait for your decision, @ihincks.

[t-imamichi]

I'm not sure why I cannot push a commit with reno, but I leave it here just in case.

---
features_primitives:
  - |
    Improved performance of the methods :meth:`.BitArray.slice_bits` and
    `meth:.BitArray.postselect` by introducing internal utility functions
    for bit-packing, unpacking, and slicing.

[raynelfss]

Is this still on track for 1.3?