Design discussion for OpenQASM header-file management

[jakelishman]

What should we add?

Summary

Qiskit's OpenQASM export needs a way for us to define what header files the file is defined in terms of, including letting the user override the headers we export against in order to target specific backends that may not allow gate statements for basis-gates they don't support. For parsing, we need an interface to allow the user to show us where header files that are used in their input program are.

We should be able to at least export in terms of any standard libraries of the languages and against header files that the user provides. For convenience of users generating OpenQASM that can be consumed into Qiskit's standard gates, we likely want to provide a header file that defines all of Qiskit's standard gates.

For import, the user should at least be able to point us to header files that they've used, and we should be able to find any Qiskit-defined headers in standard search paths. For greater support of existing backends, it's likely that we should accept a Python-space method for the caller to override which Instruction classes are used to represent particular OpenQASM instructions including allowing certain instructions to be defined as built-in (i.e. not needing definition) so it's easier for providers to upgrade to us.

Scope

In this document, I'm considering:

• broad strokes of the API for importing OQ2 and OQ3
• broad strokes of the API for defining the header files to export against
• structure of the header files we should define and know about

This document will not get into technical considerations of how the import or export are actually performed. It will not get into technical details of how a circuit is prepared for export to OpenQASM 2 or OpenQASM 3 for a particular set of header files - I think this is a more complex, different problem that would distract from this discussion. Instead, let us assume that any circuit to be output is already "compatible" with the header files it is being exported against, with the precise definition of "compatible" left for another issue.

Existing implementation

The new OpenQASM 2 parser qiskit.qasm2.load (and qiskit.qasm2.loads) currently does almost all of what I've written above, and it was sufficiently extensible that it's able to be used both in a more strict-to-the-spec mode and to emulate the legacy behaviour of QuantumCircuit.from_qasm_str that had many additional built-ins. The include path is modified with the include_path argument, and additional output-class overrides / additional built-ins are specified with the custom_instructions field. We expose some data attributes to make specifying the same custom instructions that the legacy importer used to quite convenient. For example, the "legacy" parser is now implemented as

from qiskit import qasm2

def from_qasm_file(path: str) -> QuantumCircuit:
    return qasm2.load(
        path,
        include_path=qasm2.LEGACY_INCLUDE_PATH,
        custom_instructions=qasm2.LEGACY_CUSTOM_INSTRUCTIONS,
        custom_classical=qasm2.LEGACY_CUSTOM_CLASSICAL,
        strict=False,
    )

The OpenQASM 2 export (currently QuantumCircuit.qasm, but #10533 will make it available as qiskit.qasm2.dump{,s} for consistency) does not support any of the above details for controlling the export, and the qelib1.inc file that it claims to include is actually a modified Qiskit version that does not match the initial definition of that file (see #4312).

The OpenQASM 3 parser qiskit.qasm3.load is currently in its initial-support phase, since it was pending this sort of discussion happening before we committed to any API for extensibility. It does not support any of the above import discussion yet.

OpenQASM 3 export qiskit.qasm3.dump has some limited support for writing out different include headers, but it doesn't actually read those files to know what's in them, so they don't affect the export other than adding extra include lines. One can specify additional basis_gates that do not trigger gate declarations in the main file, which can (a little awkwardly) in conjunction with includes be used to ensure that if a circuit is already transpiled for a backend, it will output a valid OQ3 form. Exporting a circuit transpiled for an sx, rz, ecr backend currently looks like something like this:

from qiskit import qasm3

with open("out.qasm", "w") as fptr:
    qasm3.dump(
        transpiled,
        fptr,
        includes=("backend.inc",),
        basis_gates=("sx", "rz", "ecr"),
    )

where basis_gates must be given because nothing is known about what might be in backend.inc.

Proposal for parsing

In general, I think the API that the new OQ2 parser has here is good and extensible (though obviously I'm biased). I think it's worth separating out a few concerns:

• specifying where to look for headers
• parsing the headers to discover what they define
• defining which Python objects should be used for given OpenQASM definitions

I think the first and third should be specified purely as inputs to the importer, and the parser/converter should not have opinions beyond what's specified in those options. The first imo should default to a path that includes only qelib1.inc (OQ2) as defined in the OQ2 paper or stdgates.inc (OQ3), and a set of header files that include additional Qiskit standard gates. The which-Python-objects field should just map Qiskit standard gates.

qasm2.loads already supports all these options, but would need to learn some additional module data that includes mappings for all non-variadic Qiskit standard gates rather than just the legacy ones the previous exporter used. This could become qasm2.QISKIT_STANDARD_GATES, which would be in the format that custom_instructions expects, with none of the gates defined as built-in.

qasm3.loads would need to gain a custom_instructions field and an include_path field to support this, and whatever importer is in use internally would need to add support for these.

Proposal for exporting

This is the area we currently have no real support for. Let us begin by defining some auxiliary structures:

class IncludeFile:
    relative_path: str
    includes: list[IncludeFile]
    definitions: list[GateDefinition]

class GateDefinition:
    name: str
    num_angles: int
    num_qubits: int

Qiskit will ship with defaults for these for qelib1.inc, stdgates.inc and any Qiskit-defined headers (see next section). We may supply convenience constructors such as IncludeFile.from_file and IncludeFile.from_str that have similar signatures to qasm2.load (and would likely use similar machinery).

Now, I would propose that the interface to the exporter looks something like:

def dump(
    circuit: QuantumCircuit,
    stream: io.TextIO,
    *,
    includes: Iterable[IncludeFile],
    basis_gates: Iterable[GateDefinition],
    gate_mapping: Mapping[str, GateDefinition],
): ...

Of the parameters:

• includes would be produced using the IncludeFile.from_str etc helpers discussed above
• basis_gates would provide additional gates that should be treated as pre-defined, even if there is no true definition. This generates technically invalid OpenQASM, but for practical purposes, it will likely help link with how known OpenQASM-supporting backends function right now.
• gate_mapping would be manual overrides that map Qiskit Operation.name strings to GateDefinition entries that exist in one of the include files or the basis gates, and we'd begin by assuming the trivial mapping Operation.name == GateDefinition.name and error if any gates had incompatible signatures.

I have no particular preference about whether we'd want to allow additional user-convenience types that are automatically parsed into one of these types.

I do not think that the low-level interface specifically needs to be qasm2.dump (etc). We could use an architecture where each module has an Exporter object with dump and dumps methods that is the low-level interface whose constructor accepts only what I laid out above (and the dump method takes only a circuit and a stream), and the module-root dump and dumps are accept more types, including the automatic parsing, and simply construct a one-off Exporter and call a method on it. This has the advantage of having a clear object that manages output for a particular backend and can be re-used without re-parsing header files or needing to carry around a configuration manually.

Proposal for Qiskit header-file definitions

We previously have had tension between wanting to use qelib1.inc, and wanting a header file that includes all of Qiskit's standard gates (#4312). I propose:

• qelib1.inc reverts to the exact content of the paper. stdgates.inc is more formally a standard library, and we should never deviate from the OpenQASM 3 definition.
• Qiskit begins defining files that include all its standard gates. This can be either assuming that stdgates.inc/qelib1.inc will or won't be included in addition (or we can provide files that are suitable for both). Qiskit OQ2/OQ3 export may be set to export against these files.
• Qiskit's defining files are versioned in some form, to support us adding or removing gates from them in the future. We keep all old versions of these files around forever for backwards compatibility.
• we provide something like qasm2.DEFAULT_INCLUDE_PATH as a data attribute that lets users find copies of the header files that we might use.

For header-file versioning, there are two options that jump to mind:

• a simple counter that we just increment whenever we change the file, such as qiskit/v1.qasm, qiskit/v2.qasm, etc.
• the Qiskit minor version, such as qiskit/0_45.qasm, qiskit/1_0.qasm, etc. Since most minors will not change the file, we can either store a symlink to the last time it changed (in order to have all minors have an associated header) or only add a new file when it would change, so Qiskit 1.2 might still export in terms of qiskit/1_0.qasm.

I'd propose we distribute the Qiskit Python package's directory structure as something like

qiskit/
  - circuit/
  - qasm2/
    - include/
      - qelib1.inc
      - qiskit/
        - 0_45.qasm
  - qasm3/
    - include/
      - stdgates.inc
      - qiskit/
        - 0_45.qasm

so DEFAULT_INCLUDE_PATH can be something simple like Path(qiskit.__file__).parent / "qasm2" / "include", etc.

#6125 has attempted something in this direction before, but it didn't include the versioning provisions of this proposal, and without the additional import and export handling proposed in this PR, it would have been missing a fair amount of the usability at the time.

Versioning the Qiskit-defined header files lets us expand ourselves in the future without interfering with the concerns that #4312 has. The OpenQASM 2 and OpenQASM 3 include paths are separated because the syntax isn't entirely compatible (different built-in gates, for one), and we wouldn't want conflicts.

See also

• new qasm library #4312
• standard gates in qiskit in its own inc file #6125
• Allow custom built-in gate specification qiskit-qasm3-import#15

[1ucian0]

class IncludeFile:
    relative_path: str
    includes: list[IncludeFile]

An IncludeFile represents a list of files or a single one? Is the recursive definition intentional?

[jakelishman]

A single one, but an include file might include other files.

[1ucian0]

This proposal makes total sense to me. With respect at the versioning alternatives, I think I prefer only major increments (like qiskit/v1.qasm).

[jakelishman]

From the meeting: Matt also voiced support for the versions of the header files being decoupled from the Qiskit version number, i.e. in support of qiskit/v1.qasm, etc.

[1ucian0]

@kdk can we consider this accepted and give green light to move forward?

[graham-atom]

will this allow for importing custom gates from openqasm3 text as well? I am having trouble with this and the current loads function from qiskit.qasm3 since it does not support custom gates

[jakelishman]

This issue alone won't do that, but as part of solving it, we'd certainly have to put all the pieces in place to make that possible.

Also, I agree, we really should bring the OQ3 importer up-to-speed with the OQ2 importer in that regard. Originally we didn't because the OQ3 importer was meant to be a stop-gap solution that would be replaced by https://github.com/Qiskit/openqasm3_parser and we didn't want to commit to an interface we couldn't support. Then our priorities shifted, and that project got stalled (though we're beginning the process of picking it back up). It'll still be a while before that's fully ready to take over, though, and the general interface of the CustomGate stuff has already proved itself in use through the OQ2 importer.

[graham-atom]

ok! is there another issue I can track that will follow progress on the OQ3 parser parity anywhere?