Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add package benchmarks #97

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

Omar-Elrefaei
Copy link
Contributor

@Omar-Elrefaei Omar-Elrefaei commented Feb 2, 2025

This is a draft for benchmarks as discussed here #64.

So far I covered

  1. time to import, latency
  2. simple object instantiations
  3. arithmetic operations
  4. linear algebra
  5. symbolic manipulation (expand, simplify)
  6. expressing in optics/clifford backend

Please check if I missed any major features.

If there is any real usage notebook you'd like me to extract usages from I'd be happy to.
Also if you can hint me at specific directions of what a "hot loop" would look like, I could stress test that use case. I'm just not sure, for example, whether multiplication of hundreds of kets or ops is relevant or too contrived.

@Omar-Elrefaei Omar-Elrefaei changed the title add package benchmarks (#64) add package benchmarks Feb 2, 2025
@Omar-Elrefaei
Copy link
Contributor Author

Ping because this is a draft @Krastanov

Copy link
Member

@Krastanov Krastanov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! Thank you for spending time on it!

The only other thing that would be valuable to include is something that tests issues related to the fact that we are not using abstract data types (like Unityper, SumTypes, Expronicon, or Moshi). Because of the lack of this feature, manipulating large trees is actually somewhat expensive and it would be valuable to keep track of that expense.

So a holistic benchmark for that can look something like:

function make_large_tree_with_plenty_of_reallocations(layers)
    expr_op = identity
    expr_ket = X1
    for _ in 1:layers
        expr_op = rand([X,Y,Z,H,...])+rand([...])*expr_op
        expr_ket = expr_op*expr_ket + rand([X1,X2,Z1,Z2,...])
    end
    return expr_op, expr_ket
end


SUITE["large_tree"]["construct"]["10_layers"] = make_large_tree_with_plenty_of_reallocations(10)

We are not including tensor products here or superoperators, but that is probably fine until we invest time in actually using abstract data types.

benchmark/benchmarks.jl Show resolved Hide resolved
benchmark/benchmarks.jl Outdated Show resolved Hide resolved
@Omar-Elrefaei Omar-Elrefaei marked this pull request as ready for review February 5, 2025 14:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants