Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add noisy simulator backend to qsharp #1673

Merged
merged 68 commits into from
Jul 12, 2024
Merged

Conversation

orpuente-MS
Copy link
Contributor

@orpuente-MS orpuente-MS commented Jun 27, 2024

This PR does two things:

  • Adds a noisy simulator backend to the qsharp repo.
  • Exposes a thin PyO3 interface around the simulator.

Copy link

Benchmark for 44b23a6

Click to view benchmark
Test Base PR %
Array append evaluation 328.2±2.98µs 333.4±2.46µs +1.58%
Array literal evaluation 169.1±1.69µs 185.7±0.44µs +9.82%
Array update evaluation 412.9±3.87µs 416.4±2.39µs +0.85%
Core + Standard library compilation 19.7±0.07ms 19.9±0.48ms +1.02%
Deutsch-Jozsa evaluation 5.1±0.06ms 5.2±0.04ms +1.96%
Large file parity evaluation 34.3±0.11ms 34.5±0.22ms +0.58%
Large input file compilation 11.9±0.08ms 12.0±0.08ms +0.84%
Large input file compilation (interpreter) 46.0±0.76ms 46.1±1.21ms +0.22%
Large nested iteration 32.5±0.46ms 32.9±0.75ms +1.23%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1550.4±43.82µs 1559.4±33.52µs +0.58%
Perform Runtime Capabilities Analysis (RCA) on large file sample 7.8±0.11ms 7.9±0.13ms +1.28%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1408.6±34.95µs 1412.7±37.97µs +0.29%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 27.8±0.17ms 28.0±0.18ms +0.72%
Teleport evaluation 90.4±3.81µs 92.2±3.76µs +1.99%

Copy link

Benchmark for f2fad52

Click to view benchmark
Test Base PR %
Array append evaluation 326.6±2.02µs 331.1±4.82µs +1.38%
Array literal evaluation 168.7±0.73µs 186.7±1.35µs +10.67%
Array update evaluation 406.6±1.71µs 410.7±3.22µs +1.01%
Core + Standard library compilation 20.0±0.28ms 20.2±0.43ms +1.00%
Deutsch-Jozsa evaluation 5.2±0.05ms 5.2±0.06ms 0.00%
Large file parity evaluation 34.3±0.10ms 34.4±0.41ms +0.29%
Large input file compilation 12.2±0.13ms 12.2±0.18ms 0.00%
Large input file compilation (interpreter) 48.2±0.84ms 48.1±0.98ms -0.21%
Large nested iteration 31.8±0.35ms 32.1±0.30ms +0.94%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1554.1±36.92µs 1554.4±37.84µs +0.02%
Perform Runtime Capabilities Analysis (RCA) on large file sample 7.8±0.11ms 8.0±0.11ms +2.56%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1415.7±45.37µs 1413.4±42.01µs -0.16%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 27.9±0.21ms 28.0±0.22ms +0.36%
Teleport evaluation 92.3±3.43µs 92.3±4.23µs 0.00%

Copy link

Benchmark for 0880d24

Click to view benchmark
Test Base PR %
Array append evaluation 335.3±5.67µs 327.2±2.35µs -2.42%
Array literal evaluation 186.2±0.69µs 168.8±1.19µs -9.34%
Array update evaluation 415.8±6.35µs 408.3±2.56µs -1.80%
Core + Standard library compilation 21.3±0.47ms 19.9±0.28ms -6.57%
Deutsch-Jozsa evaluation 5.2±0.10ms 5.2±0.06ms 0.00%
Large file parity evaluation 34.4±0.13ms 34.5±0.58ms +0.29%
Large input file compilation 12.6±0.44ms 12.2±0.12ms -3.17%
Large input file compilation (interpreter) 50.4±1.08ms 48.6±1.21ms -3.57%
Large nested iteration 32.4±0.44ms 32.1±0.40ms -0.93%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1543.4±31.16µs 1554.2±56.53µs +0.70%
Perform Runtime Capabilities Analysis (RCA) on large file sample 7.7±0.09ms 7.7±0.14ms 0.00%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1403.8±28.34µs 1417.8±73.08µs +1.00%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 28.0±1.35ms 27.8±0.17ms -0.71%
Teleport evaluation 92.6±3.66µs 92.9±4.49µs +0.32%

Copy link

Benchmark for 2f8dc3f

Click to view benchmark
Test Base PR %
Array append evaluation 326.9±1.90µs 326.5±3.13µs -0.12%
Array literal evaluation 169.1±1.16µs 176.8±12.65µs +4.55%
Array update evaluation 409.8±7.54µs 411.0±19.25µs +0.29%
Core + Standard library compilation 22.5±0.90ms 22.0±0.81ms -2.22%
Deutsch-Jozsa evaluation 5.1±0.06ms 5.2±0.06ms +1.96%
Large file parity evaluation 34.5±0.09ms 34.6±0.47ms +0.29%
Large input file compilation 13.5±0.81ms 13.1±0.38ms -2.96%
Large input file compilation (interpreter) 52.4±1.57ms 52.4±3.03ms 0.00%
Large nested iteration 32.0±0.65ms 31.9±0.73ms -0.31%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1602.5±143.40µs 1616.4±135.88µs +0.87%
Perform Runtime Capabilities Analysis (RCA) on large file sample 8.1±0.14ms 8.2±0.17ms +1.23%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1472.3±157.58µs 1470.1±102.65µs -0.15%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 28.8±0.39ms 28.6±0.39ms -0.69%
Teleport evaluation 91.5±3.84µs 90.7±3.91µs -0.87%

Copy link

Benchmark for 1f4c98d

Click to view benchmark
Test Base PR %
Array append evaluation 331.5±21.40µs 329.4±2.11µs -0.63%
Array literal evaluation 169.8±1.02µs 168.7±1.15µs -0.65%
Array update evaluation 409.5±3.31µs 408.2±1.66µs -0.32%
Core + Standard library compilation 19.9±0.90ms 19.9±0.88ms 0.00%
Deutsch-Jozsa evaluation 5.2±0.06ms 5.2±0.09ms 0.00%
Large file parity evaluation 34.3±0.08ms 34.4±0.73ms +0.29%
Large input file compilation 12.2±0.30ms 12.3±0.50ms +0.82%
Large input file compilation (interpreter) 48.0±4.32ms 47.4±1.09ms -1.25%
Large nested iteration 32.0±0.25ms 31.9±0.09ms -0.31%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1546.7±32.50µs 1543.3±30.57µs -0.22%
Perform Runtime Capabilities Analysis (RCA) on large file sample 7.9±0.12ms 7.8±0.09ms -1.27%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1420.9±80.28µs 1402.3±28.10µs -1.31%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 27.8±0.29ms 27.9±0.75ms +0.36%
Teleport evaluation 90.1±3.37µs 90.6±3.44µs +0.55%

Copy link

Benchmark for 7bee5af

Click to view benchmark
Test Base PR %
Array append evaluation 329.3±3.56µs 326.1±1.06µs -0.97%
Array literal evaluation 186.4±1.68µs 169.0±1.46µs -9.33%
Array update evaluation 409.9±3.12µs 407.2±3.71µs -0.66%
Core + Standard library compilation 19.7±0.12ms 19.6±0.16ms -0.51%
Deutsch-Jozsa evaluation 5.1±0.07ms 5.2±0.06ms +1.96%
Large file parity evaluation 34.4±0.14ms 34.6±0.17ms +0.58%
Large input file compilation 11.9±0.09ms 12.0±0.11ms +0.84%
Large input file compilation (interpreter) 46.8±0.72ms 47.3±0.82ms +1.07%
Large nested iteration 32.4±0.36ms 31.8±0.14ms -1.85%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1540.8±29.99µs 1543.8±31.62µs +0.19%
Perform Runtime Capabilities Analysis (RCA) on large file sample 7.7±0.07ms 7.7±0.06ms 0.00%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1400.2±30.03µs 1406.2±27.74µs +0.43%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 27.8±0.25ms 27.9±0.30ms +0.36%
Teleport evaluation 90.0±3.30µs 91.8±9.16µs +2.00%

@orpuente-MS orpuente-MS marked this pull request as ready for review July 2, 2024 18:29
Copy link

Benchmark for 0bfbffc

Click to view benchmark
Test Base PR %
Array append evaluation 334.3±47.70µs 339.0±56.85µs +1.41%
Array literal evaluation 169.8±1.26µs 168.9±1.15µs -0.53%
Array update evaluation 407.7±2.90µs 408.6±2.42µs +0.22%
Core + Standard library compilation 21.5±0.68ms 21.9±0.89ms +1.86%
Deutsch-Jozsa evaluation 4.8±0.04ms 4.8±0.05ms 0.00%
Large file parity evaluation 34.1±0.19ms 34.2±0.34ms +0.29%
Large input file compilation 13.5±0.46ms 12.9±0.52ms -4.44%
Large input file compilation (interpreter) 52.2±1.40ms 50.5±2.18ms -3.26%
Large nested iteration 32.2±0.50ms 31.9±0.27ms -0.93%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1565.8±53.01µs 1588.1±106.22µs +1.42%
Perform Runtime Capabilities Analysis (RCA) on large file sample 7.9±0.14ms 7.8±0.16ms -1.27%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1422.9±34.92µs 1430.6±31.61µs +0.54%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 27.9±0.21ms 28.2±0.53ms +1.08%
Teleport evaluation 91.2±3.16µs 90.4±3.51µs -0.88%

Copy link

Benchmark for d1e1e27

Click to view benchmark
Test Base PR %
Array append evaluation 333.4±2.23µs 330.2±1.20µs -0.96%
Array literal evaluation 186.2±1.01µs 169.2±0.94µs -9.13%
Array update evaluation 410.0±1.19µs 408.1±1.20µs -0.46%
Core + Standard library compilation 21.4±1.08ms 21.5±0.37ms +0.47%
Deutsch-Jozsa evaluation 4.8±0.05ms 4.8±0.06ms 0.00%
Large file parity evaluation 34.1±0.16ms 34.2±0.49ms +0.29%
Large input file compilation 12.8±0.37ms 12.3±0.11ms -3.91%
Large input file compilation (interpreter) 48.9±2.15ms 49.3±1.17ms +0.82%
Large nested iteration 32.1±0.21ms 32.1±0.45ms 0.00%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1558.0±37.34µs 1581.2±82.98µs +1.49%
Perform Runtime Capabilities Analysis (RCA) on large file sample 7.8±0.06ms 7.8±0.29ms 0.00%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1417.6±29.62µs 1434.4±35.86µs +1.19%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 28.0±0.35ms 28.2±0.26ms +0.71%
Teleport evaluation 91.0±3.36µs 90.3±4.42µs -0.77%

Copy link

Benchmark for 36dc380

Click to view benchmark
Test Base PR %
Array append evaluation 331.6±4.87µs 330.8±3.34µs -0.24%
Array literal evaluation 168.5±1.20µs 178.0±3.84µs +5.64%
Array update evaluation 408.1±4.80µs 412.1±3.46µs +0.98%
Core + Standard library compilation 22.8±0.81ms 24.2±0.93ms +6.14%
Deutsch-Jozsa evaluation 4.8±0.04ms 4.8±0.04ms 0.00%
Large file parity evaluation 34.1±0.16ms 34.3±0.84ms +0.59%
Large input file compilation 14.0±0.74ms 13.8±0.51ms -1.43%
Large input file compilation (interpreter) 54.0±1.24ms 53.5±1.25ms -0.93%
Large nested iteration 32.0±0.76ms 31.9±0.61ms -0.31%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1593.4±137.74µs 1598.7±117.04µs +0.33%
Perform Runtime Capabilities Analysis (RCA) on large file sample 8.3±0.14ms 8.1±0.31ms -2.41%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1462.0±147.44µs 1469.0±152.32µs +0.48%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 28.6±0.21ms 28.7±1.13ms +0.35%
Teleport evaluation 90.2±3.35µs 90.7±7.25µs +0.55%

Copy link

Benchmark for 99aac15

Click to view benchmark
Test Base PR %
Array append evaluation 330.9±6.03µs 335.4±2.56µs +1.36%
Array literal evaluation 168.4±1.24µs 187.0±0.93µs +11.05%
Array update evaluation 409.6±5.99µs 413.9±2.92µs +1.05%
Core + Standard library compilation 22.0±0.46ms 21.9±0.46ms -0.45%
Deutsch-Jozsa evaluation 4.8±0.05ms 4.8±0.05ms 0.00%
Large file parity evaluation 34.1±0.16ms 34.1±0.23ms 0.00%
Large input file compilation 12.6±0.19ms 12.8±0.43ms +1.59%
Large input file compilation (interpreter) 50.4±1.35ms 51.0±1.47ms +1.19%
Large nested iteration 32.0±0.52ms 32.5±0.46ms +1.56%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1567.6±60.44µs 1567.1±87.83µs -0.03%
Perform Runtime Capabilities Analysis (RCA) on large file sample 7.9±0.27ms 7.9±0.11ms 0.00%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1430.9±51.20µs 1421.7±57.10µs -0.64%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 28.0±0.17ms 28.2±0.20ms +0.71%
Teleport evaluation 90.6±3.13µs 90.1±2.99µs -0.55%

Copy link

Benchmark for 6aa7f29

Click to view benchmark
Test Base PR %
Array append evaluation 333.5±3.73µs 330.0±2.75µs -1.05%
Array literal evaluation 186.8±3.48µs 169.2±1.42µs -9.42%
Array update evaluation 412.0±1.37µs 408.2±1.97µs -0.92%
Core + Standard library compilation 23.3±1.69ms 23.3±1.28ms 0.00%
Deutsch-Jozsa evaluation 4.8±0.06ms 4.8±0.06ms 0.00%
Large file parity evaluation 34.2±0.18ms 34.6±0.62ms +1.17%
Large input file compilation 13.5±0.79ms 14.6±1.30ms +8.15%
Large input file compilation (interpreter) 53.7±4.12ms 52.3±2.90ms -2.61%
Large nested iteration 32.5±0.30ms 32.7±2.03ms +0.62%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1645.0±194.94µs 1633.0±174.16µs -0.73%
Perform Runtime Capabilities Analysis (RCA) on large file sample 8.5±0.55ms 8.4±0.34ms -1.18%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1491.0±134.23µs 1491.3±236.06µs +0.02%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 29.1±0.68ms 30.0±1.31ms +3.09%
Teleport evaluation 90.9±3.30µs 90.7±3.41µs -0.22%

Copy link

Benchmark for 89e0383

Click to view benchmark
Test Base PR %
Array append evaluation 336.9±9.10µs 348.2±9.82µs +3.35%
Array literal evaluation 188.8±0.70µs 169.6±1.99µs -10.17%
Array update evaluation 416.6±7.43µs 425.6±9.68µs +2.16%
Core + Standard library compilation 21.3±0.35ms 21.3±0.30ms 0.00%
Deutsch-Jozsa evaluation 4.8±0.04ms 4.8±0.19ms 0.00%
Large file parity evaluation 34.2±0.71ms 34.1±0.18ms -0.29%
Large input file compilation 12.3±0.58ms 12.4±0.27ms +0.81%
Large input file compilation (interpreter) 48.2±1.10ms 51.3±2.03ms +6.43%
Large nested iteration 32.4±0.17ms 33.5±0.66ms +3.40%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1563.8±55.42µs 1562.3±36.92µs -0.10%
Perform Runtime Capabilities Analysis (RCA) on large file sample 7.9±0.13ms 7.8±0.07ms -1.27%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1419.5±34.60µs 1424.8±32.80µs +0.37%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 27.8±0.19ms 28.1±1.01ms +1.08%
Teleport evaluation 90.9±3.20µs 90.6±3.12µs -0.33%

@billti
Copy link
Member

billti commented Jul 12, 2024

I just noticed the top-level folder name has a dash in (noisy-simulator), whereas our others (language_server, resource_estimator) use underscores. Maybe change this for consistency also @orpuente-MS .

@orpuente-MS
Copy link
Contributor Author

I just noticed the top-level folder name has a dash in (noisy-simulator), whereas our others (language_server, resource_estimator) use underscores. Maybe change this for consistency also @orpuente-MS .

Thanks, I changed the folder name and also the name of the crate in the Cargo.toml to use underscores.

Copy link

Benchmark for 4054eff

Click to view benchmark
Test Base PR %
Array append evaluation 331.7±2.01µs 337.4±8.11µs +1.72%
Array literal evaluation 169.1±1.39µs 189.7±0.94µs +12.18%
Array update evaluation 409.9±2.37µs 417.3±7.70µs +1.81%
Core + Standard library compilation 22.8±0.53ms 23.6±0.70ms +3.51%
Deutsch-Jozsa evaluation 4.8±0.05ms 4.8±0.05ms 0.00%
Large file parity evaluation 34.2±0.20ms 34.4±0.46ms +0.58%
Large input file compilation 13.2±0.31ms 13.0±0.40ms -1.52%
Large input file compilation (interpreter) 52.2±1.16ms 52.1±1.13ms -0.19%
Large nested iteration 32.1±0.29ms 32.5±0.77ms +1.25%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1600.3±101.59µs 1587.9±92.18µs -0.77%
Perform Runtime Capabilities Analysis (RCA) on large file sample 8.1±0.15ms 8.2±0.19ms +1.23%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1462.2±143.55µs 1449.6±87.18µs -0.86%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 28.6±0.27ms 28.4±0.44ms -0.70%
Teleport evaluation 90.3±3.18µs 91.2±4.03µs +1.00%

@orpuente-MS orpuente-MS enabled auto-merge July 12, 2024 19:42
Copy link

Benchmark for 915dc1c

Click to view benchmark
Test Base PR %
Array append evaluation 328.9±1.92µs 329.7±2.90µs +0.24%
Array literal evaluation 168.8±1.89µs 168.4±1.23µs -0.24%
Array update evaluation 407.9±11.33µs 406.6±1.81µs -0.32%
Core + Standard library compilation 22.9±0.75ms 21.2±0.41ms -7.42%
Deutsch-Jozsa evaluation 4.8±0.05ms 4.8±0.06ms 0.00%
Large file parity evaluation 34.2±0.19ms 34.3±0.74ms +0.29%
Large input file compilation 13.9±0.35ms 13.0±0.46ms -6.47%
Large input file compilation (interpreter) 52.4±1.15ms 48.7±1.23ms -7.06%
Large nested iteration 31.9±0.48ms 31.8±0.21ms -0.31%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1606.8±128.52µs 1562.0±72.42µs -2.79%
Perform Runtime Capabilities Analysis (RCA) on large file sample 8.2±0.15ms 7.8±0.08ms -4.88%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1483.3±171.36µs 1416.4±38.17µs -4.51%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 29.0±0.20ms 28.1±0.33ms -3.10%
Teleport evaluation 90.3±3.14µs 92.4±3.08µs +2.33%

@orpuente-MS orpuente-MS added this pull request to the merge queue Jul 12, 2024
Merged via the queue into main with commit 312af69 Jul 12, 2024
19 checks passed
@orpuente-MS orpuente-MS deleted the oscarpuente/noisy-simulator branch July 12, 2024 20:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants