Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix display of the runnable scenario for protected values #251

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

n-osborne
Copy link
Collaborator

Fixes #248

As suggested in the issue, the PR add a variant with three constructors to Ortac_runtime_qcheck_stm making it possible to pass the information whether the result is a value, a protected value or an exception from the generated tests to the runtime.

We don't really have tests for failing tests, which explains why we didn't spot the bug earlier. I'll open an issue for that, to keep this PR small and focused on the fix.

@n-osborne
Copy link
Collaborator Author

We now have this behaviour in case of a test failure (here adding let get a i = get a (i+1) to plugins/qcheck-stm/test/array.ml):

λ dune build @launchtests
File "plugins/qcheck-stm/test/dune.inc", lines 49-52, characters 0-75:
49 | (rule
50 |  (alias launchtests)
51 |  (action
52 |   (run %{dep:array_stm_tests.exe} -v)))
random seed: 276043608
generated error fail pass / total     time test name
[✗]    2    0    1    1 / 1000     0.0s Array STM tests (generating)

--- Failure --------------------------------------------------------------------

Test Array STM tests failed (15 shrink steps):

   protect (fun () -> set <sut> 0 '\186')
   protect (fun () -> get <sut> 0)


+++ Messages ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Messages for test Array STM tests:

Gospel specification violation in function get

  File "array.mli", line 12, characters 12-37:
    a = List.nth t.contents i
  
when executing the following sequence of operations:

  open Array
  let protect f = try Ok (f ()) with e -> Error e
  let sut0 = make 16 'a'
  let _ = protect (fun () -> set sut0 0 '\186')
  (* returned Ok (()) *)
  let r = protect (fun () -> get sut0 0)
  assert (r = Ok '\186')
  (* returned Ok ('a') *)
  

================================================================================
failure (1 tests failed, 0 tests errored, ran 1 tests)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[qcheck-stm] Incorrect runnable scenario on normal behaviour when function could have raised an exception
1 participant