Evaluation of CompiledCode for testing. #7072

Unisay · 2025-05-06T09:59:52Z

Closes https://github.com/IntersectMBO/plutus-private/issues/1540

(You might find it easier to review commit-by-commit).

Split monolitic PlutusTx.Test module into a few smaller sub-modules such that it re-exports functions from sub-modules for backward-compatibility.:
- PlutusTx.Test.Golden (golden* functions)
- PlutusTx.Text.Run.Uplc (functionality to run PLC)
- PlutusTx.Text.Run.Code (functionality to run CompiledCode, new)
Updated existing tests to evaluate CompiledCode directly, making them a bit simpler.
Removed the plutus-tx-test-util package for several reasons:
- It contains only single module PlutusTx.Test.Util.Compiled, its an overkill to create a dedicated package for it.
- We already have the plutus-tx-testlib package meant to host plutus testing functionality.
Added a section to the Plinth User Guide (docusaurus) with documentation how to run compiled code.
- accompanied by the executable example.

Copilot

Copilot reviewed 13 out of 33 changed files in this pull request and generated no comments.

Files not reviewed (20)

cabal.project: Language not supported
nix/project.nix: Language not supported
plutus-benchmark/common/PlutusBenchmark/Common.hs: Language not supported
plutus-benchmark/plutus-benchmark.cabal: Language not supported
plutus-benchmark/script-contexts/test/V1/Spec.hs: Language not supported
plutus-benchmark/script-contexts/test/V2/Spec.hs: Language not supported
plutus-benchmark/script-contexts/test/V3/Spec.hs: Language not supported
plutus-ledger-api/plutus-ledger-api.cabal: Language not supported
plutus-ledger-api/test-plugin/Spec/Data/MintValue/V3.hs: Language not supported
plutus-ledger-api/test-plugin/Spec/MintValue/V3.hs: Language not supported
plutus-ledger-api/test-plugin/Spec/Value/WithCurrencySymbol.hs: Language not supported
plutus-tx-plugin/plutus-tx-plugin.cabal: Language not supported
plutus-tx-plugin/test/AssocMap/Properties1.hs: Language not supported
plutus-tx-plugin/test/AssocMap/Properties2.hs: Language not supported
plutus-tx-plugin/test/AssocMap/Properties3.hs: Language not supported
plutus-tx-plugin/test/List/Properties1.hs: Language not supported
plutus-tx-plugin/test/List/Properties2.hs: Language not supported
plutus-tx-plugin/test/Plugin/NoTrace/Lib.hs: Language not supported
plutus-tx-plugin/test/Plugin/NoTrace/Spec.hs: Language not supported
plutus-tx-plugin/test/ShortCircuit/Spec.hs: Language not supported

Unisay · 2025-05-06T13:51:41Z

plutus-tx/testlib/PlutusTx/Test/Golden.hs

+{-# LANGUAGE TypeApplications      #-}
+{-# LANGUAGE TypeOperators         #-}
+
+module PlutusTx.Test.Golden (


The code has been moved here from PlutusTx.Test without changes.

zliu41

I'll take a closer look soon but I don't like putting this in plutus-tx-testlib. First, it's a library with utilities for our internal testing, not for the users to test their own code. There may be some overlap but they are not quite the same. Second, it may not be used exclusively for testing - users may depend on it in their own libraries or executables that serve other purposes. I'd just put it in the plutus-tx package, and call it "PlutusTx.Run" or something like that.

zliu41

This PR does multiple things - not just "Evaluation of CompiledCode for testing" but also lots of code reorganization. I find it difficult to review because I wanted to focus on the former, but the latter makes it much harder to do so - for example, "Updated existing tests to evaluate CompiledCode directly" - where is that, exactly?

In the future I'd highly recommend not doing both together.

doc/docusaurus/docs/using-plinth/evaluating-plinth.md

zliu41 · 2025-05-12T00:17:00Z

doc/docusaurus/docs/using-plinth/evaluating-plinth.md

+
+```haskell
+result :: CodeResult
+result = evaluateCompiledCode compiledCode


The cost model and parameters depend on MajorProtocolVersion and ledger language version (V1, V2, V3), so I'd expect evaluateCompiledCode to take them as parameters. It could also take arguments to compiledCode as a parameter.

I see a trade-off here: on one hand I want to expose the most simple API to the users, and the less decisions they have to make - the better it is. On the other - we want it to be flexible: e.g. users can decide which CEK Parameters to use, but it means that CEK Parameters parameter needs to be exposed.
I decided to hardcode the defaultCekParametersForTesting thus limiting the function applicability to testing.
From your comment I read that you'd go different route: ask users to chose the CEK parameters, directly or indirectly. I am thinking maybe we can have two functions for both sides of the trade-off...

As I said in a comment above, I don't think running CompiledCode is only useful for testing. Even when someone uses it for testing, it is still important to use the accurate parameters - maybe they are trying to troubleshoot a budget exceeded error, and you can't reproduce it unless you use the correct parameters!

And one may also want to see the budget of their script, or see whether it exceeds the budget limit. None of these can be done unless using the correct parameters.

the less decisions they have to make

The "decision"s are just the PV and language version that one wants to run their script in - two natural numbers - I don't think it's too much to ask.

I am thinking maybe we can have two functions for both sides of the trade-off...

You can make it take Maybe (PV, LV). When passed Nothing, don't bother showing the budget in the result, because it won't be meaningful.

doc/docusaurus/docs/using-plinth/evaluating-plinth.md

plutus-tx-plugin/test/size/Main.hs

Unisay · 2025-05-12T08:05:24Z

This PR does multiple things - not just "Evaluation of CompiledCode for testing" but also lots of code reorganization.

True.

I find it difficult to review because I wanted to focus on the former, but the latter makes it much harder to do so

I am sorry to hear that it still hard to review: I tried my best splitting changes in smaller meaningful commits for you to review individually.

for example, "Updated existing tests to evaluate CompiledCode directly" - where is that, exactly?

Benchmarks are updated in this commit:
3a01b0c

Plugin and Ledger API are in this one:
e3b9326

In the future I'd highly recommend not doing both together.

What is an alternative approach to making a series of dependent changes that you recommend?

zliu41

What is an alternative approach to making a series of dependent changes that you recommend?

This is discussed in the contributing guide - you can either open the second PR after the first is merged, or open the second PR against the first PR.

This is much better than trying to carefully organize the commits in a single PR, and asking reviewers to review commit-by-commit. The latter has too many problems:

Most people don't do this. Often times they prefer to commit and push whenever convenient - e.g., before the lunch break, when they finish their day, etc.
Going back and altering and earlier commit is non-trivial - it's much easier to add new commits.
When addressing comments, you could make a new commit that significantly alter the changes introduced in a previous commit. Then if a new reviewer comes in and starts reviewing it commit-by-commit, they'll have to first review the earlier commit, only to realize that the later commit rendered it largely obsolete.
When merging into master the commits are squashed so you'd get a big master commit.

zliu41 · 2025-05-12T21:57:21Z

doc/docusaurus/docs/using-plinth/evaluating-plinth.md

+
+```haskell
+result :: CodeResult
+result = evaluateCompiledCode compiledCode


As I said in a comment above, I don't think running CompiledCode is only useful for testing. Even when someone uses it for testing, it is still important to use the accurate parameters - maybe they are trying to troubleshoot a budget exceeded error, and you can't reproduce it unless you use the correct parameters!

And one may also want to see the budget of their script, or see whether it exceeds the budget limit. None of these can be done unless using the correct parameters.

the less decisions they have to make

The "decision"s are just the PV and language version that one wants to run their script in - two natural numbers - I don't think it's too much to ask.

I am thinking maybe we can have two functions for both sides of the trade-off...

You can make it take Maybe (PV, LV). When passed Nothing, don't bother showing the budget in the result, because it won't be meaningful.

doc/docusaurus/docs/using-plinth/evaluating-plinth.md

…iledCode

…tusTx.Test.Util.Compiled` to the plutus-tx-testlib.

…separately.

Unisay self-assigned this May 6, 2025

Unisay force-pushed the yura/run-compiled-code branch from e5242b2 to 51245f0 Compare May 6, 2025 10:00

Unisay temporarily deployed to github-pages May 6, 2025 10:00 — with GitHub Actions Inactive

Unisay added the Don't look here yet label May 6, 2025

Unisay force-pushed the yura/run-compiled-code branch from 51245f0 to 3261543 Compare May 6, 2025 12:32

Unisay temporarily deployed to github-pages May 6, 2025 12:32 — with GitHub Actions Inactive

Unisay temporarily deployed to github-pages May 6, 2025 13:14 — with GitHub Actions Inactive

Unisay requested a review from Copilot May 6, 2025 13:16

Copilot AI reviewed May 6, 2025

View reviewed changes

Unisay force-pushed the yura/run-compiled-code branch from 64d3e90 to 8f85c13 Compare May 6, 2025 13:23

Unisay temporarily deployed to github-pages May 6, 2025 13:23 — with GitHub Actions Inactive

Unisay temporarily deployed to github-pages May 6, 2025 13:35 — with GitHub Actions Inactive

Unisay temporarily deployed to github-pages May 6, 2025 13:42 — with GitHub Actions Inactive

Unisay commented May 6, 2025

View reviewed changes

Unisay temporarily deployed to github-pages May 7, 2025 17:14 — with GitHub Actions Inactive

Unisay force-pushed the yura/run-compiled-code branch from 0900feb to a72fb70 Compare May 9, 2025 08:55

Unisay temporarily deployed to github-pages May 9, 2025 08:55 — with GitHub Actions Inactive

Unisay marked this pull request as ready for review May 9, 2025 08:55

Unisay removed the Don't look here yet label May 9, 2025

Unisay requested a review from a team May 9, 2025 09:06

Unisay temporarily deployed to github-pages May 9, 2025 09:09 — with GitHub Actions Inactive

zliu41 reviewed May 9, 2025

View reviewed changes

zliu41 reviewed May 12, 2025

View reviewed changes

Unisay temporarily deployed to github-pages May 12, 2025 08:34 — with GitHub Actions Inactive

zliu41 reviewed May 12, 2025

View reviewed changes

Unisay force-pushed the yura/run-compiled-code branch from 443d2c5 to 63f14c6 Compare May 13, 2025 10:43

Unisay temporarily deployed to github-pages May 13, 2025 10:43 — with GitHub Actions Inactive

Unisay temporarily deployed to github-pages May 13, 2025 16:57 — with GitHub Actions Inactive

Unisay force-pushed the yura/run-compiled-code branch from fa14879 to a37b1ee Compare May 15, 2025 11:28

Unisay had a problem deploying to github-pages May 15, 2025 11:28 — with GitHub Actions Failure

Unisay force-pushed the yura/run-compiled-code branch from a37b1ee to 70fe442 Compare May 15, 2025 11:43

Unisay temporarily deployed to github-pages May 15, 2025 11:43 — with GitHub Actions Inactive

Unisay added 16 commits May 16, 2025 10:10

chore: extract PlutusTx.Test.Orphans

a879f94

refactor: move fitsUnder to where its used.

1efb5e8

refactor: extract module containing functions to run UPLC (tests)

9067d42

refactor: extract module with golden testing functions

203fc26

Restructure PlutusTx.Test, add module with functions to evaluate Comp…

055f585

…iledCode

remove plutus-tx-test-util package, moving its only inhabitant `Plu…

8d66715

…tusTx.Test.Util.Compiled` to the plutus-tx-testlib.

Benchmarks use assertions directly on CompiledCode

006d4f7

remove unused runTermCek

34cd850

Compare CompiledCode with Haskell directly

630bd76

Don't expose unsafeRunTermCek

d6d6be5

Remove unused haskellValueToTerm function

0943e90

how to evaluate compiled code: user guide doc + example

1983114

Changelog entry

8537f08

Rename CodeResult to EvalResult with its accessor functions.

633e6a0

Improve "evaluating plinth" example: clarify application and lifting …

203975e

…separately.

refactor: extract code to display execution budget

e7bc91d

Unisay force-pushed the yura/run-compiled-code branch from 70fe442 to e7bc91d Compare May 16, 2025 08:25

Unisay temporarily deployed to github-pages May 16, 2025 08:25 — with GitHub Actions Inactive

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Evaluation of CompiledCode for testing. #7072

Evaluation of CompiledCode for testing. #7072

Unisay commented May 6, 2025 •

edited

Loading

Copilot AI left a comment

Unisay May 6, 2025

zliu41 left a comment

zliu41 left a comment

zliu41 May 12, 2025

Unisay May 12, 2025

zliu41 May 12, 2025

Unisay commented May 12, 2025

zliu41 left a comment

zliu41 May 12, 2025

Evaluation of CompiledCode for testing. #7072

Are you sure you want to change the base?

Evaluation of CompiledCode for testing. #7072

Conversation

Unisay commented May 6, 2025 • edited Loading

Copilot AI left a comment

Choose a reason for hiding this comment

Unisay May 6, 2025

Choose a reason for hiding this comment

zliu41 left a comment

Choose a reason for hiding this comment

zliu41 left a comment

Choose a reason for hiding this comment

zliu41 May 12, 2025

Choose a reason for hiding this comment

Unisay May 12, 2025

Choose a reason for hiding this comment

zliu41 May 12, 2025

Choose a reason for hiding this comment

Unisay commented May 12, 2025

zliu41 left a comment

Choose a reason for hiding this comment

zliu41 May 12, 2025

Choose a reason for hiding this comment

Unisay commented May 6, 2025 •

edited

Loading