Verifier cli: consumer version selectors - multiple --consumer-version-selectors arguments are confusing

[lautwe]

We are using the pact_verifier_cli to verify pacts.

The consumer-version-selectors described in pact documentation are showcasing combinations of consumer version selectors.
The code examples for the various languages raised the impression the combination of mainBranch and deployedOrReleased selector in one JSON-String being a valid one.

Trying the combination of mainBranch and releasedOrDeployed in one JSON string as argument:
./pact_verifier_cli-macos-aarch64 --pretty-log --provider-name SomeProvider --hostname localhost --port 8080 --broker-url https://someBrokerUrl --provider-branch someBranch --provider-version someVersion --build-url 'https://someCiUrl --token someToken --consumer-version-selectors '{"mainBranch":true, "deployedOrReleased": true}'

Returns the following error message:

No pacts found for provider 'SomeProvider' matching the given consumer version selectors
in pact broker 'https://someBrokerUrl': 
failed validation - ["consumerVersionSelectors: cannot specify mainBranch=true with any
other criteria apart from consumer (at index 0),
cannot specify the field mainBranch with the field deployedOrReleased (at index 0)"]

mainBranch and deployedOrReleased are not a combination in one JSON-String.

Looking at pact_verifier_cli --help it quotes the following:

--consumer-version-selectors <consumer-version-selectors>
Consumer version selectors to use when fetching pacts from the Broker. 
Accepts a JSON string as per https://docs.pact.io/pact_broker/advanced_topics/consumer_version_selectors/. 
Can be repeated.

The error message and the help hint at passing two arguments to the cli:
./pact_verifier_cli-macos-aarch64 --pretty-log --provider-name SomeProvider --hostname localhost --port 8080 --broker-url https://someBrokerUrl --provider-branch someBranch --provider-version someVersion --build-url 'https://someCiUrl --token someToken --consumer-version-selectors '{"mainBranch":true}' --consumer-version-selectors '{"deployedOrReleased":true}'

This returns the following result:

The pact at https://SomeBroker/pacts/provider/SomeProvider/consumer/SomeConsumer/pact-version/49c9caece4a5db51ef36f286c4496affa42bdbbf is being verified because the pact content belongs to the consumer versions matching the following criteria:
 * consumer version(s) currently deployed to someEnv (1.0.1.6b98091-SNAPSHOT)
 
 The pact at https://SomeBroker/pacts/provider/SomeProvider/consumer/SomeConsumer/pact-version/2d33ed09552d908941fc2ed6e621f76abc9c3c11 is being verified because the pact content belongs to the consumer versions matching the following criteria:
* latest version from the main branch 'someBranch' (some-version)

This one looks good.

Let's look at another use case:
Developing locally/verifying new/changed contracts and CI being triggered we also want to verify against one contract of a consumer only and thus combine consumer and branch consumer version selectors.

With the knowledge above we first used two --consumer-version-selectors arguments:
./pact_verifier_cli-macos-aarch64 --pretty-log  --provider-name SomeProvider --hostname localhost --port 8080 --broker-url https://someBrokerUrl --provider-branch someBranch --provider-version someVersion --build-url 'https://someCiUrl' --token someToken --consumer-version-selectors '{"consumer“:“someConsumer“}' --consumer-version-selectors '{"branch“:“someConsumer“}'

This yields the following error:

2024-06-26T19:41:29.457729Z ERROR pact_verifier: No pacts found for provider ’SomeProvider` matching the given consumer version selectors in pact broker 'https://SomeBrokerURL': failed validation - ["consumerVersionSelectors: must specify a value for environment or tag or branch, or specify mainBranch=true, matchingBranch=true, latest=true, deployed=true, released=true or deployedOrReleased=true (at index 0)"]
    at pact_verifier/src/lib.rs:1321 on main

 2024-06-26T19:41:29.458292Z ERROR pact_verifier: Failed to load pact - No pacts found for provider `SomeProvider` matching the given consumer version selectors in pact broker 'https://SomeBrokerURL: failed validation - ["consumerVersionSelectors: must specify a value for environment or tag or branch, or specify mainBranch=true, matchingBranch=true, latest=true, deployed=true, released=true or deployedOrReleased=true (at index 0)"]
    at pact_verifier/src/lib.rs:1103 on main

Reading the message and rethinking, it makes sense to combine them.

./pact_verifier_cli-macos-aarch64 --pretty-log  --provider-name SomeProvider --hostname localhost --port 8080 --broker-url https://someBrokerUrl --provider-branch someBranch --provider-version someVersion --build-url 'https://someCiUrl' --token someToken --consumer-version-selectors '{"consumer“:“someConsumer“, "branch“:“someConsumer“}'

This yields the following criteria being matched:

The pact at https://SomeBroker/pacts/provider/SomeProvider/consumer/SomeConsumer/pact-version/49c9caece4a5db51ef36f286c4496affa42bdbbf is being verified because the pact content belongs to the consumer versions matching the following criteria:
    * latest version of SomeConsumer from branch 'someBranch' (1.0.1.6b98091-SNAPSHOT)

This actually matches what we expect.

The behaviour regarding combining the version selectors confused me. In one case I can not combine them, in the other I need to combine them.

Is this behaviour regarding the combination as you designed, or should the first case also be an allowed combination?

Tested with:

./pact_verifier_cli-macos-aarch64 --version
pact verifier version   : v1.1.3
pact specification      : v4.0
models version          : v1.2.2

[rholshausen]

Not sure how we can help here (from the pact_verifier_cli point of view). The Pact Broker authors decided to make the consumer version selector interface one where the CLIs and language implementations are dumb, and all the smarts are in the Pact Broker. I argued against this, as I thought it was a poor user experience.

All the pact_verifier_cli can do is validate that the selectors passed on the command line are valid JSON. The schematics about how the selectors can be used and combined are built into the Pact Broker. You should ask your question on that project.

[lautwe]

Ok thank you! :) Will close this.