Pact Behavior in CI/CD Pipeline for Provider Tests

[pozitron-collab]

Hello,

I have a query regarding Pact behavior in our CI/CD pipeline when executing provider tests. In our provider tests, we specify a target base URL, and Pact makes HTTP requests to that base URL. Typically, this is done when the application is hosted somewhere, and we explicitly specify the host URL.

When running the provider application locally, we have the ability to mock external dependencies and database calls, allowing Pact to make HTTP calls to the locally running application.

Now, my question relates to the execution of provider tests within a CI/CD pipeline. I'm interested in understanding how and where Pact makes the HTTP call during this process. Specifically, I'm wondering if there's a way for Pact to validate a contract when the provider application is not running anywhere. Or is it mandatory to start the application in an environment for Pact to successfully verify the contract?

Additionally, our requirement is to perform contract validation before the deploy step. In other words, we want to ensure that the recent changes comply with all consumer expectations before deploying the provider service. Could you provide insights into how this can be achieved within the Pact framework?

Thank you for your assistance.

[YOU54F]

Hey up,

You do need to start the provider application for Pact to be able to issue HTTP requests to it. In CI/CD you would normally start your provider programatically and then run your Pact provider verification tests, tearing down your provider application after the tests have completed.

Here is a Java springboot provider example https://docs.pactflow.io/docs/examples/java/provider-springboot/

[pozitron-collab]

Hey @YOU54F ,
Does this imply that I must initiate the Provider service under all circumstances? In this context, we can achieve this within a Dockerized container in the CI pipeline. Our current setup involves microservices developed in Java and Golang, accompanied by numerous dependencies such as database connections and Kafka. Given that Pact utilizes HTTP requests to validate the contract, the provider service application needs to be operational to handle these HTTP calls.
Is there a method to eliminate dependencies and launch the application (both in Java Spring Boot and Golang)? We are open to mocking external API and database calls for this purpose.

Please note that we aim to utilize Pact as a 'Safe as Provider' approach. This means that whenever a service undergoes a deployment, we intend to retrieve all its consumer contracts from the Pact broker and subsequently replay each contract within its CI pipeline, all of which should occur before the deployment step.

[YOU54F]

Is there a method to eliminate dependencies and launch the application (both in Java Spring Boot and Golang)? We are open to mocking external API and database calls for this purpose.

yes, you mention it below

When running the provider application locally, we have the ability to mock external dependencies and database calls, allowing Pact to make HTTP calls to the locally running application.

It entirely depends on how you build your application, but this is the most preferable way to test your consumer and provider (prior to deployment)

Please note that we aim to utilize Pact as a 'Safe as Provider' approach. This means that whenever a service undergoes a deployment, we intend to retrieve all its consumer contracts from the Pact broker and subsequently replay each contract within its CI pipeline, all of which should occur before the deployment step.

Related to this point, all of the Pact documentation and guides point towards this approach, although the Pact's wouldn't need to be replayed at the time of deployment, as they will have already been pre-verified when the consumer contract is either newly published, or has changed from its original publication.

Providers will utilise webhooks to trigger provider builds against the newly published/changed contract

https://docs.pact.io/pact_broker/webhooks#the-contract-requiring-verification-published-event

which means that at the point of deploying the software to an environment, you are just querying a result and not triggering a test run. Testing is performed pre-deployment of software to an environment

I'd suggest running through the pact nirvana guide or pact workshop to get a feel for the recommended flow.

https://docs.pact.io/pact_nirvana

[pozitron-collab]

Certainly, @YOU54F, I comprehend the recommended workflow of verifying the contract: when it's published to the broker, the provider service receives the contract through a webhook, verifies it, and then returns the result to the Pact broker.

My primary interest lies in testing whether my provider service aligns well with its changes, rather than focusing on testing the consumer service and ensuring their expectations are met.

In essence, when I deploy a service, I aim to identify all contracts where the current service is defined as the Provider. Upon receiving these contracts, the service should verify each one. Essentially, this process indirectly ensures that the changes in the current service are compatible with all its downstream services. Is it feasible to achieve this with Pact?

[YOU54F]

yes when a provider makes changes and runs in ci/cd it would use consumer version selectors to retrieve relevant consumer contracts.

the most recommended is main branch of consumers, plus and deployed or released versions.

https://docs.pact.io/pact_broker/advanced_topics/consumer_version_selectors#recommended

I would note that is not the job of the consumer contract tests to functionally test the provider, and therefore these pact verification tests would be supplementary to functional tests of your provider codebase driven by the provider team.

There is some overlap, but that is a by product and not the aim.

In essence, when I deploy a service, I aim to identify all contracts where the current service is defined as the Provider. Upon receiving these contracts, the service should verify each one. Essentially, this process indirectly ensures that the changes in the current service are compatible with all its downstream services. Is it feasible to achieve this with Pact?

Again these changes would be tested pre-deployment, and the verification result assigned to your versioned code. At the point of deployment into an environment, the matrix is queried with can-i-deploy which will inform you that your dependencies are in sync, or not. You wouldn't need to the run the test suite at the point of deployment.