rust testcontainer framework #41

kpwebb · 2025-02-10T17:35:41Z

Here's my initial shot at a testcontainers implementation for the Rust SDK.

I've got the core functionality working, but wanted to share while in progress to see if adding a new "restate-test-utils" crate to the SDK is ok. Also wanted to solicit feedback on ways to extend the test implementation (e.g. mirroring Temporal's approach), per discussions on Discord.

Here's the current test implementation for a service as defined in tests/test_container.rs

#[tokio::test]
async fn test_container_image() {

    let test_container = TestContainer::new().await.unwrap();

    // uses higher port number to avoid collisions
    // with non-test instances running locally
    let host_port:u16 = 19080;
    let host_address = format!("0.0.0.0:{}", host_port);

    println!("starting host");
    // boot restate server
    tokio::spawn(async move {
        HttpServer::new(
            Endpoint::builder()
                .bind(MyServiceImpl.serve())
                .build(),
        ).listen_and_serve(host_address.parse().unwrap()).await;
    });

    use restate_sdk::service::Discoverable;
    let my_service:Service = ServeMyService::<MyServiceImpl>::discover();

    let registered = test_container.register(host_port).await;
    assert!(registered.is_ok());

    TestContainer::delay(1000).await;

    let response = test_container.invoke(my_service, "my_handler".to_string()).await;

}

slinkydeveloper · 2025-02-10T17:56:57Z

This is awesome, i'll give a look in the next days!

slinkydeveloper

just left few initial comments, thanks for the contribution!

restate-test-utils/src/test_utils.rs

restate-test-utils/Cargo.toml

restate-test-utils/src/test_utils.rs

slinkydeveloper · 2025-02-10T18:03:55Z

restate-test-utils/src/test_utils.rs

+}
+
+
+pub struct TestContainer {


It would be nice if the test infra bootstraps the service endpoint server too, and then on Drop stops the server.

slinkydeveloper · 2025-02-10T18:05:08Z

restate-test-utils/src/test_utils.rs

+        tokio::time::sleep(Duration::from_millis(milliseconds)).await;
+    }
+
+    pub async fn invoke(&self, service:Service, handler:String) -> Result<(), HandlerError> {


Rather than this, i would start the user services http server and register them when bootstrapping the test infra, then just provide a url here, and let people use the client they want to send test requests

kpwebb · 2025-02-11T16:01:36Z

@slinkydeveloper thanks for the feedback! Just pushed an update that captures the above. Here's the new test implementation:

#[tokio::test]
async fn test_container_image() {

    let mut test_container = TestContainer::new("docker.io/restatedev/restate", "latest").await.unwrap();

    let endpoint = Endpoint::builder()
                    .bind(MyServiceImpl.serve())
                    .build();

    test_container.serve_endpoint(endpoint).await;

    // optionally insert a delays via tokio sleep
    TestContainer::delay(1000).await;

    // optionally call invoke on service handlers
    use restate_sdk::service::Discoverable;
    let my_service:Service = ServeMyService::<MyServiceImpl>::discover();
    let invoke_response = test_container.invoke(my_service, "my_handler").await;

    assert!(invoke_response.is_ok());

    println!("invoke response:");
    println!("{}", invoke_response.unwrap().text().await.unwrap());

}

slinkydeveloper

Hi! Thanks again for the contribution, I had some time to take a look, it's getting there, I think the API needs another pass

slinkydeveloper · 2025-02-18T10:20:10Z

tests/test_container.rs

@@ -0,0 +1,58 @@
+use restate_test_utils::test_utils::TestContainer;


Could you move this test in restate-test-utils? This way you don't need a dependency on restate-test-utils in the root Cargo.toml. I'm afraid this might cause some issues

slinkydeveloper · 2025-02-18T10:21:21Z

restate-test-utils/src/test_utils.rs

+            .await?
+            .json::<VersionResponse>()
+            .await?;
+


Maybe add a log line here (with tracing::info) saying restate container started with the forwarded ports

slinkydeveloper · 2025-02-18T10:21:44Z

restate-test-utils/src/test_utils.rs

+      // todo cleanup on drop?
+      // testcontainers-rs already implements stop/rm on drop]
+      // https://docs.rs/testcontainers/latest/testcontainers/
+      //


you probably need to stop the hyper server here on drop

slinkydeveloper · 2025-02-18T10:22:57Z

restate-test-utils/src/test_utils.rs

+        let mut container_stdout = container.stdout(true);
+        // Spawn a task to copy data from the AsyncBufRead to stdout
+        let stdout_logging = task::spawn(async move {
+            let mut stdout = io::stdout();
+            if let Err(e) = io::copy(&mut container_stdout, &mut stdout).await {
+                eprintln!("Error copying data: {}", e);
+            }
+        });
+
+        let mut container_stderr = container.stderr(true);
+        // Spawn a task to copy data from the AsyncBufRead to stderr
+        let stderr_logging = task::spawn(async move {
+            let mut stderr = io::stderr();
+            if let Err(e) = io::copy(&mut container_stderr, &mut stderr).await {
+                eprintln!("Error copying data: {}", e);
+            }
+        });


IMO this should be printed using the tracing::debug macro. People can then configure a test tracing_subscriber using https://docs.rs/tracing-test/latest/tracing_test/

I took a shot at doing this via tracing::debug and thought it was way too verbose. As an alternative I added a .with_container_logging() option to the builder, but happy to drop back to tracing::debug if preferred.

slinkydeveloper · 2025-02-18T10:28:28Z

tests/test_container.rs

+async fn test_container_image() {
+
+    let mut test_container = TestContainer::new("docker.io/restatedev/restate", "latest").await.unwrap();
+
+    let endpoint = Endpoint::builder()
+                    .bind(MyServiceImpl.serve())
+                    .build();
+
+    test_container.serve_endpoint(endpoint).await;
+
+    // optionally insert a delays via tokio sleep
+    TestContainer::delay(1000).await;
+
+    // optionally call invoke on service handlers
+    use restate_sdk::service::Discoverable;
+    let my_service:Service = ServeMyService::<MyServiceImpl>::discover();
+    let invoke_response = test_container.invoke(my_service, "my_handler").await;
+
+    assert!(invoke_response.is_ok());
+
+    println!("invoke response:");
+    println!("{}", invoke_response.unwrap().text().await.unwrap());
+
+}


I think we're getting there with the API, maybe let me describe better what I had in mind that aligns with the other SDKs:

For 90% of users, there should be a single API like TestEnvironment::start(Endpoint) that simply takes the endpoint, prepares the environment, registers the services etc. This API returns a TestEnvironment with a method uri() that gives me the base address to send requests to restate. Dropping TestEnvironment stops the Endpoint too. People can then decide to send requests with whatever client they want, be it reqwest, or Implement ingress client #42. For example, something like that:

#[tokio::test] async fn test_container_image() { let endpoint = Endpoint::builder() .bind(MyServiceImpl.serve()) .build(); let mut test_env = TestEnvironment::start(endpoint).await.unwrap(); let ingress_address: String = test_env.ingress_uri(); // Do whatever requests you want with reqwest, or some http test client crate like https://docs.rs/asserhttp/latest/asserhttp/#reqwest // On Drop, TestEnvironment stops both container and Endpoint }

For the remaining 10% of the people, there should be a TestEnvironment::builder().start() that let's you tune few things (such as the image name, which by default should just be `"docker.io/restatedev/restate:latest".

slinkydeveloper · 2025-02-18T10:32:34Z

restate-test-utils/src/test_utils.rs

+        let host_port:u16 = 19080;
+        let host_address = format!("0.0.0.0:{}", host_port);


You need to start the TcpListener on port 0, this way you get a random port assigned. Look at the implementation of HttpServer::listen_and_serve/HttpServer::serve_with_cancel

slinkydeveloper · 2025-02-18T10:34:01Z

restate-test-utils/src/test_utils.rs

+impl TestContainer {
+
+    //"docker.io/restatedev/restate", "latest"
+    pub async fn new(image:&str, version:&str) -> Result<Self, HandlerError> {


You should use anyhow::Error here and everywhere else in this file, HandlerError is an error type reserved to be used only inside restate service handlers.

slinkydeveloper · 2025-02-18T10:38:35Z

restate-test-utils/Cargo.toml

+futures = "0.3.31"
+http = "1.2.0"
+nu-ansi-term = "0.50.1"
+reqwest = {version= "0.12.12", features = ["json"]}
+restate-sdk = { version = "0.3.2", path = ".." }
+restate-sdk-shared-core = "0.2.0"
+serde = "1.0.217"
+serde_json = "1.0.138"
+testcontainers = "0.23.1"
+tokio = "1.43.0"


Could you also re-check which dependencies you need here? I think some of those are not needed

slinkydeveloper · 2025-02-18T10:38:44Z

restate-test-utils/Cargo.toml

+edition = "2021"
+description = "Test Utilities for Restate SDK for Rust"
+license = "MIT"
+repository = "https://github.com/restatedev/sdk-rust/test-utils"


Just https://github.com/restatedev/sdk-rust is fine

slinkydeveloper · 2025-02-18T10:39:16Z

restate-test-utils/Cargo.toml

@@ -0,0 +1,21 @@
+[package]
+name = "restate-test-utils"


Maybe rename the package to restate-test-env and the containing directory to test-env

slinkydeveloper

This is awesome. Run just check locally to fix linting/formatting issues and then we should be good to go!

slinkydeveloper · 2025-02-19T07:35:34Z

test-env/src/lib.rs

+    max_admin_api_version:u32
+}
+
+pub struct TestContainerBuilder {


implement Default too calling new

slinkydeveloper · 2025-02-19T07:37:25Z

test-env/src/lib.rs

+        while let Err(_) = client.get(format!("{}/discover", self.endpoint_server_url.as_ref().unwrap()))
+                                .header("accept", "application/vnd.restate.endpointmanifest.v1+json")
+                                .send().await {
+            tokio::time::sleep(Duration::from_secs(1)).await;
+
+            warn!("retrying endpoint server");
+        }


Put some retry limit here? like after 10 times stop retrying

slinkydeveloper · 2025-02-19T07:37:35Z

test-env/src/lib.rs

+        client.get(format!("{}/discover", self.endpoint_server_url.as_ref().unwrap()))
+            .header("accept", "application/vnd.restate.endpointmanifest.v1+json")
+            .send().await?;


Why this one too?

slinkydeveloper · 2025-02-19T07:37:49Z

test-env/src/lib.rs

+        while let Err(_) = client.get(format!("{}/discover", self.endpoint_server_url.as_ref().unwrap()))
+                                .header("accept", "application/vnd.restate.endpointmanifest.v1+json")
+                                .send().await {
+            tokio::time::sleep(Duration::from_secs(1)).await;


I think you can lower this down to 100 msecs, and retry for at most 5 seconds.

slinkydeveloper · 2025-02-19T07:39:46Z

test-env/tests/test_container.rs

+    let ingress_url = test_container.ingress_url();
+
+    // call container ingress url for /MyService/my_handler
+    let resposne = reqwest::Client::new().post(format!("{}/MyService/my_handler", ingress_url))


kpwebb · 2025-02-19T13:59:13Z

thanks again for the recs and sorry for not running lint/fmt! should be good to go now

slinkydeveloper

One minor nit, and I merge

slinkydeveloper · 2025-02-19T16:23:42Z

test-env/Cargo.toml

@@ -0,0 +1,21 @@
+[package]
+name = "test-env"


this name is the crate name on crates.io, so it needs to be restate-sdk-test-env (unlike the directory name, which test-env is just fine)

slinkydeveloper · 2025-02-19T16:24:02Z

README.md

+### Testing
+
+The SDK uses [Testcontainers](https://rust.testcontainers.org/) to support integration testing using a Docker-deployed restate server.
+The `test-env` crate provides a framework for initializing the test environment, and an integration test example in `test-env/tests/test_container.rs`.


Suggested change

The `test-env` crate provides a framework for initializing the test environment, and an integration test example in `test-env/tests/test_container.rs`.

The `restate-sdk-test-env` crate provides a framework for initializing the test environment, and an integration test example in `test-env/tests/test_container.rs`.

README.md

initial testcontinaters framework

0521ff2

jackkleeman requested a review from slinkydeveloper February 10, 2025 17:44

slinkydeveloper reviewed Feb 10, 2025

View reviewed changes

updated test framework

b46ad68

kpwebb added 2 commits February 11, 2025 11:04

fix text example output

9451953

fix strings

cffc3fa

slinkydeveloper reviewed Feb 18, 2025

View reviewed changes

update testcontainer api + move to test-env crate

475ae7d

slinkydeveloper reviewed Feb 19, 2025

View reviewed changes

cleanup / lint + fmt fixes

44a4255

kpwebb added 5 commits February 19, 2025 09:03

update comment in test example

4c46cdc

fix comment

bf7f48d

more comment fixes

24129fa

add testing to README

88ef9a4

readme formatting

af3613a

slinkydeveloper approved these changes Feb 19, 2025

View reviewed changes

fix crate name

a4856c7

slinkydeveloper merged commit a656187 into restatedev:main Feb 20, 2025
2 checks passed

slinkydeveloper mentioned this pull request Feb 20, 2025

Mock context for unit testing #44

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

rust testcontainer framework #41

rust testcontainer framework #41

kpwebb commented Feb 10, 2025 •

edited

Loading

slinkydeveloper commented Feb 10, 2025

slinkydeveloper left a comment

slinkydeveloper Feb 10, 2025

slinkydeveloper Feb 10, 2025

kpwebb commented Feb 11, 2025 •

edited

Loading

slinkydeveloper left a comment

slinkydeveloper Feb 18, 2025

slinkydeveloper Feb 18, 2025

slinkydeveloper Feb 18, 2025

slinkydeveloper Feb 18, 2025

kpwebb Feb 19, 2025

slinkydeveloper Feb 18, 2025

slinkydeveloper Feb 18, 2025

slinkydeveloper Feb 18, 2025

slinkydeveloper Feb 18, 2025

slinkydeveloper Feb 18, 2025

slinkydeveloper Feb 18, 2025

slinkydeveloper left a comment

slinkydeveloper Feb 19, 2025

slinkydeveloper Feb 19, 2025

slinkydeveloper Feb 19, 2025

slinkydeveloper Feb 19, 2025

slinkydeveloper Feb 19, 2025

kpwebb commented Feb 19, 2025

slinkydeveloper left a comment

slinkydeveloper Feb 19, 2025

slinkydeveloper Feb 19, 2025

		@@ -0,0 +1,58 @@
		use restate_test_utils::test_utils::TestContainer;

		let host_port:u16 = 19080;
		let host_address = format!("0.0.0.0:{}", host_port);

	The `test-env` crate provides a framework for initializing the test environment, and an integration test example in `test-env/tests/test_container.rs`.
	The `restate-sdk-test-env` crate provides a framework for initializing the test environment, and an integration test example in `test-env/tests/test_container.rs`.

rust testcontainer framework #41

rust testcontainer framework #41

Conversation

kpwebb commented Feb 10, 2025 • edited Loading

slinkydeveloper commented Feb 10, 2025

slinkydeveloper left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kpwebb commented Feb 11, 2025 • edited Loading

slinkydeveloper left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

slinkydeveloper left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kpwebb commented Feb 19, 2025

slinkydeveloper left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kpwebb commented Feb 10, 2025 •

edited

Loading

kpwebb commented Feb 11, 2025 •

edited

Loading