diff --git a/docs/06-concepts/18-testing/01-get-started.md b/docs/06-concepts/18-testing/01-get-started.md new file mode 100644 index 00000000..eebf3293 --- /dev/null +++ b/docs/06-concepts/18-testing/01-get-started.md @@ -0,0 +1,212 @@ +# Get started + +Serverpod provides simple but feature rich test tools to make testing your backend a breeze. + +:::info + +For Serverpod Mini projects, everything related to the database in this guide can be ignored. + +::: + +
+ Have an existing project? Follow these steps first! +

+For existing non-Mini projects, a few extra things need to be done: +1. Add the `server_test_tools_path` key to `config/generator.yaml`. Without this key, the test tools file is not generated. The default location for the generated file is `integration_test/test_tools/serverpod_test_tools.dart`, but this can be set to any path (though should be outside of `lib` as per Dart's test conventions). + +2. New projects now come with a test profile in `docker-compose.yaml`. This is not strictly mandatory, but is recommended to ensure that the testing state is never polluted. Add the snippet below to the `docker-compose.yaml` file in the server directory: + +```yaml +# Test services +postgres_test: + image: postgres:16.3 + ports: + - '9090:5432' + environment: + POSTGRES_USER: postgres_test + POSTGRES_DB: projectname_test + POSTGRES_PASSWORD: "" + volumes: + - projectname_data:/var/lib/postgresql/data + profiles: + - '' # Default profile + - test +redis_test: + image: redis:6.2.6 + ports: + - '9091:6379' + command: redis-server --requirepass "" + environment: + - REDIS_REPLICATION_MODE=master + profiles: + - '' # Default profile + - test +``` + +

+Or copy the complete file here. +

+ +```yaml +services: + # Development services + postgres: + image: postgres:16.3 + ports: + - '8090:5432' + environment: + POSTGRES_USER: postgres + POSTGRES_DB: projectname + POSTGRES_PASSWORD: "" + volumes: + - projectname_data:/var/lib/postgresql/data + profiles: + - '' # Default profile + - dev + redis: + image: redis:6.2.6 + ports: + - '8091:6379' + command: redis-server --requirepass "" + environment: + - REDIS_REPLICATION_MODE=master + profiles: + - '' # Default profile + - dev + + # Test services + postgres_test: + image: postgres:16.3 + ports: + - '9090:5432' + environment: + POSTGRES_USER: postgres_test + POSTGRES_DB: projectname_test + POSTGRES_PASSWORD: "" + volumes: + - projectname_data:/var/lib/postgresql/data + profiles: + - '' # Default profile + - test + redis_test: + image: redis:6.2.6 + ports: + - '9091:6379' + command: redis-server --requirepass "" + environment: + - REDIS_REPLICATION_MODE=master + profiles: + - '' # Default profile + - test + +volumes: + projectname_data: +``` + +

+
+3. Create a `test.yaml` file and add it to the `config` directory: + +```yaml +# This is the configuration file for your local test environment. By +# default, it runs a single server on port 8090. To set up your server, you will +# need to add the name of the database you are connecting to and the user name. +# The password for the database is stored in the config/passwords.yaml. +# +# When running your server locally, the server ports are the same as the public +# facing ports. + +# Configuration for the main API test server. +apiServer: + port: 9080 + publicHost: localhost + publicPort: 9080 + publicScheme: http + +# Configuration for the Insights test server. +insightsServer: + port: 9081 + publicHost: localhost + publicPort: 9081 + publicScheme: http + +# Configuration for the web test server. +webServer: + port: 9082 + publicHost: localhost + publicPort: 9082 + publicScheme: http + +# This is the database setup for your test server. +database: + host: localhost + port: 9090 + name: projectname_test + user: postgres + +# This is the setup for your Redis test instance. +redis: + enabled: false + host: localhost + port: 9091 +``` + +4. Add this entry to `config/passwords.yaml` + +```yaml +test: + database: '' + redis: '' +``` + +That's it, the project setup should be ready to start using the test tools! +

+
+ +Go to the server directory and generate the test tools by running `serverpod generate --experimental-features testTools`. The default location for the generated file is `integration_test/test_tools/serverpod_test_tools.dart`. The folder name `integration_test` is chosen to differentiate from unit tests (see the [best practises section](best-practises#unit-and-integration-tests) for more information on this). + +The generated file exports a `withServerpod` helper that enables you to call your endpoints directly like regular functions: + +```dart +// Import the generated file, it contains everything you need. +import 'test_tools/serverpod_test_tools.dart'; + +void main() { + withServerpod('Given Example endpoint', (sessionBuilder, endpoints) { + test('when calling `hello` then should return greeting', () async { + final greeting = + await endpoints.example.hello(sessionBuilder, 'Michael'); + expect(greeting, 'Hello, Michael!'); + }); + }); +} +``` + +A few things to note from the above example: + +- The test tools should be imported from the generated test tools file and not the `serverpod_test` package. +- The `withServerpod` callback takes two parameters: `sessionBuilder` and `endpoints`. + - `sessionBuilder` is used to build a `session` object that represents the server state during an endpoint call and is used to set up scenarios. + - `endpoints` contains all your Serverpod endpoints and lets you call them. + +:::tip + +The location of the test tools can be changed by changing the `server_test_tools_path` key in `config/generator.yaml`. If you remove the `server_test_tools_path` key, the test tools will stop being generated. + +::: + +Before the test can be run the Postgres and Redis also have to be started: + +```bash +docker-compose up --build --detach +``` + +By default this starts up both the `development` and `test` profiles. To only start one profile, simply add `--profile test` to the command. + +Now the test is ready to be run: + +```bash +dart test integration_test +``` + +Happy testing! diff --git a/docs/06-concepts/18-testing/02-the-basics.md b/docs/06-concepts/18-testing/02-the-basics.md new file mode 100644 index 00000000..ddd24ba3 --- /dev/null +++ b/docs/06-concepts/18-testing/02-the-basics.md @@ -0,0 +1,248 @@ +# The basics + +## Set up a test scenario + +The `withServerpod` helper provides a `sessionBuilder` that helps with setting up different scenarios for tests. To modify the session builder's properties, call its `copyWith` method. It takes the following named parameters: + +|Property|Type|Default|Description| +|:-----|:-----|:---:|:-----| +|`authentication`|`AuthenticationOverride?`|`AuthenticationOverride.unauthenticated()`|See section [Setting authenticated state](#setting-authenticated-state).| +|`enableLogging`|`bool?`|`false`|Wether logging is turned on for the session.| + +The `copyWith` method creates a new unique session builder with the provided properties. This can then be used in endpoint calls (see section [Setting authenticated state](#setting-authenticated-state) for an example). + +To build out a `Session` (to use for [database calls](#seeding-the-database) or [pass on to functions](advanced-examples#test-business-logic-that-depends-on-session)), simply call the `build` method: + +```dart +Session session = sessionBuilder.build(); +``` + +Given the properties set on the session builder through the `copyWith` method, this returns a Serverpod `Session` that has the corresponding state. + +### Setting authenticated state + +To control the authenticated state of the session, the `AuthenticationOverride` class can be used. + +To create an unauthenticated override (this is the default value for new sessions), call `AuthenticationOverride unauthenticated()`: + +```dart +static AuthenticationOverride unauthenticated(); +``` + +To create an authenticated override, call `AuthenticationOverride.authenticationInfo(...)`: + +```dart +static AuthenticationOverride authenticationInfo( + int userId, + Set scopes, { + String? authId, +}) +``` + +Pass these to `sessionBuilder.copyWith` to simulate different scenarios. Below follows an example for each case: + +```dart +withServerpod('Given AuthenticatedExample endpoint', (sessionBuilder, endpoints) { + // Corresponds to an actual user id + const int userId = 1234; + + group('when authenticated', () { + var authenticatedSessionBuilder = sessionBuilder.copyWith( + authentication: + AuthenticationOverride.authenticationInfo(userId, {Scope('user')}), + ); + + test('then calling `hello` should return greeting', () async { + final greeting = await endpoints.authenticatedExample + .hello(authenticatedSessionBuilder, 'Michael'); + expect(greeting, 'Hello, Michael!'); + }); + }); + + group('when unauthenticated', () { + var unauthenticatedSessionBuilder = sessionBuilder.copyWith( + authentication: AuthenticationOverride.unauthenticated(), + ); + + test( + 'then calling `hello` should throw `ServerpodUnauthenticatedException`', + () async { + final future = endpoints.authenticatedExample + .hello(unauthenticatedSessionBuilder, 'Michael'); + await expectLater( + future, throwsA(isA())); + }); + }); +}); +``` + +### Seeding the database + +To seed the database before tests, `build` a `session` and pass it to the database call just as in production code. + +:::info + +By default `withServerpod` does all database operations inside a transaction that is rolled back after each `test` case. See the [rollback database configuration](#rollback-database-configuration) for how to configure this behavior. + +::: + +```dart +withServerpod('Given Products endpoint', (sessionBuilder, endpoints) { + var session = sessionBuilder.build(); + + setUp(() async { + await Product.db.insert(session, [ + Product(name: 'Apple', price: 10), + Product(name: 'Banana', price: 10) + ]); + }); + + test('then calling `all` should return all products', () async { + final products = await endpoints.products.all(sessionBuilder); + expect(products, hasLength(2)); + expect(products.map((p) => p.name), contains(['Apple', 'Banana'])); + }); +}); +``` + +## Environment + +By default `withServerpod` uses the `test` run mode and the database settings will be read from `config/test.yaml`. + +It is possible to override the default run mode by setting the `runMode` setting: + +```dart +withServerpod( + 'Given Products endpoint', + (sessionBuilder, endpoints) { + /* test code */ + }, + runMode: ServerpodRunMode.development, +); +``` + +## Configuration + +The following optional configuration options are available to pass as a second argument to `withServerpod`: + +|Property|Type|Default| +|:-----|:-----|:---:| +|`rollbackDatabase`|`RollbackDatabase?`|`RollbackDatabase.afterEach`| +|`runMode`|`String?`|`ServerpodRunmode.test`| +|`enableSessionLogging`|`bool?`|`false`| +|`applyMigrations`|`bool?`|`true`| + +### `rollbackDatabase` {#rollback-database-configuration} + +By default `withServerpod` does all database operations inside a transaction that is rolled back after each `test` case. Just like the following enum describes, the behavior of the automatic rollbacks can be configured: + +```dart +/// Options for when to rollback the database during the test lifecycle. +enum RollbackDatabase { + /// After each test. This is the default. + afterEach, + + /// After all tests. + afterAll, + + /// Disable rolling back the database. + disabled, +} +``` + +There are two main reasons to change the default setting: + +1. **Scenario tests**: when consecutive `test` cases depend on each other. While generally considered an anti-pattern, it can be useful when the set up for the test group is very expensive. In this case `rollbackDatabase` can be set to `RollbackDatabase.afterAll` to ensure that the database state persists between `test` cases. At the end of the `withServerpod` scope, all database changes will be rolled back. + +2. **Concurrent transactions in endpoints**: when concurrent calls are made to `session.db.transaction` inside an endpoint, it is no longer possible for the Serverpod test tools to do these operations as part of a top level transaction. In this case this feature should be disabled by passing `RollbackDatabase.disabled`. + +```dart +Future concurrentTransactionCalls( + Session session, +) async { + await Future.wait([ + session.db.transaction((tx) => /*...*/), + // Will throw `InvalidConfigurationException` if `rollbackDatabase` + // is not set to `RollbackDatabase.disabled` in `withServerpod` + session.db.transaction((tx) => /*...*/), + ]); +} +``` + +When setting `rollbackDatabase.disabled` to be able to test `concurrentTransactionCalls`, remember that the database has to be manually cleaned up to not leak data: + +```dart +withServerpod( + 'Given ProductsEndpoint when calling concurrentTransactionCalls', + (sessionBuilder, endpoints) { + tearDownAll(() async { + var session = sessionBuilder.build(); + // If something was saved to the database in the endpoint, + // for example a `Product`, then it has to be cleaned up! + await Product.db.deleteWhere( + session, + where: (_) => Constant.bool(true), + ); + }); + + test('then should execute and commit all transactions', () async { + var result = + await endpoints.products.concurrentTransactionCalls(sessionBuilder); + // ... + }); + }, + rollbackDatabase: RollbackDatabase.disabled, +); +``` + +Additionally, when setting `rollbackDatabase.disabled`, it may also be needed to pass the `--concurrency=1` flag to the dart test runner. Otherwise multiple tests might pollute each others database state: + +```bash +dart test integration_test --concurrency=1 +``` + +For the other cases this is not an issue, as each `withServerpod` has its own transaction and will therefore be isolated. + +### `runMode` + +The run mode that Serverpod should be running in. Defaults to `test`. + +### `enableSessionLogging` + +Wether session logging should be enabled. Defaults to `false`. + +### `applyMigrations` + +Wether pending migrations should be applied when starting Serverpod. Defaults to `true`. + +## Test exceptions + +The following exceptions are exported from the generated test tools file and can be thrown by the test tools in various scenarios, see below. + +|Exception|Description| +|:-----|:-----| +|`ServerpodUnauthenticatedException`|Thrown during an endpoint method call when the user was not authenticated.| +|`ServerpodInsufficientAccessException`|Thrown during an endpoint method call when the authentication key provided did not have sufficient access.| +|`ConnectionClosedException`|Thrown during an endpoint method call if a stream connection was closed with an error. For example, if the user authentication was revoked.| +|`InvalidConfigurationException`|Thrown when an invalid configuration state is found.| + +## Test helpers + +### `flushEventQueue` + +Test helper to flush the event queue. +Useful for waiting for async events to complete before continuing the test. + +```dart +Future flushEventQueue(); +``` + +For example, if depending on a generator function to execute up to its `yield`, then the +event queue can be flushed to ensure the generator has executed up to that point: + +```dart +var stream = endpoints.someEndoint.generatorFunction(session); +await flushEventQueue(); +``` + +See also [this complete example](advanced-examples#multiple-users-interacting-with-a-shared-stream). diff --git a/docs/06-concepts/18-testing/03-advanced-examples.md b/docs/06-concepts/18-testing/03-advanced-examples.md new file mode 100644 index 00000000..8266ea88 --- /dev/null +++ b/docs/06-concepts/18-testing/03-advanced-examples.md @@ -0,0 +1,98 @@ +# Advanced examples + +## Test business logic that depends on `Session` + +It is common to break out business logic into modules and keep it separate from the endpoints. If such a module depends on a `Session` object (e.g to interact with the database), then the `withServerpod` helper can still be used and the second `endpoint` argument can simply be ignored: + +```dart +withServerpod('Given decreasing product quantity when quantity is zero', ( + sessionBuilder, + _, +) { + var session = sessionBuilder.build(); + + setUp(() async { + await Product.db.insertRow(session, [ + Product( + id: 123, + name: 'Apple', + quantity: 0, + ), + ]); + }); + + test('then should throw `InvalidOperationException`', + () async { + var future = ProductsBusinessLogic.updateQuantity( + session, + id: 123, + decrease: 1, + ); + + await expectLater(future, throwsA(isA())); + }); +}); +``` + +## Multiple users interacting with a shared stream + +For cases where there are multiple users reading from or writing to a stream, such as real-time communication, it can be helpful to validate this behavior in tests. + +Given the following simplified endpoint: + +```dart +class CommunicationExampleEndpoint { + static const sharedStreamName = 'shared-stream'; + Future postNumberToSharedStream(Session session, int number) async { + await session.messages + .postMessage(sharedStreamName, SimpleData(num: number)); + } + + Stream listenForNumbersOnSharedStream(Session session) async* { + var sharedStream = + session.messages.createStream(sharedStreamName); + + await for (var message in sharedStream) { + yield message.num; + } + } +} +``` + +Then a test to verify this behavior can be written as below. Note the call to the `flushEventQueue` helper (exported by the test tools), which ensures that `listenForNumbersOnSharedStream` executes up to its first `yield` statement before continuing with the test. This guarantees that the stream was registered by Serverpod before messages are posted to it. + +```dart +withServerpod('Given CommunicationExampleEndpoint', (sessionBuilder, endpoints) { + const int userId1 = 1; + const int userId2 = 2; + + test( + 'when calling postNumberToSharedStream and listenForNumbersOnSharedStream ' + 'with different sessions then number should be echoed', + () async { + var userSession1 = sessionBuilder.copyWith( + authentication: AuthenticationOverride.authenticationInfo( + userId1, + {}, + ), + ); + var userSession2 = sessionBuilder.copyWith( + authentication: AuthenticationOverride.authenticationInfo( + userId2, + {}, + ), + ); + + var stream = + endpoints.testTools.listenForNumbersOnSharedStream(userSession1); + // Wait for `listenForNumbersOnSharedStream` to execute up to its + // `yield` statement before continuing + await flushEventQueue(); + + await endpoints.testTools.postNumberToSharedStream(userSession2, 111); + await endpoints.testTools.postNumberToSharedStream(userSession2, 222); + + await expectLater(stream.take(2), emitsInOrder([111, 222])); + }); +}); +``` diff --git a/docs/06-concepts/18-testing/04-best-practises.md b/docs/06-concepts/18-testing/04-best-practises.md new file mode 100644 index 00000000..045ec373 --- /dev/null +++ b/docs/06-concepts/18-testing/04-best-practises.md @@ -0,0 +1,125 @@ +--- +# Don't display do's and don'ts in the table of contents +toc_max_heading_level: 2 +--- + +# Best practises + +## Imports + +While it's possible to import types and test helpers from the `serverpod_test`, it's completely redundant. The generated file exports everything that is needed. Adding an additional import is just unnecessary noise and will likely also be flagged as duplicated imports by the Dart linter. + +### Don't + +```dart +import 'serverpod_test_tools.dart'; +// Don't import `serverpod_test` directly. +import 'package:serverpod_test/serverpod_test.dart'; ❌ +``` + +### Do + +```dart +// Only import the generated test tools file. +// It re-exports all helpers and types that are needed. +import 'serverpod_test_tools.dart'; ✅ +``` + +### Database clean up + +Unless configured otherwise, by default `withServerpod` does all database operations inside a transaction that is rolled back after each `test` (see [the configuration options](the-basics#rollback-database-configuration) for more info on this behavior). + +### Don't + +```dart +withServerpod('Given ProductsEndpoint', (sessionBuilder, endpoints) { + var session = sessionBuilder.build(); + + setUp(() async { + await Product.db.insertRow(session, Product(name: 'Apple', price: 10)); + }); + + tearDown(() async { + await Product.db.deleteWhere( ❌ // Unnecessary clean up + session, + where: (_) => Constant.bool(true), + ); + }); + + // ... +}); +``` + +### Do + +```dart +withServerpod('Given ProductsEndpoint', (sessionBuilder, endpoints) { + var session = sessionBuilder.build(); + + setUp(() async { + await Product.db.insertRow(session, Product(name: 'Apple', price: 10)); + }); + + ✅ // Clean up can be omitted since the transaction is rolled back after each by default + + // ... +}); +``` + +## Calling endpoints + +While it's technically possible to instantiate an endpoint class and call its methods directly with a Serverpod `Session`, it's advised that you do not. The reason is that lifecycle events and validation that should happen before or after an endpoint method is called is taken care of by the framework. Calling endpoint methods directly would circumvent that and the code would not behave like production code. Using the test tools guarantees that the way endpoints behave during tests is the same as in production. + +### Don't + +```dart +void main() { + // ❌ Don't instantiate endpoints directly + var exampleEndpoint = ExampleEndpoint(); + + withServerpod('Given Example endpoint', ( + sessionBuilder, + _ /* not using the provided endpoints */, + ) { + var session = sessionBuilder.build(); + + test('when calling `hello` then should return greeting', () async { + // ❌ Don't call and endpoint method directly on the endpoint class. + final greeting = await exampleEndpoint.hello(session, 'Michael'); + expect(greeting, 'Hello, Michael!'); + }); + }); +} +``` + +### Do + +```dart +void main() { + withServerpod('Given Example endpoint', (sessionBuilder, endpoints) { + var session = sessionBuilder.build(); + + test('when calling `hello` then should return greeting', () async { + // ✅ Use the provided `endpoints` to call the endpoint that should be tested. + final greeting = + await endpoints.example.hello(session, 'Michael'); + expect(greeting, 'Hello, Michael!'); + }); + }); +} +``` + +## Unit and integration tests + +It is significantly easier to navigate a project if the different types of tests are clearly separated. + +### Don't + +❌ Mix different types of tests together. + +### Do + +✅ Have a clear structure for the different types of test. Serverpod recommends the following two folders in the `server`: + +- `test`: Unit tests. +- `integration_test`: Tests for endpoints or business logic modules using the `withServerpod` helper. diff --git a/docs/06-concepts/18-testing/_category_.json b/docs/06-concepts/18-testing/_category_.json new file mode 100644 index 00000000..82138d51 --- /dev/null +++ b/docs/06-concepts/18-testing/_category_.json @@ -0,0 +1,4 @@ +{ + "label": "Testing", + "collapsed": true +} \ No newline at end of file