Skip to content

Commit

Permalink
Merge pull request #42535 from holly-cummins/dev-service-docs
Browse files Browse the repository at this point in the history
Add "how to write dev services" documentation
  • Loading branch information
gsmet authored Aug 28, 2024
2 parents 601a2fa + 3386f3a commit c46100d
Show file tree
Hide file tree
Showing 13 changed files with 141 additions and 12 deletions.
2 changes: 1 addition & 1 deletion docs/src/main/asciidoc/cache-redis-reference.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ include::{includes}/extension-status.adoc[]

When using Redis as the backend for Quarkus cache, each cached item will be stored in Redis:

- The backend uses the _<default>_ Redis client (if not configured otherwise), so make sure it's configured (or use the xref:redis-dev-services.adoc[redis dev service])
- The backend uses the _<default>_ Redis client (if not configured otherwise), so make sure it's configured (or use the xref:redis-dev-services.adoc[Redis Dev Service])
- the Redis key is built as follows: `cache:$cache-name:$cache-key`, where `cache-key` is the key the application uses.
- the value is encoded to JSON if needed

Expand Down
4 changes: 2 additions & 2 deletions docs/src/main/asciidoc/databases-dev-services.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,7 @@ quarkus.datasource.db-kind=mysql
quarkus.datasource.devservices.volumes."/local/test/data"=/var/lib/mysql
----

When starting Dev Services (for example, in tests or in dev mode), you will see that the folder "/local/test/data" will be created at your file sytem and that will contain all the database data. When rerunning again the same dev services, this data will contain all the data you might have created beforehand.
When starting Dev Services (for example, in tests or in dev mode), you will see that the folder "/local/test/data" will be created at your file sytem and that will contain all the database data. When rerunning again the same Dev Services, this data will contain all the data you might have created beforehand.

[IMPORTANT]
====
Expand All @@ -190,7 +190,7 @@ Overriding the MariaDB/MySQL configuration would be done as follows:
quarkus.datasource.devservices.container-properties.TC_MY_CNF=testcontainers/mysql-conf
----

This support is database specific and needs to be implemented in each dev service specifically.
This support is database specific and needs to be implemented in each Dev Service specifically.

== Connect To Database Run as a Dev Service

Expand Down
2 changes: 1 addition & 1 deletion docs/src/main/asciidoc/dev-services.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ https://github.com/quarkusio/quarkus/tree/main/docs/src/main/asciidoc
= Dev Services Overview
include::_attributes.adoc[]
:categories: core
:summary: An introduction to dev services and a list of all extensions that support Dev Services and their configuration options.
:summary: An introduction to Dev Services and a list of all extensions that support Dev Services and their configuration options.
:topics: dev-services,dev-mode,testing

== What Are Dev Services?
Expand Down
123 changes: 123 additions & 0 deletions docs/src/main/asciidoc/extension-writing-dev-service.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
////
This document is maintained in the main Quarkus repository
and pull requests should be submitted there:
https://github.com/quarkusio/quarkus/tree/main/docs/src/main/asciidoc
////
[id="extension-writing-dev-service"]
= Writing a Dev Service
include::_attributes.adoc[]
:categories: writing-extensions
:diataxis-type: howto
:topics: extensions
////
////


== Prerequisites

- You should already have an xref:building-my-first-extension.adoc[extension structure] in place
- You should have a containerised version of your external service (not all Dev Services rely on containers, but most do)

== Creating a Dev Service

If your extension provides APIs for connecting to an external service, it's a good idea to provide a xref:dev-services.adoc[Dev Service] implementation.

To create a Dev Service, add a new build step into the extension processor class that returns a `DevServicesResultBuildItem`.
Here, the link:https://hub.docker.com/_/hello-world`hello-world` image is used, but you should set up the right image for your service.

[source%nowrap,java]
----
@BuildStep(onlyIfNot = IsNormal.class, onlyIf = GlobalDevServicesConfig.Enabled.class) {
public DevServicesResultBuildItem createContainer() {
DockerImageName dockerImageName = DockerImageName.parse("hello-world");
GenericContainer container = new GenericContainer<>(dockerImageName)
.withExposedPorts(SERVICE_PORT, OTHER_SERVICE_PORT)
.waitingFor(Wait.forLogMessage(".*" + "Started" + ".*", 1))
.withReuse(true);
container.start();
String newUrl = "http://" + container.getHost() + ":" + container.getMappedPort(SERVICE_PORT);
Map<String, String> configOverrides = Map.of("some-service.base-url", newUrl);
return new DevServicesResultBuildItem.RunningDevService(FEATURE, container.getContainerId(),
container::close, configOverrides)
.toBuildItem();
}
----

With this code, you should be able to see your container starting if you add your extension to a test application and run `quarkus dev`.
However, the application will not be able to connect to it, because no ports are exposed. To expose ports, add `withExposedPorts` to the container construction.
For example,

[source%nowrap,java]
----
GenericContainer container = new GenericContainer<>(dockerImageName)
.withExposedPorts(SERVICE_PORT, OTHER_SERVICE_PORT);
----

Testcontainers will map these ports to random ports on the host. This avoids port conflicts, but presents a new problem – how do applications connect to the service in the container?

To allow applications to connect, the extension should override the default configuration for the service with the mapped ports.
This must be done after starting the container.
For example,

[source%nowrap,java]
----
container.start();
Map<String, String> configOverrides = Map.of("some-service.base-url",
"http://" + container.getHost() + ":" + container.getMappedPort(SERVICE_PORT));
----

Other configuration overrides may be included in the same map.

== Waiting for the container to start

You should add a `.waitingFor` call to the container construction, to wait for the container to start. For example

[source%nowrap,java]
----
.waitingFor(Wait.forLogMessage(".*" + "Started" + ".*", 1))
----

Waiting for a port to be open is another option. See the link:https://java.testcontainers.org/features/startup_and_waits/[Testcontainers documentation] for a full discussion of wait strategies.

== Configuring the Dev Service

To configure the Dev Service launch process, your build step can accept a `ConfigPhase.BUILD_TIME` config class in its constructor.
For example,

[source%nowrap,java]
----
@BuildStep(onlyIfNot = IsNormal.class, onlyIf = GlobalDevServicesConfig.Enabled.class) {
public DevServicesResultBuildItem createContainer(MyConfig config) {
----

You may wish to use this config to set a fixed port, or set an image name, for example.

[source%nowrap,java]
----
if (config.port.isPresent()) {
container.setPortBindings(List.of(config.port.get() + ":" + SERVICE_PORT));
}
----

== Controlling re-use

In dev mode, with live reload, Quarkus may restart frequently. By default, this will also restart test containers.
Quarkus restarts are usually very fast, but containers may take much longer to restart.
To prevent containers restarting on every code change, you can mark the container as reusable:

[source%nowrap,java]
----
.withReuse(true)
----

Some Dev Services implement sophisticated reuse logic in which they track the state of the container in the processor itself.
You may need this if your service has more complex requirements, or needs sharing across instances.


== References

- xref:dev-services.adoc[Dev services overview]
- xref:writing-extensions.adoc[Guide to writing extensions]
2 changes: 1 addition & 1 deletion docs/src/main/asciidoc/getting-started-dev-services.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -306,7 +306,7 @@ But what about production? You won't want to use Dev Services in production.
In fact, Quarkus only starts Dev Services in dev and test modes.

Wouldn't it be nice to configure an external database,
but have it *only* used in production, so you could still use dev services the rest of the time?
but have it *only* used in production, so you could still use Dev Services the rest of the time?

Add a `%prod.`
prefix to the database configuration. This means the configuration
Expand Down
2 changes: 1 addition & 1 deletion docs/src/main/asciidoc/getting-started.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -491,7 +491,7 @@ include::{generated-dir}/config/quarkus-info.adoc[opts=optional, leveloffset=+2]

This guide covered the creation of an application using Quarkus.
However, there is much more.
We recommend continuing the journey by creating xref:getting-started-dev-services.adoc[your second Quarkus application], with dev services and persistence.
We recommend continuing the journey by creating xref:getting-started-dev-services.adoc[your second Quarkus application], with Dev Services and persistence.
You can learn about creating a native executable and packaging it in a container with the xref:building-native-image.adoc[building a native executable guide].
If you are interested in reactive, we recommend the xref:getting-started-reactive.adoc[getting started with reactive guide], where you can see how to implement reactive applications with Quarkus.

Expand Down
2 changes: 1 addition & 1 deletion docs/src/main/asciidoc/infinispan-dev-services.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ link:https://infinispan.org/tutorials/simple/simple_tutorials.html#cross-site-re

== Multiple Dev Services for named connections
The Infinispan Client extension supports connecting to more than one Infinispan Cluster with
the named connections. If you need to spin an additional dev service for a connection name, configure
the named connections. If you need to spin an additional Dev Service for a connection name, configure
at least on property in the application properties:

[source,properties]
Expand Down
2 changes: 1 addition & 1 deletion docs/src/main/asciidoc/kafka-dev-services.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ Dev Services for Kafka supports https://redpanda.com[Redpanda], https://github/o
and https://strimzi.io[Strimzi] (in https://github.com/apache/kafka/blob/trunk/config/kraft/README.md[Kraft] mode) images.

**Redpanda** is a Kafka compatible event streaming platform.
Because it provides a fast startup times, dev services defaults to Redpanda images from `vectorized/redpanda`.
Because it provides a fast startup times, Dev Services defaults to Redpanda images from `vectorized/redpanda`.
You can select any version from https://hub.docker.com/r/vectorized/redpanda.

**kafka-native** provides images of standard Apache Kafka distribution compiled to native binary using Quarkus and GraalVM.
Expand Down
2 changes: 1 addition & 1 deletion docs/src/main/asciidoc/kafka-schema-registry-avro.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -620,7 +620,7 @@ The `quarkus-apicurio-registry-avro` extension depends on recent versions of Api
and most versions of Apicurio Registry server and client are backwards compatible.
For some you need to make sure that the client used by Serdes is compatible with the server.

For example, with Apicurio dev service if you set the image name to use version `2.1.5.Final`:
For example, with Apicurio Dev Service if you set the image name to use version `2.1.5.Final`:

[source,properties]
----
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -648,7 +648,7 @@ The `quarkus-apicurio-registry-json-schema` extension depends on recent versions
and most versions of Apicurio Registry server and client are backwards compatible.
For some you need to make sure that the client used by Serdes is compatible with the server.

For example, with Apicurio dev service if you set the image name to use version `2.1.5.Final`:
For example, with Apicurio Dev Service if you set the image name to use version `2.1.5.Final`:

[source,properties]
----
Expand Down
2 changes: 1 addition & 1 deletion docs/src/main/asciidoc/observability-devservices.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ include::_attributes.adoc[]
:topics: observability,grafana,lgtm,prometheus,victoriametrics,jaeger,otel,otlp
:extensions: io.quarkus:quarkus-observability-devservices

We are already familiar with xref:dev-services.adoc[Dev Service] concept, but in the case of Observability we need a way to orchestrate and connect more than a single dev service, usually a whole stack of them; e.g. a metrics agent periodically scraping application for metrics, pushing them into timeseries database, and Grafana feeding graphs of this timeseries data.
We are already familiar with xref:dev-services.adoc[Dev Service] concept, but in the case of Observability we need a way to orchestrate and connect more than a single Dev Service, usually a whole stack of them; e.g. a metrics agent periodically scraping application for metrics, pushing them into timeseries database, and Grafana feeding graphs of this timeseries data.

With this in mind, we added a new concept of Dev Resource, an adapter between Dev Service concept and https://testcontainers.com/[Testcontainers]. And since we now have fine-grained services - with the Dev Resource per container, we can take this even further, allowing the user to choose the way to use this new Dev Resource concept:

Expand Down
2 changes: 1 addition & 1 deletion docs/src/main/asciidoc/opentelemetry-metrics.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,7 @@ First we need to start a system to visualise the OpenTelemetry data.

=== See the data

==== Grafana-OTel-LGTM dev service
==== Grafana-OTel-LGTM Dev Service
You can use the xref:observability-devservices-lgtm.adoc[Grafana-OTel-LGTM] devservice.

This Dev service includes a Grafana for visualizing data, Loki to store logs, Tempo to store traces and Prometheus to store metrics.
Expand Down
6 changes: 6 additions & 0 deletions docs/src/main/asciidoc/writing-extensions.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -1978,6 +1978,12 @@ in your runtime module, and add a `META-INF/services/io.quarkus.dev.spi.HotRepla
On startup the `setupHotDeployment` method will be called, and you can use the provided `io.quarkus.dev.spi.HotReplacementContext`
to initiate a scan for changed files.

==== Dev Services

Where extensions use an external service, adding a Dev Service can improve the user experience in development and test modes.
See xref:extension-writing-dev-service.adoc[how to write a Dev Service] for more details.


=== Testing Extensions

Testing of Quarkus extensions should be done with the `io.quarkus.test.QuarkusUnitTest` JUnit 5 extension.
Expand Down

0 comments on commit c46100d

Please sign in to comment.