From 4e9211ec7c0b7525278a7519e4f6eed19f4e236a Mon Sep 17 00:00:00 2001 From: David Mueller Date: Mon, 19 Aug 2024 15:55:52 -0400 Subject: [PATCH 01/62] add example #7459 --- .../pages/feature/mpTelemetry-2.0/examples.adoc | 16 ++++++++++++++++ .../pages/feature/mpTelemetry/examples.adoc | 16 ++++++++++++++++ 2 files changed, 32 insertions(+) create mode 100644 modules/reference/pages/feature/mpTelemetry-2.0/examples.adoc create mode 100644 modules/reference/pages/feature/mpTelemetry/examples.adoc diff --git a/modules/reference/pages/feature/mpTelemetry-2.0/examples.adoc b/modules/reference/pages/feature/mpTelemetry-2.0/examples.adoc new file mode 100644 index 0000000000..5e1f82ab9d --- /dev/null +++ b/modules/reference/pages/feature/mpTelemetry-2.0/examples.adoc @@ -0,0 +1,16 @@ +== Examples + +=== Collect logs from a specified source + +To enable the MicroProfile Telemetry feature to collect logs from different sources in the Open Liberty runtime environment, configure the `source` attribute for the `mpTelemetry` element with a comma-separated list of comma-separated log sources. + +[source,xml] +---- + +---- + +* The `mpTelemetry` configuration element is optional. If you do not specify ity or the `source` attribute, the default configuration source is `message`. + +* If the source attribute is empty (`source=“”`), no sources are collected, and nothing is routed to the configured OpenTelemetry logs exporter. + +* The attributes and values are case insensitive. diff --git a/modules/reference/pages/feature/mpTelemetry/examples.adoc b/modules/reference/pages/feature/mpTelemetry/examples.adoc new file mode 100644 index 0000000000..5e1f82ab9d --- /dev/null +++ b/modules/reference/pages/feature/mpTelemetry/examples.adoc @@ -0,0 +1,16 @@ +== Examples + +=== Collect logs from a specified source + +To enable the MicroProfile Telemetry feature to collect logs from different sources in the Open Liberty runtime environment, configure the `source` attribute for the `mpTelemetry` element with a comma-separated list of comma-separated log sources. + +[source,xml] +---- + +---- + +* The `mpTelemetry` configuration element is optional. If you do not specify ity or the `source` attribute, the default configuration source is `message`. + +* If the source attribute is empty (`source=“”`), no sources are collected, and nothing is routed to the configured OpenTelemetry logs exporter. + +* The attributes and values are case insensitive. From 31cca5aba6a053fbb2ea44ce90cca6d3eeb1c5d3 Mon Sep 17 00:00:00 2001 From: David Mueller Date: Tue, 20 Aug 2024 12:12:30 -0400 Subject: [PATCH 02/62] add event tables #7459 --- modules/ROOT/pages/mptel-log-events-list.adoc | 147 ++++++++++++++++++ .../pages/feature/mpTelemetry/examples.adoc | 6 +- 2 files changed, 150 insertions(+), 3 deletions(-) create mode 100644 modules/ROOT/pages/mptel-log-events-list.adoc diff --git a/modules/ROOT/pages/mptel-log-events-list.adoc b/modules/ROOT/pages/mptel-log-events-list.adoc new file mode 100644 index 0000000000..75ec60225a --- /dev/null +++ b/modules/ROOT/pages/mptel-log-events-list.adoc @@ -0,0 +1,147 @@ +// Copyright (c) 2018 IBM Corporation and others. +// Licensed under Creative Commons Attribution-NoDerivatives +// 4.0 International (CC BY-ND 4.0) +// https://creativecommons.org/licenses/by-nd/4.0/ +// +// Contributors: +// IBM Corporation +// +:page-layout: general-reference +:page-type: general +:seo-title: MicroProfile Telemetry logging events - OpenLiberty.io +:seo-description: The MicroProfile Telemetry logging events that can be captured from the Open Liberty server runtime environment and applications. += MicroProfile Telemetry log events reference list + +MicroProfile Telemetry can collect logging events from the server runtime and applications. You can use these events to gather and analyze data that can help to better understand the behavior of applications. + +MicroProfile Telemetry can collect the following types of events: + +* <> +* <> +* <> + +== Message events +The following table provides the fields for message events and a description for each field: + +.Message event fields +[options="header"] +|======================= +| Liberty message event fields | Description | Maps to OTel Logs Data Model | OTel Java Agent JUL Mapping +| type | A string that identifies the type of event. | Attributes[`io.openliberty.type`] | N/A +| host | Host name of the server that is the source of the event. | Resource[`host.name`] | Resource[`host.name`] +| message | The message from the log record, starting with the message ID. | Body | body +| ibm_threadId | Thread ID in the log line, for example, 00000015. | Attributes[`thread.id`] | N/A +| ibm_datetime | Time at which the event occurred. | TimeStamp | TimeStamp +| ibm_messageId | Message ID in the log line. | Attributes[`io.openliberty.message_id`] | N/A +| module | Logger name from the log record. | Attributes[`io.openliberty.module`] | N/A +| loglevel | Severity indicator | SeverityText | SeverityText +| ibm_methodName | Method name from the log record. | Attributes[`io.openliberty.method_name`] | N/A +| ibm_className | Class name from the log record. | Attributes[`io.openliberty.class_name`] | N/A +| ibm_sequence | Sequence number of the event,. | Attributes[`io.openliberty.sequence`] | N/A +| ext_thread | Name of the thread that is the source of the event. | Attributes[`thread.name`] | N/A +| ext_appName | Name of the application that logged the message. | Attributes[`io.openliberty.ext.app_name`] | N/A +|======================= + +The following example shows the OpenTelemetry Collector output for a message event from an application: + +---- +timeUnixNano="1717512929163000000" +observeTimeUnixNano="1717512929602000000" +severityNumber="13" +severityText="W" +body=" BADAP0001W: Test Warning." +traceId="a4fa4f34167d2bcbcfb94588d37b332" +spanId="63522117c3492d42" +Attributes: + io.openliberty.type="liberty_message" + io.openliberty.message_id="BADAP0001W" + io.openliberty.sequence="1717512929163_0000000000012 + io.openliberty.module="com.ibm.ws.lumberjack.badness.Angry" + thread.id="65" + thread.name="Default Executor-thread-2" + io.openliberty.ext.app_name="badapp" +---- + +== Trace events +The following table provides the fields for trace events and a description for each field: + +.Trace event fields +[options="header"] +|======================= +| Liberty trace event fields | Description | Maps to OTel Logs Data Model | OTel Java Agent JUL Mapping +| type | A string that identifies the type of event. | Attributes[`io.openliberty.type`] | N/A +| host | Host name of the server that is the source of the event. | Resource[`host.name`] | Resource[`host.name`] +| message | The message from the log record, starting with the message ID. | Body | body +| ibm_threadId | Thread ID in the log line, for example, 00000015. | Attributes[`thread.id`] | N/A +| ibm_datetime | Time at which the event occurred. | TimeStamp | TimeStamp +| ibm_messageId | Message ID in the log line. | Attributes[`io.openliberty.message_id`] | N/A +| module | Logger name from the log record. | Attributes[`io.openliberty.module`] | N/A +| loglevel | Severity indicator | SeverityText | SeverityText +| ibm_methodName | Method name from the log record. | Attributes[`io.openliberty.method_name`] | N/A +| ibm_className | Class name from the log record. | Attributes[`io.openliberty.class_name`] | N/A +| ibm_sequence | Sequence number of the event,. | Attributes[`io.openliberty.sequence`] | N/A +| ext_thread | Name of the thread that is the source of the event. | Attributes[`thread.name`] | N/A +| ext_appName | Name of the application that logged the message. | Attributes[`io.openliberty.ext.app_name`] | N/A +|======================= + +The following example shows the OpenTelemetry Collector output for a trace event from an application: + +---- +timeUnixNano="11234512231363000000" +observeTimeUnixNano="17123512929602000000" +severityNumber="1" +severityText="3" +body="finest trace" +traceId="3fced7fbfbc062349c66a8c1a37c31b2" +spanId="4be39c323e375d9e" +Attributes: + io.openliberty.type="liberty_trace" + io.openliberty.sequence="12312363_0000000000021" + io.openliberty.module="com.test.app" + thread.id="79" + thread.name="Default Executor-thread-12" + io.openliberty.ext.app_name="testapp" +---- + +== FFDC events +The following table provides the fields for the first failure data capture (FFDC) events and a description for each field: + +.FFDC event fields +|======================= +| Liberty FFDC event fields | Description | Maps to OTel Logs Data Model +| type | A string that identifies the type of event. | Attributes[`io.openliberty.type`] +| host | Host name of the server that is the source of the event. | Resource[`host.name`] +| ibm_datetime | Time at which the event occurred. | TimeStamp +| message | The message from the exception that triggered the event. | body and Attributes[`exception.message`] +| ibm_className | The class that emitted the FFDC event. | Attributes[`io.openliberty.class_name`] +| ibm_exceptionName | The exception that is reported in the FFDC event. | Attributes[`exception.type`] +| ibm_probeID | The unique identifier of the FFDC point within the class. | Attributes[`io.openliberty.probe_id`] +| ibm_threadId | The thread ID of the FFDC event. | Attributes[`thread.id`] +| ibm_stackTrace | The stack trace of the FFDC event. | Attributes[`exception.stacktrace`] +| ibm_objectDetails | The incident details for the FFDC event. | Attributes[`io.openliberty.object_details`] +| ibm_sequence | Sequence number of the event, which is useful for sorting records with the same timestamp. | Attributes[`io.openliberty.sequence`] +|======================= + +The following example shows the OpenTelemetry Collector output for an FFDC event from an application: + +---- +timeUnixNano=“1232335231363000000” +observeTimeUnixNano=“123455512929602000000” +severityNumber=“13” +body="Cannot invoke "java.lang.String.toString()" because "myString" is null" +traceId=“3fced7fbfbc062349c66a8c1a37c31b2” +spanId=“4be39c323e375d9e” +Attributes: + io.openliberty.type="liberty_ffdc" + exception.message="Cannot invoke "java.lang.String.toString()" because "myString" is null" + exception.stacktrace="java.lang.NullPointerException: Cannot invoke "java.lang.String.toString()" because "myString" is null\n\tat io.openliberty.microprofile.telemetry.logging.internal.container.fat.MpTelemetryLogApp.MpTelemetryServlet.doGet(MpTelemetryServlet.java:53) ..." + exception.type= "java.lang.NullPointerException" + io.openliberty.class_name="io.openliberty.microprofile.telemetry.logging.internal.container.fat.MpTelemetryLogApp" + io.openliberty.object_details="Object type = io.openliberty.microprofile.telemetry.logging.internal.container.fat.MpTelemetryLogApp\n tc = class com.ibm.websphere.ras.TraceComponent@7baa4ff8 ... + io.openliberty.probe_id="62" + io.openliberty.sequence=“1723140654466_00000000000011" + thread.id=“92” + thread.name=“Default Executor-thread-12” +---- + +// assisted by watsonx latest with latest GenAI contribution: llama-3-70b-instruct diff --git a/modules/reference/pages/feature/mpTelemetry/examples.adoc b/modules/reference/pages/feature/mpTelemetry/examples.adoc index 5e1f82ab9d..0d488ad4a0 100644 --- a/modules/reference/pages/feature/mpTelemetry/examples.adoc +++ b/modules/reference/pages/feature/mpTelemetry/examples.adoc @@ -1,15 +1,15 @@ == Examples -=== Collect logs from a specified source +=== Collect logs from specified sources -To enable the MicroProfile Telemetry feature to collect logs from different sources in the Open Liberty runtime environment, configure the `source` attribute for the `mpTelemetry` element with a comma-separated list of comma-separated log sources. +To enable the MicroProfile Telemetry feature to collect logs from different sources in the Open Liberty runtime environment, configure the `source` attribute for the config:mpTelemetry element with a comma-separated list of log sources. [source,xml] ---- ---- -* The `mpTelemetry` configuration element is optional. If you do not specify ity or the `source` attribute, the default configuration source is `message`. +* The `mpTelemetry` configuration element is optional. If you do not specify it, or if you do not include the `source` attribute, the default configuration source is `message`. * If the source attribute is empty (`source=“”`), no sources are collected, and nothing is routed to the configured OpenTelemetry logs exporter. From 6cc668c6fcd646af712731feed911809db285110 Mon Sep 17 00:00:00 2001 From: David Mueller Date: Tue, 20 Aug 2024 14:23:27 -0400 Subject: [PATCH 03/62] add trace level table #7459 --- .../ROOT/pages/log-trace-configuration.adoc | 2 ++ .../ROOT/pages/microprofile-telemetry.adoc | 2 +- modules/ROOT/pages/microprofile.adoc | 6 ++--- modules/ROOT/pages/mptel-log-events-list.adoc | 27 +++++++++++++++++++ modules/ROOT/pages/observability.adoc | 8 +++--- .../reference/pages/diff/mp-50-60-diff.adoc | 2 +- .../feature/mpTelemetry/description.adoc | 2 +- .../pages/feature/mpTelemetry/examples.adoc | 2 +- 8 files changed, 39 insertions(+), 12 deletions(-) diff --git a/modules/ROOT/pages/log-trace-configuration.adoc b/modules/ROOT/pages/log-trace-configuration.adoc index 5ed5d18cc2..ea98bed529 100644 --- a/modules/ROOT/pages/log-trace-configuration.adoc +++ b/modules/ROOT/pages/log-trace-configuration.adoc @@ -669,3 +669,5 @@ The `trace.log` file is created and logs are written to it only if one or more t |all |All events are logged. If you create custom levels, `all` includes those levels, and can provide a more detailed trace than finest. |=== + +For information about mapping Open Liberty log levels to OpenTelemetry severity numbers, see xref:mptel-log-events-list.adoc#sevlev[OpenTelemetry severity number mapping with Open Liberty log levels]. diff --git a/modules/ROOT/pages/microprofile-telemetry.adoc b/modules/ROOT/pages/microprofile-telemetry.adoc index e176427de3..60f116a11e 100644 --- a/modules/ROOT/pages/microprofile-telemetry.adoc +++ b/modules/ROOT/pages/microprofile-telemetry.adoc @@ -10,7 +10,7 @@ :seo-description: :page-layout: general-reference :page-type: general -= Enable distributed tracing with MicroProfile Telemetry += Enable observability with MicroProfile Telemetry In microservice applications, sources of latency or inaccuracy can be difficult to determine because relationships and dependencies among the constituent services are not always obvious. MicroProfile Telemetry helps you collect data on the paths that application requests take through services. diff --git a/modules/ROOT/pages/microprofile.adoc b/modules/ROOT/pages/microprofile.adoc index 7a8a545a62..b2fe2ccdc1 100644 --- a/modules/ROOT/pages/microprofile.adoc +++ b/modules/ROOT/pages/microprofile.adoc @@ -27,7 +27,7 @@ Writing code more quickly isn't the only reason that MicroProfile is popular. It Although a microservice architecture provides many benefits, it also introduces new challenges not apparent in traditional monolithic applications. MicroProfile addresses these challenges so that you can easily develop cloud-native applications. These challenges include toleration of service failures, end-to-end security for an authenticated user request that flows through a set of microservices, and problem determination for requests that span many services. //// - + == Developing RESTful, scalable, observable microservices: MicroProfile layers of functionality MicroProfile can be organized into three layers of functionality. The bottom layer represents RESTful services. The middle layer is for writing microservices that can be operated and managed at scale. The top layer contains tools to help you write observable microservices that enable you to detect and diagnose issues. @@ -122,9 +122,9 @@ For more information, see xref:microservice-observability-metrics.adoc[Microserv === Enable distributed tracing of your microservices link:https://projects.eclipse.org/projects/technology.microprofile/releases/microprofile-telemetry-1.0/plan%E2%80%A8%E2%80%A822.0.0.10-bet[MicroProfile Telemetry] is based on the https://opentelemetry.io/[OpenTelemetry project], which is a collection of open source vendor-agnostic tools, APIs, and SDKs for creating and managing trace data. -In Open Liberty, MicroPRofile Telemetry is supported by the feature:mpTelemetry[display=MicroProfile Telemetry] feature. You can export the data that this feature collects to tracing systems such as Jaeger or Zipkin. For more information, see xref:microprofile-telemetry.adoc[Enable distributed tracing with MicroProfile Telemetry]. +In Open Liberty, MicroPRofile Telemetry is supported by the feature:mpTelemetry[display=MicroProfile Telemetry] feature. You can export the data that this feature collects to tracing systems such as Jaeger or Zipkin. For more information, see xref:microprofile-telemetry.adoc[Enable observability with MicroProfile Telemetry]. -MicroProfile Telemetry replaces MicroProfile OpenTracing. For information about migrating your applications from MicroProfile OpenTracing to MicroProfile Telemetry, see xref:reference:diff/mp-50-60-diff.adoc#telemetry[Differences between MicroProfile Telemetry 1.0 and MicroProfile OpenTracing 3.0]. +MicroProfile Telemetry replaces MicroProfile OpenTracing. For information about migrating your applications from MicroProfile OpenTracing to MicroProfile Telemetry, see xref:reference:diff/mp-50-60-diff.adoc#telemetry[Differences between MicroProfile Telemetry 1.0 and MicroProfile OpenTracing 3.0]. == Other MicroProfile features In addition to the three layers of MicroProfile features that comprise the MicroProfile stack, these additional MicroProfile features are implemented in the Open Liberty runtime but are not yet part of an official MicroProfile release. These features extend the MicroProfile core capabilities, giving you enhanced control over concurrency and reactive programming in your microservice-based applications. diff --git a/modules/ROOT/pages/mptel-log-events-list.adoc b/modules/ROOT/pages/mptel-log-events-list.adoc index 75ec60225a..f55ec91751 100644 --- a/modules/ROOT/pages/mptel-log-events-list.adoc +++ b/modules/ROOT/pages/mptel-log-events-list.adoc @@ -14,12 +14,16 @@ MicroProfile Telemetry can collect logging events from the server runtime and applications. You can use these events to gather and analyze data that can help to better understand the behavior of applications. +For more information, see xref:microprofile-telemetry.adoc[Enable observability with MicroProfile Telemetry] + MicroProfile Telemetry can collect the following types of events: * <> * <> * <> +For information about trace log detail levels and how Open Liberty settings map to OpenTelemetry severity numbers, see <>. + == Message events The following table provides the fields for message events and a description for each field: @@ -144,4 +148,27 @@ Attributes: thread.name=“Default Executor-thread-12” ---- +[#sevlev] +== OpenTelemetry severity number mapping with Open Liberty log levels + +You can set the level of detail that your Open Liberty logs collect by configuring the `tracSpecification` attribute for the config:logging[] attribute in your `server.xml` file. For more information about Liberty log levels, see xref:log-trace-configuration.adoc#log_details[Trace log detail levels]. + +The following table describes how the different Liberty log levels map to severity numbers and short names from the OpenTelemetry specification. + +.Open Liberty log levels and OpenTelemetry severity numbers +[options="header"] +|======================= +| Liberty log levels | OpenTelemetry severity short Name | OpenTelemetry severity meaning | OpenTelemetry severity number +| fatal | FATAL | A fatal error such as application or system crash. | 21 +| severe | ERROR | An error event. Something went wrong. | 17 +| warning | WARN | A warning event. Not an error but is likely more important than an informational event. | 13 +| audit | INFO2 | An informational event. Indicates that an event happened. | 10 +| info | INFO | - | 9 +| config | DEBUG4 | A debugging event. | 8 +| detail | DEBUG3 | - | 7 +| fine | DEBUG2 | - | 6 +| finer | DEBUG | - | 5 +| finest | TRACE | A fine-grained debugging event. Typically disabled in default configurations. | 1 +|======================= + // assisted by watsonx latest with latest GenAI contribution: llama-3-70b-instruct diff --git a/modules/ROOT/pages/observability.adoc b/modules/ROOT/pages/observability.adoc index c29b2ae986..d8dbeb033e 100644 --- a/modules/ROOT/pages/observability.adoc +++ b/modules/ROOT/pages/observability.adoc @@ -11,7 +11,7 @@ :seo-description: Building observability into applications externalizes the internal status of a system so operations teams can monitor systems more effectively. Open Liberty provides a robust framework for building observable applications and integrates with numerous third party monitoring tools. :page-layout: general-reference :page-type: general -= Observability += Observability When applications are observable, operations teams can identify and understand the root causes of bugs, bottlenecks, and other inefficiencies. Open Liberty provides a robust framework for developing observable applications and integrates with numerous third-party monitoring tools. @@ -21,14 +21,12 @@ xref:microservice-observability-metrics.adoc[Microservice observability with met Open Liberty uses MicroProfile Metrics to provide metrics that describe the internal state of many Open Liberty components. MicroProfile Metrics provides a `/metrics` endpoint from which you can access all metrics that are emitted by an Open Liberty server and deployed applications. xref:micrometer-metrics.adoc[Choose your own monitoring tools with MicroProfile Metrics]:: -The Open Liberty MicroProfile Metrics 5.0 feature uses embedded Micrometer metrics technology to transfer metric data to multiple third-party monitoring systems. Starting in MicroProfile Metrics 5.0, you can instrument your code in a standardized way to visualize metrics results in the monitoring system of your choice. +The Open Liberty MicroProfile Metrics 5.0 feature uses embedded Micrometer metrics technology to transfer metric data to multiple third-party monitoring systems. Starting in MicroProfile Metrics 5.0, you can instrument your code in a standardized way to visualize metrics results in the monitoring system of your choice. xref:health-check-microservices.adoc[Health checks for microservices]:: A health check is a special REST API implementation that you can use to validate the status of a microservice and its dependencies. With MicroProfile Health, microservices can self-check their health and publish their overall status to a defined endpoint. -xref:microprofile-telemetry.adoc[Enable distributed tracing with MicroProfile Telemetry]:: +xref:microprofile-telemetry.adoc[Enable observability with MicroProfile Telemetry]:: In microservice applications, sources of latency or inaccuracy can be difficult to determine because relationships and dependencies among the constituent services are not always obvious. MicroProfile Telemetry helps you collect data on the paths that application requests take through services. For hands-on tutorials on different observability configurations for Open Liberty, check out our link:/guides/#observability[Observability guides]. - - diff --git a/modules/reference/pages/diff/mp-50-60-diff.adoc b/modules/reference/pages/diff/mp-50-60-diff.adoc index 548c333f81..794339cc49 100644 --- a/modules/reference/pages/diff/mp-50-60-diff.adoc +++ b/modules/reference/pages/diff/mp-50-60-diff.adoc @@ -130,7 +130,7 @@ In MicroProfile 6.0, MicroProfile Telemetry 1.0 replaces MicroProfile OpenTracin === Configuration changes -In addition to replacing the `mpOpenTracing-3.0` feature with the `mpTelemtry-1.0` feature in your `server.xml` file, you must provide configuration to allow the MicroProfile Telemetry feature to connect to your distributed trace service. For more information, see xref:ROOT:microprofile-telemetry.adoc[Enable distributed tracing with MicroProfile Telemetry]. +In addition to replacing the `mpOpenTracing-3.0` feature with the `mpTelemtry-1.0` feature in your `server.xml` file, you must provide configuration to allow the MicroProfile Telemetry feature to connect to your distributed trace service. For more information, see xref:ROOT:microprofile-telemetry.adoc[Enable observability with MicroProfile Telemetry]. You do not need to package a client for your particular tracing service. MicroProfile Telemetry includes exporters for Zipkin, the OpenTelemetry Protocol (OTLP) (which is used by Jaeger since v1.35), and the older Jaeger protocol. If you need to export to a different service, you can provide a custom exporter by using the link:https://www.javadoc.io/doc/io.opentelemetry/opentelemetry-sdk-extension-autoconfigure-spi/1.19.0/io/opentelemetry/sdk/autoconfigure/spi/traces/ConfigurableSpanExporterProvider.html[ConfigurableSpanExporterProvider service provider interface (SPI)]. diff --git a/modules/reference/pages/feature/mpTelemetry/description.adoc b/modules/reference/pages/feature/mpTelemetry/description.adoc index 631db96078..37849acb52 100644 --- a/modules/reference/pages/feature/mpTelemetry/description.adoc +++ b/modules/reference/pages/feature/mpTelemetry/description.adoc @@ -1,3 +1,3 @@ For information about migrating from MicroProfile Open Tracing to MicroProfile Open Telemetry, see xref:diff/mp-50-60-diff.adoc#telemetry[Differences between MicroProfile Telemetry 1.0 and MicroProfile OpenTracing 3.0]. -For information about configuring MicroProfile Telemetry, see xref:ROOT:microprofile-telemetry.adoc[Enable distributed tracing with MicroProfile Telemetry]. \ No newline at end of file +For information about configuring MicroProfile Telemetry, see xref:ROOT:microprofile-telemetry.adoc[Enable observability with MicroProfile Telemetry]. diff --git a/modules/reference/pages/feature/mpTelemetry/examples.adoc b/modules/reference/pages/feature/mpTelemetry/examples.adoc index 0d488ad4a0..2f0ca17cad 100644 --- a/modules/reference/pages/feature/mpTelemetry/examples.adoc +++ b/modules/reference/pages/feature/mpTelemetry/examples.adoc @@ -2,7 +2,7 @@ === Collect logs from specified sources -To enable the MicroProfile Telemetry feature to collect logs from different sources in the Open Liberty runtime environment, configure the `source` attribute for the config:mpTelemetry element with a comma-separated list of log sources. +To enable the MicroProfile Telemetry feature to collect logs from different sources in the Open Liberty runtime environment, configure the `source` attribute for the config:mpTelemetry[] element with a comma-separated list of log sources. [source,xml] ---- From 6d68c293bd1316dc7781f1a923759a31fa74fc28 Mon Sep 17 00:00:00 2001 From: David Mueller Date: Wed, 21 Aug 2024 15:20:49 -0400 Subject: [PATCH 04/62] more edits #7459 --- .../pages/microprofile-config-properties.adoc | 9 +- .../ROOT/pages/microprofile-telemetry.adoc | 204 ++++-------------- modules/ROOT/pages/mptel-log-events-list.adoc | 12 +- modules/ROOT/pages/telemetry-trace.adoc | 176 +++++++++++++++ 4 files changed, 233 insertions(+), 168 deletions(-) create mode 100644 modules/ROOT/pages/telemetry-trace.adoc diff --git a/modules/ROOT/pages/microprofile-config-properties.adoc b/modules/ROOT/pages/microprofile-config-properties.adoc index b737b673bf..2ebf3596f8 100644 --- a/modules/ROOT/pages/microprofile-config-properties.adoc +++ b/modules/ROOT/pages/microprofile-config-properties.adoc @@ -616,6 +616,8 @@ This property or `*/mp-rest/url` is considered required. However, some implement The following MicroProfile Config properties can be specified when MicroProfile Telemetry is enabled. This table lists the most important properties. For the full list, see the https://download.eclipse.org/microprofile/microprofile-telemetry-1.0/tracing/microprofile-telemetry-tracing-spec-1.0.html#_configuration[MicroProfile Telemetry specification]. +For more information about collecting Open Liberty logs, metrics, and traces, see xref:microprofile-telemetry.adoc[Enable observability with MicroProfile Telemetry]. + .MicroProfile Config properties for MicroProfile Telemetry [options="header"] |=== @@ -642,7 +644,7 @@ The following MicroProfile Config properties can be specified when MicroProfile |`otel.exporter.zipkin.endpoint=http://localhost:9411/api/v2/spans` |`otel.sdk.disabled` -|Enables tracing by MicroProfile Telemetry. The default is `true`. +|Enables logs, metrics, and traces to be sent to OpenTelemetry. The default is `true`. |Application startup |`otel.sdk.disabled=false` @@ -651,6 +653,11 @@ The following MicroProfile Config properties can be specified when MicroProfile |Application startup |`otel.service.name=system` +|`otel.logs.exporter` +|You can use this property to change where the logs that MicroProfile Telemetry collects are exported. For more information, see xref:microprofile-telemetry.adoc#logs[Configuring Open Liberty to use MicroProfile Telemetry to collect logs] +|Application startup +|`otel.traces.exporter=console` + |`otel.traces.exporter` |Sets the exporter that is used to collect traces. Possible values are `otlp`, `zipkin`, `jaeger`, or `logging`. The default value is `otlp`. For the Jaeger trace service versions 1.35 and later, the `otlp` exporter value is recommended, rather than `jaeger`. |Application startup diff --git a/modules/ROOT/pages/microprofile-telemetry.adoc b/modules/ROOT/pages/microprofile-telemetry.adoc index 60f116a11e..7f6c3b1465 100644 --- a/modules/ROOT/pages/microprofile-telemetry.adoc +++ b/modules/ROOT/pages/microprofile-telemetry.adoc @@ -22,24 +22,31 @@ MicroProfile Telemetry replaces MicroProfile OpenTracing. For more information a The following sections explain how to prepare your Open Liberty runtime and application code to use MicroProfile Telemetry. -- <<#ol-config, Configuring Open Liberty to use MicroProfile Telemetry>> +- <<#traces, Configuring Open Liberty to use MicroProfile Telemetry>> - <<#code, Code instrumentation>> - <<#trouble, Troubleshooting MicroProfile Telemetry>> -[#ol-config] -== Configuring Open Liberty to use MicroProfile Telemetry +[#global] +== Enabling MicroProfile Telemetry for Open Liberty -To enable MicroProfile Telemetry in your Open Liberty runtime, you must add the MicroProfile Telemetry feature to your `server.xml` file and specify MicroProfile Config properties to configure how MicroProfile Telemetry collects and exports traces. +To enable MicroProfile Telemetry in your Open Liberty runtime, you must add the MicroProfile Telemetry feature to your `server.xml` file and enable the OpenTelemetry SDK by specifying the `otel.sdk.disabled=false` MicroProfile Config property or the `OTEL_SDK_DISABLED=false` environment variable. in any of the xref:external-configuration.adoc#default[config sources that are available to MicroProfile Config]. -. Add a version of the feature:mpTelemetry[display=MicroProfile Telemetry] feature to your `server.xml` file. +. Enable the feature:mpTelemetry[display=MicroProfile Telemetry] feature in your `server.xml` file. To export metrics or logs, you must enable `mpTelemetry-2.0` or later. -. Enable tracing and configure a trace storage system by specifying MicroProfile Config properties. +. OpenTelemetry is disabled by default. To enable the generation of traces, set the `otel.sdk.disabled=false` property. + -You can configure how MicroProfile Telemetry collects and exports traces by specifying configuration properties in any of the xref:external-configuration.adoc#default[config sources that are available to MicroProfile Config]. If you choose to set these configuration properties by using environment variables, make the key name uppercase and convert any punctuation to underscores. For example, the `otel.sdk.disabled=false` property is equivalent to the `OTEL_SDK_DISABLED=false` environment variable. +You can configure how MicroProfile Telemetry collects and exports logs, metrics, and traces by specifying configuration properties in any of the xref:external-configuration.adoc#default[config sources that are available to MicroProfile Config]. Alternatively, if you choose to set these configuration properties by using environment variables, make the key name uppercase and convert any punctuation to underscores. For example, the `otel.sdk.disabled=false` property is equivalent to the `OTEL_SDK_DISABLED=false` environment variable. -.. OpenTelemetry is disabled by default. To enable the generation of traces, set the `otel.sdk.disabled=false` property. +[#traces] +== Configuring Open Liberty to use MicroProfile Telemetry to collect traces + +To enable MicroProfile Telemetry in your Open Liberty runtime, you must add the MicroProfile Telemetry feature to your `server.xml` file, enable the OpenTelemetry SDK. Optionally, you specify MicroProfile Config properties to configure how MicroProfile Telemetry collects and exports traces. + +. Enable the MicroProfile Telemetry feature 2.0 or later and specify a MicroProfile Config property or environment variable to enable the OpenTelemetry SDK. + -.. Configure a trace storage system by specifying an exporter definition that includes the exporter type and the endpoint to connect to. +For more information, see <<#global,Enabling MicroProfile Telemetry for Open Liberty>> + +. Configure a trace storage system by specifying an exporter definition that includes the exporter type and the endpoint to connect to. + For example, to use a Jaeger server, you might add configuration similar to the following example to your `bootstrap.properties` file: + @@ -59,7 +66,7 @@ otel.traces.exporter=zipkin otel.exporter.zipkin.endpoint=http://localhost:9411/api/v2/spans ---- -.. Optionally, set other MicroProfile Config properties to configure trace details. +. Optionally, set other MicroProfile Config properties to configure trace details. + For example, if you want to export traces to Open Liberty log files, set the following property: + @@ -72,173 +79,48 @@ For more information about the available properties, see xref:microprofile-confi . Depending on how you choose to instrument your application code for tracing, further configuration might be required. + -For more information, see the following section. - -[#code] -== Code instrumentation - -After you enable MicroProfile Telemetry and configure a trace server, you can instrument tracing in your application code by using one of the following methods: +For more information, see xref:telemetry-trace.adoc#t[Code instrumentation for MicroProfile Telemetry tracing]. -* <<#auto, Automatic instrumentation>>, which automatically traces your JAX-RS or RESTful web services. -* <<#manual, Manual instrumentation>>, which allows you to start and end telemetry spans manually for applications that are not JAX-RS or RESTful web services. -* <<#agent, Agent instrumentation>>, which automatically adds telemetry to popular open source libraries. However, agent instrumentation also imposes certain <<#limit, limitations>>. +[#logs] +== Configuring Open Liberty to use MicroProfile Telemetry to collect logs -[#auto] -=== Automatic instrumentation - -With automatic instrumentation, you can observe traces without modifying the source code in your applications. All you need to do is configure runtime as described in <<#ol-config,Configuring Open Liberty to use MicroProfile Telemetry>>. However, automatic instrumentation is available only for JAX-RS and Jakarta RESTful web service applications. - -In Open Liberty version 23.0.0.11 and later, spans are automatically generated for incoming HTTP requests, including static files, servlets, and JSPs. - -//// -To start emitting traces with automatic instrumentation, enable the MicroProfile Telemetry feature in your `server.xml` file by adding `mpTelemetry-1.0` or `mpTelemetry-1.1` to your server.xml file. By default, MicroProfile Telemetry tracing is off. To enable tracing, specify the `otel.sdk.disabled=false` MicroProfile Config property and any exporter configuration that your tracing service requires. +To enable MicroProfile Telemetry in your Open Liberty runtime, you must add the MicroProfile Telemetry feature to your `server.xml` file, enable the OpenTelemetry SDK. Optionally, you can specify MicroProfile Config properties to configure how MicroProfile Telemetry collects and exports logs. -For example, to export traces to a Jaeger server with the OpenTelemetry Protocol (OTLP) enabled, add the following entries to your `bootstrap.properties` file. - -[source,properties] ----- -otel.sdk.disabled=false -otel.traces.exporter=otlp -otel.exporter.otlp.endpoint=http://localhost:4317/ ----- +. Enable the MicroProfile Telemetry feature 2.0 or later and specify a MicroProfile Config property or environment variable to enable the OpenTelemetry SDK. ++ +For more information, see <<#global,Enabling MicroProfile Telemetry for Open Liberty>> -To export traces to a Zipkin server, you can use the following properties instead: +. Optionally, configure the log sources that MicroProfile Telemetry collects and where the logs are exported ++ +You can configure how MicroProfile Telemetry collects and exports logs by specifying configuration properties in any of the xref:external-configuration.adoc#default[config sources that are available to MicroProfile Config]. If you choose to set these configuration properties by using environment variables, make the key name uppercase and convert any punctuation to underscores. For example, the `otel.sdk.disabled=false` property is equivalent to the `OTEL_SDK_DISABLED=false` environment variable. +.. Change the log exporter that MicroProfile Telemtry uses. ++ +By default, all OpenTelemetry data is exported to link:https://opentelemetry.io/docs/languages/java/exporters/#otlp[OTLP]. You can change this setting by specifying the `otel.logs.exporter` property. ++ +For example, to send logs to the `console.log` file for debugging purposes, you might add configuration similar to the following example to your `bootstrap.properties` file: ++ [source,properties] ---- otel.sdk.disabled=false -otel.traces.exporter=zipkin -otel.exporter.zipkin.endpoint=http://localhost:9411/api/v2/spans +otel.logs.exporter=console ---- -//// - -[#manual] -=== Manual instrumentation - -Automatic instrumentation is available only for JAX-RS and Jakarta RESTful web service applications. To create spans for other operations, such as database calls, you can add manual instrumentation to the source code for those operations by using the https://www.javadoc.io/doc/io.opentelemetry/opentelemetry-api/1.19.0/io/opentelemetry/api/trace/package-summary.html[OpenTelemetry API]. However, before you manually instrument your code, you must complete the following prerequisites. - -. Enable MicroProfile Telemetry, as described in <<#ol-config,Configuring Open Liberty to use MicroProfile Telemetry>>. - -. xref:class-loader-library-config.adoc#3rd-party[Enable third-party APIs] for your application by adding the following code in your `server.xml` file: + -[source,xml] ----- - - - ----- - -. Add the `opentelemetry` API and OpenTelemetry instrumentation annotations as a provided dependency to your build path. For example, with Maven, add the following code to your `pom.xml` file. +If it is set to `console`, all the logs are exported to standard out (`stdout`) and the `console.log` file. The `console.log` file the contains the usual logs, along with duplicate OpenTelemetry-mapped logs. This setting is only for debugging purposes because the the `console.log`file does not roll over and could affect performance if it gets too large. + -[source,xml] ----- - - io.opentelemetry - opentelemetry-api - 1.19.0 - provided - - - io.opentelemetry.instrumentation - opentelemetry-instrumentation-annotations - 1.19.0-alpha - provided - ----- - -After you complete those prerequisites, you're ready to instrument your code. The following examples show configuration options with the OpenTelemetry API. - -- Add extra information, such as the user ID, to the current span. Any information that you add to a span is visible when you look at traces on your trace server. +.. Configure MicroProfile Telemetry to use the OpenTelemetry Batch LogRecord Processor. + -[source,java] ----- -private static final AttributeKey USER_ID_ATTR = AttributeKey.stringKey("userId"); - -@Inject private Span currentSpan; - -@GET -public String myMethod() { - ... - currentSpan.setAttribute(USER_ID_ATTR, getUserId()); - ... -} ----- - -- Create a subspan around a particular operation, such as querying a database. This subspan shows you how long it took and the order in which it occurred relative to other spans. +By default, the SimpleLogRecordProcessor is enabled, so the records are sent immediately. However, if you want to send the records in batches, you can also configure the following logging specific Batch LogRecord Processor properties: + -[source,java] ----- -@Inject private Tracer tracer; - -@GET -public String myMethod() { - ... - Span newSpan = tracer.spanBuilder("QueryDatabase").startSpan(); - try (Scope s = newSpan.makeCurrent()) { - queryDatabase(); - } finally { - newSpan.end(); - } - ... -} ----- - -- Annotate methods in any Jakarta CDI beans by using the `@WithSpan` annotation. This annotation creates a new Span and establishes any required relationships with the current trace context. You can annotate method parameters with the `@SpanAttribute` annotation to indicate which method parameters are part of the trace, as shown in the following example. +* `otel.blrp.schedule.delay` +* `otel.blrp.max.queue.size` +* `otel.blrp.max.export.batch.size` +* `otel.blrp.export.timeout` + -[source,java] ----- -@ApplicationScoped -class SpanBean { +For more information, see link:https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#batch-logrecord-processor[Batch LogRecord Processor]. - @WithSpan("name") - void spanName() { - ... - } - - @WithSpan - void spanArgs(@SpanAttribute(value = "arg") String arg) { - ... - } -} ----- - -==== Considerations for using manual instrumentation - -The following important considerations apply to manual instrumentation. - -- You must call the `.end()` method on any span you create, otherwise the span is not recorded. -- The current span is used as the parent for any new spans that are created. Therefore, when you create a span, you usually also want to make it current. However, you must close the `Scope` instance that is returned by the `Span.makeCurrent()` method. You can close a `Scope` instance by specifying a try-with-resources block, as shown in the previous example for creating a subspan. -- Because Liberty supports per-application configuration, it does not support `GlobalOpenTelemetry`. Using that class does not produce any telemetry data. -- If you set any properties by using environment variables, including those in the `server.env` file, the keys must be in uppercase and all punctuation must be replaced by an underscore. Values must be written normally. - -For more information, see the https://opentelemetry.io/docs/instrumentation/java/manual[OpenTelemetry manual instrumentation documentation]. However, remember when you use the MicroProfile Telemetry feature in Open Liberty, you must obtain the `OpenTelemetry` and `Tracer` objects by injecting them, not by creating your own. Furthermore, be aware that this documentation includes information for the OpenTelemetry Metrics and Logging APIs, which are not supported by MicroProfile Telemetry. - -[#agent] -=== Agent instrumentation -The https://github.com/open-telemetry/opentelemetry-java-instrumentation[OpenTelemetry Instrumentation for Java] project provides a Java agent JAR file that can be attached to any Java 8+ application. This file dynamically injects bytecode that adds telemetry support to popular open source libraries and frameworks. If you are using any of the https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/docs/supported-libraries.md#libraries--frameworks[supported libraries] in your application, you can use this agent with Open Liberty to instrument them. - -To enable the Java agent on your Open Liberty runtime, https://github.com/open-telemetry/opentelemetry-java-instrumentation#getting-started[download the latest agent version from OpenTelemetry] and add the following line to your `jvm.options` file. - -[subs=+quotes] ----- --javaagent: _path/to/opentelemetry-javaagent.jar_ ----- - -You can configure the agent with https://github.com/open-telemetry/opentelemetry-java-instrumentation#configuring-the-agent[environment variables and system properties]. You can find a list of supported libraries and frameworks in the https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/docs/supported-libraries.md#libraries--frameworks[OpenTelemetry Java instrumentation documentation]. - -[#limit] -==== Limitations of agent instrumentation - -The OpenTelemetry Java agent is a tool that is provided by the OpenTelemetry project. Although it is compatible with Open Liberty, it is a separate project and is subject to the following limitations. +For more information about the available properties, see xref:microprofile-config-properties.adoc#telemetry[MicroProfile Config properties: MicroProfile Telemetry]. -* Configuration works differently when you use the agent. https://opentelemetry.io/docs/instrumentation/java/automatic/agent-config/[Configuration of the agent is well documented], but the following aspects are different from configuration without the agent: - ** Configuration is shared between all applications that are deployed to the server. - ** Configuration properties are only read from system properties and environment variables. They are not read from MicroProfile Config configuration sources. - ** Because the agent reads its configuration early in the startup process, system properties are not read from the `bootstrap.properties` file. Alternatively, you can set system properties in the `jvm.options` file by using the following syntax: `-Dname=value` - ** Implementations of https://www.javadoc.io/doc/io.opentelemetry/opentelemetry-sdk-extension-autoconfigure-spi/1.19.0/index.html[SPI extensions] within applications are ignored. For more information, see the https://opentelemetry.io/docs/instrumentation/java/automatic/agent-config/#extensions[agent documentation for providing SPI extensions]. -* When you use the agent, it takes over the instrumentation of REST calls and methods that are annotated with `@WithSpan`. As a result, the created spans might be slightly different. -* The agent is not compatible with https://www.ibm.com/docs/en/was-liberty/base?topic=security-java-2[Java 2 security]. -* Open Liberty uses many open source libraries internally. Some of these libraries might be automatically instrumented by the agent. [#trouble] == Troubleshooting MicroProfile Telemetry diff --git a/modules/ROOT/pages/mptel-log-events-list.adoc b/modules/ROOT/pages/mptel-log-events-list.adoc index f55ec91751..7540683cc4 100644 --- a/modules/ROOT/pages/mptel-log-events-list.adoc +++ b/modules/ROOT/pages/mptel-log-events-list.adoc @@ -14,7 +14,7 @@ MicroProfile Telemetry can collect logging events from the server runtime and applications. You can use these events to gather and analyze data that can help to better understand the behavior of applications. -For more information, see xref:microprofile-telemetry.adoc[Enable observability with MicroProfile Telemetry] +For more information, see xref:microprofile-telemetry.adoc[Enable observability with MicroProfile Telemetry]. MicroProfile Telemetry can collect the following types of events: @@ -22,15 +22,15 @@ MicroProfile Telemetry can collect the following types of events: * <> * <> -For information about trace log detail levels and how Open Liberty settings map to OpenTelemetry severity numbers, see <>. +For information about trace log detail levels and how Open Liberty settings map to OpenTelemetry severity numbers, see <>. == Message events -The following table provides the fields for message events and a description for each field: +The following table provides the fields for Liberty message events, a description for each field, and the mapping to the OpenTelemetry logs data model and Java agent: .Message event fields [options="header"] |======================= -| Liberty message event fields | Description | Maps to OTel Logs Data Model | OTel Java Agent JUL Mapping +| Liberty message event fields | Description | Maps to OTel logs data model | OTel Java agent (JUL) mapping | type | A string that identifies the type of event. | Attributes[`io.openliberty.type`] | N/A | host | Host name of the server that is the source of the event. | Resource[`host.name`] | Resource[`host.name`] | message | The message from the log record, starting with the message ID. | Body | body @@ -67,7 +67,7 @@ Attributes: ---- == Trace events -The following table provides the fields for trace events and a description for each field: +The following table provides the fields for Liberty trace events, a description for each field, and the mapping to the OpenTelemetry logs data model and Java agent: .Trace event fields [options="header"] @@ -108,7 +108,7 @@ Attributes: ---- == FFDC events -The following table provides the fields for the first failure data capture (FFDC) events and a description for each field: +The following table provides the fields for the first failure data capture (FFDC) events, a description for each fiel, and the mapping to the OpenTelemetry logs data model: .FFDC event fields |======================= diff --git a/modules/ROOT/pages/telemetry-trace.adoc b/modules/ROOT/pages/telemetry-trace.adoc new file mode 100644 index 0000000000..585bec6ed8 --- /dev/null +++ b/modules/ROOT/pages/telemetry-trace.adoc @@ -0,0 +1,176 @@ +// Copyright (c) 2022 IBM Corporation and others. +// Licensed under Creative Commons Attribution-NoDerivatives +// 4.0 International (CC BY-ND 4.0) +// https://creativecommons.org/licenses/by-nd/4.0/ +// +// Contributors: +// IBM Corporation +// +:page-description: +:seo-description: +:page-layout: general-reference +:page-type: general += Code instrumentation for MicroProfile Telemetry tracing + +After you enable MicroProfile Telemetry and configure a trace server, you can instrument tracing in your application code. You can instrument your code automatically, manually, or by using the Java agent. + +* <<#auto, Automatic instrumentation>>, which automatically traces your JAX-RS or RESTful web services. +* <<#manual, Manual instrumentation>>, which allows you to start and end telemetry spans manually for applications that are not JAX-RS or RESTful web services. +* <<#agent, Agent instrumentation>>, which automatically adds telemetry to popular open source libraries. However, agent instrumentation also imposes certain <<#limit, limitations>>. + +[#auto] +== Automatic instrumentation + +With automatic instrumentation, you can observe traces without modifying the source code in your applications. All you need to do is configure runtime as described in <<#ol-config,Configuring Open Liberty to use MicroProfile Telemetry>>. However, automatic instrumentation is available only for JAX-RS and Jakarta RESTful web service applications. + +In Open Liberty version 23.0.0.11 and later, spans are automatically generated for incoming HTTP requests, including static files, servlets, and JSPs. + +//// +To start emitting traces with automatic instrumentation, enable the MicroProfile Telemetry feature in your `server.xml` file by adding `mpTelemetry-1.0` or `mpTelemetry-1.1` to your server.xml file. By default, MicroProfile Telemetry tracing is off. To enable tracing, specify the `otel.sdk.disabled=false` MicroProfile Config property and any exporter configuration that your tracing service requires. + +For example, to export traces to a Jaeger server with the OpenTelemetry Protocol (OTLP) enabled, add the following entries to your `bootstrap.properties` file. + +[source,properties] +---- +otel.sdk.disabled=false +otel.traces.exporter=otlp +otel.exporter.otlp.endpoint=http://localhost:4317/ +---- + +To export traces to a Zipkin server, you can use the following properties instead: + +[source,properties] +---- +otel.sdk.disabled=false +otel.traces.exporter=zipkin +otel.exporter.zipkin.endpoint=http://localhost:9411/api/v2/spans +---- +//// + +[#manual] +== Manual instrumentation + +Automatic instrumentation is available only for JAX-RS and Jakarta RESTful web service applications. To create spans for other operations, such as database calls, you can add manual instrumentation to the source code for those operations by using the https://www.javadoc.io/doc/io.opentelemetry/opentelemetry-api/1.19.0/io/opentelemetry/api/trace/package-summary.html[OpenTelemetry API]. However, before you manually instrument your code, you must complete the following prerequisites. + +. Enable MicroProfile Telemetry, as described in <<#ol-config,Configuring Open Liberty to use MicroProfile Telemetry>>. + +. xref:class-loader-library-config.adoc#3rd-party[Enable third-party APIs] for your application by adding the following code in your `server.xml` file: ++ +[source,xml] +---- + + + +---- + +. Add the `opentelemetry` API and OpenTelemetry instrumentation annotations as a provided dependency to your build path. For example, with Maven, add the following code to your `pom.xml` file. ++ +[source,xml] +---- + + io.opentelemetry + opentelemetry-api + 1.19.0 + provided + + + io.opentelemetry.instrumentation + opentelemetry-instrumentation-annotations + 1.19.0-alpha + provided + +---- + +After you complete those prerequisites, you're ready to instrument your code. The following examples show configuration options with the OpenTelemetry API. + +- Add extra information, such as the user ID, to the current span. Any information that you add to a span is visible when you look at traces on your trace server. ++ +[source,java] +---- +private static final AttributeKey USER_ID_ATTR = AttributeKey.stringKey("userId"); + +@Inject private Span currentSpan; + +@GET +public String myMethod() { + ... + currentSpan.setAttribute(USER_ID_ATTR, getUserId()); + ... +} +---- + +- Create a subspan around a particular operation, such as querying a database. This subspan shows you how long it took and the order in which it occurred relative to other spans. ++ +[source,java] +---- +@Inject private Tracer tracer; + +@GET +public String myMethod() { + ... + Span newSpan = tracer.spanBuilder("QueryDatabase").startSpan(); + try (Scope s = newSpan.makeCurrent()) { + queryDatabase(); + } finally { + newSpan.end(); + } + ... +} +---- + +- Annotate methods in any Jakarta CDI beans by using the `@WithSpan` annotation. This annotation creates a new Span and establishes any required relationships with the current trace context. You can annotate method parameters with the `@SpanAttribute` annotation to indicate which method parameters are part of the trace, as shown in the following example. ++ +[source,java] +---- +@ApplicationScoped +class SpanBean { + + @WithSpan("name") + void spanName() { + ... + } + + @WithSpan + void spanArgs(@SpanAttribute(value = "arg") String arg) { + ... + } +} +---- + +=== Considerations for manual instrumentation + +The following important considerations apply to manual instrumentation. + +- You must call the `.end()` method on any span you create, otherwise the span is not recorded. +- The current span is used as the parent for any new spans that are created. Therefore, when you create a span, you usually also want to make it current. However, you must close the `Scope` instance that is returned by the `Span.makeCurrent()` method. You can close a `Scope` instance by specifying a try-with-resources block, as shown in the previous example for creating a subspan. +- Because Liberty supports per-application configuration, it does not support `GlobalOpenTelemetry`. Using that class does not produce any telemetry data. +- If you set any properties by using environment variables, including those in the `server.env` file, the keys must be in uppercase and all punctuation must be replaced by an underscore. Values must be written normally. + +For more information, see the https://opentelemetry.io/docs/instrumentation/java/manual[OpenTelemetry manual instrumentation documentation]. However, remember when you use the MicroProfile Telemetry feature in Open Liberty, you must obtain the `OpenTelemetry` and `Tracer` objects by injecting them, not by creating your own. Furthermore, be aware that this documentation includes information for the OpenTelemetry Metrics and Logging APIs, which are not supported by MicroProfile Telemetry. + +[#agent] +== Agent instrumentation +The https://github.com/open-telemetry/opentelemetry-java-instrumentation[OpenTelemetry Instrumentation for Java] project provides a Java agent JAR file that can be attached to any Java 8+ application. This file dynamically injects bytecode that adds telemetry support to popular open source libraries and frameworks. If you are using any of the https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/docs/supported-libraries.md#libraries--frameworks[supported libraries] in your application, you can use this agent with Open Liberty to instrument them. + +To enable the Java agent on your Open Liberty runtime, https://github.com/open-telemetry/opentelemetry-java-instrumentation#getting-started[download the latest agent version from OpenTelemetry] and add the following line to your `jvm.options` file. + +[subs=+quotes] +---- +-javaagent: _path/to/opentelemetry-javaagent.jar_ +---- + +You can configure the agent with https://github.com/open-telemetry/opentelemetry-java-instrumentation#configuring-the-agent[environment variables and system properties]. You can find a list of supported libraries and frameworks in the https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/docs/supported-libraries.md#libraries--frameworks[OpenTelemetry Java instrumentation documentation]. + +[#limit] +=== Limitations of agent instrumentation + +The OpenTelemetry Java agent is a tool that is provided by the OpenTelemetry project. Although it is compatible with Open Liberty, it is a separate project and is subject to the following limitations. + +* Configuration works differently when you use the agent. https://opentelemetry.io/docs/instrumentation/java/automatic/agent-config/[Configuration of the agent is well documented], but the following aspects are different from configuration without the agent: + ** Configuration is shared between all applications that are deployed to the server. + ** Configuration properties are only read from system properties and environment variables. They are not read from MicroProfile Config configuration sources. + ** Because the agent reads its configuration early in the startup process, system properties are not read from the `bootstrap.properties` file. Alternatively, you can set system properties in the `jvm.options` file by using the following syntax: `-Dname=value` + ** Implementations of https://www.javadoc.io/doc/io.opentelemetry/opentelemetry-sdk-extension-autoconfigure-spi/1.19.0/index.html[SPI extensions] within applications are ignored. For more information, see the https://opentelemetry.io/docs/instrumentation/java/automatic/agent-config/#extensions[agent documentation for providing SPI extensions]. +* When you use the agent, it takes over the instrumentation of REST calls and methods that are annotated with `@WithSpan`. As a result, the created spans might be slightly different. +* The agent is not compatible with https://www.ibm.com/docs/en/was-liberty/base?topic=security-java-2[Java 2 security]. +* Open Liberty uses many open source libraries internally. Some of these libraries might be automatically instrumented by the agent. From 8fe99eccf33d64f8ead39929ba619e60c5834a36 Mon Sep 17 00:00:00 2001 From: David Mueller Date: Thu, 22 Aug 2024 12:18:38 -0400 Subject: [PATCH 05/62] edits #7459 --- .../ROOT/pages/microprofile-telemetry.adoc | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/modules/ROOT/pages/microprofile-telemetry.adoc b/modules/ROOT/pages/microprofile-telemetry.adoc index 7f6c3b1465..2442b076cd 100644 --- a/modules/ROOT/pages/microprofile-telemetry.adoc +++ b/modules/ROOT/pages/microprofile-telemetry.adoc @@ -22,9 +22,12 @@ MicroProfile Telemetry replaces MicroProfile OpenTracing. For more information a The following sections explain how to prepare your Open Liberty runtime and application code to use MicroProfile Telemetry. -- <<#traces, Configuring Open Liberty to use MicroProfile Telemetry>> -- <<#code, Code instrumentation>> -- <<#trouble, Troubleshooting MicroProfile Telemetry>> + +- <<#global, Enabling MicroProfile Telemetry for Open Liberty>> +- <<#traces, Collecting traces>> +- <<#logs, Collecting logs>> +- <<#metrics, Collecting metrics>> +- <<#trouble, Troubleshooting>> [#global] == Enabling MicroProfile Telemetry for Open Liberty @@ -112,12 +115,12 @@ If it is set to `console`, all the logs are exported to standard out (`stdout`) + By default, the SimpleLogRecordProcessor is enabled, so the records are sent immediately. However, if you want to send the records in batches, you can also configure the following logging specific Batch LogRecord Processor properties: + -* `otel.blrp.schedule.delay` -* `otel.blrp.max.queue.size` -* `otel.blrp.max.export.batch.size` -* `otel.blrp.export.timeout` +* `otel.blrp.schedule.delay` or `OTEL_BLRP_SCHEDULE_DELAY` +* `otel.blrp.max.queue.size` or `OTEL_BLRP_MAX_QUEUE_SIZE` +* `otel.blrp.max.export.batch.size` or `OTEL_BLRP_MAX_EXPORT_BATCH_SIZE` +* `otel.blrp.export.timeout` or `OTEL_BLRP_EXPORT_TIMEOUT` + -For more information, see link:https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#batch-logrecord-processor[Batch LogRecord Processor]. +For more information, see the OpenTelemetry link:https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#batch-logrecord-processor[Batch LogRecord Processor] documentation. For more information about the available properties, see xref:microprofile-config-properties.adoc#telemetry[MicroProfile Config properties: MicroProfile Telemetry]. From 89c52277f1840b212ce2d90d0682625431423c4c Mon Sep 17 00:00:00 2001 From: Ramkumar K Date: Fri, 23 Aug 2024 17:08:40 +0530 Subject: [PATCH 06/62] 7472-Hostname verification-1 7472-Hostname verification-1 #7472 --- .../pages/feature/transportSecurity/examples.adoc | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/modules/reference/pages/feature/transportSecurity/examples.adoc b/modules/reference/pages/feature/transportSecurity/examples.adoc index 797686e2f8..d92623fb87 100644 --- a/modules/reference/pages/feature/transportSecurity/examples.adoc +++ b/modules/reference/pages/feature/transportSecurity/examples.adoc @@ -7,7 +7,7 @@ - <<#configure-specific, Configure specific TLS protocols>> - <<#outbound, Configure outbound TLS>> - <<#certs, Provide certificates from an environment variable or a file>> - +- <<#hostverify, Hostname verification>> [#config] === Configure transport layer security (TLS) @@ -168,3 +168,14 @@ cert_defaultKeyStore="-----BEGIN CERTIFICATE----- .... -----END CERTIFICATE-----" ---- + +[#hostverify] +=== Hostname verification + +Open Liberty performs hostname verification on SSL certificates. If you want to disable the hostname verification functionality, the `verifyHostname` attribute within the `ssl` tag must be initialized to `false`. + +[source,xml] +---- + +---- + From f05d6dd3be8e3f94fa7b87fcf0c475eed58a8b8c Mon Sep 17 00:00:00 2001 From: David Mueller Date: Fri, 23 Aug 2024 12:34:12 -0400 Subject: [PATCH 07/62] add mptelemetry tables #7466 --- modules/ROOT/pages/metrics-list.adoc | 262 ++++++++++++++++++++++++++- 1 file changed, 260 insertions(+), 2 deletions(-) diff --git a/modules/ROOT/pages/metrics-list.adoc b/modules/ROOT/pages/metrics-list.adoc index 3f752c69f3..91d7fe498e 100644 --- a/modules/ROOT/pages/metrics-list.adoc +++ b/modules/ROOT/pages/metrics-list.adoc @@ -30,11 +30,15 @@ The metrics reference tables list and describe all the metrics that are available for Open Liberty. Use metric data to effectively monitor the status of your microservice systems. +The metrics reference tables list and describe all the metrics that are available for Open Liberty with MicroProfile Metrics and MicroProfile Telemetry 2.0 and later. Use metric data to effectively monitor the status of your microservice systems. + +You can obtain metrics from applications, the Open Liberty runtime, and the Java virtual machine (JVM). They can be gathered and stored in database tools, such as [Prometheus](https://prometheus.io/), and displayed on dashboards, such as [Grafana](https://grafana.com/). For more information about building observability into your applications with MicroProfile Metrics, see [Microservice observability with metrics](https://openliberty.io/docs/latest/microservice-observability-metrics.html). For more information about integrating MicroProfile Metrics 5.0 with Micrometer to send metric data to third-party monitoring systems, see [Choose your own monitoring tools with MicroProfile Metrics](https://openliberty.io/docs/latest/micrometer-metrics.html). + You can obtain metrics from applications, the Open Liberty runtime, and the Java virtual machine (JVM). They can be gathered and stored in database tools, such as https://prometheus.io/[Prometheus], and displayed on dashboards, such as https://grafana.com/[Grafana]. For more information about building observability into your applications, see xref:microservice-observability-metrics.adoc[Microservice observability with metrics]. For more information about integrating MicroProfile Metrics 5.0 with Micrometer to send metric data to third-party monitoring systems, see xref:micrometer-metrics.adoc[Choose your own monitoring tools with MicroProfile Metrics]. -== Base and vendor metrics +== MicroProfile Metrics base and vendor metrics When the feature:mpMetrics[display="MicroProfile metrics"] feature is enabled, a set of base metrics is always reported. You can augment this set of metrics by collecting vendor metrics, which are available from different monitoring components within your Open Liberty server. When you enable both the feature:monitor[display="Performance Monitoring"] and the MicroProfile metrics feature, you enable the reporting of vendor metrics on the `/metrics` endpoint. The Performance Monitoring feature retrieves the statistical data from all available monitoring components. The `REST` base metrics that were introduced in MicroProfile Metrics 2.3 also rely on the Performance Monitoring feature. Before MicroProfile Metrics 2.3, the Performance Monitoring feature had to be explicitly configured. In MicroProfile Metrics 2.3, and later, the Performance Monitoring feature is automatically enabled by MicroProfile Metrics during startup. == Filter metrics to gather only the data you need @@ -523,6 +527,14 @@ This metric is a counter. |{grpc-server-metric-features} |{mp-2-3} +|http.server.request.duration +|http_server_request_duration{error_type="",http_request_method="",http_response_status_code="",http_route="",mp_scope="vendor",network_protocol_version="",server_address="",server_port="",url_scheme=""} +|The duration of HTTP server requests. This metric is a timer. /(seconds) +|`HTTP` +| feature:mpMetrics[display=MicroProfile Metrics] +| feature:mpMetrics-5.0[display=MicroProfile Metrics 5.0] + + |jaxws.client.checkedApplicationFaults.total{endpoint=} |jaxws_client_checkedApplicationFaults_total{endpoint="",mp_scope="vendor"} |The number of checked application faults. @@ -662,7 +674,7 @@ This metric is a gauge. |mp_messaging_message_count{channel="",mp_scope="base"} |The number of messages sent on the named channel. |Base metric, but available only when MP Metrics and MP Reactive Messaging features are enabled. -|{connector-metric-features} +|{connector-metric-features} |{mp-4-0} |requestTiming.activeRequestCount @@ -1582,6 +1594,252 @@ This metric is a gauge. |=== +== MicroProfile Telemetry 2.0 and later metrics reference + +OpenTelemetry just sends metric data to compatible OpenTelemetry protocol (OTLP) receivers. Prometheus formatted metrics can be reported by the Open Telemetry collector, but I will replace that column in favor for the attributes (i.e. tags/labels) used. +(Also not sure hot to format a list in the table, so I will list it with commas. +NOTE NOTE: Perhaps we can have "two" tables for HTTP + JVM and the rest. This is because HTTP and JVM actually follow the Open Telemetry HTTP semantic conventions lsited for HTTP and JVM. The others are ones we created for Open liberty. + +The following table lists and describes the metrics that are available for Open Liberty for MicroProfile Metrics 4.0 and earlier. + +// Assisted by watsonx latest genai contribution llama-3-70b +[#telem-table] +.MicroProfile Telemetry 2.0 and later metrics reference +[options="header"cols="3,4a,6a,2,2,2"] +|=== +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] name | Attributes | Type and description | Monitoring component | Features required | Version introduced + +| `http.server.request.duration` +|* `http.request.method` +* `url.scheme` +* `errory.type` (conditionally required) +** `http.response.status_code` +* `http.route` +* `network.protocol.version` +* `server.address` +* `server.port` +| Duration of HTTP server requests. This metric is a Histogram. / (seconds). This histogram has the following explicit bucket boundaries [ 0.005, 0.01, 0.025, 0.05, 0.075, 0.1, 0.25, 0.5, 0.75, 1, 2.5, 5, 7.5, 10 ] +| HTTP +| MicroProfile Telemetry +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.connection_pool.handle.count +| io.openliberty.datasource.name="" +| The number of connections that are in use. This number might include multiple connections that are shared from a single managed connection. This metric is an `ObservableLongUpDownCounter` counter. / ({connection_handle}) +| ConnectionPool +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.connection_pool.connection.created +| io.openliberty.datasource.name="" +| The total number of managed connections that were created since the pool creation. This metric is an `ObservableLongCounter` counter. / ({connection_handle}) +| ConnectionPool +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.connection_pool.connection.destroyed +| io.openliberty.datasource.name="" +| The total number of managed connections that were destroyed since the pool creation. This metric is an `ObservableLongCounter` counter. / ({connection_handle}) +| ConnectionPool +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.connection_pool.connection.free +| io.openliberty.datasource.name="" +| The number of managed connections that are available. This metric is an `ObservableLongUpDownCounter` counter. / ({connection_handle}) +| ConnectionPool +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.connection_pool.connection.use_time +| io.openliberty.datasource.name="" +| The amount of time connections were used for. This metric is an DoubleHistogram / (seconds). This histogram has the following explicit bucket boundaries [ 0.001, 0.005, 0.01, 0.05, 0.1, 0.5, 1, 5, 10] +| ConnectionPool +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.connection_pool.connection.count +| io.openliberty.datasource.name="" +| The current sum of managed connections in the pool. This sum includes managed connections that are available and those that are in use. A single managed connection that is shared by multiple connections only counts once. This metric is an `ObservableLongUpDownCounter` counter / ({connection_handle}) +| ConnectionPool | feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.connection_pool.connection.wait_time +| io.openliberty.datasource.name="" +| The amount of time that connection requests waited for a connection. This metric is an `DoubleHistogram` histogram / (seconds). This histogram has the following explicit bucket boundaries [ 0.001, 0.005, 0.01, 0.05, 0.1, 0.5, 1, 5, 10] +| ConnectionPool +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.request_timing.active +| n/a +| The number of servlet requests that are currently running. This metric is an `ObservableLongUpDownCounter` counter. / ({request}) +| RequestTiming +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.request_timing.hung +| n/a +| The number of servlet requests that are currently hung. This metric is an `ObservableLongUpDownCounter` counter. / ({request}) +| RequestTiming +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.request_timing.processed +| n/a +| The number of servlet requests since the server started. This metric is an `ObservableLongCounter` counter. / ({request}) +| RequestTiming +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.request_timing.slow +| n/a +| The number of servlet requests that are currently running but are slow. This metric is an `ObservableLongUpDownCounter` counter. / ({request}) +| RequestTiming +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.session.active +| io.openliberty.app.name="" +| The number of concurrently active sessions. A session is considered active if the application server is processing a request that uses that user session. This metric is an `ObservableLongUpDownCounter` counter. / ({session}) +| Session +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.session.created +| io.openliberty.app.name="" +| The number of sessions logged in since this metric was enabled. This metric is an `ObservableLongCounter` counter. / ({session}) +| Session +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.session.invalidated +| io.openliberty.app.name="" +| The number of sessions logged out since this metric was enabled. This metric is an `ObservableLongCounter` counter. / ({session}) +| Session +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.session.invalidated_by_timeout +| io.openliberty.app.name="" +| The number of sessions logged out because of a timeout since this metric was enabled. This metric is an `ObservableLongCounter` counter. / ({session}) +| Session +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.session.live +| io.openliberty.app.name="" +| The number of users that are currently logged in. This metric is an `ObservableLongUpDownCounter` counter. / ({session}) +| Session +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.threadpool.active_threads +| io.openliberty.threadpool.name="" +| The number of threads that are actively running tasks. This metric is an `ObservableLongUpDownCounter` counter. / ({thread}) +| ThreadPool +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.threadpool.size +| io.openliberty.threadpool.name="" +| The size of the thread pool. This metric is an `ObservableLongUpDownCounter` counter. / ({thread}) +| ThreadPool +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| jvm.memory.used +| vm.memory.pool.name="" , -jvm.memory.type= +| Measure of memory used. This metric is an `UpDownCounter` counter. /(bytes) +| n/a +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| jvm.memory.committed +| * vm.memory.pool.name="" +* jvm.memory.type= +| Measure of memory committed. This metric is an `UpDownCounter` counter. /(bytes) +| n/a +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| jvm.memory.limit +| * vm.memory.pool.name="" +* jvm.memory.type= +| Measure of max obtainable memory. This metric is an `UpDownCounter` counter. /(bytes) +| n/a +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| jvm.memory.used_after_last_gc +| * vm.memory.pool.name="" +* jvm.memory.type= +| Measure of memory used, as measured after the most recent garbage collection event on this pool. This metric is an `UpDownCounter` counter. /(bytes) +| n/a +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| jvm.gc.duration +| * jvm.gc.action= +* jvm.gc.name= +| Duration of JVM garbage collection actions. This metric is an Histogram. / (seconds). This histogram has the following explicit bucket boundaries [ 0.01, 0.1, 1, 10 ] +| n/a +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| jvm.thread.count +| * *jvm.thread.daemon= +* jvm.thread.state= +| Number of executing platform threads. This metric is an `UpDownCounter` counter. /({thread}) +| n/a +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| jvm.class.loaded +| n/a +| Number of classes loaded since JVM start. This metric is a Counter. / ({class}) +| n/a +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + + +| jvm.class.unloaded +| n/a +| Number of classes unloaded since JVM start. This metric is a Counter. / ({class}) +| n/a +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| jvm.class.count +| n/a +| Number of classes currently loaded. This metric is an `UpDownCounter` counter. /({class}) +| n/a +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| jvm.cpu.time +| n/a +| CPU time used by the process as reported by the JVM. This metric is a Counter +| n/a +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| jvm.cpu.count +| n/a +| Number of processors available to the Java virtual machine. This metric is an `UpDownCounter` counter. /({cpu}) +| n/a +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| jvm.cpu.recent_utilization +| n/a +| Recent CPU utilization for the process as reported by the JVM. This metric is a Gauge. +| n/a +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +|=== + {empty} + From ebea80b3ecea08fd1f6ece0d1336b32a4fe6e081 Mon Sep 17 00:00:00 2001 From: David Mueller Date: Fri, 23 Aug 2024 13:30:13 -0400 Subject: [PATCH 08/62] add info to main topic #7466 --- modules/ROOT/pages/jmx-metrics-list.adoc | 60 +++++++++++++++++++ .../ROOT/pages/microprofile-telemetry.adoc | 21 ++++++- 2 files changed, 80 insertions(+), 1 deletion(-) diff --git a/modules/ROOT/pages/jmx-metrics-list.adoc b/modules/ROOT/pages/jmx-metrics-list.adoc index 756708515f..45970cea2b 100644 --- a/modules/ROOT/pages/jmx-metrics-list.adoc +++ b/modules/ROOT/pages/jmx-metrics-list.adoc @@ -21,6 +21,7 @@ The following sections list the JMX metrics that are available for different mon * <> * <> +* <> * <> * <> * <> @@ -31,6 +32,7 @@ The following sections list the JMX metrics that are available for different mon * <> * <> + [#jvm-stats] == JVM monitoring: JvmStats MXBean You can use the `JvmStats` MXBean to monitor the JVM in Open Liberty. @@ -111,6 +113,64 @@ The object name of the MXBean for these attributes is `WebSphere:type=RequestTim |=== +[#http-stats] +== HTTP request monitoring + +You can use the `HttpServerStats` MXBean to monitor HTTP requests that are made to the Open Liberty server. Performance data is available for each HTTP request made to the server and each unique combination of request method, response status, and HTTP route has its own MXBean. + +The following attributes are available for the `HttpServerStats` MXBean. The object name of the MXBean for these attributes is `WebSphere:type=HttpServerStats,name=*:` + +// Assisted by watsonx latest genai contribution llama-3-70b +.HttpServerStats MXBean attributes +[%header,cols="6,3,12"] +|=== +| MXBean attribute | Units | Description +| Duration +| nanoseconds +| The cumulative duration, in nanoseconds, of the requests made to this combination of request method, response status, and HTTP route + +| `Count ` +| n/a +| The cumulative count of requests made to this combination of request method, response status, and HTTP route + +| `RequestMethod` +| n/a +| The request method used for the request + +| `ResponseStatus` +| n/a +| The response status of the request + +| `HttpRoute` +| n/a +| The HTTP route of the request + +| `Scheme` +| n/a +| The URL Scheme used for the request + +| `NetworkProtocolName` +| n/a +| The network protocol name used for the request + +| `NetworkProtocolVersion` +| n/a +| The network protocol version used for the request + +| `ServerName` +| n/a +| The server name the request was made to + +| `ServerPort` +| n/a +| The server port the request was made to + +| `ErrorType` +| n/a +| Error encountered if it exists +|=== + + [#servlet-stats] == Servlet monitoring: ServletStats MXBean You can use the `ServletStats` MXBean to monitor web applications in Open Liberty. diff --git a/modules/ROOT/pages/microprofile-telemetry.adoc b/modules/ROOT/pages/microprofile-telemetry.adoc index 2442b076cd..32bf5b552a 100644 --- a/modules/ROOT/pages/microprofile-telemetry.adoc +++ b/modules/ROOT/pages/microprofile-telemetry.adoc @@ -97,7 +97,7 @@ For more information, see <<#global,Enabling MicroProfile Telemetry for Open Lib + You can configure how MicroProfile Telemetry collects and exports logs by specifying configuration properties in any of the xref:external-configuration.adoc#default[config sources that are available to MicroProfile Config]. If you choose to set these configuration properties by using environment variables, make the key name uppercase and convert any punctuation to underscores. For example, the `otel.sdk.disabled=false` property is equivalent to the `OTEL_SDK_DISABLED=false` environment variable. -.. Change the log exporter that MicroProfile Telemtry uses. +.. Change the log exporter that MicroProfile Telemetry uses. + By default, all OpenTelemetry data is exported to link:https://opentelemetry.io/docs/languages/java/exporters/#otlp[OTLP]. You can change this setting by specifying the `otel.logs.exporter` property. + @@ -124,6 +124,25 @@ For more information, see the OpenTelemetry link:https://opentelemetry.io/docs/s For more information about the available properties, see xref:microprofile-config-properties.adoc#telemetry[MicroProfile Config properties: MicroProfile Telemetry]. +[#metricss] +== Configuring Open Liberty to use MicroProfile Telemetry to collect logs + +To enable MicroProfile Telemetry to collect and export metrics in your Open Liberty runtime, add the MicroProfile Telemetry 2.0 feature to your `server.xml` file and enable the OpenTelemetry SDK. Optionally, you can specify MicroProfile Config properties to configure how MicroProfile Telemetry collects and exports metrics. + +. Enable the MicroProfile Telemetry feature 2.0 or later and specify a MicroProfile Config property or environment variable to enable the OpenTelemetry SDK. ++ +For more information, see <<#global,Enabling MicroProfile Telemetry for Open Liberty>> ++ +To separately configure multiple applications in a server, you can configure OpenTelemetry with application configuration. However, you cannot collect runtime-level metrics this way. + +. Optionally, change the log exporter that MicroProfile Telemetry uses. ++ +By default, all OpenTelemetry data is exported to link:https://opentelemetry.io/docs/languages/java/exporters/#otlp[OTLP]. You can change this setting by specifying the `otel.metrics.exporter` property or the `OTEL_METRICS_EXPORTER` environment variable. + +. Optionally, change the metric export interval. ++ +By default, metric data is exported at an interval of 60 seconds. To modify the export interval, specify the `otel.metric.export.interval` property or the `OTEL_METRIC_EXPORT_INTERVAL` environment variable. Specify the value in milliseconds. For more information, see link:https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#periodic-exporting-metricreader[Periodic exporter MetricReader] in the OpenTelemetry documentation. + [#trouble] == Troubleshooting MicroProfile Telemetry From 65c52425e5a03661552ce3080a008d1d53e132e5 Mon Sep 17 00:00:00 2001 From: David Mueller Date: Fri, 23 Aug 2024 14:49:22 -0400 Subject: [PATCH 09/62] metrics info #7470 --- .../ROOT/pages/microprofile-telemetry.adoc | 55 ++++++++++++++++++- 1 file changed, 52 insertions(+), 3 deletions(-) diff --git a/modules/ROOT/pages/microprofile-telemetry.adoc b/modules/ROOT/pages/microprofile-telemetry.adoc index 32bf5b552a..e66b0ab3ad 100644 --- a/modules/ROOT/pages/microprofile-telemetry.adoc +++ b/modules/ROOT/pages/microprofile-telemetry.adoc @@ -125,24 +125,69 @@ For more information, see the OpenTelemetry link:https://opentelemetry.io/docs/s For more information about the available properties, see xref:microprofile-config-properties.adoc#telemetry[MicroProfile Config properties: MicroProfile Telemetry]. [#metricss] -== Configuring Open Liberty to use MicroProfile Telemetry to collect logs +== Configuring Open Liberty to use MicroProfile Telemetry to collect metrics To enable MicroProfile Telemetry to collect and export metrics in your Open Liberty runtime, add the MicroProfile Telemetry 2.0 feature to your `server.xml` file and enable the OpenTelemetry SDK. Optionally, you can specify MicroProfile Config properties to configure how MicroProfile Telemetry collects and exports metrics. . Enable the MicroProfile Telemetry feature 2.0 or later and specify a MicroProfile Config property or environment variable to enable the OpenTelemetry SDK. + For more information, see <<#global,Enabling MicroProfile Telemetry for Open Liberty>> -+ -To separately configure multiple applications in a server, you can configure OpenTelemetry with application configuration. However, you cannot collect runtime-level metrics this way. . Optionally, change the log exporter that MicroProfile Telemetry uses. + By default, all OpenTelemetry data is exported to link:https://opentelemetry.io/docs/languages/java/exporters/#otlp[OTLP]. You can change this setting by specifying the `otel.metrics.exporter` property or the `OTEL_METRICS_EXPORTER` environment variable. ++ +For example, to export metrics to Open Liberty log files, specify the following property: ++ +---- +otel.metrics.exporter=logging +---- . Optionally, change the metric export interval. + By default, metric data is exported at an interval of 60 seconds. To modify the export interval, specify the `otel.metric.export.interval` property or the `OTEL_METRIC_EXPORT_INTERVAL` environment variable. Specify the value in milliseconds. For more information, see link:https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#periodic-exporting-metricreader[Periodic exporter MetricReader] in the OpenTelemetry documentation. +To separately configure multiple applications in a server, you can configure OpenTelemetry with application configuration. However, you cannot collect runtime-level metrics this way. + +Depending on how you choose to instrument your application code for metrics, further configuration might be required. + + + +=== Custom metrics + +You can use the metrics API to define custom metrics in your application code, as shown in the following example: + +[source,java] +---- +class WithCounter { + @Inject + Meter meter; + + private LongCounter counter; + + @PostConstruct + public void init() { + counter = meter + .counterBuilder("new_subscriptions") + .setDescription("Number of new subscriptions") + .setUnit("1") + .build(); + } + + void subscribe(String plan) { + counter.add(1, + Attributes.of(AttributeKey.stringKey("plan"), plan)); + } +} +---- + +In this example, `Meter` is used to define an instrument, in this case a Counter. Application code then can record measurement values along with additional attributes. Measurement aggregations are computed separately for each unique combination of attributes. + +For a full list of available metrics, see link:https://opentelemetry.io/docs/specs/otel/metrics/api/#meter-operations[Meter operations] in the OpenTelemetry documentation. + + + + [#trouble] == Troubleshooting MicroProfile Telemetry @@ -172,6 +217,10 @@ You receive the CWMOT5003W message that the application attempted to acquire Mic Review the application to see why it attempted to use MicroProfile Telemetry after it shut down. Actions that might trigger MicroProfile Telemetry include calling a method that is annotated with `@WithSpan` or making a request with a JAX-RS Client or MP Rest Client. +You receive either of the CWMOT5006W or CWMOT5007 warning message that there is conflicting configuration for otel.sdk.disabled:: + +Specify the settings to enable or disable OpenTelemetry instances by using either environment variables or MicroProfile Config sources, but not both. If you see these warnings, the other MicroProfile Config source to look at is your `server.xml` file. + //// + Different versions of the MicroProfile Telemetry feature are compatible with different MicroProfile versions, Jakarta and Java Enterprise Editions, and the Open Liberty umbrella features that support them. Both feature:mpTelemetry-1.0[] and feature:mpTelemetry-1.1[] are compatible with feature:jakartaee-10.0[] and feature:microProfile-6.0[]. However, `mpTelemetry-1.1` is also compatible with the following earlier umbrella features: From c2a07a98948bbf104065f78d660103fb99bd82d0 Mon Sep 17 00:00:00 2001 From: David Mueller Date: Mon, 26 Aug 2024 10:54:17 -0400 Subject: [PATCH 10/62] edits #7469 --- .../ROOT/pages/microprofile-telemetry.adoc | 38 +++++++++---------- modules/ROOT/pages/observability.adoc | 2 +- 2 files changed, 19 insertions(+), 21 deletions(-) diff --git a/modules/ROOT/pages/microprofile-telemetry.adoc b/modules/ROOT/pages/microprofile-telemetry.adoc index e66b0ab3ad..b36d59aaa1 100644 --- a/modules/ROOT/pages/microprofile-telemetry.adoc +++ b/modules/ROOT/pages/microprofile-telemetry.adoc @@ -12,13 +12,9 @@ :page-type: general = Enable observability with MicroProfile Telemetry -In microservice applications, sources of latency or inaccuracy can be difficult to determine because relationships and dependencies among the constituent services are not always obvious. MicroProfile Telemetry helps you collect data on the paths that application requests take through services. +MicroProfile Telemetry helps you collect n and analyze data on the paths that application requests take through services. With MicroProfile Telemetry 2.0 and later, you can manage logs, metrics, and traces in a standardized way by using the OpenTelemetry protocol. -One way to increase observability of an application is by emitting traces. Traces represent requests and consist of multiple spans. A span represents a single operation in a request. It includes a name, time-related data, log messages, and metadata to collect data about what happens during a transaction. - -link:https://projects.eclipse.org/projects/technology.microprofile/releases/microprofile-telemetry-1.0/plan%E2%80%A8%E2%80%A822.0.0.10-bet[MicroProfile Telemetry] is based on the https://opentelemetry.io/[OpenTelemetry project], a collection of open source vendor-independent tools, APIs, and SDKs for creating and managing trace data. You can enable MicroProfile Telemetry for Open Liberty by adding the feature:mpTelemetry[display=MicroProfile Telemetry] feature to your `server.xml` file and configuring the feature to connect to your distributed trace service. - -MicroProfile Telemetry replaces MicroProfile OpenTracing. For more information about migrating your applications from MicroProfile OpenTracing to MicroProfile Telemetry, see xref:reference:diff/mp-50-60-diff.adoc#telemetry[Differences between MicroProfile Telemetry 1.0 and MicroProfile OpenTracing 3.0]. +MicroProfile Telemetry is based on the https://opentelemetry.io/[OpenTelemetry project], a collection of open source vendor-independent tools, APIs, and SDKs for creating and managing trace data. You can enable MicroProfile Telemetry for Open Liberty by adding the feature:mpTelemetry[display=MicroProfile Telemetry] feature to your `server.xml` file and configuring the feature to connect to your distributed trace service. The following sections explain how to prepare your Open Liberty runtime and application code to use MicroProfile Telemetry. @@ -32,7 +28,7 @@ The following sections explain how to prepare your Open Liberty runtime and appl [#global] == Enabling MicroProfile Telemetry for Open Liberty -To enable MicroProfile Telemetry in your Open Liberty runtime, you must add the MicroProfile Telemetry feature to your `server.xml` file and enable the OpenTelemetry SDK by specifying the `otel.sdk.disabled=false` MicroProfile Config property or the `OTEL_SDK_DISABLED=false` environment variable. in any of the xref:external-configuration.adoc#default[config sources that are available to MicroProfile Config]. +To enable MicroProfile Telemetry in your Open Liberty runtime, you must add the MicroProfile Telemetry feature to your `server.xml` file and enable the OpenTelemetry SDK. . Enable the feature:mpTelemetry[display=MicroProfile Telemetry] feature in your `server.xml` file. To export metrics or logs, you must enable `mpTelemetry-2.0` or later. @@ -43,13 +39,15 @@ You can configure how MicroProfile Telemetry collects and exports logs, metrics, [#traces] == Configuring Open Liberty to use MicroProfile Telemetry to collect traces -To enable MicroProfile Telemetry in your Open Liberty runtime, you must add the MicroProfile Telemetry feature to your `server.xml` file, enable the OpenTelemetry SDK. Optionally, you specify MicroProfile Config properties to configure how MicroProfile Telemetry collects and exports traces. +One way to increase observability of an application is by emitting traces. Traces represent requests and consist of multiple spans. A span represents a single operation in a request. It includes a name, time-related data, log messages, and metadata to collect data about what happens during a transaction. -. Enable the MicroProfile Telemetry feature 2.0 or later and specify a MicroProfile Config property or environment variable to enable the OpenTelemetry SDK. +MicroProfile Telemetry replaces MicroProfile OpenTracing. For more information about migrating your applications from MicroProfile OpenTracing to MicroProfile Telemetry, see xref:reference:diff/mp-50-60-diff.adoc#telemetry[Differences between MicroProfile Telemetry 1.0 and MicroProfile OpenTracing 3.0]. + +. Enable the MicroProfile Telemetry feature and specify a MicroProfile Config property or environment variable to enable the OpenTelemetry SDK. + For more information, see <<#global,Enabling MicroProfile Telemetry for Open Liberty>> -. Configure a trace storage system by specifying an exporter definition that includes the exporter type and the endpoint to connect to. +. Configure a trace storage system by specifying an exporter definition that includes the exporter type and endpoint. + For example, to use a Jaeger server, you might add configuration similar to the following example to your `bootstrap.properties` file: + @@ -87,15 +85,15 @@ For more information, see xref:telemetry-trace.adoc#t[Code instrumentation for M [#logs] == Configuring Open Liberty to use MicroProfile Telemetry to collect logs -To enable MicroProfile Telemetry in your Open Liberty runtime, you must add the MicroProfile Telemetry feature to your `server.xml` file, enable the OpenTelemetry SDK. Optionally, you can specify MicroProfile Config properties to configure how MicroProfile Telemetry collects and exports logs. +To enable MicroProfile Telemetry to collect and export logs in your Open Liberty runtime, add the MicroProfile Telemetry feature 2.0 or later to your `server.xml` file and enable the OpenTelemetry SDK. Optionally, you can specify MicroProfile Config properties to configure how MicroProfile Telemetry collects and exports logs. . Enable the MicroProfile Telemetry feature 2.0 or later and specify a MicroProfile Config property or environment variable to enable the OpenTelemetry SDK. + -For more information, see <<#global,Enabling MicroProfile Telemetry for Open Liberty>> +For more information, see <<#global,Enabling MicroProfile Telemetry for Open Liberty>>. -. Optionally, configure the log sources that MicroProfile Telemetry collects and where the logs are exported +. Optionally, configure the log sources that MicroProfile Telemetry collects and where the logs are exported. + -You can configure how MicroProfile Telemetry collects and exports logs by specifying configuration properties in any of the xref:external-configuration.adoc#default[config sources that are available to MicroProfile Config]. If you choose to set these configuration properties by using environment variables, make the key name uppercase and convert any punctuation to underscores. For example, the `otel.sdk.disabled=false` property is equivalent to the `OTEL_SDK_DISABLED=false` environment variable. +You can configure how MicroProfile Telemetry collects and exports logs by specifying configuration properties in any of the xref:external-configuration.adoc#default[config sources that are available to MicroProfile Config], or by specifying environment variables. .. Change the log exporter that MicroProfile Telemetry uses. + @@ -109,11 +107,11 @@ otel.sdk.disabled=false otel.logs.exporter=console ---- + -If it is set to `console`, all the logs are exported to standard out (`stdout`) and the `console.log` file. The `console.log` file the contains the usual logs, along with duplicate OpenTelemetry-mapped logs. This setting is only for debugging purposes because the the `console.log`file does not roll over and could affect performance if it gets too large. +If it is set to `console`, all the logs are exported to standard out (`stdout`) and the `console.log` file. The `console.log` file the contains the usual logs, along with duplicate OpenTelemetry-mapped logs. This setting is only for debugging purposes because the `console.log` file does not roll over and might affect performance if it gets too large. + .. Configure MicroProfile Telemetry to use the OpenTelemetry Batch LogRecord Processor. + -By default, the SimpleLogRecordProcessor is enabled, so the records are sent immediately. However, if you want to send the records in batches, you can also configure the following logging specific Batch LogRecord Processor properties: +By default, the SimpleLogRecordProcessor is enabled, so the records are sent immediately. However, if you want to send the records in batches, you can also configure the following logging-specific Batch LogRecord Processor properties or environment variables: + * `otel.blrp.schedule.delay` or `OTEL_BLRP_SCHEDULE_DELAY` * `otel.blrp.max.queue.size` or `OTEL_BLRP_MAX_QUEUE_SIZE` @@ -124,7 +122,7 @@ For more information, see the OpenTelemetry link:https://opentelemetry.io/docs/s For more information about the available properties, see xref:microprofile-config-properties.adoc#telemetry[MicroProfile Config properties: MicroProfile Telemetry]. -[#metricss] +[#metrics] == Configuring Open Liberty to use MicroProfile Telemetry to collect metrics To enable MicroProfile Telemetry to collect and export metrics in your Open Liberty runtime, add the MicroProfile Telemetry 2.0 feature to your `server.xml` file and enable the OpenTelemetry SDK. Optionally, you can specify MicroProfile Config properties to configure how MicroProfile Telemetry collects and exports metrics. @@ -181,9 +179,9 @@ class WithCounter { } ---- -In this example, `Meter` is used to define an instrument, in this case a Counter. Application code then can record measurement values along with additional attributes. Measurement aggregations are computed separately for each unique combination of attributes. +In this example, `Meter` is used to define an instrument, in this case a Counter. Application code then can record measurement values along with other attributes. Measurement aggregations are computed separately for each unique combination of attributes. -For a full list of available metrics, see link:https://opentelemetry.io/docs/specs/otel/metrics/api/#meter-operations[Meter operations] in the OpenTelemetry documentation. +For a full list of available metrics, see link:https://opentelemetry.io/docs/specs/otel/metrics/api/#meter-operations[Meter operations] in the OpenTelemetry documentation. @@ -217,7 +215,7 @@ You receive the CWMOT5003W message that the application attempted to acquire Mic Review the application to see why it attempted to use MicroProfile Telemetry after it shut down. Actions that might trigger MicroProfile Telemetry include calling a method that is annotated with `@WithSpan` or making a request with a JAX-RS Client or MP Rest Client. -You receive either of the CWMOT5006W or CWMOT5007 warning message that there is conflicting configuration for otel.sdk.disabled:: +You receive either of the CWMOT5006W or CWMOT5007 warning message that conflicting configuration is specified for otel.sdk.disabled:: Specify the settings to enable or disable OpenTelemetry instances by using either environment variables or MicroProfile Config sources, but not both. If you see these warnings, the other MicroProfile Config source to look at is your `server.xml` file. diff --git a/modules/ROOT/pages/observability.adoc b/modules/ROOT/pages/observability.adoc index d8dbeb033e..b520d4c20a 100644 --- a/modules/ROOT/pages/observability.adoc +++ b/modules/ROOT/pages/observability.adoc @@ -27,6 +27,6 @@ xref:health-check-microservices.adoc[Health checks for microservices]:: A health check is a special REST API implementation that you can use to validate the status of a microservice and its dependencies. With MicroProfile Health, microservices can self-check their health and publish their overall status to a defined endpoint. xref:microprofile-telemetry.adoc[Enable observability with MicroProfile Telemetry]:: -In microservice applications, sources of latency or inaccuracy can be difficult to determine because relationships and dependencies among the constituent services are not always obvious. MicroProfile Telemetry helps you collect data on the paths that application requests take through services. +In microservice applications, sources of latency or inaccuracy can be difficult to determine because relationships and dependencies among the constituent services are not always obvious. MicroProfile Telemetry helps you collect data on the paths that application requests take through services. With MicroProfile Telemetry 2.0 and later, you can manage logs, metrics, and traces in a standardized way by using the OpenTelemetry protocol. For hands-on tutorials on different observability configurations for Open Liberty, check out our link:/guides/#observability[Observability guides]. From 6e5ed6c1c92f243457f43bc8a08220df9205a14a Mon Sep 17 00:00:00 2001 From: David Mueller Date: Mon, 26 Aug 2024 11:13:49 -0400 Subject: [PATCH 11/62] edits #7459 #7470 #7466 --- .../pages/introduction-monitoring-metrics.adoc | 14 +++++++++----- modules/ROOT/pages/log-management.adoc | 4 +++- modules/ROOT/pages/log-trace-configuration.adoc | 6 ++++++ modules/ROOT/pages/microprofile-telemetry.adoc | 3 --- 4 files changed, 18 insertions(+), 9 deletions(-) diff --git a/modules/ROOT/pages/introduction-monitoring-metrics.adoc b/modules/ROOT/pages/introduction-monitoring-metrics.adoc index 7b69c970d3..cd61c8b0c7 100644 --- a/modules/ROOT/pages/introduction-monitoring-metrics.adoc +++ b/modules/ROOT/pages/introduction-monitoring-metrics.adoc @@ -22,14 +22,18 @@ Open Liberty uses MicroProfile Metrics to expose metrics that describe the inter Developers can also use the MicroProfile Metrics API to expose metrics from their applications. For more information about adding these metrics to your applications, see xref:microservice-observability-metrics.adoc#add[Adding metrics to your applications] and the link:/guides/microprofile-metrics.html[Providing metrics from a microservice] guide. For a list of all REST endpoint-style metrics that are available for Open Liberty, see the xref:metrics-list.adoc[Metrics reference list]. +[#mptel] +== Collect and export metrics with MicroProfile Telemetry +When you enable the MicroProfile Telemetry feature version 2.0 and later, you can manage your logs, metrics, and traces in a standardized way with the OpenTelemetry protocol. For more information, see xref:microprofile-telemetry[Enable observability with MicroProfile Telemetry]. + [#endpoint] == MicroProfile Metrics and the metrics endpoint -The MicroProfile Metrics feature provides a `/metrics` REST interface that conforms to the MicroProfile Metrics specification. You can access MicroProfile Metrics with Open Liberty by feature:mpMetrics[display=enabling the MicroProfile Metrics feature]. Real-time values of all metrics are available by calling the `/metrics` endpoint, which provides formatted metric data that can be consumed as time-series data by external monitoring tools. +The MicroProfile Metrics feature provides a `/metrics` REST interface that conforms to the MicroProfile Metrics specification. You can access MicroProfile Metrics with Open Liberty by feature:mpMetrics[display=enabling the MicroProfile Metrics feature]. Real-time values of all metrics are available by calling the `/metrics` endpoint, which provides formatted metric data that can be consumed as time-series data by external monitoring tools. When an application is running, you can view metric data from any browser by accessing the `/metrics` endpoint, for example, `\https://localhost:9443/metrics`, where `9443` is the port number for your application. You can narrow the scope of the metrics by using the `scope` query parameter with the `/metrics` endpoint to specify the metric registry scope. For example, to view the metric data for the `base` metric registry, use the `/metrics?scope=base` endpoint. This pattern is used to access all runtime-provided scopes (`base`, `application`, and `vendor`) and any xref:microservice-observability-metrics.adoc#customscope[user-defined metric registry scopes]. For MicroProfile Metrics 4.0 and earlier, only runtime-provided scopes are available and path parameters are used to narrow down metric data. The endpoints are `/metrics/base`, `/metrics/application`, and `/metrics/vendor`. -By default, MicroProfile Metrics provides metrics data in Prometheus format. Prometheus format is a representation of the metrics that is compatible with the https://prometheus.io/[Prometheus monitoring tool], which is an open source metrics scraper, data store, and basic visualization tool. In MicroProfile metrics 4.0 and earlier, JSON format is also available. +By default, MicroProfile Metrics provides metrics data in Prometheus format. Prometheus format is a representation of the metrics that is compatible with the https://prometheus.io/[Prometheus monitoring tool], which is an open source metrics scraper, data store, and basic visualization tool. In MicroProfile metrics 4.0 and earlier, JSON format is also available. The format that each response uses depends on the HTTP accept header of the corresponding request. Prometheus format is returned for requests with a `text/plain` accept header. @@ -37,9 +41,9 @@ In MicroProfile metrics 4.0 and earlier, JSON format is returned for requests wi Metrics endpoints differ in syntax and output between MicroProfile Metrics 5.0 and MicroProfile Metrics 4.0 and earlier. MicroProfile Metrics 5.0 also xref:micrometer-metrics.adoc[uses embedded Micrometer metrics technology] to enable third-party monitoring systems. For more information, see xref:reference:diff/mp-50-60-diff.adoc#metrics[Differences between MicroProfile Metrics 5.0 and 4.0] -=== Metrics endpoints in MicroProfile Metrics 5.0 +=== Metrics endpoints in MicroProfile Metrics 5.0 -MicroProfile Metrics 5.0 does not support JSON-formatted output. Furthermore, syntax for the endpoints URLs uses query string parameters for the scope and metric name values. +MicroProfile Metrics 5.0 does not support JSON-formatted output. Furthermore, syntax for the endpoints URLs uses query string parameters for the scope and metric name values. The following table displays the different endpoints that can be accessed with a `GET` request in MicroProfile Metrics 5.0 to provide metrics in Prometheus format. @@ -64,7 +68,7 @@ The following table displays the different endpoints that can be accessed with a |Returns metrics that match the metric name for the specified scope. |=== -=== Metrics endpoints in MicroProfile Metrics 4.0 and earlier +=== Metrics endpoints in MicroProfile Metrics 4.0 and earlier The following table displays the different endpoints that can be accessed with a GET request in MicroProfile Metrics 4.0 and earlier to provide metrics in Prometheus or JSON format. diff --git a/modules/ROOT/pages/log-management.adoc b/modules/ROOT/pages/log-management.adoc index e1ed71d568..5f5de87e4c 100644 --- a/modules/ROOT/pages/log-management.adoc +++ b/modules/ROOT/pages/log-management.adoc @@ -14,6 +14,8 @@ Logs are a record of events that occur during the operation of your servers. You can use logs to diagnose problems and understand the behavior of applications. +When you enable the MicroProfile Telemetry feature version 2.0 and later, you can manage your logs, metrics, and traces in a standardized way with the OpenTelemetry protocol. For more information, see xref:microprofile-telemetry[Enable observability with MicroProfile Telemetry]. + Open Liberty servers and applications produce various logs that can be used for different forms of analysis. These logs include the message logs, trace logs, first failure data capture (FFDC) logs, xref:audit-logs.adoc[audit logs], and access logs: .Descriptions of logs @@ -27,7 +29,7 @@ Open Liberty servers and applications produce various logs that can be used for |Access logs |Record every HTTP request that is made to the server. These logs are useful for traffic analysis. |=== -For more information, see xref:json-log-events-list.adoc[JSON log events reference list]. +For more information, see xref:json-log-events-list.adoc[JSON log events reference list] and xref:mptel-log-events-list.adoc#[MicroProfile Telemetry log events reference list]. You can aggregate logs from all servers with log analysis tools so that the logs can be easily searched and analyzed. When logs are aggregated, you can use log analysis tools to determine which events occur in proximity to each other, which servers encounter the same problems, and when problems first occur. diff --git a/modules/ROOT/pages/log-trace-configuration.adoc b/modules/ROOT/pages/log-trace-configuration.adoc index ea98bed529..c85e21d9e3 100644 --- a/modules/ROOT/pages/log-trace-configuration.adoc +++ b/modules/ROOT/pages/log-trace-configuration.adoc @@ -33,6 +33,7 @@ You can use the `user.timezone` JVM property to set the time zone for your appli The following sections provide more information about configuring your Open Liberty logs: +* <<#mptel,Logging with MicroProfile Telemetry>> * <<#configuaration,Logging configuration>> * <<#storage,Log file storage management>> * <<#log_formats,Log formats>> @@ -46,6 +47,11 @@ The following sections provide more information about configuring your Open Libe * <<#settings,Configuration settings by source>> * <<#log_details,Trace log detail levels>> +[#mptel] +== Logging with MicroProfile Telemetry + +When you enable the MicroProfile Telemetry feature version 2.0 and later, you can manage your logs, metrics, and traces in a standardized way with the OpenTelemetry protocol. For more information, see xref:microprofile-telemetry[Enable observability with MicroProfile Telemetry]. + [#configuaration] == Logging configuration The logging component can be controlled through the server configuration. The logging component can be fully configured in your `server.xml` file by the `logging` element. However, logging is initialized before the `server.xml` file is processed so configuring logging through the `server.xml` file can result in early log entries that use a different log configuration from later ones. To avoid this problem, you can provide much of the logging configuration in the `boostrap.properties` file and in some cases by using environment variables. Avoid specifying different values for the same configuration property in both the bootstrap.properties and the server.xml file. For more information about logging configuration settings, see the <<#settings,Configuration settings by source>> table. diff --git a/modules/ROOT/pages/microprofile-telemetry.adoc b/modules/ROOT/pages/microprofile-telemetry.adoc index b36d59aaa1..3225b6add6 100644 --- a/modules/ROOT/pages/microprofile-telemetry.adoc +++ b/modules/ROOT/pages/microprofile-telemetry.adoc @@ -184,9 +184,6 @@ In this example, `Meter` is used to define an instrument, in this case a Counter For a full list of available metrics, see link:https://opentelemetry.io/docs/specs/otel/metrics/api/#meter-operations[Meter operations] in the OpenTelemetry documentation. - - - [#trouble] == Troubleshooting MicroProfile Telemetry The following information can help you determine the cause of common problems and error messages. From 1ae4cee5139a4ac198e541b5e9687eda638c48fb Mon Sep 17 00:00:00 2001 From: David Mueller Date: Mon, 26 Aug 2024 11:22:00 -0400 Subject: [PATCH 12/62] Update instanton.adoc --- modules/ROOT/pages/instanton.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/ROOT/pages/instanton.adoc b/modules/ROOT/pages/instanton.adoc index f80d6cd453..485f250db9 100644 --- a/modules/ROOT/pages/instanton.adoc +++ b/modules/ROOT/pages/instanton.adoc @@ -375,7 +375,7 @@ For more information, see the Red Hat documentation for link:https://docs.opensh [#supported-features] == Open Liberty InstantOn supported features -InstantOn supports a subset of Open Liberty features. If a feature is enabled that InstantOn does not support, a failure occurs when you try to take a checkpoint of an application process. InstantOn supports the following Jakarta EE and MicroProfile xref:reference:feature/feature-overview.adoc#conv[convenience features]: +InstantOn supports a subset of Open Liberty features. If a feature is enabled that InstantOn does not support, a failure occurs when you try to take a checkpoint of an application process. InstantOn supports the following Jakarta EE and MicroProfile xref:reference:feature/feature-overview.adoc#conv[convenience features] and all the features that they enable: - Jakarta EE Web Profile versions feature:webProfile-8.0[display=8.0] and later - MicroProfile versions feature:microProfile-4.1[display=4.1] and later From c1232de5b5d98be6b8c9c1b133bfe89fa72b0d96 Mon Sep 17 00:00:00 2001 From: David Mueller Date: Mon, 26 Aug 2024 12:15:44 -0400 Subject: [PATCH 13/62] edits per review #7459 --- .../pages/microprofile-config-properties.adoc | 20 +++++++++++++++++++ .../ROOT/pages/microprofile-telemetry.adoc | 13 +++++++++++- modules/ROOT/pages/mptel-log-events-list.adoc | 9 +++++---- modules/ROOT/pages/observability.adoc | 6 +++--- 4 files changed, 40 insertions(+), 8 deletions(-) diff --git a/modules/ROOT/pages/microprofile-config-properties.adoc b/modules/ROOT/pages/microprofile-config-properties.adoc index 2ebf3596f8..abe0e48f2a 100644 --- a/modules/ROOT/pages/microprofile-config-properties.adoc +++ b/modules/ROOT/pages/microprofile-config-properties.adoc @@ -623,6 +623,26 @@ For more information about collecting Open Liberty logs, metrics, and traces, se |=== |Name |Description |When the runtime reads the property |Example +|`otel.blrp.export.timeout` +|Sets the delay interval in milliseconds between two consecutive exports from the Batch LogRecord Processor. The default is `30000`. +|Application startup +|`otel.blrp.export.timeout=50000` + +|`otel.blrp.schedule.delay` +|Sets the maximum allowed time in milliseconds to export data from the Batch LogRecord Processor. The default is `1000`. +|Application startup +|`otel.blrp.schedule.delay=5000` + +|`otel.blrp.max.queue.size` +|Sets the maximum queue size for the Batch LogRecord Processor. The default is `2048`. +|Application startup +|`otel.blrp.max.queue.size=5000` + +|`otel.blrp.max.export.batch.size` +|Sets the maximum batch size for the Batch LogRecord Processor. The default is `512`. The value must be Must be less than or equal to the value for `otel.blrp.max.queue.size`. +|Application startup +|`otel.blrp.max.export.batch.size=1024` + |`otel.exporter.jaeger.endpoint` |Sets the endpoint for the Jaeger exporter. The default is \http://localhost:14250. |Application startup diff --git a/modules/ROOT/pages/microprofile-telemetry.adoc b/modules/ROOT/pages/microprofile-telemetry.adoc index 3225b6add6..70380d852d 100644 --- a/modules/ROOT/pages/microprofile-telemetry.adoc +++ b/modules/ROOT/pages/microprofile-telemetry.adoc @@ -91,6 +91,17 @@ To enable MicroProfile Telemetry to collect and export logs in your Open Liberty + For more information, see <<#global,Enabling MicroProfile Telemetry for Open Liberty>>. +. Optionally, enable the MicroProfile Telemetry feature to collect logs from different sources in the Open Liberty runtime environment. ++ +Configure the `source` attribute for the `mpTelemetry` element with a comma-separated list of comma-separated log sources: ++ +[source,xml] +---- + +---- ++ +The `mpTelemetry` configuration element is optional. If you do not specify it, or if you do not include the `source` attribute, the default configuration source is `message`. For more information, see feature:mpTelemetry-2.0[display=Collect logs from a specified source]. + . Optionally, configure the log sources that MicroProfile Telemetry collects and where the logs are exported. + You can configure how MicroProfile Telemetry collects and exports logs by specifying configuration properties in any of the xref:external-configuration.adoc#default[config sources that are available to MicroProfile Config], or by specifying environment variables. @@ -107,7 +118,7 @@ otel.sdk.disabled=false otel.logs.exporter=console ---- + -If it is set to `console`, all the logs are exported to standard out (`stdout`) and the `console.log` file. The `console.log` file the contains the usual logs, along with duplicate OpenTelemetry-mapped logs. This setting is only for debugging purposes because the `console.log` file does not roll over and might affect performance if it gets too large. +If you set this property to `console`, all the logs are exported to standard out (`stdout`) and the `console.log` file. The `console.log` file contains the usual logs, along with duplicate OpenTelemetry-mapped logs. This setting is only for debugging purposes because the `console.log` file does not roll over and might affect performance if it gets too large. If you set this property to `none`, no logs are exported + .. Configure MicroProfile Telemetry to use the OpenTelemetry Batch LogRecord Processor. + diff --git a/modules/ROOT/pages/mptel-log-events-list.adoc b/modules/ROOT/pages/mptel-log-events-list.adoc index 7540683cc4..bef1c703bf 100644 --- a/modules/ROOT/pages/mptel-log-events-list.adoc +++ b/modules/ROOT/pages/mptel-log-events-list.adoc @@ -30,7 +30,7 @@ The following table provides the fields for Liberty message events, a descriptio .Message event fields [options="header"] |======================= -| Liberty message event fields | Description | Maps to OTel logs data model | OTel Java agent (JUL) mapping +| Liberty message event fields | Description | Maps to OpenTelemetry logs data model | OpenTelemetry Java agent (JUL) mapping | type | A string that identifies the type of event. | Attributes[`io.openliberty.type`] | N/A | host | Host name of the server that is the source of the event. | Resource[`host.name`] | Resource[`host.name`] | message | The message from the log record, starting with the message ID. | Body | body @@ -72,7 +72,7 @@ The following table provides the fields for Liberty trace events, a description .Trace event fields [options="header"] |======================= -| Liberty trace event fields | Description | Maps to OTel Logs Data Model | OTel Java Agent JUL Mapping +| Liberty trace event fields | Description | Maps to OpenTelemetry Logs Data Model | OpenTelemetry Java Agent JUL Mapping | type | A string that identifies the type of event. | Attributes[`io.openliberty.type`] | N/A | host | Host name of the server that is the source of the event. | Resource[`host.name`] | Resource[`host.name`] | message | The message from the log record, starting with the message ID. | Body | body @@ -111,8 +111,9 @@ Attributes: The following table provides the fields for the first failure data capture (FFDC) events, a description for each fiel, and the mapping to the OpenTelemetry logs data model: .FFDC event fields +[options="header"] |======================= -| Liberty FFDC event fields | Description | Maps to OTel Logs Data Model +| Liberty FFDC event fields | Description | Maps to OpenTelemetry Logs Data Model | type | A string that identifies the type of event. | Attributes[`io.openliberty.type`] | host | Host name of the server that is the source of the event. | Resource[`host.name`] | ibm_datetime | Time at which the event occurred. | TimeStamp @@ -151,7 +152,7 @@ Attributes: [#sevlev] == OpenTelemetry severity number mapping with Open Liberty log levels -You can set the level of detail that your Open Liberty logs collect by configuring the `tracSpecification` attribute for the config:logging[] attribute in your `server.xml` file. For more information about Liberty log levels, see xref:log-trace-configuration.adoc#log_details[Trace log detail levels]. +You can set the level of detail that your Open Liberty logs collect by configuring the `traceSpecification` attribute for the config:logging[] attribute in your `server.xml` file. For more information about Liberty log levels, see xref:log-trace-configuration.adoc#log_details[Trace log detail levels]. The following table describes how the different Liberty log levels map to severity numbers and short names from the OpenTelemetry specification. diff --git a/modules/ROOT/pages/observability.adoc b/modules/ROOT/pages/observability.adoc index b520d4c20a..178f74097f 100644 --- a/modules/ROOT/pages/observability.adoc +++ b/modules/ROOT/pages/observability.adoc @@ -17,6 +17,9 @@ When applications are observable, operations teams can identify and understand t The following topics provide information to help you develop observable applications with Open Liberty. +xref:microprofile-telemetry.adoc[Enable observability with MicroProfile Telemetry]:: +In microservice applications, sources of latency or inaccuracy can be difficult to determine because relationships and dependencies among the constituent services are not always obvious. MicroProfile Telemetry helps you collect data on the paths that application requests take through services. With MicroProfile Telemetry 2.0 and later, you can manage logs, metrics, and traces in a standardized way by using the OpenTelemetry protocol. + xref:microservice-observability-metrics.adoc[Microservice observability with metrics]:: Open Liberty uses MicroProfile Metrics to provide metrics that describe the internal state of many Open Liberty components. MicroProfile Metrics provides a `/metrics` endpoint from which you can access all metrics that are emitted by an Open Liberty server and deployed applications. @@ -26,7 +29,4 @@ The Open Liberty MicroProfile Metrics 5.0 feature uses embedded Micrometer metri xref:health-check-microservices.adoc[Health checks for microservices]:: A health check is a special REST API implementation that you can use to validate the status of a microservice and its dependencies. With MicroProfile Health, microservices can self-check their health and publish their overall status to a defined endpoint. -xref:microprofile-telemetry.adoc[Enable observability with MicroProfile Telemetry]:: -In microservice applications, sources of latency or inaccuracy can be difficult to determine because relationships and dependencies among the constituent services are not always obvious. MicroProfile Telemetry helps you collect data on the paths that application requests take through services. With MicroProfile Telemetry 2.0 and later, you can manage logs, metrics, and traces in a standardized way by using the OpenTelemetry protocol. - For hands-on tutorials on different observability configurations for Open Liberty, check out our link:/guides/#observability[Observability guides]. From 44a57ae2273c9d9609278f408066219a118714a2 Mon Sep 17 00:00:00 2001 From: David Mueller Date: Mon, 26 Aug 2024 12:16:36 -0400 Subject: [PATCH 14/62] Update microprofile-config-properties.adoc --- modules/ROOT/pages/microprofile-config-properties.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/ROOT/pages/microprofile-config-properties.adoc b/modules/ROOT/pages/microprofile-config-properties.adoc index abe0e48f2a..d2bb536459 100644 --- a/modules/ROOT/pages/microprofile-config-properties.adoc +++ b/modules/ROOT/pages/microprofile-config-properties.adoc @@ -624,7 +624,7 @@ For more information about collecting Open Liberty logs, metrics, and traces, se |Name |Description |When the runtime reads the property |Example |`otel.blrp.export.timeout` -|Sets the delay interval in milliseconds between two consecutive exports from the Batch LogRecord Processor. The default is `30000`. +|Sets the delay interval in milliseconds between two consecutive exports from the link:https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#batch-logrecord-processor[Batch LogRecord Processor]. The default is `30000`. |Application startup |`otel.blrp.export.timeout=50000` From ece8caf863695bf7ae9e68ce201bc47ca712e9e0 Mon Sep 17 00:00:00 2001 From: David Mueller Date: Mon, 26 Aug 2024 12:17:45 -0400 Subject: [PATCH 15/62] Update log-trace-configuration.adoc --- modules/ROOT/pages/log-trace-configuration.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/ROOT/pages/log-trace-configuration.adoc b/modules/ROOT/pages/log-trace-configuration.adoc index c85e21d9e3..dbfd064b72 100644 --- a/modules/ROOT/pages/log-trace-configuration.adoc +++ b/modules/ROOT/pages/log-trace-configuration.adoc @@ -613,7 +613,7 @@ If you specify a value of `false`, the date and time are formatted according to [#log_details] == Trace log detail levels -You can specify the `traceSpecification` attribute for the logging element in your `server.xml` file to selectively enable trace and set the trace log detail level for specific components in your server configuration. +You can specify the `traceSpecification` attribute for the `logging` element in your `server.xml` file to selectively enable trace and set the trace log detail level for specific components in your server configuration. The following example specifies a trace specification for two groups of components. [source,xml] From 5affbde0959296141a810fffe23095b05fb5cbd6 Mon Sep 17 00:00:00 2001 From: David Mueller Date: Mon, 26 Aug 2024 13:25:57 -0400 Subject: [PATCH 16/62] edits #7459 --- modules/ROOT/pages/microprofile-telemetry.adoc | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/modules/ROOT/pages/microprofile-telemetry.adoc b/modules/ROOT/pages/microprofile-telemetry.adoc index 70380d852d..a36fc209ff 100644 --- a/modules/ROOT/pages/microprofile-telemetry.adoc +++ b/modules/ROOT/pages/microprofile-telemetry.adoc @@ -14,7 +14,7 @@ MicroProfile Telemetry helps you collect n and analyze data on the paths that application requests take through services. With MicroProfile Telemetry 2.0 and later, you can manage logs, metrics, and traces in a standardized way by using the OpenTelemetry protocol. -MicroProfile Telemetry is based on the https://opentelemetry.io/[OpenTelemetry project], a collection of open source vendor-independent tools, APIs, and SDKs for creating and managing trace data. You can enable MicroProfile Telemetry for Open Liberty by adding the feature:mpTelemetry[display=MicroProfile Telemetry] feature to your `server.xml` file and configuring the feature to connect to your distributed trace service. +MicroProfile Telemetry is based on the https://opentelemetry.io/[OpenTelemetry project], a collection of open source vendor-independent tools, APIs, and SDKs for creating and managing logs, metrics, and trace data. The following sections explain how to prepare your Open Liberty runtime and application code to use MicroProfile Telemetry. @@ -91,7 +91,11 @@ To enable MicroProfile Telemetry to collect and export logs in your Open Liberty + For more information, see <<#global,Enabling MicroProfile Telemetry for Open Liberty>>. -. Optionally, enable the MicroProfile Telemetry feature to collect logs from different sources in the Open Liberty runtime environment. +. Optionally, configure the log sources that MicroProfile Telemetry collects and where the logs are exported. ++ +You can configure how MicroProfile Telemetry collects and exports logs by specifying configuration properties in any of the xref:external-configuration.adoc#default[config sources that are available to MicroProfile Config], or by specifying environment variables. + +.. Enable MicroProfile Telemetry to collect logs from different sources in the Open Liberty runtime environment. + Configure the `source` attribute for the `mpTelemetry` element with a comma-separated list of comma-separated log sources: + @@ -102,10 +106,6 @@ Configure the `source` attribute for the `mpTelemetry` element with a comma-sepa + The `mpTelemetry` configuration element is optional. If you do not specify it, or if you do not include the `source` attribute, the default configuration source is `message`. For more information, see feature:mpTelemetry-2.0[display=Collect logs from a specified source]. -. Optionally, configure the log sources that MicroProfile Telemetry collects and where the logs are exported. -+ -You can configure how MicroProfile Telemetry collects and exports logs by specifying configuration properties in any of the xref:external-configuration.adoc#default[config sources that are available to MicroProfile Config], or by specifying environment variables. - .. Change the log exporter that MicroProfile Telemetry uses. + By default, all OpenTelemetry data is exported to link:https://opentelemetry.io/docs/languages/java/exporters/#otlp[OTLP]. You can change this setting by specifying the `otel.logs.exporter` property. @@ -118,7 +118,7 @@ otel.sdk.disabled=false otel.logs.exporter=console ---- + -If you set this property to `console`, all the logs are exported to standard out (`stdout`) and the `console.log` file. The `console.log` file contains the usual logs, along with duplicate OpenTelemetry-mapped logs. This setting is only for debugging purposes because the `console.log` file does not roll over and might affect performance if it gets too large. If you set this property to `none`, no logs are exported +If you set this property to `console`, all the logs are exported to standard out (`stdout`) and the `console.log` file. The `console.log` file contains the usual logs, along with duplicate OpenTelemetry-mapped logs. This setting is only for debugging purposes because the `console.log` file does not roll over and might affect performance if it gets too large. If you set this property to `none`, no logs are exported to OpenTelemetry. + .. Configure MicroProfile Telemetry to use the OpenTelemetry Batch LogRecord Processor. + From cc322907e50eb6e2186fceecc07fbf7256fdaf77 Mon Sep 17 00:00:00 2001 From: David Mueller Date: Mon, 26 Aug 2024 13:47:47 -0400 Subject: [PATCH 17/62] edits #7466 --- modules/ROOT/pages/jmx-metrics-list.adoc | 3 ++- modules/ROOT/pages/metrics-list.adoc | 21 +++++++-------------- 2 files changed, 9 insertions(+), 15 deletions(-) diff --git a/modules/ROOT/pages/jmx-metrics-list.adoc b/modules/ROOT/pages/jmx-metrics-list.adoc index 45970cea2b..8dc9f29047 100644 --- a/modules/ROOT/pages/jmx-metrics-list.adoc +++ b/modules/ROOT/pages/jmx-metrics-list.adoc @@ -32,6 +32,7 @@ The following sections list the JMX metrics that are available for different mon * <> * <> +When you enable the feature:mpTelemetry-2.0[display=MicroProfile Telemetry] feature version 2.0 and later, you can manage your logs, metrics, and traces in a standardized way with the OpenTelemetry protocol. For more information, see xref:microprofile-telemetry[Enable observability with MicroProfile Telemetry]. [#jvm-stats] == JVM monitoring: JvmStats MXBean @@ -129,7 +130,7 @@ The following attributes are available for the `HttpServerStats` MXBean. The obj | nanoseconds | The cumulative duration, in nanoseconds, of the requests made to this combination of request method, response status, and HTTP route -| `Count ` +| `Count` | n/a | The cumulative count of requests made to this combination of request method, response status, and HTTP route diff --git a/modules/ROOT/pages/metrics-list.adoc b/modules/ROOT/pages/metrics-list.adoc index 91d7fe498e..4f83bc3d42 100644 --- a/modules/ROOT/pages/metrics-list.adoc +++ b/modules/ROOT/pages/metrics-list.adoc @@ -32,11 +32,9 @@ Use metric data to effectively monitor the status of your microservice systems. The metrics reference tables list and describe all the metrics that are available for Open Liberty with MicroProfile Metrics and MicroProfile Telemetry 2.0 and later. Use metric data to effectively monitor the status of your microservice systems. -You can obtain metrics from applications, the Open Liberty runtime, and the Java virtual machine (JVM). They can be gathered and stored in database tools, such as [Prometheus](https://prometheus.io/), and displayed on dashboards, such as [Grafana](https://grafana.com/). For more information about building observability into your applications with MicroProfile Metrics, see [Microservice observability with metrics](https://openliberty.io/docs/latest/microservice-observability-metrics.html). For more information about integrating MicroProfile Metrics 5.0 with Micrometer to send metric data to third-party monitoring systems, see [Choose your own monitoring tools with MicroProfile Metrics](https://openliberty.io/docs/latest/micrometer-metrics.html). +You can obtain metrics from applications, the Open Liberty runtime, and the Java virtual machine (JVM). They can be gathered and stored in database tools, such as [Prometheus](https://prometheus.io/), and displayed on dashboards, such as [Grafana](https://grafana.com/). For more information about building observability into your applications with MicroProfile Metrics, see xref:microservice-observability-metrics.adoc[Microservice observability with metrics]. For more information about integrating MicroProfile Metrics 5.0 with Micrometer to send metric data to third-party monitoring systems, see xref:micrometer-metrics.adoc[Choose your own monitoring tools with MicroProfile Metrics][Choose your own monitoring tools with MicroProfile Metrics]. -You can obtain metrics from applications, the Open Liberty runtime, and the Java virtual machine (JVM). -They can be gathered and stored in database tools, such as https://prometheus.io/[Prometheus], and displayed on dashboards, such as https://grafana.com/[Grafana]. -For more information about building observability into your applications, see xref:microservice-observability-metrics.adoc[Microservice observability with metrics]. For more information about integrating MicroProfile Metrics 5.0 with Micrometer to send metric data to third-party monitoring systems, see xref:micrometer-metrics.adoc[Choose your own monitoring tools with MicroProfile Metrics]. +When you enable the feature:mpTelemetry-2.0[display=MicroProfile Telemetry] feature version 2.0 and later, you can manage your logs, metrics, and traces in a standardized way with the OpenTelemetry protocol. For more information, see xref:microprofile-telemetry[Enable observability with MicroProfile Telemetry]. == MicroProfile Metrics base and vendor metrics When the feature:mpMetrics[display="MicroProfile metrics"] feature is enabled, a set of base metrics is always reported. You can augment this set of metrics by collecting vendor metrics, which are available from different monitoring components within your Open Liberty server. When you enable both the feature:monitor[display="Performance Monitoring"] and the MicroProfile metrics feature, you enable the reporting of vendor metrics on the `/metrics` endpoint. The Performance Monitoring feature retrieves the statistical data from all available monitoring components. The `REST` base metrics that were introduced in MicroProfile Metrics 2.3 also rely on the Performance Monitoring feature. Before MicroProfile Metrics 2.3, the Performance Monitoring feature had to be explicitly configured. In MicroProfile Metrics 2.3, and later, the Performance Monitoring feature is automatically enabled by MicroProfile Metrics during startup. @@ -67,15 +65,14 @@ To disable all monitoring components, add the following code to your `server.xml == Metrics reference tables -The following two tables list the available metrics when you use <<#metrics-table-mp-metrics-5, MicroProfile Metrics 5.0>> or <<#metrics-table,MicroProfile Metrics 4.0 (or earlier versions)>>. -In both tables, the Prometheus metric names are listed after each metric. +The following tables list the available metrics when you use <<#metrics-table-mp-metrics-5, MicroProfile Metrics 5.0>>, <<#metrics-table,MicroProfile Metrics 4.0 (or earlier versions)>>, or <<#telem-table, MicroProfile Telemetry 2.0 and later>>. In each table, the Prometheus metric names are listed after each metric. The tables also list the metric types, metric units, and descriptions of all metrics that are available for Open Liberty. For the vendor metrics, the associated monitoring component that you can use to filter the metric is also included. The **Features required** column of the table includes the feature or features that must be enabled to obtain that metric data. The **Version introduced** column specifies the minimum version of the feature that you must enable to collect the metric. === Units in the metrics reference tables -For both tables, metric units are included along with both the metric names and Prometheus names. In MicroProfile 4.0 and earlier, these units can differ because the Prometheus exporter scales metrics to a base unit. +Metric units are included along with both the metric names and Prometheus names. In MicroProfile 4.0 and earlier, these units can differ because the Prometheus exporter scales metrics to a base unit. For example, while the `ft..bulkhead.executionDuration` metric is recorded internally in nanoseconds, the Prometheus output is in seconds. In MicroProfile Metrics 5.0, the unit that is associated with the metric is what is reflected in the Prometheus output. The metrics are not scaled to a base unit. @@ -1596,18 +1593,14 @@ This metric is a gauge. == MicroProfile Telemetry 2.0 and later metrics reference -OpenTelemetry just sends metric data to compatible OpenTelemetry protocol (OTLP) receivers. Prometheus formatted metrics can be reported by the Open Telemetry collector, but I will replace that column in favor for the attributes (i.e. tags/labels) used. -(Also not sure hot to format a list in the table, so I will list it with commas. -NOTE NOTE: Perhaps we can have "two" tables for HTTP + JVM and the rest. This is because HTTP and JVM actually follow the Open Telemetry HTTP semantic conventions lsited for HTTP and JVM. The others are ones we created for Open liberty. - -The following table lists and describes the metrics that are available for Open Liberty for MicroProfile Metrics 4.0 and earlier. +When the MicroProfile Telemetry feature 2.0 and later is enabled, Open Liberty can sends metric data to compatible OpenTelemetry protocol (OTLP) receivers. For more information, see xref:microprofile-telemetry[Enable observability with MicroProfile Telemetry]. // Assisted by watsonx latest genai contribution llama-3-70b [#telem-table] .MicroProfile Telemetry 2.0 and later metrics reference [options="header"cols="3,4a,6a,2,2,2"] |=== -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] name | Attributes | Type and description | Monitoring component | Features required | Version introduced +| MicroProfile Telemetry 2.0 name | Attributes | Type and description | Monitoring component | Features required | Version introduced | `http.server.request.duration` |* `http.request.method` @@ -1620,7 +1613,7 @@ The following table lists and describes the metrics that are available for Open * `server.port` | Duration of HTTP server requests. This metric is a Histogram. / (seconds). This histogram has the following explicit bucket boundaries [ 0.005, 0.01, 0.025, 0.05, 0.075, 0.1, 0.25, 0.5, 0.75, 1, 2.5, 5, 7.5, 10 ] | HTTP -| MicroProfile Telemetry +| feature:mpTelemetry[display=MicroProfile Telemetry] | feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] | io.openliberty.connection_pool.handle.count From 0b1cc6685a77ef24591fa5f2e186e2a739ec366f Mon Sep 17 00:00:00 2001 From: David Mueller Date: Mon, 26 Aug 2024 14:42:01 -0400 Subject: [PATCH 18/62] edits #7466 --- modules/ROOT/pages/metrics-list.adoc | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/modules/ROOT/pages/metrics-list.adoc b/modules/ROOT/pages/metrics-list.adoc index 4f83bc3d42..1ccdab14a8 100644 --- a/modules/ROOT/pages/metrics-list.adoc +++ b/modules/ROOT/pages/metrics-list.adoc @@ -32,7 +32,7 @@ Use metric data to effectively monitor the status of your microservice systems. The metrics reference tables list and describe all the metrics that are available for Open Liberty with MicroProfile Metrics and MicroProfile Telemetry 2.0 and later. Use metric data to effectively monitor the status of your microservice systems. -You can obtain metrics from applications, the Open Liberty runtime, and the Java virtual machine (JVM). They can be gathered and stored in database tools, such as [Prometheus](https://prometheus.io/), and displayed on dashboards, such as [Grafana](https://grafana.com/). For more information about building observability into your applications with MicroProfile Metrics, see xref:microservice-observability-metrics.adoc[Microservice observability with metrics]. For more information about integrating MicroProfile Metrics 5.0 with Micrometer to send metric data to third-party monitoring systems, see xref:micrometer-metrics.adoc[Choose your own monitoring tools with MicroProfile Metrics][Choose your own monitoring tools with MicroProfile Metrics]. +You can obtain metrics from applications, the Open Liberty runtime, and the Java virtual machine (JVM). They can be gathered and stored in database tools, such as link:https://prometheus.io/[Prometheus], and displayed on dashboards, such as link:https://grafana.com/[Grafana]. For more information about building observability into your applications with MicroProfile Metrics, see xref:microservice-observability-metrics.adoc[Microservice observability with metrics]. For more information about integrating MicroProfile Metrics 5.0 with Micrometer to send metric data to third-party monitoring systems, see xref:micrometer-metrics.adoc[Choose your own monitoring tools with MicroProfile Metrics]. When you enable the feature:mpTelemetry-2.0[display=MicroProfile Telemetry] feature version 2.0 and later, you can manage your logs, metrics, and traces in a standardized way with the OpenTelemetry protocol. For more information, see xref:microprofile-telemetry[Enable observability with MicroProfile Telemetry]. @@ -65,7 +65,13 @@ To disable all monitoring components, add the following code to your `server.xml == Metrics reference tables -The following tables list the available metrics when you use <<#metrics-table-mp-metrics-5, MicroProfile Metrics 5.0>>, <<#metrics-table,MicroProfile Metrics 4.0 (or earlier versions)>>, or <<#telem-table, MicroProfile Telemetry 2.0 and later>>. In each table, the Prometheus metric names are listed after each metric. +The metrics reference tables list the available metrics when you use one of the following solutions: + +* <<#metrics-table-mp-metrics-5, MicroProfile Metrics 5.0>> +* <<#metrics-table,MicroProfile Metrics 4.0 (or earlier versions)>> +* <<#telem-table, MicroProfile Telemetry 2.0 and later>> + +In each table, the Prometheus metric names are listed after each metric. The tables also list the metric types, metric units, and descriptions of all metrics that are available for Open Liberty. For the vendor metrics, the associated monitoring component that you can use to filter the metric is also included. The **Features required** column of the table includes the feature or features that must be enabled to obtain that metric data. The **Version introduced** column specifies the minimum version of the feature that you must enable to collect the metric. From b703e6372358b836cc7440e32c70be2e5b7a2402 Mon Sep 17 00:00:00 2001 From: Ramkumar K Date: Tue, 27 Aug 2024 22:13:51 +0530 Subject: [PATCH 19/62] 7496-Document the image refresh cycle-1 7496-Document the image refresh cycle-1 #7496 --- modules/ROOT/pages/container-images.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/ROOT/pages/container-images.adoc b/modules/ROOT/pages/container-images.adoc index 64c1a46de2..cbf5b36804 100644 --- a/modules/ROOT/pages/container-images.adoc +++ b/modules/ROOT/pages/container-images.adoc @@ -38,7 +38,7 @@ Select from the following container image tags for the latest Open Liberty relea These tags pull the latest version because they do not specify a version number. You can specify a version other than the latest version in the image tag. If you do not specify a tag value, `latest` is used by default. For more information about these tag values, see the <<#tags,Container image tags naming conventions>> section. -For more information on the full list of supported Open Liberty container images, see the link:https://github.com/OpenLiberty/ci.docker/blob/main/docs/icr-images.md[IBM Container Registry (ICR) page on the Open Liberty GitHub repository]. +After the initial release of the Liberty container images, they are refreshed regularly (every 1-2 weeks) to include fixes for the operating system (OS) and Java. For more information on the full list of supported Open Liberty container images, see the link:https://github.com/OpenLiberty/ci.docker/blob/main/docs/icr-images.md[IBM Container Registry (ICR) page on the Open Liberty GitHub repository]. If you have an IBM Cloud ID, you can generate a list of the currently available images in ICR by using the link:https://cloud.ibm.com/docs/cli?topic=cli-getting-started[IBM Cloud CLI] with the link:https://cloud.ibm.com/docs/cli?topic=cli-install-devtools-manually#idt-install-container-registry-cli-plugin[IBM Cloud Container Registry CLI plug-in] installed. Authentication is required only to list the images. You can pull images from ICR without authentication. To list the images, run the following commands. From 84f763d12c8f3c5524348029d9a12727c8821581 Mon Sep 17 00:00:00 2001 From: Ramkumar K Date: Tue, 27 Aug 2024 22:41:34 +0530 Subject: [PATCH 20/62] 7496-Document the image refresh cycle-2 7496-Document the image refresh cycle-2 #7496 --- modules/ROOT/pages/container-images.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/ROOT/pages/container-images.adoc b/modules/ROOT/pages/container-images.adoc index cbf5b36804..b1d97cddac 100644 --- a/modules/ROOT/pages/container-images.adoc +++ b/modules/ROOT/pages/container-images.adoc @@ -15,7 +15,7 @@ A container image is a layered executable software package that contains everything you need to build and run a containerized application. You can use an Open Liberty container image to build an application image that meets your development needs. -Open Liberty container images are available from link:https://www.ibm.com/cloud/container-registry[IBM Container Registry] (ICR). You can pull images from ICR without authentication. +Open Liberty container images are available from the IBM Container Registry (ICR). You can pull images from ICR without authentication. == Container images on IBM Container Registry From 25023cfaa46e9c5c27d97b7031f88c334b69a4d1 Mon Sep 17 00:00:00 2001 From: David Mueller Date: Tue, 27 Aug 2024 14:21:09 -0400 Subject: [PATCH 21/62] edits per review #7459 --- modules/ROOT/pages/microprofile-telemetry.adoc | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/modules/ROOT/pages/microprofile-telemetry.adoc b/modules/ROOT/pages/microprofile-telemetry.adoc index a36fc209ff..7795c83fe5 100644 --- a/modules/ROOT/pages/microprofile-telemetry.adoc +++ b/modules/ROOT/pages/microprofile-telemetry.adoc @@ -12,7 +12,7 @@ :page-type: general = Enable observability with MicroProfile Telemetry -MicroProfile Telemetry helps you collect n and analyze data on the paths that application requests take through services. With MicroProfile Telemetry 2.0 and later, you can manage logs, metrics, and traces in a standardized way by using the OpenTelemetry protocol. +MicroProfile Telemetry helps you collect and analyze data on the paths that application requests take through services. With MicroProfile Telemetry 2.0 and later, you can manage logs, metrics, and traces in a standardized way by using the OpenTelemetry protocol. MicroProfile Telemetry is based on the https://opentelemetry.io/[OpenTelemetry project], a collection of open source vendor-independent tools, APIs, and SDKs for creating and managing logs, metrics, and trace data. @@ -39,7 +39,7 @@ You can configure how MicroProfile Telemetry collects and exports logs, metrics, [#traces] == Configuring Open Liberty to use MicroProfile Telemetry to collect traces -One way to increase observability of an application is by emitting traces. Traces represent requests and consist of multiple spans. A span represents a single operation in a request. It includes a name, time-related data, log messages, and metadata to collect data about what happens during a transaction. +One way to increase observability of an application is by emitting traces. Traces represent requests and consist of multiple spans. A span represents a single operation in a request. It includes a name, time-related data, log messages, and metadata about what happens during a transaction. MicroProfile Telemetry replaces MicroProfile OpenTracing. For more information about migrating your applications from MicroProfile OpenTracing to MicroProfile Telemetry, see xref:reference:diff/mp-50-60-diff.adoc#telemetry[Differences between MicroProfile Telemetry 1.0 and MicroProfile OpenTracing 3.0]. @@ -49,7 +49,7 @@ For more information, see <<#global,Enabling MicroProfile Telemetry for Open Lib . Configure a trace storage system by specifying an exporter definition that includes the exporter type and endpoint. + -For example, to use a Jaeger server, you might add configuration similar to the following example to your `bootstrap.properties` file: +For example, to use a Jaeger server or to forward data to an OpenTelemetry collector, you might add configuration similar to the following example to your `bootstrap.properties` file: + [source,properties] ---- From 7196da5227b64886a5cce63d5d28ad8f274fdbf9 Mon Sep 17 00:00:00 2001 From: David Mueller Date: Tue, 27 Aug 2024 14:46:53 -0400 Subject: [PATCH 22/62] Update microprofile-telemetry.adoc --- modules/ROOT/pages/microprofile-telemetry.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/modules/ROOT/pages/microprofile-telemetry.adoc b/modules/ROOT/pages/microprofile-telemetry.adoc index 7795c83fe5..d879df675e 100644 --- a/modules/ROOT/pages/microprofile-telemetry.adoc +++ b/modules/ROOT/pages/microprofile-telemetry.adoc @@ -32,7 +32,7 @@ To enable MicroProfile Telemetry in your Open Liberty runtime, you must add the . Enable the feature:mpTelemetry[display=MicroProfile Telemetry] feature in your `server.xml` file. To export metrics or logs, you must enable `mpTelemetry-2.0` or later. -. OpenTelemetry is disabled by default. To enable the generation of traces, set the `otel.sdk.disabled=false` property. +. OpenTelemetry is disabled by default. To enable the generation of traces, metrics, and logs, set the `otel.sdk.disabled=false` property. + You can configure how MicroProfile Telemetry collects and exports logs, metrics, and traces by specifying configuration properties in any of the xref:external-configuration.adoc#default[config sources that are available to MicroProfile Config]. Alternatively, if you choose to set these configuration properties by using environment variables, make the key name uppercase and convert any punctuation to underscores. For example, the `otel.sdk.disabled=false` property is equivalent to the `OTEL_SDK_DISABLED=false` environment variable. @@ -73,7 +73,7 @@ For example, if you want to export traces to Open Liberty log files, set the fol + [source,properties] ---- -otel.traces.exporter=logging +otel.traces.exporter=console ---- + For more information about the available properties, see xref:microprofile-config-properties.adoc#telemetry[MicroProfile Config properties: MicroProfile Telemetry]. From 98a78951b5d4c4828a8e175e29442d3504fa8d27 Mon Sep 17 00:00:00 2001 From: David Mueller Date: Tue, 27 Aug 2024 14:50:18 -0400 Subject: [PATCH 23/62] Update security-vulnerabilities.adoc --- modules/ROOT/pages/security-vulnerabilities.adoc | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/modules/ROOT/pages/security-vulnerabilities.adoc b/modules/ROOT/pages/security-vulnerabilities.adoc index 133b60f338..9dd0d291a1 100644 --- a/modules/ROOT/pages/security-vulnerabilities.adoc +++ b/modules/ROOT/pages/security-vulnerabilities.adoc @@ -28,6 +28,13 @@ The `CWWKF0012I` message uses the word "installed", but it lists features that a |=== |CVE |CVSS score by X-Force® |Vulnerability assessment |Versions affected |Version fixed |Notes +|http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-50314[CVE-2023-50314] +|5.3 +|Information disclosure +|17.0.0.3 - 24.0.0.8 +|24.0.0.9 +| + |http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-22354[CVE-2024-22354] |7.0 |XML External Entity (XXE) injection From a800578d490f6e97fbe4b3b6c391a34568f15aec Mon Sep 17 00:00:00 2001 From: Ramkumar K Date: Wed, 28 Aug 2024 11:37:13 +0530 Subject: [PATCH 24/62] 7472-Hostname verification-2 7472-Hostname verification-2 #7472 --- modules/reference/pages/feature/transportSecurity/examples.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/reference/pages/feature/transportSecurity/examples.adoc b/modules/reference/pages/feature/transportSecurity/examples.adoc index d92623fb87..4267ec8d59 100644 --- a/modules/reference/pages/feature/transportSecurity/examples.adoc +++ b/modules/reference/pages/feature/transportSecurity/examples.adoc @@ -176,6 +176,6 @@ Open Liberty performs hostname verification on SSL certificates. If you want to [source,xml] ---- - + ---- From 6741a89558f496b89c335a19ee627adfebedc87a Mon Sep 17 00:00:00 2001 From: David Mueller Date: Wed, 28 Aug 2024 14:20:26 -0400 Subject: [PATCH 25/62] add app-runtime level explanation #7459 --- modules/ROOT/pages/microprofile-telemetry.adoc | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/modules/ROOT/pages/microprofile-telemetry.adoc b/modules/ROOT/pages/microprofile-telemetry.adoc index d879df675e..1d7d0aa79d 100644 --- a/modules/ROOT/pages/microprofile-telemetry.adoc +++ b/modules/ROOT/pages/microprofile-telemetry.adoc @@ -34,7 +34,13 @@ To enable MicroProfile Telemetry in your Open Liberty runtime, you must add the . OpenTelemetry is disabled by default. To enable the generation of traces, metrics, and logs, set the `otel.sdk.disabled=false` property. + -You can configure how MicroProfile Telemetry collects and exports logs, metrics, and traces by specifying configuration properties in any of the xref:external-configuration.adoc#default[config sources that are available to MicroProfile Config]. Alternatively, if you choose to set these configuration properties by using environment variables, make the key name uppercase and convert any punctuation to underscores. For example, the `otel.sdk.disabled=false` property is equivalent to the `OTEL_SDK_DISABLED=false` environment variable. +You can configure how MicroProfile Telemetry collects and exports logs, metrics, and traces by specifying configuration properties in any of the config sources that are available to MicroProfile Config. Alternatively, if you choose to set these configuration properties by using environment variables, make the key name uppercase and convert any punctuation to underscores. For example, the `otel.sdk.disabled=false` property is equivalent to the `OTEL_SDK_DISABLED=false` environment variable. ++ +Depending on how you enable the OpenTelemetry SDK, you can configure a single instance for the runtime and all applications, or configure a separate instance for each application. Any runtime-level configuration settings, such as environment variables or system properties, apply to the runtime and all applications, unless an application overrides those settings in a higher ordinal value configuration source. +Although application-level configuration gives you more fine-grained control over the data that you collect, it does not collect any runtime-level metrics and might require manual instrumentation in your code. Runtime-level configuration does not require you to modify your application code. For more information about configuration sources and their ordinal values, see xref:external-configuration.adoc#default[Default configuration sources]. + + + [#traces] == Configuring Open Liberty to use MicroProfile Telemetry to collect traces @@ -156,8 +162,6 @@ otel.metrics.exporter=logging + By default, metric data is exported at an interval of 60 seconds. To modify the export interval, specify the `otel.metric.export.interval` property or the `OTEL_METRIC_EXPORT_INTERVAL` environment variable. Specify the value in milliseconds. For more information, see link:https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#periodic-exporting-metricreader[Periodic exporter MetricReader] in the OpenTelemetry documentation. -To separately configure multiple applications in a server, you can configure OpenTelemetry with application configuration. However, you cannot collect runtime-level metrics this way. - Depending on how you choose to instrument your application code for metrics, further configuration might be required. From e7058253d0581143cbb05cf862a1c06b3a20b497 Mon Sep 17 00:00:00 2001 From: David Mueller Date: Wed, 28 Aug 2024 14:24:39 -0400 Subject: [PATCH 26/62] Update microprofile-telemetry.adoc --- modules/ROOT/pages/microprofile-telemetry.adoc | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/modules/ROOT/pages/microprofile-telemetry.adoc b/modules/ROOT/pages/microprofile-telemetry.adoc index 1d7d0aa79d..f4fd6852d6 100644 --- a/modules/ROOT/pages/microprofile-telemetry.adoc +++ b/modules/ROOT/pages/microprofile-telemetry.adoc @@ -36,8 +36,9 @@ To enable MicroProfile Telemetry in your Open Liberty runtime, you must add the + You can configure how MicroProfile Telemetry collects and exports logs, metrics, and traces by specifying configuration properties in any of the config sources that are available to MicroProfile Config. Alternatively, if you choose to set these configuration properties by using environment variables, make the key name uppercase and convert any punctuation to underscores. For example, the `otel.sdk.disabled=false` property is equivalent to the `OTEL_SDK_DISABLED=false` environment variable. + -Depending on how you enable the OpenTelemetry SDK, you can configure a single instance for the runtime and all applications, or configure a separate instance for each application. Any runtime-level configuration settings, such as environment variables or system properties, apply to the runtime and all applications, unless an application overrides those settings in a higher ordinal value configuration source. -Although application-level configuration gives you more fine-grained control over the data that you collect, it does not collect any runtime-level metrics and might require manual instrumentation in your code. Runtime-level configuration does not require you to modify your application code. For more information about configuration sources and their ordinal values, see xref:external-configuration.adoc#default[Default configuration sources]. +Depending on how you enable the OpenTelemetry SDK, you can configure a single instance for the runtime and all applications, or configure a separate instance for each application. Any runtime-level configuration settings, such as environment variables or system properties, apply to the runtime and all applications, unless an application overrides those settings in a higher ordinal value configuration source. For more information about configuration sources and their ordinal values, see xref:external-configuration.adoc#default[Default configuration sources]. ++ +Although application-level configuration gives you more fine-grained control over the data that you collect, it does not collect any runtime-level data and might require manual instrumentation in your code. Runtime-level configuration does not require you to modify your application code. From 41db0343d946ced6ac10b2ba0f247686f4a44ff2 Mon Sep 17 00:00:00 2001 From: Ramkumar K Date: Thu, 29 Aug 2024 14:48:56 +0530 Subject: [PATCH 27/62] 7472-Hostname verification-3 7472-Hostname verification-3 #7472 --- .../feature/transportSecurity/examples.adoc | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/modules/reference/pages/feature/transportSecurity/examples.adoc b/modules/reference/pages/feature/transportSecurity/examples.adoc index 4267ec8d59..2d049bb316 100644 --- a/modules/reference/pages/feature/transportSecurity/examples.adoc +++ b/modules/reference/pages/feature/transportSecurity/examples.adoc @@ -172,10 +172,22 @@ cert_defaultKeyStore="-----BEGIN CERTIFICATE----- [#hostverify] === Hostname verification -Open Liberty performs hostname verification on SSL certificates. If you want to disable the hostname verification functionality, the `verifyHostname` attribute within the `ssl` tag must be initialized to `false`. +In Open Liberty, hostname and IP address verification are enabled by default. This verification is enforced for target servers in all SSL connections by using the Open Liberty socket factories. However, you can specify a list of hostnames, IP addresses, or both to skip verification. + +To disable hostname verification entirely, set the `verifyHostname` attribute within the `ssl` tag to `false`. + +[source,xml] +---- + +---- + +When hostname verification is enabled (`verifyHostname="true"`), you can specify hostnames or IP addresses to be skipped for verification by using the `skipHostnameVerificationForHosts` attribute within the `ssl` tag. [source,xml] ---- - + ---- +Additionally, hostname verification for only HTTP connections can be controlled separately by using the `httpHostNameVerification` attribute within the `sslDefault` tag. If `httpHostNameVerification` attribute is set to `true` and `verifyHostname` attribute is set to `false`, hostname verification can still be enforced on HTTP connections, but not on other connections. + + From 1a60da8ade01d8a7357bc4d8807fc7ab6f3f10f8 Mon Sep 17 00:00:00 2001 From: David Mueller Date: Thu, 29 Aug 2024 08:43:59 -0400 Subject: [PATCH 28/62] fix links on container image page --- modules/ROOT/pages/container-images.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/modules/ROOT/pages/container-images.adoc b/modules/ROOT/pages/container-images.adoc index 64c1a46de2..2e8f95281d 100644 --- a/modules/ROOT/pages/container-images.adoc +++ b/modules/ROOT/pages/container-images.adoc @@ -15,7 +15,7 @@ A container image is a layered executable software package that contains everything you need to build and run a containerized application. You can use an Open Liberty container image to build an application image that meets your development needs. -Open Liberty container images are available from link:https://www.ibm.com/cloud/container-registry[IBM Container Registry] (ICR). You can pull images from ICR without authentication. +Open Liberty container images are available from IBM Container Registry. You can pull images from ICR without authentication. == Container images on IBM Container Registry @@ -40,7 +40,7 @@ These tags pull the latest version because they do not specify a version number. For more information on the full list of supported Open Liberty container images, see the link:https://github.com/OpenLiberty/ci.docker/blob/main/docs/icr-images.md[IBM Container Registry (ICR) page on the Open Liberty GitHub repository]. -If you have an IBM Cloud ID, you can generate a list of the currently available images in ICR by using the link:https://cloud.ibm.com/docs/cli?topic=cli-getting-started[IBM Cloud CLI] with the link:https://cloud.ibm.com/docs/cli?topic=cli-install-devtools-manually#idt-install-container-registry-cli-plugin[IBM Cloud Container Registry CLI plug-in] installed. Authentication is required only to list the images. You can pull images from ICR without authentication. To list the images, run the following commands. +If you have an IBM Cloud ID, you can generate a list of the currently available images in ICR by using the link:https://cloud.ibm.com/docs/cli?topic=cli-getting-started[IBM Cloud CLI] with the link:https://cloud.ibm.com/docs/Registry?topic=Registry-containerregcli[IBM Cloud Container Registry CLI plug-in] installed. Authentication is required only to list the images. You can pull images from ICR without authentication. To list the images, run the following commands. ---- ibmcloud login From d476bfb84a18eeeb3f4c399155461a5e527a3542 Mon Sep 17 00:00:00 2001 From: David Mueller Date: Thu, 29 Aug 2024 08:44:34 -0400 Subject: [PATCH 29/62] Update container-images.adoc --- modules/ROOT/pages/container-images.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/modules/ROOT/pages/container-images.adoc b/modules/ROOT/pages/container-images.adoc index 2e8f95281d..64c1a46de2 100644 --- a/modules/ROOT/pages/container-images.adoc +++ b/modules/ROOT/pages/container-images.adoc @@ -15,7 +15,7 @@ A container image is a layered executable software package that contains everything you need to build and run a containerized application. You can use an Open Liberty container image to build an application image that meets your development needs. -Open Liberty container images are available from IBM Container Registry. You can pull images from ICR without authentication. +Open Liberty container images are available from link:https://www.ibm.com/cloud/container-registry[IBM Container Registry] (ICR). You can pull images from ICR without authentication. == Container images on IBM Container Registry @@ -40,7 +40,7 @@ These tags pull the latest version because they do not specify a version number. For more information on the full list of supported Open Liberty container images, see the link:https://github.com/OpenLiberty/ci.docker/blob/main/docs/icr-images.md[IBM Container Registry (ICR) page on the Open Liberty GitHub repository]. -If you have an IBM Cloud ID, you can generate a list of the currently available images in ICR by using the link:https://cloud.ibm.com/docs/cli?topic=cli-getting-started[IBM Cloud CLI] with the link:https://cloud.ibm.com/docs/Registry?topic=Registry-containerregcli[IBM Cloud Container Registry CLI plug-in] installed. Authentication is required only to list the images. You can pull images from ICR without authentication. To list the images, run the following commands. +If you have an IBM Cloud ID, you can generate a list of the currently available images in ICR by using the link:https://cloud.ibm.com/docs/cli?topic=cli-getting-started[IBM Cloud CLI] with the link:https://cloud.ibm.com/docs/cli?topic=cli-install-devtools-manually#idt-install-container-registry-cli-plugin[IBM Cloud Container Registry CLI plug-in] installed. Authentication is required only to list the images. You can pull images from ICR without authentication. To list the images, run the following commands. ---- ibmcloud login From eef01997bfedfc517541f1d84583f6dcc5692a8d Mon Sep 17 00:00:00 2001 From: David Mueller Date: Thu, 29 Aug 2024 08:48:54 -0400 Subject: [PATCH 30/62] Update container-images.adoc --- modules/ROOT/pages/container-images.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/ROOT/pages/container-images.adoc b/modules/ROOT/pages/container-images.adoc index b1d97cddac..596fd7c4bb 100644 --- a/modules/ROOT/pages/container-images.adoc +++ b/modules/ROOT/pages/container-images.adoc @@ -40,7 +40,7 @@ These tags pull the latest version because they do not specify a version number. After the initial release of the Liberty container images, they are refreshed regularly (every 1-2 weeks) to include fixes for the operating system (OS) and Java. For more information on the full list of supported Open Liberty container images, see the link:https://github.com/OpenLiberty/ci.docker/blob/main/docs/icr-images.md[IBM Container Registry (ICR) page on the Open Liberty GitHub repository]. -If you have an IBM Cloud ID, you can generate a list of the currently available images in ICR by using the link:https://cloud.ibm.com/docs/cli?topic=cli-getting-started[IBM Cloud CLI] with the link:https://cloud.ibm.com/docs/cli?topic=cli-install-devtools-manually#idt-install-container-registry-cli-plugin[IBM Cloud Container Registry CLI plug-in] installed. Authentication is required only to list the images. You can pull images from ICR without authentication. To list the images, run the following commands. +If you have an IBM Cloud ID, you can generate a list of the currently available images in ICR by using the link:https://cloud.ibm.com/docs/cli?topic=cli-getting-started[IBM Cloud CLI] with the link:https://cloud.ibm.com/docs/Registry?topic=Registry-containerregcli[IBM Cloud Container Registry CLI plug-in] installed. Authentication is required only to list the images. You can pull images from ICR without authentication. To list the images, run the following commands. ---- ibmcloud login From 1e2adde39070ee0c2cf0193239b0d12c42a5380e Mon Sep 17 00:00:00 2001 From: David Mueller Date: Thu, 29 Aug 2024 14:42:55 -0400 Subject: [PATCH 31/62] Update microprofile-config-properties.adoc --- .../pages/microprofile-config-properties.adoc | 26 ++++++++++--------- 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/modules/ROOT/pages/microprofile-config-properties.adoc b/modules/ROOT/pages/microprofile-config-properties.adoc index d2bb536459..f57ac29e23 100644 --- a/modules/ROOT/pages/microprofile-config-properties.adoc +++ b/modules/ROOT/pages/microprofile-config-properties.adoc @@ -616,6 +616,8 @@ This property or `*/mp-rest/url` is considered required. However, some implement The following MicroProfile Config properties can be specified when MicroProfile Telemetry is enabled. This table lists the most important properties. For the full list, see the https://download.eclipse.org/microprofile/microprofile-telemetry-1.0/tracing/microprofile-telemetry-tracing-spec-1.0.html#_configuration[MicroProfile Telemetry specification]. +The runtime reads properties in the following table at either application startup or runtime startup, depending on when the OpenTelemetry instance initializes. In cases where each application is configured to use a separate OpenTelemetry instance, the runtime reads the properties at application startup. In cases where all applications on the runtime share a single OpenTelemetry instance, the runtime reads the properties at runtime startup. + For more information about collecting Open Liberty logs, metrics, and traces, see xref:microprofile-telemetry.adoc[Enable observability with MicroProfile Telemetry]. .MicroProfile Config properties for MicroProfile Telemetry @@ -625,62 +627,62 @@ For more information about collecting Open Liberty logs, metrics, and traces, se |`otel.blrp.export.timeout` |Sets the delay interval in milliseconds between two consecutive exports from the link:https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#batch-logrecord-processor[Batch LogRecord Processor]. The default is `30000`. -|Application startup +|Application startup or runtime startup |`otel.blrp.export.timeout=50000` |`otel.blrp.schedule.delay` |Sets the maximum allowed time in milliseconds to export data from the Batch LogRecord Processor. The default is `1000`. -|Application startup +|Application startup or runtime startup |`otel.blrp.schedule.delay=5000` |`otel.blrp.max.queue.size` |Sets the maximum queue size for the Batch LogRecord Processor. The default is `2048`. -|Application startup +|Application startup or runtime startup |`otel.blrp.max.queue.size=5000` |`otel.blrp.max.export.batch.size` |Sets the maximum batch size for the Batch LogRecord Processor. The default is `512`. The value must be Must be less than or equal to the value for `otel.blrp.max.queue.size`. -|Application startup +|Application startup or runtime startup |`otel.blrp.max.export.batch.size=1024` |`otel.exporter.jaeger.endpoint` |Sets the endpoint for the Jaeger exporter. The default is \http://localhost:14250. -|Application startup +|Application startup or runtime startup |`otel.exporter.jaeger.endpoint=http://localhost:14250` |`otel.exporter.jaeger.timeout` |Sets the maximum time, in milliseconds, that the Jaeger exporter waits for each batch export. The default is `10000`. -|Application startup +|Application startup or runtime startup |`otel.exporter.jaeger.timeout=10000` |`otel.exporter.otlp.endpoint` |Sets the endpoint for the OpenTelemetry Protocol (otlp) exporter. The default is \http://localhost:4317 -|Application startup +|Application startup or runtime startup |`otel.exporter.otlp.endpoint=http://localhost:4317` |`otel.exporter.zipkin.endpoint` |Sets the endpoint for the Zipkin exporter. The default is \http://localhost:9411/api/v2/spans. -|Application startup +|Application startup or runtime startup |`otel.exporter.zipkin.endpoint=http://localhost:9411/api/v2/spans` |`otel.sdk.disabled` |Enables logs, metrics, and traces to be sent to OpenTelemetry. The default is `true`. -|Application startup +|Application startup or runtime startup |`otel.sdk.disabled=false` |`otel.service.name` |Sets of the name of the service that MicroProfile Telemetry is tracing. -|Application startup +|Application startup or runtime startup |`otel.service.name=system` |`otel.logs.exporter` |You can use this property to change where the logs that MicroProfile Telemetry collects are exported. For more information, see xref:microprofile-telemetry.adoc#logs[Configuring Open Liberty to use MicroProfile Telemetry to collect logs] -|Application startup +|Application startup or runtime startup |`otel.traces.exporter=console` |`otel.traces.exporter` |Sets the exporter that is used to collect traces. Possible values are `otlp`, `zipkin`, `jaeger`, or `logging`. The default value is `otlp`. For the Jaeger trace service versions 1.35 and later, the `otlp` exporter value is recommended, rather than `jaeger`. -|Application startup +|Application startup or runtime startup |`otel.traces.exporter=otlp` |=== From a8f77f587beb90b0f6b3af47c956561257130e05 Mon Sep 17 00:00:00 2001 From: Ramkumar K Date: Mon, 2 Sep 2024 14:41:29 +0530 Subject: [PATCH 32/62] 7472-Hostname verification-ut-cmts-1 7472-Hostname verification-ut-cmts-1 #7472 --- .../ROOT/pages/secure-communication-tls.adoc | 4 ++ modules/ROOT/pages/troubleshooting.adoc | 41 +++++++++++++++++++ .../feature/transportSecurity/examples.adoc | 23 +++-------- 3 files changed, 51 insertions(+), 17 deletions(-) diff --git a/modules/ROOT/pages/secure-communication-tls.adoc b/modules/ROOT/pages/secure-communication-tls.adoc index ea10d5c0e4..9b4c041e4a 100644 --- a/modules/ROOT/pages/secure-communication-tls.adoc +++ b/modules/ROOT/pages/secure-communication-tls.adoc @@ -52,6 +52,10 @@ Open Liberty does not check certificate revocations and instead relies on the un Before the wide adoption of the TLS protocol, SSL was the standard protocol to secure web communications. Over time, many security vulnerabilities were identified for SSL and it is no longer in widespread use. However, for historical reasons, the term SSL is often used to refer to encrypted network connections even when TLS is in use. In Open Liberty, the term SSL is still used in feature and configuration names, even though TLS is the underlying protocol. Effectively, SSL is now a synonym for TLS. +NOTE: Hostname verification is enabled by default. + +For more information on Hostname verification, see feature:transportSecurity[display=Transport Security]. + == See also https://tools.ietf.org/html/rfc8446[RFC 8446: The Transport Layer Security (TLS) Protocol] diff --git a/modules/ROOT/pages/troubleshooting.adoc b/modules/ROOT/pages/troubleshooting.adoc index 192b4ac582..acfa9e0f63 100644 --- a/modules/ROOT/pages/troubleshooting.adoc +++ b/modules/ROOT/pages/troubleshooting.adoc @@ -273,6 +273,47 @@ Exception thrown while trying to read configuration and update ManagedServiceFac This error occurs when a keystore element exists in the configuration without an ID field. If you use a minimal TLS configuration, set the `ID` field to `defaultKeyStore`. +=== You receive the CWPKI0824E message that SSL handshake failure due to hostname verification error + +If you try to access a URL, you might see the following message. + +---- +CWPKI0824E: SSL HANDSHAKE FAILURE: Host name verification error while connecting to host [testServer.com]. The host name used to access the server does not match the server certificate's [Subject Alternative Name [dnsName:server1.com, ipAddress:11.22.33.444]]. The extended error message from the SSL handshake exception is: [No subject alternative names matching host name testServer.com]. +---- + +Hostname and IP address verification is a critical security check that prevents man-in-the-middle attacks by making sure that the client connects to the correct server. However, hostname verification can fail during an SSL handshake. + +The following list provides common reasons that hostname verification fails. + +==== Mismatched hostnames + +The hostname that is specified in the client’s URL does not match the Common Name (CN) or Subject Alternative Name (SAN) in the server’s SSL certificate. + +==== Incorrect SSL configuration + +The SSL configuration on the server might be set up with a certificate that does not include the correct hostname. + +==== Configuration error in client + +The client might be configured with an incorrect URL or might be using a deprecated hostname. + +You can resolve the hostname verification failure by addressing the following areas. + +- Verify the hostname or IP address: Check that the hostname or IP address in the URL that you are using matches the SAN or CN in the server's SSL certificate. If the URL is incorrect, update it with the correct hostname. + +- Review your SSL configuration: Make sure that the server SSL certificate is configured correctly. The SSL certificate must contain the SAN or CN of the hostname that the client is connecting to. + +- If the security of your environment is not impacted, you can skip hostname verification for specific hostnames, IP addresses, or both. Set the `skipHostnameVerificationForHosts` attribute in the SSL config to the specific hostnames, IP addresses, or both that you want to skip verification for. Separate multiple entries with commas. + +- Disable hostname verification temporarily when this security check is not a concern, such as in nonproduction environments, by setting the SSL config element with the attribute `verifyHostname` to `false`. +The following message is then displayed: + +---- +CWPKI0063W: Hostname verification is disabled for [mySSLConfig]. TLS/SSL connections do not check server identities to verify that the client is communicating with the correct server. +---- + +NOTE: Avoid disabling hostname verification for production environments as it can compromise security. + [#Troubleshooting_TAI] == Troubleshooting Trust Association Interceptor diff --git a/modules/reference/pages/feature/transportSecurity/examples.adoc b/modules/reference/pages/feature/transportSecurity/examples.adoc index 2d049bb316..1a75b10eb7 100644 --- a/modules/reference/pages/feature/transportSecurity/examples.adoc +++ b/modules/reference/pages/feature/transportSecurity/examples.adoc @@ -7,7 +7,7 @@ - <<#configure-specific, Configure specific TLS protocols>> - <<#outbound, Configure outbound TLS>> - <<#certs, Provide certificates from an environment variable or a file>> -- <<#hostverify, Hostname verification>> +- <<#hostverify, Hostname and IP address verification>> [#config] === Configure transport layer security (TLS) @@ -170,24 +170,13 @@ cert_defaultKeyStore="-----BEGIN CERTIFICATE----- ---- [#hostverify] -=== Hostname verification +=== Hostname and IP address verification -In Open Liberty, hostname and IP address verification are enabled by default. This verification is enforced for target servers in all SSL connections by using the Open Liberty socket factories. However, you can specify a list of hostnames, IP addresses, or both to skip verification. +In Open Liberty, hostname and IP address verification are enabled by default. This verification is enforced for target servers in all SSL connections through Open Liberty socket factories. However, you can specify a list of hostnames, IP addresses, or both to skip verification. -To disable hostname verification entirely, set the `verifyHostname` attribute within the `ssl` tag to `false`. +The verification makes sure that the hostname or IP address in the URL matches the Subject Alternative Name (SAN) in the SSL certificate of the server. If the SAN is not found, the property makes sure that the hostname in the URL matches the Common Name (CN). If a mismatch exists, the SSL connection is rejected. -[source,xml] ----- - ----- - -When hostname verification is enabled (`verifyHostname="true"`), you can specify hostnames or IP addresses to be skipped for verification by using the `skipHostnameVerificationForHosts` attribute within the `ssl` tag. - -[source,xml] ----- - ----- - -Additionally, hostname verification for only HTTP connections can be controlled separately by using the `httpHostNameVerification` attribute within the `sslDefault` tag. If `httpHostNameVerification` attribute is set to `true` and `verifyHostname` attribute is set to `false`, hostname verification can still be enforced on HTTP connections, but not on other connections. +Typically, during hostname verification, when the hostname is used in the request, it checks against the DNSName entry in the SAN. If the SAN does not contain a DNSName entry, hostname verification uses the certificate owner's common name (CN). When an IP address is used in the request, hostname verification relies on the IP address information in the SAN only. +For more information, see xref:ROOT:pages/troubleshooting.adoc#Troubleshooting_SSL[Troubleshooting SSL and TLS] From 72a7d1812b7e5c7690e7e743e10506c51bfe7c8a Mon Sep 17 00:00:00 2001 From: Ramkumar K Date: Tue, 3 Sep 2024 16:35:27 +0530 Subject: [PATCH 33/62] 7472-Hostname verification-ut-cmts-2 7472-Hostname verification-ut-cmts-2 #7472 --- modules/reference/pages/feature/transportSecurity/examples.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/reference/pages/feature/transportSecurity/examples.adoc b/modules/reference/pages/feature/transportSecurity/examples.adoc index 1a75b10eb7..6f68f5adc7 100644 --- a/modules/reference/pages/feature/transportSecurity/examples.adoc +++ b/modules/reference/pages/feature/transportSecurity/examples.adoc @@ -178,5 +178,5 @@ The verification makes sure that the hostname or IP address in the URL matches t Typically, during hostname verification, when the hostname is used in the request, it checks against the DNSName entry in the SAN. If the SAN does not contain a DNSName entry, hostname verification uses the certificate owner's common name (CN). When an IP address is used in the request, hostname verification relies on the IP address information in the SAN only. -For more information, see xref:ROOT:pages/troubleshooting.adoc#Troubleshooting_SSL[Troubleshooting SSL and TLS] +For more information, see xref:pages/troubleshooting.adoc#Troubleshooting_SSL[Troubleshooting SSL and TLS] From 830c12abc220d43a47cec992501e156d3f9e935c Mon Sep 17 00:00:00 2001 From: David Mueller Date: Tue, 3 Sep 2024 10:53:32 -0400 Subject: [PATCH 34/62] edits per review #7459 --- .../ROOT/pages/microprofile-telemetry.adoc | 62 ++++++++++++++++--- 1 file changed, 55 insertions(+), 7 deletions(-) diff --git a/modules/ROOT/pages/microprofile-telemetry.adoc b/modules/ROOT/pages/microprofile-telemetry.adoc index f4fd6852d6..56dccda9fa 100644 --- a/modules/ROOT/pages/microprofile-telemetry.adoc +++ b/modules/ROOT/pages/microprofile-telemetry.adoc @@ -28,19 +28,67 @@ The following sections explain how to prepare your Open Liberty runtime and appl [#global] == Enabling MicroProfile Telemetry for Open Liberty -To enable MicroProfile Telemetry in your Open Liberty runtime, you must add the MicroProfile Telemetry feature to your `server.xml` file and enable the OpenTelemetry SDK. +To enable MicroProfile Telemetry in your Open Liberty runtime, you must add the MicroProfile Telemetry feature to your `server.xml` file and enable the OpenTelemetry SDK.OpenTelemetry is disabled by default. Depending on your needs, you can configure OpenTelemetry SDK instances at the runtime level or the application level. + +Runtime-level configuration:: +Runtime-level configuration collects and emits telemetry from the runtime and all applications. However, this configuration is preferable when you are running a single application per Liberty because all telemetry emitted is labeled as coming from the same service. + +You can configure your runtime instance in a few different sources, including your `jvm.options`, `bootstrap.properties`, or `server.env` files. You do not need to modify your application configuration to configure your runtime instance. + +Application-level configuration:: +Configuring application-level OpenTelemetry instances gives you more fine-grained control over the data you collect for each application. This option also supports compatibility with earlier versions of MicroProfile Telemetry, which created only application SDK instances. Runtime-level data is not collected or emitted with application-level instances. + +You can configure your application-level OpenTelemetry instances in the applications `microprofile-config.properties` files or the `appProperties` attribute in the `server.xml` file for each application. When you create application-level instances, any system and server environment properties overwrite your application-level configuration. For more information about configuration sources and their ordinal values, see xref:external-configuration.adoc#default[Default configuration sources]. . Enable the feature:mpTelemetry[display=MicroProfile Telemetry] feature in your `server.xml` file. To export metrics or logs, you must enable `mpTelemetry-2.0` or later. -. OpenTelemetry is disabled by default. To enable the generation of traces, metrics, and logs, set the `otel.sdk.disabled=false` property. -+ -You can configure how MicroProfile Telemetry collects and exports logs, metrics, and traces by specifying configuration properties in any of the config sources that are available to MicroProfile Config. Alternatively, if you choose to set these configuration properties by using environment variables, make the key name uppercase and convert any punctuation to underscores. For example, the `otel.sdk.disabled=false` property is equivalent to the `OTEL_SDK_DISABLED=false` environment variable. +. To enable the generation of traces, metrics, and logs, set the `otel.sdk.disabled=false` property at the runtime level or the application level. + +.. To enable the OpenTelemetry SDK at the runtime level, set the `otel.sdk.disabled=false property` as a system property, for example, in the `bootstrap.properties` file: + -Depending on how you enable the OpenTelemetry SDK, you can configure a single instance for the runtime and all applications, or configure a separate instance for each application. Any runtime-level configuration settings, such as environment variables or system properties, apply to the runtime and all applications, unless an application overrides those settings in a higher ordinal value configuration source. For more information about configuration sources and their ordinal values, see xref:external-configuration.adoc#default[Default configuration sources]. +---- +otel.sdk.disabled=false +otel.service.name=RT1 +---- + -Although application-level configuration gives you more fine-grained control over the data that you collect, it does not collect any runtime-level data and might require manual instrumentation in your code. Runtime-level configuration does not require you to modify your application code. - +The `otel.service.name` property creates a name for any telemetry that this OpenTelemetry SDK instance collects and emits. +.. To enable the OpenTelemetry SDK at the application level, set the `otel.sdk.disabled=false property` in the `microprofile-config.properties` file of each application. ++ +Application 1 `microprofile-config.properties` file: ++ +---- +otel.sdk.disabled=false +otel.service.name=A1 +---- ++ +Application 2 `microprofile-config.properties` file: ++ +---- +otel.sdk.disabled=false +otel.service.name=A2 +---- ++ +This configuration creates all telemetry from App1 with the service name A1, and from App2 with the service name A2. It omits all runtime-level telemetry. ++ +At runtime initialization, if the `otel.sdk.disabled` property is set to false, the runtime-level OpenTelemetry SDK instance is created. If the runtime instance is not enabled and `otel.sdk.disabled=false` is specified at the application level, an application-level instance is created during application initialization. ++ +You can also use a combination of application-level and runtime-level configuration. Regardless of whether a runtime instance is created, any configuration in the system properties and server environment variables takes precedence over application-level configuration. Therefore, you can configure common application configuration at the runtime level in the server environment variables or system properties and make application-specific changes in your application configuration. ++ +For example, you can set the OpenTelemetry exporter at the runtime-level in the `bootstrap.properties` file: ++ +---- +otel.logs.exporter=myExporter +---- ++ +Then, enable an application-level SDK instance in the `microprofile-config.properties` file for each application in the runtime: ++ +---- +otel.sdk.disabled=false +otel.service.name= +---- ++ +This configuration enables an application-level instance and a custom service name for each configured application, while all applications in the runtime use the `myExporter` OpenTelemetry exporter. [#traces] From 18336c5c5c63381d8fadbff52c7082c958043865 Mon Sep 17 00:00:00 2001 From: David Mueller Date: Tue, 3 Sep 2024 10:57:12 -0400 Subject: [PATCH 35/62] Update microprofile-telemetry.adoc --- modules/ROOT/pages/microprofile-telemetry.adoc | 2 ++ 1 file changed, 2 insertions(+) diff --git a/modules/ROOT/pages/microprofile-telemetry.adoc b/modules/ROOT/pages/microprofile-telemetry.adoc index 56dccda9fa..80f02deb83 100644 --- a/modules/ROOT/pages/microprofile-telemetry.adoc +++ b/modules/ROOT/pages/microprofile-telemetry.adoc @@ -52,6 +52,8 @@ otel.service.name=RT1 ---- + The `otel.service.name` property creates a name for any telemetry that this OpenTelemetry SDK instance collects and emits. ++ +If you choose to set configuration properties by using environment variables in your `server.env` file, make the key name uppercase and convert any punctuation to underscores. For example, the `otel.sdk.disabled=false` property is equivalent to the `OTEL_SDK_DISABLED=false` environment variable. .. To enable the OpenTelemetry SDK at the application level, set the `otel.sdk.disabled=false property` in the `microprofile-config.properties` file of each application. + From 561528067913946b174407705d897787d23adeb6 Mon Sep 17 00:00:00 2001 From: David Mueller Date: Tue, 3 Sep 2024 12:00:32 -0400 Subject: [PATCH 36/62] edits --- .../ROOT/pages/microprofile-telemetry.adoc | 20 ++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/modules/ROOT/pages/microprofile-telemetry.adoc b/modules/ROOT/pages/microprofile-telemetry.adoc index 80f02deb83..5a4439db31 100644 --- a/modules/ROOT/pages/microprofile-telemetry.adoc +++ b/modules/ROOT/pages/microprofile-telemetry.adoc @@ -28,21 +28,27 @@ The following sections explain how to prepare your Open Liberty runtime and appl [#global] == Enabling MicroProfile Telemetry for Open Liberty -To enable MicroProfile Telemetry in your Open Liberty runtime, you must add the MicroProfile Telemetry feature to your `server.xml` file and enable the OpenTelemetry SDK.OpenTelemetry is disabled by default. Depending on your needs, you can configure OpenTelemetry SDK instances at the runtime level or the application level. +OpenTelemetry is disabled by default. Depending on your needs, you can configure OpenTelemetry SDK instances at the runtime level or the application level. Runtime-level configuration:: Runtime-level configuration collects and emits telemetry from the runtime and all applications. However, this configuration is preferable when you are running a single application per Liberty because all telemetry emitted is labeled as coming from the same service. - ++ You can configure your runtime instance in a few different sources, including your `jvm.options`, `bootstrap.properties`, or `server.env` files. You do not need to modify your application configuration to configure your runtime instance. Application-level configuration:: Configuring application-level OpenTelemetry instances gives you more fine-grained control over the data you collect for each application. This option also supports compatibility with earlier versions of MicroProfile Telemetry, which created only application SDK instances. Runtime-level data is not collected or emitted with application-level instances. - ++ You can configure your application-level OpenTelemetry instances in the applications `microprofile-config.properties` files or the `appProperties` attribute in the `server.xml` file for each application. When you create application-level instances, any system and server environment properties overwrite your application-level configuration. For more information about configuration sources and their ordinal values, see xref:external-configuration.adoc#default[Default configuration sources]. + + +To enable MicroProfile Telemetry in your Open Liberty runtime, you must add the MicroProfile Telemetry feature to your `server.xml` file and enable the OpenTelemetry SDK. + . Enable the feature:mpTelemetry[display=MicroProfile Telemetry] feature in your `server.xml` file. To export metrics or logs, you must enable `mpTelemetry-2.0` or later. -. To enable the generation of traces, metrics, and logs, set the `otel.sdk.disabled=false` property at the runtime level or the application level. +. Set the `otel.sdk.disabled=false` property at the runtime level or the application level. ++ +At runtime initialization, if the `otel.sdk.disabled` property is set to false, the runtime-level OpenTelemetry SDK instance is created. If the runtime instance is not enabled and `otel.sdk.disabled=false` is specified at the application level, an application-level instance is created during application initialization. .. To enable the OpenTelemetry SDK at the runtime level, set the `otel.sdk.disabled=false property` as a system property, for example, in the `bootstrap.properties` file: + @@ -72,10 +78,10 @@ otel.service.name=A2 ---- + This configuration creates all telemetry from App1 with the service name A1, and from App2 with the service name A2. It omits all runtime-level telemetry. + +.. Optionally, use a combination of application-level and runtime-level configuration. + -At runtime initialization, if the `otel.sdk.disabled` property is set to false, the runtime-level OpenTelemetry SDK instance is created. If the runtime instance is not enabled and `otel.sdk.disabled=false` is specified at the application level, an application-level instance is created during application initialization. -+ -You can also use a combination of application-level and runtime-level configuration. Regardless of whether a runtime instance is created, any configuration in the system properties and server environment variables takes precedence over application-level configuration. Therefore, you can configure common application configuration at the runtime level in the server environment variables or system properties and make application-specific changes in your application configuration. +Regardless of whether a runtime instance is created, any configuration in the system properties and server environment variables takes precedence over application-level configuration. Therefore, you can configure common application configuration at the runtime level in the server environment variables or system properties and make application-specific changes in your application configuration. + For example, you can set the OpenTelemetry exporter at the runtime-level in the `bootstrap.properties` file: + From 1caf5a0ed95154b2e2304a59f484d1897b90fc28 Mon Sep 17 00:00:00 2001 From: David Mueller Date: Tue, 3 Sep 2024 13:42:40 -0400 Subject: [PATCH 37/62] Update microprofile-telemetry.adoc --- modules/ROOT/pages/microprofile-telemetry.adoc | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/modules/ROOT/pages/microprofile-telemetry.adoc b/modules/ROOT/pages/microprofile-telemetry.adoc index 5a4439db31..a286101434 100644 --- a/modules/ROOT/pages/microprofile-telemetry.adoc +++ b/modules/ROOT/pages/microprofile-telemetry.adoc @@ -31,12 +31,12 @@ The following sections explain how to prepare your Open Liberty runtime and appl OpenTelemetry is disabled by default. Depending on your needs, you can configure OpenTelemetry SDK instances at the runtime level or the application level. Runtime-level configuration:: -Runtime-level configuration collects and emits telemetry from the runtime and all applications. However, this configuration is preferable when you are running a single application per Liberty because all telemetry emitted is labeled as coming from the same service. +Runtime-level configuration collects and emits telemetry from the runtime and all applications. However, this configuration is preferable when you are running a single application per Liberty runtime because all telemetry emitted is labeled as coming from the same service. + You can configure your runtime instance in a few different sources, including your `jvm.options`, `bootstrap.properties`, or `server.env` files. You do not need to modify your application configuration to configure your runtime instance. Application-level configuration:: -Configuring application-level OpenTelemetry instances gives you more fine-grained control over the data you collect for each application. This option also supports compatibility with earlier versions of MicroProfile Telemetry, which created only application SDK instances. Runtime-level data is not collected or emitted with application-level instances. +Configuring application-level OpenTelemetry instances gives you more fine-grained control over the data you collect for each application. This option also supports compatibility with MicroProfile Telemetry 1.1 and earlier, which created only application-level SDK instances. Runtime-level data is not collected or emitted with application-level instances. + You can configure your application-level OpenTelemetry instances in the applications `microprofile-config.properties` files or the `appProperties` attribute in the `server.xml` file for each application. When you create application-level instances, any system and server environment properties overwrite your application-level configuration. For more information about configuration sources and their ordinal values, see xref:external-configuration.adoc#default[Default configuration sources]. @@ -59,7 +59,7 @@ otel.service.name=RT1 + The `otel.service.name` property creates a name for any telemetry that this OpenTelemetry SDK instance collects and emits. + -If you choose to set configuration properties by using environment variables in your `server.env` file, make the key name uppercase and convert any punctuation to underscores. For example, the `otel.sdk.disabled=false` property is equivalent to the `OTEL_SDK_DISABLED=false` environment variable. +Alternatively, you can set runtime-level configuration properties by using environment variables in your `server.env` file. For any property definition, make the key name uppercase and convert any punctuation to underscores. For example, the `otel.sdk.disabled=false` property is equivalent to the `OTEL_SDK_DISABLED=false` environment variable. .. To enable the OpenTelemetry SDK at the application level, set the `otel.sdk.disabled=false property` in the `microprofile-config.properties` file of each application. + @@ -77,11 +77,11 @@ otel.sdk.disabled=false otel.service.name=A2 ---- + -This configuration creates all telemetry from App1 with the service name A1, and from App2 with the service name A2. It omits all runtime-level telemetry. +This configuration creates all telemetry from Application 1 with the service name A1, and from Application 2 with the service name A2. It omits all runtime-level telemetry. .. Optionally, use a combination of application-level and runtime-level configuration. + -Regardless of whether a runtime instance is created, any configuration in the system properties and server environment variables takes precedence over application-level configuration. Therefore, you can configure common application configuration at the runtime level in the server environment variables or system properties and make application-specific changes in your application configuration. +Regardless of whether a runtime instance is created, any configuration in the system properties and server environment variables takes precedence over application-level configuration. Therefore, you can configure shared application configuration at the runtime level and make application-specific changes in your application configuration. + For example, you can set the OpenTelemetry exporter at the runtime-level in the `bootstrap.properties` file: + @@ -89,14 +89,14 @@ For example, you can set the OpenTelemetry exporter at the runtime-level in the otel.logs.exporter=myExporter ---- + -Then, enable an application-level SDK instance in the `microprofile-config.properties` file for each application in the runtime: +Then, enable an application-level OpenTelemetry SDK instance in the `microprofile-config.properties` file for each application in the runtime: + ---- otel.sdk.disabled=false otel.service.name= ---- + -This configuration enables an application-level instance and a custom service name for each configured application, while all applications in the runtime use the `myExporter` OpenTelemetry exporter. +This configuration enables an application-level SDK instance and custom service name for each configured application, while all applications in the runtime use the `myExporter` OpenTelemetry exporter. [#traces] From 120103364f26da0e7667cc42eace7a9bd6f062cd Mon Sep 17 00:00:00 2001 From: David Mueller Date: Tue, 3 Sep 2024 14:42:04 -0400 Subject: [PATCH 38/62] Update microprofile-telemetry.adoc --- modules/ROOT/pages/microprofile-telemetry.adoc | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/modules/ROOT/pages/microprofile-telemetry.adoc b/modules/ROOT/pages/microprofile-telemetry.adoc index a286101434..998fe3917a 100644 --- a/modules/ROOT/pages/microprofile-telemetry.adoc +++ b/modules/ROOT/pages/microprofile-telemetry.adoc @@ -28,7 +28,7 @@ The following sections explain how to prepare your Open Liberty runtime and appl [#global] == Enabling MicroProfile Telemetry for Open Liberty -OpenTelemetry is disabled by default. Depending on your needs, you can configure OpenTelemetry SDK instances at the runtime level or the application level. +To enable MicroProfile Telemetry in your Open Liberty runtime, you must add the MicroProfile Telemetry feature to your `server.xml` file and enable the OpenTelemetry SDK.OpenTelemetry is disabled by default. Depending on your needs, you can configure OpenTelemetry SDK instances at the runtime level or the application level. Runtime-level configuration:: Runtime-level configuration collects and emits telemetry from the runtime and all applications. However, this configuration is preferable when you are running a single application per Liberty runtime because all telemetry emitted is labeled as coming from the same service. @@ -36,13 +36,13 @@ Runtime-level configuration collects and emits telemetry from the runtime and al You can configure your runtime instance in a few different sources, including your `jvm.options`, `bootstrap.properties`, or `server.env` files. You do not need to modify your application configuration to configure your runtime instance. Application-level configuration:: -Configuring application-level OpenTelemetry instances gives you more fine-grained control over the data you collect for each application. This option also supports compatibility with MicroProfile Telemetry 1.1 and earlier, which created only application-level SDK instances. Runtime-level data is not collected or emitted with application-level instances. +Configuring application-level OpenTelemetry instances gives you more fine-grained control over the data you collect for each application. This option also supports compatibility earlier versions of OpenTelemetry, which created only application-level SDK instances. Runtime-level data is not collected or emitted by application-level instances. + -You can configure your application-level OpenTelemetry instances in the applications `microprofile-config.properties` files or the `appProperties` attribute in the `server.xml` file for each application. When you create application-level instances, any system and server environment properties overwrite your application-level configuration. For more information about configuration sources and their ordinal values, see xref:external-configuration.adoc#default[Default configuration sources]. +You can configure your application-level OpenTelemetry instances in the `microprofile-config.properties` file or the `appProperties` attribute for the config:application[] element in the `server.xml` file. When you create application-level instances, any system and server environment properties overwrite your application-level configuration. +For more information about configuration sources and their ordinal values, see xref:external-configuration.adoc#default[Default configuration sources]. - -To enable MicroProfile Telemetry in your Open Liberty runtime, you must add the MicroProfile Telemetry feature to your `server.xml` file and enable the OpenTelemetry SDK. +To enable MicroProfile Telemetry to collect and emit logs, metrics, and traces, complete the following steps. . Enable the feature:mpTelemetry[display=MicroProfile Telemetry] feature in your `server.xml` file. To export metrics or logs, you must enable `mpTelemetry-2.0` or later. From 43608ec9b932748c4dddea9a428586943e98c374 Mon Sep 17 00:00:00 2001 From: David Mueller Date: Wed, 4 Sep 2024 08:46:15 -0400 Subject: [PATCH 39/62] edits per review #7459 --- modules/ROOT/pages/microprofile-telemetry.adoc | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/modules/ROOT/pages/microprofile-telemetry.adoc b/modules/ROOT/pages/microprofile-telemetry.adoc index 998fe3917a..94feaba527 100644 --- a/modules/ROOT/pages/microprofile-telemetry.adoc +++ b/modules/ROOT/pages/microprofile-telemetry.adoc @@ -28,15 +28,15 @@ The following sections explain how to prepare your Open Liberty runtime and appl [#global] == Enabling MicroProfile Telemetry for Open Liberty -To enable MicroProfile Telemetry in your Open Liberty runtime, you must add the MicroProfile Telemetry feature to your `server.xml` file and enable the OpenTelemetry SDK.OpenTelemetry is disabled by default. Depending on your needs, you can configure OpenTelemetry SDK instances at the runtime level or the application level. +To enable MicroProfile Telemetry in your Open Liberty runtime, you must add the MicroProfile Telemetry feature to your `server.xml` file and enable the OpenTelemetry SDK.OpenTelemetry is disabled by default. Depending on your needs, you can configure OpenTelemetry SDK instances at the runtime level or the application level. In most cases, runtime-level configuration is preferred as it includes both runtime-level telemetry and application-specific telemetry. Runtime-level configuration:: -Runtime-level configuration collects and emits telemetry from the runtime and all applications. However, this configuration is preferable when you are running a single application per Liberty runtime because all telemetry emitted is labeled as coming from the same service. +Runtime-level configuration collects and emits telemetry from the runtime and all applications. As all telemetry from the runtime and applications is labeled as coming from the same service, this configuration works best when you are running a single application per Liberty runtime. + You can configure your runtime instance in a few different sources, including your `jvm.options`, `bootstrap.properties`, or `server.env` files. You do not need to modify your application configuration to configure your runtime instance. Application-level configuration:: -Configuring application-level OpenTelemetry instances gives you more fine-grained control over the data you collect for each application. This option also supports compatibility earlier versions of OpenTelemetry, which created only application-level SDK instances. Runtime-level data is not collected or emitted by application-level instances. +Configuring application-level OpenTelemetry instances gives you more fine-grained control over the data you collect for each application. This option also supports configuration-compatibility with MicroProfile Telemetry 1.1 and earlier, which created only application-level SDK instances. Runtime-level data is not collected or emitted by application-level instances. + You can configure your application-level OpenTelemetry instances in the `microprofile-config.properties` file or the `appProperties` attribute for the config:application[] element in the `server.xml` file. When you create application-level instances, any system and server environment properties overwrite your application-level configuration. @@ -160,7 +160,7 @@ You can configure how MicroProfile Telemetry collects and exports logs by specif .. Enable MicroProfile Telemetry to collect logs from different sources in the Open Liberty runtime environment. + -Configure the `source` attribute for the `mpTelemetry` element with a comma-separated list of comma-separated log sources: +Configure the `source` attribute for the `mpTelemetry` element with a comma-separated list of log sources: + [source,xml] ---- @@ -181,7 +181,7 @@ otel.sdk.disabled=false otel.logs.exporter=console ---- + -If you set this property to `console`, all the logs are exported to standard out (`stdout`) and the `console.log` file. The `console.log` file contains the usual logs, along with duplicate OpenTelemetry-mapped logs. This setting is only for debugging purposes because the `console.log` file does not roll over and might affect performance if it gets too large. If you set this property to `none`, no logs are exported to OpenTelemetry. +If you set this property to `console`, all the logs are exported to standard out (`stdout`) or the `console.log` file. The `console.log` file contains the usual logs, along with duplicate OpenTelemetry-mapped logs. This setting is only for debugging purposes because the `console.log` file does not roll over and might affect performance if it gets too large. If you set this property to `none`, no logs are exported. + .. Configure MicroProfile Telemetry to use the OpenTelemetry Batch LogRecord Processor. + @@ -212,7 +212,7 @@ By default, all OpenTelemetry data is exported to link:https://opentelemetry.io/ For example, to export metrics to Open Liberty log files, specify the following property: + ---- -otel.metrics.exporter=logging +otel.metrics.exporter=console ---- . Optionally, change the metric export interval. From 787ead15349d0fc1525f2e4c179c0a553a8b2673 Mon Sep 17 00:00:00 2001 From: David Mueller Date: Wed, 4 Sep 2024 09:57:03 -0400 Subject: [PATCH 40/62] Update microprofile-telemetry.adoc --- modules/ROOT/pages/microprofile-telemetry.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/ROOT/pages/microprofile-telemetry.adoc b/modules/ROOT/pages/microprofile-telemetry.adoc index 94feaba527..a487f511b5 100644 --- a/modules/ROOT/pages/microprofile-telemetry.adoc +++ b/modules/ROOT/pages/microprofile-telemetry.adoc @@ -28,7 +28,7 @@ The following sections explain how to prepare your Open Liberty runtime and appl [#global] == Enabling MicroProfile Telemetry for Open Liberty -To enable MicroProfile Telemetry in your Open Liberty runtime, you must add the MicroProfile Telemetry feature to your `server.xml` file and enable the OpenTelemetry SDK.OpenTelemetry is disabled by default. Depending on your needs, you can configure OpenTelemetry SDK instances at the runtime level or the application level. In most cases, runtime-level configuration is preferred as it includes both runtime-level telemetry and application-specific telemetry. +To enable MicroProfile Telemetry in your Open Liberty runtime, you must add the MicroProfile Telemetry feature to your `server.xml` file and enable the OpenTelemetry SDK. OpenTelemetry is disabled by default. Depending on your needs, you can configure OpenTelemetry SDK instances at the runtime level or the application level. In most cases, runtime-level configuration is preferred as it includes both runtime-level telemetry and application-specific telemetry. Runtime-level configuration:: Runtime-level configuration collects and emits telemetry from the runtime and all applications. As all telemetry from the runtime and applications is labeled as coming from the same service, this configuration works best when you are running a single application per Liberty runtime. From 41bcc225aa47d9d207be951b76993d3ddd17db1f Mon Sep 17 00:00:00 2001 From: David Mueller Date: Wed, 4 Sep 2024 12:25:44 -0400 Subject: [PATCH 41/62] edits per review #7466 --- modules/ROOT/pages/metrics-list.adoc | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/modules/ROOT/pages/metrics-list.adoc b/modules/ROOT/pages/metrics-list.adoc index 1ccdab14a8..13b921f274 100644 --- a/modules/ROOT/pages/metrics-list.adoc +++ b/modules/ROOT/pages/metrics-list.adoc @@ -1612,7 +1612,7 @@ When the MicroProfile Telemetry feature 2.0 and later is enabled, Open Liberty c |* `http.request.method` * `url.scheme` * `errory.type` (conditionally required) -** `http.response.status_code` +* `http.response.status_code` * `http.route` * `network.protocol.version` * `server.address` @@ -1755,7 +1755,7 @@ When the MicroProfile Telemetry feature 2.0 and later is enabled, Open Liberty c | feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] | jvm.memory.committed -| * vm.memory.pool.name="" +| * jvm.memory.pool.name="" * jvm.memory.type= | Measure of memory committed. This metric is an `UpDownCounter` counter. /(bytes) | n/a @@ -1787,7 +1787,7 @@ When the MicroProfile Telemetry feature 2.0 and later is enabled, Open Liberty c | feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] | jvm.thread.count -| * *jvm.thread.daemon= +| * jvm.thread.daemon= * jvm.thread.state= | Number of executing platform threads. This metric is an `UpDownCounter` counter. /({thread}) | n/a From ac25fbd73aed68914479a1b22a9d39f101337e0a Mon Sep 17 00:00:00 2001 From: David Mueller Date: Wed, 4 Sep 2024 17:49:51 -0400 Subject: [PATCH 42/62] edits per review #7459 --- .../pages/custom-mptelemetry-metrics.adoc | 46 +++ modules/ROOT/pages/metrics-list.adoc | 244 +------------- .../ROOT/pages/microprofile-telemetry.adoc | 35 -- .../ROOT/pages/mptelemetry-metrics-list.adoc | 318 ++++++++++++++++++ 4 files changed, 365 insertions(+), 278 deletions(-) create mode 100644 modules/ROOT/pages/custom-mptelemetry-metrics.adoc create mode 100644 modules/ROOT/pages/mptelemetry-metrics-list.adoc diff --git a/modules/ROOT/pages/custom-mptelemetry-metrics.adoc b/modules/ROOT/pages/custom-mptelemetry-metrics.adoc new file mode 100644 index 0000000000..3117eff2df --- /dev/null +++ b/modules/ROOT/pages/custom-mptelemetry-metrics.adoc @@ -0,0 +1,46 @@ +// Copyright (c) 2022 IBM Corporation and others. +// Licensed under Creative Commons Attribution-NoDerivatives +// 4.0 International (CC BY-ND 4.0) +// https://creativecommons.org/licenses/by-nd/4.0/ +// +// Contributors: +// IBM Corporation +// +:page-description: +:seo-description: +:page-layout: general-reference +:page-type: general += Define custom MicroProfile Telemetry metrics + +You can use the OpenTelemetry metrics API to define custom metrics in your application code. When you enable the MicroProfile Telemetry feature 2.0 or later, you can then collect and emit these metrics to customize the observability of your application. + + +The following example defines a custom counter metric: + +[source,java] +---- +class WithCounter { + @Inject + Meter meter; + + private LongCounter counter; + + @PostConstruct + public void init() { + counter = meter + .counterBuilder("new_subscriptions") + .setDescription("Number of new subscriptions") + .setUnit("1") + .build(); + } + + void subscribe(String plan) { + counter.add(1, + Attributes.of(AttributeKey.stringKey("plan"), plan)); + } +} +---- + +In this example, `Meter` is used to define an instrument, in this case a Counter. Application code then can record measurement values along with other attributes. Measurement aggregations are computed separately for each unique combination of attributes. + +For a full list of available metrics, see link:https://opentelemetry.io/docs/specs/otel/metrics/api/#meter-operations[Meter operations] in the OpenTelemetry documentation. diff --git a/modules/ROOT/pages/metrics-list.adoc b/modules/ROOT/pages/metrics-list.adoc index 13b921f274..1daa7d07eb 100644 --- a/modules/ROOT/pages/metrics-list.adoc +++ b/modules/ROOT/pages/metrics-list.adoc @@ -25,7 +25,7 @@ :ft-metric-features: feature:mpMetrics[display=MicroProfile Metrics] and feature:mpFaultTolerance[display=MicroProfile Fault Tolerance] :grpc-client-metric-features: feature:mpMetrics[display=MicroProfile Metrics] and feature:grpcClient[display=gRPC Client] :grpc-server-metric-features: feature:mpMetrics[display=MicroProfile Metrics] and feature:grpc[display=gRPC] -= Metrics reference list += MIcroProfile Metrics reference list The metrics reference tables list and describe all the metrics that are available for Open Liberty. Use metric data to effectively monitor the status of your microservice systems. @@ -69,7 +69,6 @@ The metrics reference tables list the available metrics when you use one of the * <<#metrics-table-mp-metrics-5, MicroProfile Metrics 5.0>> * <<#metrics-table,MicroProfile Metrics 4.0 (or earlier versions)>> -* <<#telem-table, MicroProfile Telemetry 2.0 and later>> In each table, the Prometheus metric names are listed after each metric. The tables also list the metric types, metric units, and descriptions of all metrics that are available for Open Liberty. For the vendor metrics, the associated monitoring component that you can use to filter the metric is also included. @@ -1597,247 +1596,6 @@ This metric is a gauge. |=== -== MicroProfile Telemetry 2.0 and later metrics reference - -When the MicroProfile Telemetry feature 2.0 and later is enabled, Open Liberty can sends metric data to compatible OpenTelemetry protocol (OTLP) receivers. For more information, see xref:microprofile-telemetry[Enable observability with MicroProfile Telemetry]. - -// Assisted by watsonx latest genai contribution llama-3-70b -[#telem-table] -.MicroProfile Telemetry 2.0 and later metrics reference -[options="header"cols="3,4a,6a,2,2,2"] -|=== -| MicroProfile Telemetry 2.0 name | Attributes | Type and description | Monitoring component | Features required | Version introduced - -| `http.server.request.duration` -|* `http.request.method` -* `url.scheme` -* `errory.type` (conditionally required) -* `http.response.status_code` -* `http.route` -* `network.protocol.version` -* `server.address` -* `server.port` -| Duration of HTTP server requests. This metric is a Histogram. / (seconds). This histogram has the following explicit bucket boundaries [ 0.005, 0.01, 0.025, 0.05, 0.075, 0.1, 0.25, 0.5, 0.75, 1, 2.5, 5, 7.5, 10 ] -| HTTP -| feature:mpTelemetry[display=MicroProfile Telemetry] -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] - -| io.openliberty.connection_pool.handle.count -| io.openliberty.datasource.name="" -| The number of connections that are in use. This number might include multiple connections that are shared from a single managed connection. This metric is an `ObservableLongUpDownCounter` counter. / ({connection_handle}) -| ConnectionPool -| feature:mpTelemetry[display=MicroProfile Telemetry] -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] - -| io.openliberty.connection_pool.connection.created -| io.openliberty.datasource.name="" -| The total number of managed connections that were created since the pool creation. This metric is an `ObservableLongCounter` counter. / ({connection_handle}) -| ConnectionPool -| feature:mpTelemetry[display=MicroProfile Telemetry] -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] - -| io.openliberty.connection_pool.connection.destroyed -| io.openliberty.datasource.name="" -| The total number of managed connections that were destroyed since the pool creation. This metric is an `ObservableLongCounter` counter. / ({connection_handle}) -| ConnectionPool -| feature:mpTelemetry[display=MicroProfile Telemetry] -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] - -| io.openliberty.connection_pool.connection.free -| io.openliberty.datasource.name="" -| The number of managed connections that are available. This metric is an `ObservableLongUpDownCounter` counter. / ({connection_handle}) -| ConnectionPool -| feature:mpTelemetry[display=MicroProfile Telemetry] -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] - -| io.openliberty.connection_pool.connection.use_time -| io.openliberty.datasource.name="" -| The amount of time connections were used for. This metric is an DoubleHistogram / (seconds). This histogram has the following explicit bucket boundaries [ 0.001, 0.005, 0.01, 0.05, 0.1, 0.5, 1, 5, 10] -| ConnectionPool -| feature:mpTelemetry[display=MicroProfile Telemetry] -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] - -| io.openliberty.connection_pool.connection.count -| io.openliberty.datasource.name="" -| The current sum of managed connections in the pool. This sum includes managed connections that are available and those that are in use. A single managed connection that is shared by multiple connections only counts once. This metric is an `ObservableLongUpDownCounter` counter / ({connection_handle}) -| ConnectionPool | feature:mpTelemetry[display=MicroProfile Telemetry] -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] - -| io.openliberty.connection_pool.connection.wait_time -| io.openliberty.datasource.name="" -| The amount of time that connection requests waited for a connection. This metric is an `DoubleHistogram` histogram / (seconds). This histogram has the following explicit bucket boundaries [ 0.001, 0.005, 0.01, 0.05, 0.1, 0.5, 1, 5, 10] -| ConnectionPool -| feature:mpTelemetry[display=MicroProfile Telemetry] -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] - -| io.openliberty.request_timing.active -| n/a -| The number of servlet requests that are currently running. This metric is an `ObservableLongUpDownCounter` counter. / ({request}) -| RequestTiming -| feature:mpTelemetry[display=MicroProfile Telemetry] -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] - -| io.openliberty.request_timing.hung -| n/a -| The number of servlet requests that are currently hung. This metric is an `ObservableLongUpDownCounter` counter. / ({request}) -| RequestTiming -| feature:mpTelemetry[display=MicroProfile Telemetry] -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] - -| io.openliberty.request_timing.processed -| n/a -| The number of servlet requests since the server started. This metric is an `ObservableLongCounter` counter. / ({request}) -| RequestTiming -| feature:mpTelemetry[display=MicroProfile Telemetry] -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] - -| io.openliberty.request_timing.slow -| n/a -| The number of servlet requests that are currently running but are slow. This metric is an `ObservableLongUpDownCounter` counter. / ({request}) -| RequestTiming -| feature:mpTelemetry[display=MicroProfile Telemetry] -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] - -| io.openliberty.session.active -| io.openliberty.app.name="" -| The number of concurrently active sessions. A session is considered active if the application server is processing a request that uses that user session. This metric is an `ObservableLongUpDownCounter` counter. / ({session}) -| Session -| feature:mpTelemetry[display=MicroProfile Telemetry] -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] - -| io.openliberty.session.created -| io.openliberty.app.name="" -| The number of sessions logged in since this metric was enabled. This metric is an `ObservableLongCounter` counter. / ({session}) -| Session -| feature:mpTelemetry[display=MicroProfile Telemetry] -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] - -| io.openliberty.session.invalidated -| io.openliberty.app.name="" -| The number of sessions logged out since this metric was enabled. This metric is an `ObservableLongCounter` counter. / ({session}) -| Session -| feature:mpTelemetry[display=MicroProfile Telemetry] -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] - -| io.openliberty.session.invalidated_by_timeout -| io.openliberty.app.name="" -| The number of sessions logged out because of a timeout since this metric was enabled. This metric is an `ObservableLongCounter` counter. / ({session}) -| Session -| feature:mpTelemetry[display=MicroProfile Telemetry] -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] - -| io.openliberty.session.live -| io.openliberty.app.name="" -| The number of users that are currently logged in. This metric is an `ObservableLongUpDownCounter` counter. / ({session}) -| Session -| feature:mpTelemetry[display=MicroProfile Telemetry] -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] - -| io.openliberty.threadpool.active_threads -| io.openliberty.threadpool.name="" -| The number of threads that are actively running tasks. This metric is an `ObservableLongUpDownCounter` counter. / ({thread}) -| ThreadPool -| feature:mpTelemetry[display=MicroProfile Telemetry] -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] - -| io.openliberty.threadpool.size -| io.openliberty.threadpool.name="" -| The size of the thread pool. This metric is an `ObservableLongUpDownCounter` counter. / ({thread}) -| ThreadPool -| feature:mpTelemetry[display=MicroProfile Telemetry] -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] - -| jvm.memory.used -| vm.memory.pool.name="" , -jvm.memory.type= -| Measure of memory used. This metric is an `UpDownCounter` counter. /(bytes) -| n/a -| feature:mpTelemetry[display=MicroProfile Telemetry] -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] - -| jvm.memory.committed -| * jvm.memory.pool.name="" -* jvm.memory.type= -| Measure of memory committed. This metric is an `UpDownCounter` counter. /(bytes) -| n/a -| feature:mpTelemetry[display=MicroProfile Telemetry] -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] - -| jvm.memory.limit -| * vm.memory.pool.name="" -* jvm.memory.type= -| Measure of max obtainable memory. This metric is an `UpDownCounter` counter. /(bytes) -| n/a -| feature:mpTelemetry[display=MicroProfile Telemetry] -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] - -| jvm.memory.used_after_last_gc -| * vm.memory.pool.name="" -* jvm.memory.type= -| Measure of memory used, as measured after the most recent garbage collection event on this pool. This metric is an `UpDownCounter` counter. /(bytes) -| n/a -| feature:mpTelemetry[display=MicroProfile Telemetry] -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] - -| jvm.gc.duration -| * jvm.gc.action= -* jvm.gc.name= -| Duration of JVM garbage collection actions. This metric is an Histogram. / (seconds). This histogram has the following explicit bucket boundaries [ 0.01, 0.1, 1, 10 ] -| n/a -| feature:mpTelemetry[display=MicroProfile Telemetry] -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] - -| jvm.thread.count -| * jvm.thread.daemon= -* jvm.thread.state= -| Number of executing platform threads. This metric is an `UpDownCounter` counter. /({thread}) -| n/a -| feature:mpTelemetry[display=MicroProfile Telemetry] -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] - -| jvm.class.loaded -| n/a -| Number of classes loaded since JVM start. This metric is a Counter. / ({class}) -| n/a -| feature:mpTelemetry[display=MicroProfile Telemetry] -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] - - -| jvm.class.unloaded -| n/a -| Number of classes unloaded since JVM start. This metric is a Counter. / ({class}) -| n/a -| feature:mpTelemetry[display=MicroProfile Telemetry] -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] - -| jvm.class.count -| n/a -| Number of classes currently loaded. This metric is an `UpDownCounter` counter. /({class}) -| n/a -| feature:mpTelemetry[display=MicroProfile Telemetry] -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] - -| jvm.cpu.time -| n/a -| CPU time used by the process as reported by the JVM. This metric is a Counter -| n/a -| feature:mpTelemetry[display=MicroProfile Telemetry] -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] - -| jvm.cpu.count -| n/a -| Number of processors available to the Java virtual machine. This metric is an `UpDownCounter` counter. /({cpu}) -| n/a -| feature:mpTelemetry[display=MicroProfile Telemetry] -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] - -| jvm.cpu.recent_utilization -| n/a -| Recent CPU utilization for the process as reported by the JVM. This metric is a Gauge. -| n/a -| feature:mpTelemetry[display=MicroProfile Telemetry] -| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] - -|=== {empty} + diff --git a/modules/ROOT/pages/microprofile-telemetry.adoc b/modules/ROOT/pages/microprofile-telemetry.adoc index a487f511b5..d485f2ec09 100644 --- a/modules/ROOT/pages/microprofile-telemetry.adoc +++ b/modules/ROOT/pages/microprofile-telemetry.adoc @@ -221,41 +221,6 @@ By default, metric data is exported at an interval of 60 seconds. To modify the Depending on how you choose to instrument your application code for metrics, further configuration might be required. - - -=== Custom metrics - -You can use the metrics API to define custom metrics in your application code, as shown in the following example: - -[source,java] ----- -class WithCounter { - @Inject - Meter meter; - - private LongCounter counter; - - @PostConstruct - public void init() { - counter = meter - .counterBuilder("new_subscriptions") - .setDescription("Number of new subscriptions") - .setUnit("1") - .build(); - } - - void subscribe(String plan) { - counter.add(1, - Attributes.of(AttributeKey.stringKey("plan"), plan)); - } -} ----- - -In this example, `Meter` is used to define an instrument, in this case a Counter. Application code then can record measurement values along with other attributes. Measurement aggregations are computed separately for each unique combination of attributes. - -For a full list of available metrics, see link:https://opentelemetry.io/docs/specs/otel/metrics/api/#meter-operations[Meter operations] in the OpenTelemetry documentation. - - [#trouble] == Troubleshooting MicroProfile Telemetry The following information can help you determine the cause of common problems and error messages. diff --git a/modules/ROOT/pages/mptelemetry-metrics-list.adoc b/modules/ROOT/pages/mptelemetry-metrics-list.adoc new file mode 100644 index 0000000000..c8e980ef99 --- /dev/null +++ b/modules/ROOT/pages/mptelemetry-metrics-list.adoc @@ -0,0 +1,318 @@ +// Copyright (c) 2019, 2022 IBM Corporation and others. +// Licensed under Creative Commons Attribution-NoDerivatives +// 4.0 International (CC BY-ND 4.0) +// https://creativecommons.org/licenses/by-nd/4.0/ +// +// Contributors: +// IBM Corporation +// +:page-description: The metrics contained in this reference list are all available for Open Liberty. Use metric data to effectively monitor the status of your microservice systems. +:seo-title: Metrics reference list - openliberty.io +:seo-description: The metrics contained in this reference list are all available for Open Liberty. Use metric data to effectively monitor the status of your microservice systems. +:page-layout: general-reference +:page-type: general +:mp-1-0: feature:mpMetrics-1.0[display=MicroProfile Metrics 1.0] +:mp-1-1: feature:mpMetrics-1.1[display=MicroProfile Metrics 1.1] +:mp-2-0: feature:mpMetrics-2.0[display=MicroProfile Metrics 2.0] +:mp-2-3: feature:mpMetrics-2.3[display=MicroProfile Metrics 2.3] +:mp-3-0: feature:mpMetrics-3.0[display=MicroProfile Metrics 3.0] +:mp-4-0: feature:mpMetrics-4.0[display=MicroProfile Metrics 4.0] +:mp-5-0: feature:mpMetrics-5.0[display=MicroProfile Metrics 5.0] +:mp-ft-3-0: feature:mpFaultTolerance-3.0[display=MicroProfile Fault Tolerance 3.0] +:connector-metric-features: feature:mpMetrics[display=MicroProfile Metrics] and feature:mpReactiveMessaging[display=MicroProfile Reactive Messaging 3.0] +:base-metric-features: feature:mpMetrics[display=MicroProfile Metrics] +:vendor-metric-features: feature:mpMetrics[display=MicroProfile Metrics] +:ft-metric-features: feature:mpMetrics[display=MicroProfile Metrics] and feature:mpFaultTolerance[display=MicroProfile Fault Tolerance] +:grpc-client-metric-features: feature:mpMetrics[display=MicroProfile Metrics] and feature:grpcClient[display=gRPC Client] +:grpc-server-metric-features: feature:mpMetrics[display=MicroProfile Metrics] and feature:grpc[display=gRPC] += MicroProfile Telemetry metrics reference list + +The MicroProfile Telemetry metrics reference table lists and describes all the metrics that are available for Open Liberty when you enable the MicroProfile Telemetry feature 2.0 or later. +Use metric data to effectively monitor the status of your microservice systems. + +You can obtain metrics from applications, the Open Liberty runtime, and the Java virtual machine (JVM). They can be gathered and stored in database tools, such as link:https://prometheus.io/[Prometheus], and displayed on dashboards, such as link:https://grafana.com/[Grafana]. + +When you enable the feature:mpTelemetry-2.0[display=MicroProfile Telemetry] feature version 2.0 and later, you can manage your logs, metrics, and traces in a standardized way with the OpenTelemetry protocol. For more information, see xref:microprofile-telemetry[Enable observability with MicroProfile Telemetry]. + + +== Filter metrics to gather only the data you need +By default, all monitoring components are enabled. If your server is collecting more metrics data than you need, you can improve the server performance by collecting only those vendor metrics that you intend to use. To configure only a subset of vendor metrics to be reported, specify the components that you want to monitor in the `filter` attribute for the `monitor` configuration element in your `server.xml` file. You can identify the relevant monitoring component for each vendor metric by referencing the **Monitoring component** column of the metrics reference tables. + +To enable only the monitoring components that are used by MicroProfile Telemetry, add the following code to your `server.xml` file. + +[source,xml] +---- + +---- + +To disable all vendor metrics, but to keep the `REST` base metrics configure the `server.xml`, as shown in the following example: + +[source,xml] +---- + +---- + +To disable all monitoring components, add the following code to your `server.xml` file: + +[source,xml] +---- + +---- + + +== MicroProfile Telemetry 2.0 and later metrics reference + +When the MicroProfile Telemetry feature 2.0 and later is enabled, Open Liberty can sends metric data to compatible OpenTelemetry protocol (OTLP) receivers. For more information, see xref:microprofile-telemetry[Enable observability with MicroProfile Telemetry]. + +In the table, the Prometheus metric names are listed after each metric. +The tables also list the metric types, metric units, and descriptions of all metrics that are available for Open Liberty. For the vendor metrics, the associated monitoring component that you can use to filter the metric is also included. +The **Features required** column of the table includes the feature or features that must be enabled to obtain that metric data. +The **Version introduced** column specifies the minimum version of the feature that you must enable to collect the metric. + +// Assisted by watsonx latest genai contribution llama-3-70b +[#telem-table] +.MicroProfile Telemetry 2.0 and later metrics reference +[options="header"cols="3,4a,6a,2,2,2"] +|=== +| MicroProfile Telemetry 2.0 name | Attributes | Type and description | Monitoring component | Features required | Version introduced + +| `http.server.request.duration` +|* `http.request.method` +* `url.scheme` +* `errory.type` (conditionally required) +* `http.response.status_code` +* `http.route` +* `network.protocol.version` +* `server.address` +* `server.port` +| Duration of HTTP server requests. This metric is a Histogram. / (seconds). This histogram has the following explicit bucket boundaries [ 0.005, 0.01, 0.025, 0.05, 0.075, 0.1, 0.25, 0.5, 0.75, 1, 2.5, 5, 7.5, 10 ] +| HTTP +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.connection_pool.handle.count +| io.openliberty.datasource.name="" +| The number of connections that are in use. This number might include multiple connections that are shared from a single managed connection. This metric is an `ObservableLongUpDownCounter` counter. / ({connection_handle}) +| ConnectionPool +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.connection_pool.connection.created +| io.openliberty.datasource.name="" +| The total number of managed connections that were created since the pool creation. This metric is an `ObservableLongCounter` counter. / ({connection_handle}) +| ConnectionPool +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.connection_pool.connection.destroyed +| io.openliberty.datasource.name="" +| The total number of managed connections that were destroyed since the pool creation. This metric is an `ObservableLongCounter` counter. / ({connection_handle}) +| ConnectionPool +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.connection_pool.connection.free +| io.openliberty.datasource.name="" +| The number of managed connections that are available. This metric is an `ObservableLongUpDownCounter` counter. / ({connection_handle}) +| ConnectionPool +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.connection_pool.connection.use_time +| io.openliberty.datasource.name="" +| The amount of time connections were used for. This metric is an DoubleHistogram / (seconds). This histogram has the following explicit bucket boundaries [ 0.001, 0.005, 0.01, 0.05, 0.1, 0.5, 1, 5, 10] +| ConnectionPool +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.connection_pool.connection.count +| io.openliberty.datasource.name="" +| The current sum of managed connections in the pool. This sum includes managed connections that are available and those that are in use. A single managed connection that is shared by multiple connections only counts once. This metric is an `ObservableLongUpDownCounter` counter / ({connection_handle}) +| ConnectionPool | feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.connection_pool.connection.wait_time +| io.openliberty.datasource.name="" +| The amount of time that connection requests waited for a connection. This metric is an `DoubleHistogram` histogram / (seconds). This histogram has the following explicit bucket boundaries [ 0.001, 0.005, 0.01, 0.05, 0.1, 0.5, 1, 5, 10] +| ConnectionPool +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.request_timing.active +| n/a +| The number of servlet requests that are currently running. This metric is an `ObservableLongUpDownCounter` counter. / ({request}) +| RequestTiming +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.request_timing.hung +| n/a +| The number of servlet requests that are currently hung. This metric is an `ObservableLongUpDownCounter` counter. / ({request}) +| RequestTiming +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.request_timing.processed +| n/a +| The number of servlet requests since the server started. This metric is an `ObservableLongCounter` counter. / ({request}) +| RequestTiming +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.request_timing.slow +| n/a +| The number of servlet requests that are currently running but are slow. This metric is an `ObservableLongUpDownCounter` counter. / ({request}) +| RequestTiming +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.session.active +| io.openliberty.app.name="" +| The number of concurrently active sessions. A session is considered active if the application server is processing a request that uses that user session. This metric is an `ObservableLongUpDownCounter` counter. / ({session}) +| Session +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.session.created +| io.openliberty.app.name="" +| The number of sessions logged in since this metric was enabled. This metric is an `ObservableLongCounter` counter. / ({session}) +| Session +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.session.invalidated +| io.openliberty.app.name="" +| The number of sessions logged out since this metric was enabled. This metric is an `ObservableLongCounter` counter. / ({session}) +| Session +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.session.invalidated_by_timeout +| io.openliberty.app.name="" +| The number of sessions logged out because of a timeout since this metric was enabled. This metric is an `ObservableLongCounter` counter. / ({session}) +| Session +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.session.live +| io.openliberty.app.name="" +| The number of users that are currently logged in. This metric is an `ObservableLongUpDownCounter` counter. / ({session}) +| Session +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.threadpool.active_threads +| io.openliberty.threadpool.name="" +| The number of threads that are actively running tasks. This metric is an `ObservableLongUpDownCounter` counter. / ({thread}) +| ThreadPool +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| io.openliberty.threadpool.size +| io.openliberty.threadpool.name="" +| The size of the thread pool. This metric is an `ObservableLongUpDownCounter` counter. / ({thread}) +| ThreadPool +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| jvm.memory.used +| vm.memory.pool.name="" , -jvm.memory.type= +| Measure of memory used. This metric is an `UpDownCounter` counter. /(bytes) +| n/a +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| jvm.memory.committed +| * jvm.memory.pool.name="" +* jvm.memory.type= +| Measure of memory committed. This metric is an `UpDownCounter` counter. /(bytes) +| n/a +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| jvm.memory.limit +| * vm.memory.pool.name="" +* jvm.memory.type= +| Measure of max obtainable memory. This metric is an `UpDownCounter` counter. /(bytes) +| n/a +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| jvm.memory.used_after_last_gc +| * vm.memory.pool.name="" +* jvm.memory.type= +| Measure of memory used, as measured after the most recent garbage collection event on this pool. This metric is an `UpDownCounter` counter. /(bytes) +| n/a +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| jvm.gc.duration +| * jvm.gc.action= +* jvm.gc.name= +| Duration of JVM garbage collection actions. This metric is an Histogram. / (seconds). This histogram has the following explicit bucket boundaries [ 0.01, 0.1, 1, 10 ] +| n/a +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| jvm.thread.count +| * jvm.thread.daemon= +* jvm.thread.state= +| Number of executing platform threads. This metric is an `UpDownCounter` counter. /({thread}) +| n/a +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| jvm.class.loaded +| n/a +| Number of classes loaded since JVM start. This metric is a Counter. / ({class}) +| n/a +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + + +| jvm.class.unloaded +| n/a +| Number of classes unloaded since JVM start. This metric is a Counter. / ({class}) +| n/a +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| jvm.class.count +| n/a +| Number of classes currently loaded. This metric is an `UpDownCounter` counter. /({class}) +| n/a +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| jvm.cpu.time +| n/a +| CPU time used by the process as reported by the JVM. This metric is a Counter +| n/a +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| jvm.cpu.count +| n/a +| Number of processors available to the Java virtual machine. This metric is an `UpDownCounter` counter. /({cpu}) +| n/a +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +| jvm.cpu.recent_utilization +| n/a +| Recent CPU utilization for the process as reported by the JVM. This metric is a Gauge. +| n/a +| feature:mpTelemetry[display=MicroProfile Telemetry] +| feature:mpTelemetry-2.0[display=MicroProfile Telemetry 2.0] + +|=== + +{empty} + + + + +== See also + +* Guide: link:/guides/microprofile-metrics.html[Providing metrics from a microservice] +* xref:reference:diff/mp-21-22-diff.adoc[Differences between MicroProfile 2.1 and 2.2] +* xref:microservice-observability-metrics.adoc[Microservice observability with metrics] From 83f562e60942716efd2cc8853bf6d78f5462955b Mon Sep 17 00:00:00 2001 From: David Mueller Date: Thu, 5 Sep 2024 09:53:33 -0400 Subject: [PATCH 43/62] add links #7459 --- modules/ROOT/pages/microprofile-telemetry.adoc | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/modules/ROOT/pages/microprofile-telemetry.adoc b/modules/ROOT/pages/microprofile-telemetry.adoc index d485f2ec09..839e87d860 100644 --- a/modules/ROOT/pages/microprofile-telemetry.adoc +++ b/modules/ROOT/pages/microprofile-telemetry.adoc @@ -196,6 +196,8 @@ For more information, see the OpenTelemetry link:https://opentelemetry.io/docs/s For more information about the available properties, see xref:microprofile-config-properties.adoc#telemetry[MicroProfile Config properties: MicroProfile Telemetry]. +For information about Liberty message event fields for MicroProfile Telemetry, see xref:mptel-log-events-list.adoc[MicroProfile Telemetry log events reference list]. + [#metrics] == Configuring Open Liberty to use MicroProfile Telemetry to collect metrics @@ -219,7 +221,9 @@ otel.metrics.exporter=console + By default, metric data is exported at an interval of 60 seconds. To modify the export interval, specify the `otel.metric.export.interval` property or the `OTEL_METRIC_EXPORT_INTERVAL` environment variable. Specify the value in milliseconds. For more information, see link:https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#periodic-exporting-metricreader[Periodic exporter MetricReader] in the OpenTelemetry documentation. -Depending on how you choose to instrument your application code for metrics, further configuration might be required. +Depending on how you choose to instrument your application code for metrics, further configuration might be required. For information about defining your own metrics, see xref:custom-mptelemetry-metrics.adoc[Define custom MicroProfile Telemetry metrics]. + +For a list of metrics that are available for Open Liberty when you enable the MicroProfile Telemetry feature 2.0 or later, see xref:mptelemetry-metrics-list.adoc[MicroProfile Telemetry metrics reference list]. [#trouble] == Troubleshooting MicroProfile Telemetry From b282935b8d137e376c34ecaac1188e65ebe4ac44 Mon Sep 17 00:00:00 2001 From: David Mueller Date: Thu, 5 Sep 2024 09:58:48 -0400 Subject: [PATCH 44/62] remove exrtra example --- .../pages/feature/mpTelemetry/examples.adoc | 16 ---------------- 1 file changed, 16 deletions(-) delete mode 100644 modules/reference/pages/feature/mpTelemetry/examples.adoc diff --git a/modules/reference/pages/feature/mpTelemetry/examples.adoc b/modules/reference/pages/feature/mpTelemetry/examples.adoc deleted file mode 100644 index 2f0ca17cad..0000000000 --- a/modules/reference/pages/feature/mpTelemetry/examples.adoc +++ /dev/null @@ -1,16 +0,0 @@ -== Examples - -=== Collect logs from specified sources - -To enable the MicroProfile Telemetry feature to collect logs from different sources in the Open Liberty runtime environment, configure the `source` attribute for the config:mpTelemetry[] element with a comma-separated list of log sources. - -[source,xml] ----- - ----- - -* The `mpTelemetry` configuration element is optional. If you do not specify it, or if you do not include the `source` attribute, the default configuration source is `message`. - -* If the source attribute is empty (`source=“”`), no sources are collected, and nothing is routed to the configured OpenTelemetry logs exporter. - -* The attributes and values are case insensitive. From 8847905f266f0f7b0a1fb8121a756fc99b195be6 Mon Sep 17 00:00:00 2001 From: David Mueller Date: Thu, 5 Sep 2024 10:08:12 -0400 Subject: [PATCH 45/62] Update metrics-list.adoc --- modules/ROOT/pages/metrics-list.adoc | 4 ---- 1 file changed, 4 deletions(-) diff --git a/modules/ROOT/pages/metrics-list.adoc b/modules/ROOT/pages/metrics-list.adoc index 1daa7d07eb..673bb8e49c 100644 --- a/modules/ROOT/pages/metrics-list.adoc +++ b/modules/ROOT/pages/metrics-list.adoc @@ -30,12 +30,8 @@ The metrics reference tables list and describe all the metrics that are available for Open Liberty. Use metric data to effectively monitor the status of your microservice systems. -The metrics reference tables list and describe all the metrics that are available for Open Liberty with MicroProfile Metrics and MicroProfile Telemetry 2.0 and later. Use metric data to effectively monitor the status of your microservice systems. - You can obtain metrics from applications, the Open Liberty runtime, and the Java virtual machine (JVM). They can be gathered and stored in database tools, such as link:https://prometheus.io/[Prometheus], and displayed on dashboards, such as link:https://grafana.com/[Grafana]. For more information about building observability into your applications with MicroProfile Metrics, see xref:microservice-observability-metrics.adoc[Microservice observability with metrics]. For more information about integrating MicroProfile Metrics 5.0 with Micrometer to send metric data to third-party monitoring systems, see xref:micrometer-metrics.adoc[Choose your own monitoring tools with MicroProfile Metrics]. -When you enable the feature:mpTelemetry-2.0[display=MicroProfile Telemetry] feature version 2.0 and later, you can manage your logs, metrics, and traces in a standardized way with the OpenTelemetry protocol. For more information, see xref:microprofile-telemetry[Enable observability with MicroProfile Telemetry]. - == MicroProfile Metrics base and vendor metrics When the feature:mpMetrics[display="MicroProfile metrics"] feature is enabled, a set of base metrics is always reported. You can augment this set of metrics by collecting vendor metrics, which are available from different monitoring components within your Open Liberty server. When you enable both the feature:monitor[display="Performance Monitoring"] and the MicroProfile metrics feature, you enable the reporting of vendor metrics on the `/metrics` endpoint. The Performance Monitoring feature retrieves the statistical data from all available monitoring components. The `REST` base metrics that were introduced in MicroProfile Metrics 2.3 also rely on the Performance Monitoring feature. Before MicroProfile Metrics 2.3, the Performance Monitoring feature had to be explicitly configured. In MicroProfile Metrics 2.3, and later, the Performance Monitoring feature is automatically enabled by MicroProfile Metrics during startup. From 430bf05eabf08b7fd27e47d77dd8eaf4daf8d8a6 Mon Sep 17 00:00:00 2001 From: David Mueller Date: Thu, 5 Sep 2024 11:16:39 -0400 Subject: [PATCH 46/62] edits #7459 --- modules/ROOT/pages/custom-mptelemetry-metrics.adoc | 1 + modules/ROOT/pages/observability.adoc | 13 +++++++++---- modules/ROOT/pages/telemetry-trace.adoc | 2 +- 3 files changed, 11 insertions(+), 5 deletions(-) diff --git a/modules/ROOT/pages/custom-mptelemetry-metrics.adoc b/modules/ROOT/pages/custom-mptelemetry-metrics.adoc index 3117eff2df..371eaae7e1 100644 --- a/modules/ROOT/pages/custom-mptelemetry-metrics.adoc +++ b/modules/ROOT/pages/custom-mptelemetry-metrics.adoc @@ -14,6 +14,7 @@ You can use the OpenTelemetry metrics API to define custom metrics in your application code. When you enable the MicroProfile Telemetry feature 2.0 or later, you can then collect and emit these metrics to customize the observability of your application. +For more information about collecting and emitting metrics with MicroPRofile Telemetry, see xref:microprofile-telemetry.adoc#metrics[Configuring Open Liberty to use MicroProfile Telemetry to collect metrics]. The following example defines a custom counter metric: diff --git a/modules/ROOT/pages/observability.adoc b/modules/ROOT/pages/observability.adoc index 178f74097f..276f63dc94 100644 --- a/modules/ROOT/pages/observability.adoc +++ b/modules/ROOT/pages/observability.adoc @@ -15,12 +15,11 @@ When applications are observable, operations teams can identify and understand the root causes of bugs, bottlenecks, and other inefficiencies. Open Liberty provides a robust framework for developing observable applications and integrates with numerous third-party monitoring tools. -The following topics provide information to help you develop observable applications with Open Liberty. +With MicroProfile Telemetry 2.0 and later, you can manage logs, metrics, and traces in a standardized way by using the OpenTelemetry protocol. For more information, see xref:microprofile-telemetry.adoc[Enable observability with MicroProfile Telemetry]. -xref:microprofile-telemetry.adoc[Enable observability with MicroProfile Telemetry]:: -In microservice applications, sources of latency or inaccuracy can be difficult to determine because relationships and dependencies among the constituent services are not always obvious. MicroProfile Telemetry helps you collect data on the paths that application requests take through services. With MicroProfile Telemetry 2.0 and later, you can manage logs, metrics, and traces in a standardized way by using the OpenTelemetry protocol. +The following topics provide information to help you develop observable applications with Open Liberty. -xref:microservice-observability-metrics.adoc[Microservice observability with metrics]:: +xref:microservice-observability-metrics.adoc[Microservice observability with MicroProfile metrics]:: Open Liberty uses MicroProfile Metrics to provide metrics that describe the internal state of many Open Liberty components. MicroProfile Metrics provides a `/metrics` endpoint from which you can access all metrics that are emitted by an Open Liberty server and deployed applications. xref:micrometer-metrics.adoc[Choose your own monitoring tools with MicroProfile Metrics]:: @@ -29,4 +28,10 @@ The Open Liberty MicroProfile Metrics 5.0 feature uses embedded Micrometer metri xref:health-check-microservices.adoc[Health checks for microservices]:: A health check is a special REST API implementation that you can use to validate the status of a microservice and its dependencies. With MicroProfile Health, microservices can self-check their health and publish their overall status to a defined endpoint. +xref:custom-mptelemetry-metrics.adoc[Define custom MicroProfile Telemetry metrics]:: +You can use the OpenTelemetry metrics API to define custom metrics in your application code. When you enable the MicroProfile Telemetry feature 2.0 or later, you can then collect and emit these metrics to customize the observability of your application. + +xref:telemetry-trace.adoc[Code instrumentation for MicroProfile Telemetry tracing]:: +After you enable MicroProfile Telemetry and configure a trace server, you can instrument tracing in your application code. You can instrument your code automatically, manually, or by using the Java agent. + For hands-on tutorials on different observability configurations for Open Liberty, check out our link:/guides/#observability[Observability guides]. diff --git a/modules/ROOT/pages/telemetry-trace.adoc b/modules/ROOT/pages/telemetry-trace.adoc index 585bec6ed8..2a82a09924 100644 --- a/modules/ROOT/pages/telemetry-trace.adoc +++ b/modules/ROOT/pages/telemetry-trace.adoc @@ -21,7 +21,7 @@ After you enable MicroProfile Telemetry and configure a trace server, you can in [#auto] == Automatic instrumentation -With automatic instrumentation, you can observe traces without modifying the source code in your applications. All you need to do is configure runtime as described in <<#ol-config,Configuring Open Liberty to use MicroProfile Telemetry>>. However, automatic instrumentation is available only for JAX-RS and Jakarta RESTful web service applications. +With automatic instrumentation, you can observe traces without modifying the source code in your applications. All you need to do is configure runtime as described in xref:microprofile-telemetry.adoc#traces[Configuring Open Liberty to use MicroProfile Telemetry to collect traces]. However, automatic instrumentation is available only for JAX-RS and Jakarta RESTful web service applications. In Open Liberty version 23.0.0.11 and later, spans are automatically generated for incoming HTTP requests, including static files, servlets, and JSPs. From c4fae5b99701de7253b872b1718342ea66ace795 Mon Sep 17 00:00:00 2001 From: David Mueller Date: Thu, 5 Sep 2024 15:02:42 -0400 Subject: [PATCH 47/62] edits per review #7459 --- .../introduction-monitoring-metrics.adoc | 21 ++++++++++--------- modules/ROOT/pages/mptel-log-events-list.adoc | 2 +- modules/ROOT/pages/mptelemetry-logging.adoc | 19 +++++++++++++++++ .../ROOT/pages/mptelemetry-metrics-list.adoc | 11 +++++----- 4 files changed, 36 insertions(+), 17 deletions(-) create mode 100644 modules/ROOT/pages/mptelemetry-logging.adoc diff --git a/modules/ROOT/pages/introduction-monitoring-metrics.adoc b/modules/ROOT/pages/introduction-monitoring-metrics.adoc index cd61c8b0c7..4f53ad876c 100644 --- a/modules/ROOT/pages/introduction-monitoring-metrics.adoc +++ b/modules/ROOT/pages/introduction-monitoring-metrics.adoc @@ -8,26 +8,27 @@ // :page-layout: general-reference :page-type: general -:page-description: With Open Liberty, two types of metrics are available to monitor your applications, REST endpoint-style metrics that are provided by MicroProfile Metrics, and Java Management Extensions (JMX) metrics. +:page-description: With Open Liberty, three types of metrics are available to monitor your applications: OpenTelemetry metrics provided by MicroProfile Telemetry, REST endpoint-style metrics that are provided by MicroProfile Metrics, and Java Management Extensions (JMX) metrics. :seo-title: Monitoring with metrics - OpenLiberty.io -:seo-description: With Open Liberty, two types of metrics are available to monitor your applications, REST endpoint-style metrics that are provided by MicroProfile Metrics, and Java Management Extensions (JMX) metrics. +:seo-description: With Open Liberty, three types of metrics are available to monitor your applications: OpenTelemetry metrics provided by MicroProfile Telemetry, REST endpoint-style metrics that are provided by MicroProfile Metrics, and Java Management Extensions (JMX) metrics.. = Monitoring with metrics -Two kinds of metrics are available to monitor your Open Liberty applications: REST endpoint-style metrics that are provided by MicroProfile Metrics, and Java Management Extensions (JMX) metrics. MicroProfile Metrics and JMX metrics can be used separately or in conjunction to improve the observability of your application systems. +Three kinds of metrics are available to monitor your Open Liberty applications: OpenTelemetry metrics from MicroProfile Telemetry, REST endpoint-style metrics from MicroProfile Metrics, and Java Management Extensions (JMX) metrics. MicroProfile Metrics and JMX metrics can be used separately or in conjunction to improve the observability of your application systems. + +MicroProfile Telemetry 2.0 and later exports metrics to OTLP or another configured OpenTelemetry metrics exporter. You might prefer this option if you also use MicroPRofile Telemetry to manage your logs and traces. MicroProfile Metrics can be accessed by monitoring tools, such as Prometheus, or by any client that can make REST requests. <<#jmx,JMX metrics>> are suitable for use by Java-based monitoring tools that can communicate with JMX servers, or by custom JMX clients. -Open Liberty uses MicroProfile Metrics to expose metrics that describe the internal state of many Open Liberty components. -Developers can also use the MicroProfile Metrics API to expose metrics from their applications. -For more information about adding these metrics to your applications, see xref:microservice-observability-metrics.adoc#add[Adding metrics to your applications] and the link:/guides/microprofile-metrics.html[Providing metrics from a microservice] guide. For a list of all REST endpoint-style metrics that are available for Open Liberty, see the xref:metrics-list.adoc[Metrics reference list]. - -[#mptel] +[#mptelem] == Collect and export metrics with MicroProfile Telemetry -When you enable the MicroProfile Telemetry feature version 2.0 and later, you can manage your logs, metrics, and traces in a standardized way with the OpenTelemetry protocol. For more information, see xref:microprofile-telemetry[Enable observability with MicroProfile Telemetry]. +When you enable the MicroProfile Telemetry feature version 2.0 and later, you can manage your logs, metrics, and traces in a standardized way with the OpenTelemetry protocol. For more information, see xref:microprofile-telemetry.adoc[Enable observability with MicroProfile Telemetry]. [#endpoint] == MicroProfile Metrics and the metrics endpoint +Open Liberty uses MicroProfile Metrics to expose metrics that describe the internal state of many Open Liberty components. Developers can also use the MicroProfile Metrics API to expose metrics from their applications. +For more information about adding these metrics to your applications, see xref:microservice-observability-metrics.adoc#add[Adding metrics to your applications] and the link:/guides/microprofile-metrics.html[Providing metrics from a microservice] guide. For a list of all REST endpoint-style metrics that are available for Open Liberty, see the xref:metrics-list.adoc[Metrics reference list]. + The MicroProfile Metrics feature provides a `/metrics` REST interface that conforms to the MicroProfile Metrics specification. You can access MicroProfile Metrics with Open Liberty by feature:mpMetrics[display=enabling the MicroProfile Metrics feature]. Real-time values of all metrics are available by calling the `/metrics` endpoint, which provides formatted metric data that can be consumed as time-series data by external monitoring tools. When an application is running, you can view metric data from any browser by accessing the `/metrics` endpoint, for example, `\https://localhost:9443/metrics`, where `9443` is the port number for your application. @@ -39,7 +40,7 @@ The format that each response uses depends on the HTTP accept header of the corr Prometheus format is returned for requests with a `text/plain` accept header. In MicroProfile metrics 4.0 and earlier, JSON format is returned for requests with an `application/json` accept header. -Metrics endpoints differ in syntax and output between MicroProfile Metrics 5.0 and MicroProfile Metrics 4.0 and earlier. MicroProfile Metrics 5.0 also xref:micrometer-metrics.adoc[uses embedded Micrometer metrics technology] to enable third-party monitoring systems. For more information, see xref:reference:diff/mp-50-60-diff.adoc#metrics[Differences between MicroProfile Metrics 5.0 and 4.0] +Metrics endpoints differ in syntax and output between MicroProfile Metrics 5.0 and MicroProfile Metrics 4.0 and earlier. MicroProfile Metrics 5.0 also xref:micrometer-metrics.adoc[uses embedded Micrometer metrics technology] to enable third-party monitoring systems. For more information, see xref:reference:diff/mp-50-60-diff.adoc#metrics[Differences between MicroProfile Metrics 5.0 and 4.0]. === Metrics endpoints in MicroProfile Metrics 5.0 diff --git a/modules/ROOT/pages/mptel-log-events-list.adoc b/modules/ROOT/pages/mptel-log-events-list.adoc index bef1c703bf..6778871dc1 100644 --- a/modules/ROOT/pages/mptel-log-events-list.adoc +++ b/modules/ROOT/pages/mptel-log-events-list.adoc @@ -1,4 +1,4 @@ -// Copyright (c) 2018 IBM Corporation and others. +// Copyright (c) 2024 IBM Corporation and others. // Licensed under Creative Commons Attribution-NoDerivatives // 4.0 International (CC BY-ND 4.0) // https://creativecommons.org/licenses/by-nd/4.0/ diff --git a/modules/ROOT/pages/mptelemetry-logging.adoc b/modules/ROOT/pages/mptelemetry-logging.adoc new file mode 100644 index 0000000000..cbdcd9b9e5 --- /dev/null +++ b/modules/ROOT/pages/mptelemetry-logging.adoc @@ -0,0 +1,19 @@ +// Copyright (c) 2024 IBM Corporation and others. +// Licensed under Creative Commons Attribution-NoDerivatives +// 4.0 International (CC BY-ND 4.0) +// https://creativecommons.org/licenses/by-nd/4.0/ +// +// Contributors: +// IBM Corporation +// +:page-layout: general-reference +:page-type: general +:seo-title: Write logs with MicroProfile Telemetry logging- OpenLiberty.io +:seo-description: The MicroProfile Telemetry logging events that can be captured from the Open Liberty server runtime environment and applications. += Write logs with MicroProfile Telemetry logging + +You can use MicroProfile Telemetry to mange your logs, along with traces and metrics. When you enable the MicroProfile Telemetry 2.0 or later and the OpenTelemetry SDK, MicroProfile Telemetry automatically collects any logs that are sent to the `java.util.logging` API. + +Any messages that are logged at a `java.util.logging.Level` log level of `INFO` and above are considered messages. Levels below `INFO` are considered trace. By default, MicroProfile Telemetry automatically collects messages, but you can configure the sources that it collects from in your `server.xml` file. For more information, see link:/docs/latest/reference/feature/mpTelemetry-2.0.html#_collect_logs_from_a_specified_source[Collect logs from a specified source]. + +For more information about configuring MicroProfile Telemetry to collect and emit logs, metrics, and trace, see xref:microprofile-telemetry.adoc[Enable observability with MicroProfile Telemetry]. diff --git a/modules/ROOT/pages/mptelemetry-metrics-list.adoc b/modules/ROOT/pages/mptelemetry-metrics-list.adoc index c8e980ef99..ef57615ce2 100644 --- a/modules/ROOT/pages/mptelemetry-metrics-list.adoc +++ b/modules/ROOT/pages/mptelemetry-metrics-list.adoc @@ -42,14 +42,14 @@ To enable only the monitoring components that are used by MicroProfile Telemetry [source,xml] ---- - + ---- -To disable all vendor metrics, but to keep the `REST` base metrics configure the `server.xml`, as shown in the following example: +To disable all vendor metrics but keep the `Session` base metrics, add the following code to your `server.xml` file: [source,xml] ---- - + ---- To disable all monitoring components, add the following code to your `server.xml` file: @@ -62,10 +62,9 @@ To disable all monitoring components, add the following code to your `server.xml == MicroProfile Telemetry 2.0 and later metrics reference -When the MicroProfile Telemetry feature 2.0 and later is enabled, Open Liberty can sends metric data to compatible OpenTelemetry protocol (OTLP) receivers. For more information, see xref:microprofile-telemetry[Enable observability with MicroProfile Telemetry]. +When the MicroProfile Telemetry feature 2.0 and later is enabled, Open Liberty can send metric data to compatible OpenTelemetry protocol (OTLP) receivers. For more information, see xref:microprofile-telemetry[Enable observability with MicroProfile Telemetry]. -In the table, the Prometheus metric names are listed after each metric. -The tables also list the metric types, metric units, and descriptions of all metrics that are available for Open Liberty. For the vendor metrics, the associated monitoring component that you can use to filter the metric is also included. +The table lists the metric types, descriptions and attributes that are available for Open Liberty. Where applicable, the associated monitoring component that you can use to filter the metric is also included. The **Features required** column of the table includes the feature or features that must be enabled to obtain that metric data. The **Version introduced** column specifies the minimum version of the feature that you must enable to collect the metric. From 4cb04d83a4d2862707c78c175a3be929b7f57ab5 Mon Sep 17 00:00:00 2001 From: David Mueller Date: Thu, 5 Sep 2024 15:19:30 -0400 Subject: [PATCH 48/62] add container features to list #29571 --- .../pages/feature/versionless-features.adoc | 24 +++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/modules/reference/pages/feature/versionless-features.adoc b/modules/reference/pages/feature/versionless-features.adoc index 5c6c409c74..2e0e34ee65 100644 --- a/modules/reference/pages/feature/versionless-features.adoc +++ b/modules/reference/pages/feature/versionless-features.adoc @@ -264,12 +264,21 @@ Declare a platform version by using any of the three strategies that are describ |feature:faces[display=Jakarta Faces] | `faces` +|feature:facesContainer[display=Jakarta Faces Container] +| `facesContainer` + |feature:jsonb[display=Jakarta JavaScript Object Notation Binding] |`jsonb` +|feature:jsonbContainer[display=Jakarta JavaScript Object Notation Binding Container] +|`jsonb Container` + |feature:jsonp[display=Jakarta JavaScript Object Notation Processing] |`jsonp` +|feature:jsonpContainer[display=Jakarta JavaScript Object Notation Processing Container] +|`jsonpContainer` + |feature:mail[display=Jakarta Mail] |`mail` @@ -282,6 +291,9 @@ Declare a platform version by using any of the three strategies that are describ |feature:persistence[display=Jakarta Persistence] |`persistence` +|feature:persistenceContainer[display=Jakarta Persistence Container] +|`persistenceContainer` + |feature:restfulWS[display=Jakarta RESTful Web Services] |`restfulWS` @@ -390,6 +402,9 @@ Declare a platform version by using any of the three strategies that are describ |feature:jpa-2.2[display=Java Persistence API] |`jpa` +|feature:jpa-2.2[display=Java Persistence API Container] +|`jpaContainer` + |feature:jaxrs-2.1[display=Java RESTful Services] |`jaxrs` @@ -414,12 +429,21 @@ Declare a platform version by using any of the three strategies that are describ |feature:jsonb-1.0[display=JavaScript Object Notation Binding] (`javaee-8` only) |`jsonb` +|feature:jsonbContainer-1.0[display=JavaScript Object Notation Binding Container] (`javaee-8` only) +|`jsonbContainer` + |feature:jsonp-1.1[display=JavaScript Object Notation Processing] |`jsonp` +|feature:jsonpContainer-1.1[display=JavaScript Object Notation Processing Container] +|`jsonpContainer` + |feature:jsf-2.3[display=JavaServer Faces] |`jsf` +|feature:jsfContainer-2.3[display=JavaServer Faces Container] +|`jsfContainer` + |feature:jsp-2.3[display=JavaServer Pages] |`jsp` From 22175fc608ecc3a06a4172ac68a6b940425d1daf Mon Sep 17 00:00:00 2001 From: David Mueller Date: Thu, 5 Sep 2024 15:22:36 -0400 Subject: [PATCH 49/62] Update examples.adoc --- modules/reference/pages/feature/mpTelemetry-2.0/examples.adoc | 1 + 1 file changed, 1 insertion(+) diff --git a/modules/reference/pages/feature/mpTelemetry-2.0/examples.adoc b/modules/reference/pages/feature/mpTelemetry-2.0/examples.adoc index 5e1f82ab9d..d048f3e3e1 100644 --- a/modules/reference/pages/feature/mpTelemetry-2.0/examples.adoc +++ b/modules/reference/pages/feature/mpTelemetry-2.0/examples.adoc @@ -1,5 +1,6 @@ == Examples +[#logs] === Collect logs from a specified source To enable the MicroProfile Telemetry feature to collect logs from different sources in the Open Liberty runtime environment, configure the `source` attribute for the `mpTelemetry` element with a comma-separated list of comma-separated log sources. From 27de5610e860e2882229de50e7f9ae18b1a3a13c Mon Sep 17 00:00:00 2001 From: Ramkumar K Date: Fri, 6 Sep 2024 12:40:55 +0530 Subject: [PATCH 50/62] 7472-Hostname verification-Dvd-cmts-1 7472-Hostname verification-Dvd-cmts-1 #7472 --- modules/ROOT/pages/troubleshooting.adoc | 1 + .../feature/transportSecurity/examples.adoc | 20 ++++++++++++++++--- 2 files changed, 18 insertions(+), 3 deletions(-) diff --git a/modules/ROOT/pages/troubleshooting.adoc b/modules/ROOT/pages/troubleshooting.adoc index acfa9e0f63..e124645d43 100644 --- a/modules/ROOT/pages/troubleshooting.adoc +++ b/modules/ROOT/pages/troubleshooting.adoc @@ -273,6 +273,7 @@ Exception thrown while trying to read configuration and update ManagedServiceFac This error occurs when a keystore element exists in the configuration without an ID field. If you use a minimal TLS configuration, set the `ID` field to `defaultKeyStore`. +[#hostverify] === You receive the CWPKI0824E message that SSL handshake failure due to hostname verification error If you try to access a URL, you might see the following message. diff --git a/modules/reference/pages/feature/transportSecurity/examples.adoc b/modules/reference/pages/feature/transportSecurity/examples.adoc index 6f68f5adc7..a2e9682894 100644 --- a/modules/reference/pages/feature/transportSecurity/examples.adoc +++ b/modules/reference/pages/feature/transportSecurity/examples.adoc @@ -172,11 +172,25 @@ cert_defaultKeyStore="-----BEGIN CERTIFICATE----- [#hostverify] === Hostname and IP address verification -In Open Liberty, hostname and IP address verification are enabled by default. This verification is enforced for target servers in all SSL connections through Open Liberty socket factories. However, you can specify a list of hostnames, IP addresses, or both to skip verification. +Hostname and IP address verification are enabled by default. This verification is enforced for target servers in all SSL connections through Open Liberty socket factories. However, you can specify a list of hostnames, IP addresses, or both to skip verification. + +To disable hostname verification entirely, set the `verifyHostname` attribute within the `ssl` tag to `false`. + +[source,xml] +---- + +---- + +When hostname verification is enabled, you can skip verification for hostnames or IP addresses that are specified in the `skipHostnameVerificationForHosts` attribute within the `ssl` tag. + +[source,xml] +---- + +---- The verification makes sure that the hostname or IP address in the URL matches the Subject Alternative Name (SAN) in the SSL certificate of the server. If the SAN is not found, the property makes sure that the hostname in the URL matches the Common Name (CN). If a mismatch exists, the SSL connection is rejected. -Typically, during hostname verification, when the hostname is used in the request, it checks against the DNSName entry in the SAN. If the SAN does not contain a DNSName entry, hostname verification uses the certificate owner's common name (CN). When an IP address is used in the request, hostname verification relies on the IP address information in the SAN only. +Typically, during hostname verification, when the hostname is used in the request, it checks against the `DNSName` entry in the SAN. If the SAN does not contain a `DNSName` entry, hostname verification uses the certificate owner's Common Name (CN). When an IP address is used in the request, hostname verification relies on the IP address information in the SAN only. -For more information, see xref:pages/troubleshooting.adoc#Troubleshooting_SSL[Troubleshooting SSL and TLS] +For more information, see xref:ROOT/troubleshooting.adoc#Troubleshooting_SSL[Troubleshooting SSL and TLS] From bddb775db24933a0c91c03c2cf7ffe7df3520c87 Mon Sep 17 00:00:00 2001 From: Ramkumar K Date: Fri, 6 Sep 2024 14:19:43 +0530 Subject: [PATCH 51/62] 7472-Hostname verification-Dvd-cmts-2 7472-Hostname verification-Dvd-cmts-2 #7472 --- modules/reference/pages/feature/transportSecurity/examples.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/reference/pages/feature/transportSecurity/examples.adoc b/modules/reference/pages/feature/transportSecurity/examples.adoc index a2e9682894..aa774aab30 100644 --- a/modules/reference/pages/feature/transportSecurity/examples.adoc +++ b/modules/reference/pages/feature/transportSecurity/examples.adoc @@ -192,5 +192,5 @@ The verification makes sure that the hostname or IP address in the URL matches t Typically, during hostname verification, when the hostname is used in the request, it checks against the `DNSName` entry in the SAN. If the SAN does not contain a `DNSName` entry, hostname verification uses the certificate owner's Common Name (CN). When an IP address is used in the request, hostname verification relies on the IP address information in the SAN only. -For more information, see xref:ROOT/troubleshooting.adoc#Troubleshooting_SSL[Troubleshooting SSL and TLS] +For more information, see xref:ROOT:troubleshooting.adoc#hostverify[Troubleshooting SSL and TLS] From da1a0736bd7edb85bc93fc8d80f2f3af672c5e44 Mon Sep 17 00:00:00 2001 From: Ramkumar K Date: Fri, 6 Sep 2024 16:15:58 +0530 Subject: [PATCH 52/62] 7472-Hostname verification-Dvd-cmts-3 7472-Hostname verification-Dvd-cmts-3 #7472 --- modules/ROOT/pages/secure-communication-tls.adoc | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/modules/ROOT/pages/secure-communication-tls.adoc b/modules/ROOT/pages/secure-communication-tls.adoc index 9b4c041e4a..e28836f737 100644 --- a/modules/ROOT/pages/secure-communication-tls.adoc +++ b/modules/ROOT/pages/secure-communication-tls.adoc @@ -52,9 +52,7 @@ Open Liberty does not check certificate revocations and instead relies on the un Before the wide adoption of the TLS protocol, SSL was the standard protocol to secure web communications. Over time, many security vulnerabilities were identified for SSL and it is no longer in widespread use. However, for historical reasons, the term SSL is often used to refer to encrypted network connections even when TLS is in use. In Open Liberty, the term SSL is still used in feature and configuration names, even though TLS is the underlying protocol. Effectively, SSL is now a synonym for TLS. -NOTE: Hostname verification is enabled by default. - -For more information on Hostname verification, see feature:transportSecurity[display=Transport Security]. +Hostname verification is enabled by default. For more information on Hostname verification, see feature:transportSecurity[display=Transport Security]. == See also From a99cad7e51f904896a9e92f95543cf3c6b58c668 Mon Sep 17 00:00:00 2001 From: David Mueller Date: Fri, 6 Sep 2024 15:15:50 -0400 Subject: [PATCH 53/62] edits per review #7459 --- .../ROOT/pages/introduction-monitoring-metrics.adoc | 2 +- modules/ROOT/pages/mptelemetry-logging.adoc | 2 +- modules/ROOT/pages/mptelemetry-metrics-list.adoc | 12 ++---------- 3 files changed, 4 insertions(+), 12 deletions(-) diff --git a/modules/ROOT/pages/introduction-monitoring-metrics.adoc b/modules/ROOT/pages/introduction-monitoring-metrics.adoc index 4f53ad876c..eabc136145 100644 --- a/modules/ROOT/pages/introduction-monitoring-metrics.adoc +++ b/modules/ROOT/pages/introduction-monitoring-metrics.adoc @@ -16,7 +16,7 @@ Three kinds of metrics are available to monitor your Open Liberty applications: OpenTelemetry metrics from MicroProfile Telemetry, REST endpoint-style metrics from MicroProfile Metrics, and Java Management Extensions (JMX) metrics. MicroProfile Metrics and JMX metrics can be used separately or in conjunction to improve the observability of your application systems. -MicroProfile Telemetry 2.0 and later exports metrics to OTLP or another configured OpenTelemetry metrics exporter. You might prefer this option if you also use MicroPRofile Telemetry to manage your logs and traces. +MicroProfile Telemetry 2.0 and later exports metrics to OTLP or another configured OpenTelemetry metrics exporter. You might prefer this option if you also use MicroProfile Telemetry to manage your logs and traces. MicroProfile Metrics can be accessed by monitoring tools, such as Prometheus, or by any client that can make REST requests. <<#jmx,JMX metrics>> are suitable for use by Java-based monitoring tools that can communicate with JMX servers, or by custom JMX clients. diff --git a/modules/ROOT/pages/mptelemetry-logging.adoc b/modules/ROOT/pages/mptelemetry-logging.adoc index cbdcd9b9e5..4b3797b5f5 100644 --- a/modules/ROOT/pages/mptelemetry-logging.adoc +++ b/modules/ROOT/pages/mptelemetry-logging.adoc @@ -12,7 +12,7 @@ :seo-description: The MicroProfile Telemetry logging events that can be captured from the Open Liberty server runtime environment and applications. = Write logs with MicroProfile Telemetry logging -You can use MicroProfile Telemetry to mange your logs, along with traces and metrics. When you enable the MicroProfile Telemetry 2.0 or later and the OpenTelemetry SDK, MicroProfile Telemetry automatically collects any logs that are sent to the `java.util.logging` API. +You can use MicroProfile Telemetry to mange your logs, along with traces and metrics. When you enable the MicroProfile Telemetry 2.0 or later, MicroProfile Telemetry automatically collects any logs that are sent to the `java.util.logging` API. Any messages that are logged at a `java.util.logging.Level` log level of `INFO` and above are considered messages. Levels below `INFO` are considered trace. By default, MicroProfile Telemetry automatically collects messages, but you can configure the sources that it collects from in your `server.xml` file. For more information, see link:/docs/latest/reference/feature/mpTelemetry-2.0.html#_collect_logs_from_a_specified_source[Collect logs from a specified source]. diff --git a/modules/ROOT/pages/mptelemetry-metrics-list.adoc b/modules/ROOT/pages/mptelemetry-metrics-list.adoc index ef57615ce2..79b23d1044 100644 --- a/modules/ROOT/pages/mptelemetry-metrics-list.adoc +++ b/modules/ROOT/pages/mptelemetry-metrics-list.adoc @@ -36,7 +36,7 @@ When you enable the feature:mpTelemetry-2.0[display=MicroProfile Telemetry] feat == Filter metrics to gather only the data you need -By default, all monitoring components are enabled. If your server is collecting more metrics data than you need, you can improve the server performance by collecting only those vendor metrics that you intend to use. To configure only a subset of vendor metrics to be reported, specify the components that you want to monitor in the `filter` attribute for the `monitor` configuration element in your `server.xml` file. You can identify the relevant monitoring component for each vendor metric by referencing the **Monitoring component** column of the metrics reference tables. +By default, all monitoring components are enabled. If your server is collecting more metrics data than you need, you can improve the server performance by collecting only those metrics that you intend to use. To configure only a subset of metrics to be reported, specify the components that you want to monitor in the `filter` attribute for the `monitor` configuration element in your `server.xml` file. You can identify the relevant monitoring component for each metric by referencing the **Monitoring component** column of the metrics reference tables. To enable only the monitoring components that are used by MicroProfile Telemetry, add the following code to your `server.xml` file. @@ -45,7 +45,7 @@ To enable only the monitoring components that are used by MicroProfile Telemetry ---- -To disable all vendor metrics but keep the `Session` base metrics, add the following code to your `server.xml` file: +To disable all metrics but the `Session` metrics, add the following code to your `server.xml` file: [source,xml] ---- @@ -307,11 +307,3 @@ The **Version introduced** column specifies the minimum version of the feature t |=== {empty} + - - - -== See also - -* Guide: link:/guides/microprofile-metrics.html[Providing metrics from a microservice] -* xref:reference:diff/mp-21-22-diff.adoc[Differences between MicroProfile 2.1 and 2.2] -* xref:microservice-observability-metrics.adoc[Microservice observability with metrics] From b3b7be51202a032977b76f6cb09cfeddea0217df Mon Sep 17 00:00:00 2001 From: David Mueller Date: Mon, 9 Sep 2024 08:40:57 -0400 Subject: [PATCH 54/62] Update mptelemetry-logging.adoc --- modules/ROOT/pages/mptelemetry-logging.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/ROOT/pages/mptelemetry-logging.adoc b/modules/ROOT/pages/mptelemetry-logging.adoc index 4b3797b5f5..fc33ea12c9 100644 --- a/modules/ROOT/pages/mptelemetry-logging.adoc +++ b/modules/ROOT/pages/mptelemetry-logging.adoc @@ -12,7 +12,7 @@ :seo-description: The MicroProfile Telemetry logging events that can be captured from the Open Liberty server runtime environment and applications. = Write logs with MicroProfile Telemetry logging -You can use MicroProfile Telemetry to mange your logs, along with traces and metrics. When you enable the MicroProfile Telemetry 2.0 or later, MicroProfile Telemetry automatically collects any logs that are sent to the `java.util.logging` API. +You can use MicroProfile Telemetry to mange your logs, along with traces and metrics. When you enable the MicroProfile Telemetry feature version 2.0 or later, MicroProfile Telemetry automatically collects any logs that are sent to the `java.util.logging` API. Any messages that are logged at a `java.util.logging.Level` log level of `INFO` and above are considered messages. Levels below `INFO` are considered trace. By default, MicroProfile Telemetry automatically collects messages, but you can configure the sources that it collects from in your `server.xml` file. For more information, see link:/docs/latest/reference/feature/mpTelemetry-2.0.html#_collect_logs_from_a_specified_source[Collect logs from a specified source]. From 567af573cc62d14e94a0362946f542c8069e9f95 Mon Sep 17 00:00:00 2001 From: Ramkumar K Date: Mon, 9 Sep 2024 21:17:00 +0530 Subject: [PATCH 55/62] 7472-Hostname verification-Dvd-cmts-4 7472-Hostname verification-Dvd-cmts-4 #7472 --- .../reference/pages/feature/transportSecurity/examples.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/modules/reference/pages/feature/transportSecurity/examples.adoc b/modules/reference/pages/feature/transportSecurity/examples.adoc index aa774aab30..49028c03d1 100644 --- a/modules/reference/pages/feature/transportSecurity/examples.adoc +++ b/modules/reference/pages/feature/transportSecurity/examples.adoc @@ -181,7 +181,7 @@ To disable hostname verification entirely, set the `verifyHostname` attribute wi ---- -When hostname verification is enabled, you can skip verification for hostnames or IP addresses that are specified in the `skipHostnameVerificationForHosts` attribute within the `ssl` tag. +When hostname verification is enabled, you can skip verification for hostnames or IP addresses that are specified in the `skipHostnameVerificationForHosts` attribute within the `ssl` tag. The `skipHostnameVerificationForHosts` attribute is a comma-separated list of hostnames, IP addresses, or both. [source,xml] ---- @@ -192,5 +192,5 @@ The verification makes sure that the hostname or IP address in the URL matches t Typically, during hostname verification, when the hostname is used in the request, it checks against the `DNSName` entry in the SAN. If the SAN does not contain a `DNSName` entry, hostname verification uses the certificate owner's Common Name (CN). When an IP address is used in the request, hostname verification relies on the IP address information in the SAN only. -For more information, see xref:ROOT:troubleshooting.adoc#hostverify[Troubleshooting SSL and TLS] +For more information, see xref:ROOT:troubleshooting.adoc#hostverify[Troubleshooting SSL and TLS]. From 3a3f42a6343a6fc2d1c17dd0cb18f45b2341c74e Mon Sep 17 00:00:00 2001 From: David Mueller Date: Mon, 9 Sep 2024 12:16:50 -0400 Subject: [PATCH 56/62] edits from peer review #7459 --- modules/ROOT/pages/microprofile-telemetry.adoc | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/modules/ROOT/pages/microprofile-telemetry.adoc b/modules/ROOT/pages/microprofile-telemetry.adoc index 839e87d860..9ed03a835c 100644 --- a/modules/ROOT/pages/microprofile-telemetry.adoc +++ b/modules/ROOT/pages/microprofile-telemetry.adoc @@ -36,7 +36,7 @@ Runtime-level configuration collects and emits telemetry from the runtime and al You can configure your runtime instance in a few different sources, including your `jvm.options`, `bootstrap.properties`, or `server.env` files. You do not need to modify your application configuration to configure your runtime instance. Application-level configuration:: -Configuring application-level OpenTelemetry instances gives you more fine-grained control over the data you collect for each application. This option also supports configuration-compatibility with MicroProfile Telemetry 1.1 and earlier, which created only application-level SDK instances. Runtime-level data is not collected or emitted by application-level instances. +Configuring application-level OpenTelemetry instances gives you more fine-grained control over the data you collect for each application. This option also supports configuration compatibility with feature:mpTelemetry-1.1[display=MicroProfile Telemetry 1.1] and earlier, which created only application-level SDK instances. Runtime-level data is not collected or emitted by application-level instances. + You can configure your application-level OpenTelemetry instances in the `microprofile-config.properties` file or the `appProperties` attribute for the config:application[] element in the `server.xml` file. When you create application-level instances, any system and server environment properties overwrite your application-level configuration. @@ -48,9 +48,9 @@ To enable MicroProfile Telemetry to collect and emit logs, metrics, and traces, . Set the `otel.sdk.disabled=false` property at the runtime level or the application level. + -At runtime initialization, if the `otel.sdk.disabled` property is set to false, the runtime-level OpenTelemetry SDK instance is created. If the runtime instance is not enabled and `otel.sdk.disabled=false` is specified at the application level, an application-level instance is created during application initialization. +At runtime initialization, if the `otel.sdk.disabled` property is set to `false`, the runtime-level OpenTelemetry SDK instance is created. If the runtime instance is not enabled and `otel.sdk.disabled=false` is specified at the application level, an application-level instance is created during application initialization. -.. To enable the OpenTelemetry SDK at the runtime level, set the `otel.sdk.disabled=false property` as a system property, for example, in the `bootstrap.properties` file: +.. To enable the OpenTelemetry SDK at the runtime level, set the `otel.sdk.disabled=false` property as a system property, for example, in the `bootstrap.properties` file: + ---- otel.sdk.disabled=false @@ -61,7 +61,7 @@ The `otel.service.name` property creates a name for any telemetry that this Open + Alternatively, you can set runtime-level configuration properties by using environment variables in your `server.env` file. For any property definition, make the key name uppercase and convert any punctuation to underscores. For example, the `otel.sdk.disabled=false` property is equivalent to the `OTEL_SDK_DISABLED=false` environment variable. -.. To enable the OpenTelemetry SDK at the application level, set the `otel.sdk.disabled=false property` in the `microprofile-config.properties` file of each application. +.. To enable the OpenTelemetry SDK at the application level, set the `otel.sdk.disabled=false` property in the `microprofile-config.properties` file of each application. + Application 1 `microprofile-config.properties` file: + @@ -77,7 +77,7 @@ otel.sdk.disabled=false otel.service.name=A2 ---- + -This configuration creates all telemetry from Application 1 with the service name A1, and from Application 2 with the service name A2. It omits all runtime-level telemetry. +This configuration creates all telemetry from Application 1 with the service name `A1`, and from Application 2 with the service name `A2`. It omits all runtime-level telemetry. .. Optionally, use a combination of application-level and runtime-level configuration. + @@ -185,7 +185,7 @@ If you set this property to `console`, all the logs are exported to standard out + .. Configure MicroProfile Telemetry to use the OpenTelemetry Batch LogRecord Processor. + -By default, the SimpleLogRecordProcessor is enabled, so the records are sent immediately. However, if you want to send the records in batches, you can also configure the following logging-specific Batch LogRecord Processor properties or environment variables: +By default, the `SimpleLogRecordProcessor` is enabled, so the records are sent immediately. However, if you want to send the records in batches, you can also configure the following logging-specific Batch LogRecord Processor properties or environment variables: + * `otel.blrp.schedule.delay` or `OTEL_BLRP_SCHEDULE_DELAY` * `otel.blrp.max.queue.size` or `OTEL_BLRP_MAX_QUEUE_SIZE` From 21d459f99cad0b2e04629885c38835bd2606e05d Mon Sep 17 00:00:00 2001 From: David Mueller Date: Mon, 9 Sep 2024 13:32:09 -0400 Subject: [PATCH 57/62] update nav --- modules/ROOT/nav.adoc | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/modules/ROOT/nav.adoc b/modules/ROOT/nav.adoc index 0c2eb339e9..08857e25af 100644 --- a/modules/ROOT/nav.adoc +++ b/modules/ROOT/nav.adoc @@ -4,9 +4,9 @@ // Begin basics section .Basics * xref:zero-migration-architecture.adoc[Zero migration] -* xref:development-mode.adoc[Dev mode] +* xref:dev-mode.adoc[Dev mode] * xref:java-se.adoc[Java SE support] -* xref:installing-open-liberty-betas.adoc[Installing Open Liberty beta releases] +* xref:installing-open-liberty-betas.adoc[Install Open Liberty beta releases] * xref:develop-liberty-tools.adoc[Develop with Liberty Tools] * https://openliberty.io/guides/getting-started.html[Guide: Getting started with Open Liberty] @@ -18,8 +18,8 @@ * xref:container-images.adoc[Container images] ** xref:verify-signatures-for-container-images-in-open-liberty.adoc[Verify signatures for container images in Open Liberty] * xref:rest-microservices.adoc[RESTful services] - ** xref:sync-async-rest-clients.adoc[Sync and async REST clients] ** xref:rest-clients.adoc[REST clients] + ** xref:sync-async-rest-clients.adoc[Sync and async REST clients] ** xref:json-p-b.adoc[JSON-P and JSON-B] ** xref:send-receive-multipart-jaxrs.adoc[Send and receive multipart/form-data parts] ** https://openliberty.io/guides/#restful_service[Guides: RESTful services] @@ -37,6 +37,7 @@ * xref:cdi-beans.adoc[Context and Dependency Injection] ** xref:jaxrs-integration-cdi.adoc[RESTful Web Services integration with CDI] ** https://openliberty.io/guides/cdi-intro.html[Guide: Injecting dependencies into microservices] + ** xref:cdi-extension-user-feature.adoc[Creating a CDI extension inside a user feature] * xref:data-persistence.adoc[Data persistence] ** xref:relational-database-connections-JDBC.adoc[Relational database connections with JDBC] ** xref:kerberos-authentication-jdbc.adoc[Kerberos authentication for JDBC] @@ -49,10 +50,12 @@ ** xref:distributed-session-caching.adoc[Distributed session caching] ** xref:configuring-infinispan-support.adoc[Configuring Infinispan as a JCache provider] * xref:observability.adoc[Observability] - ** xref:microservice-observability-metrics.adoc[Microservice observability with metrics] + ** xref:microservice-observability-metrics.adoc[Microservice observability with MicroProfile Metrics] ** xref:micrometer-metrics.adoc[Choose your own monitoring tools with MicroProfile Metrics] ** xref:health-check-microservices.adoc[Health checks for microservices] - ** xref:microprofile-telemetry.adoc[Enable distributed tracing] + ** xref:custom-mptelemetry-metrics.adoc[Define custom MicroProfile Telemetry metrics] + ** xref:telemetry-trace.adoc[Code instrumentation for MicroProfile Telemetry tracing] + ** xref:mptelemetry-logging.adoc[Write logs with MicroProfile Telemetry logging] ** https://openliberty.io/guides/#observability[Guides: Observability] * xref:integration-testing.adoc[Testing] ** https://openliberty.io/guides/#test[Guides: Testing] @@ -85,6 +88,7 @@ ** xref:server-configuration-hardening.adoc[Server configuration] ** xref:network-hardening.adoc[Network] ** xref:application-configuration-hardening.adoc[Application configuration] +* xref:custom-password-encryption.adoc[Custom password encryption] * xref:password-encryption.adoc[Password encryption limitations] * xref:audit-logs.adoc[Audit logs] * xref:verifying-package-signatures.adoc[Verifying release package signatures] @@ -108,22 +112,26 @@ * xref:class-loader-library-config.adoc[Class loader configuration] * xref:virtual-hosts.adoc[Virtual hosts] * xref:application-bindings.adoc[Application bindings] +* xref:loose-applications.adoc[Loose applications] * https://openliberty.io/guides/#kubernetes[Guides: Kubernetes] * https://openliberty.io/guides/#cloud_deployment[Guides: Cloud deployment] // Begin operations section .Operations +* xref:microprofile-telemetry.adoc[Enable observability with MicroProfile Telemetry] * xref:log-trace-configuration.adoc[Logs] ** xref:log-management.adoc[Log management] ** xref:access-logging.adoc[HTTP access logging] ** xref:jdbc-tracing.adoc[JDBC tracing] + ** xref:mptel-log-events-list.adoc[MicroProfile Telemetry log events reference list] ** xref:json-log-events-list.adoc[JSON log events reference list] ** xref:logstash-events-list.adoc[Logstash events reference list] ** xref:audit-log-events-list-cadf.adoc[Audit log events (CADF) reference list] ** xref:analyzing-logs-elk.adoc[Analyzing JSON logs with Elastic Stack] ** xref:forwarding-logs-logstash.adoc[Forwarding logs with Logstash collector] * xref:introduction-monitoring-metrics.adoc[Metrics] - ** xref:metrics-list.adoc[Metrics reference list] + ** xref:mptelemetry-metrics-list.adoc[MicroProfile Telemetry metrics reference list] + ** xref:metrics-list.adoc[MicroProfile Metrics reference list] ** xref:configuring-jmx-connection.adoc[Configuring JMX connections] ** xref:jmx-metrics-list.adoc[JMX metrics reference list] * xref:slow-hung-request-detection.adoc[Slow and hung request detection] From 10f802227b16ce3e0aad566bd5caafcc26ce99ce Mon Sep 17 00:00:00 2001 From: David Mueller Date: Mon, 9 Sep 2024 13:47:02 -0400 Subject: [PATCH 58/62] Update nav.adoc --- modules/ROOT/nav.adoc | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-) diff --git a/modules/ROOT/nav.adoc b/modules/ROOT/nav.adoc index 08857e25af..4e274aeabb 100644 --- a/modules/ROOT/nav.adoc +++ b/modules/ROOT/nav.adoc @@ -4,9 +4,9 @@ // Begin basics section .Basics * xref:zero-migration-architecture.adoc[Zero migration] -* xref:dev-mode.adoc[Dev mode] +* xref:development-mode.adoc[Dev mode] * xref:java-se.adoc[Java SE support] -* xref:installing-open-liberty-betas.adoc[Install Open Liberty beta releases] +* xref:installing-open-liberty-betas.adoc[Installing Open Liberty beta releases] * xref:develop-liberty-tools.adoc[Develop with Liberty Tools] * https://openliberty.io/guides/getting-started.html[Guide: Getting started with Open Liberty] @@ -18,8 +18,8 @@ * xref:container-images.adoc[Container images] ** xref:verify-signatures-for-container-images-in-open-liberty.adoc[Verify signatures for container images in Open Liberty] * xref:rest-microservices.adoc[RESTful services] - ** xref:rest-clients.adoc[REST clients] ** xref:sync-async-rest-clients.adoc[Sync and async REST clients] + ** xref:rest-clients.adoc[REST clients] ** xref:json-p-b.adoc[JSON-P and JSON-B] ** xref:send-receive-multipart-jaxrs.adoc[Send and receive multipart/form-data parts] ** https://openliberty.io/guides/#restful_service[Guides: RESTful services] @@ -37,7 +37,6 @@ * xref:cdi-beans.adoc[Context and Dependency Injection] ** xref:jaxrs-integration-cdi.adoc[RESTful Web Services integration with CDI] ** https://openliberty.io/guides/cdi-intro.html[Guide: Injecting dependencies into microservices] - ** xref:cdi-extension-user-feature.adoc[Creating a CDI extension inside a user feature] * xref:data-persistence.adoc[Data persistence] ** xref:relational-database-connections-JDBC.adoc[Relational database connections with JDBC] ** xref:kerberos-authentication-jdbc.adoc[Kerberos authentication for JDBC] @@ -50,12 +49,12 @@ ** xref:distributed-session-caching.adoc[Distributed session caching] ** xref:configuring-infinispan-support.adoc[Configuring Infinispan as a JCache provider] * xref:observability.adoc[Observability] - ** xref:microservice-observability-metrics.adoc[Microservice observability with MicroProfile Metrics] - ** xref:micrometer-metrics.adoc[Choose your own monitoring tools with MicroProfile Metrics] - ** xref:health-check-microservices.adoc[Health checks for microservices] ** xref:custom-mptelemetry-metrics.adoc[Define custom MicroProfile Telemetry metrics] ** xref:telemetry-trace.adoc[Code instrumentation for MicroProfile Telemetry tracing] ** xref:mptelemetry-logging.adoc[Write logs with MicroProfile Telemetry logging] + ** xref:microservice-observability-metrics.adoc[Microservice observability with with MicroProfile Metrics] + ** xref:micrometer-metrics.adoc[Choose your own monitoring tools with MicroProfile Metrics] + ** xref:health-check-microservices.adoc[Health checks for microservices] ** https://openliberty.io/guides/#observability[Guides: Observability] * xref:integration-testing.adoc[Testing] ** https://openliberty.io/guides/#test[Guides: Testing] @@ -88,7 +87,6 @@ ** xref:server-configuration-hardening.adoc[Server configuration] ** xref:network-hardening.adoc[Network] ** xref:application-configuration-hardening.adoc[Application configuration] -* xref:custom-password-encryption.adoc[Custom password encryption] * xref:password-encryption.adoc[Password encryption limitations] * xref:audit-logs.adoc[Audit logs] * xref:verifying-package-signatures.adoc[Verifying release package signatures] @@ -112,7 +110,6 @@ * xref:class-loader-library-config.adoc[Class loader configuration] * xref:virtual-hosts.adoc[Virtual hosts] * xref:application-bindings.adoc[Application bindings] -* xref:loose-applications.adoc[Loose applications] * https://openliberty.io/guides/#kubernetes[Guides: Kubernetes] * https://openliberty.io/guides/#cloud_deployment[Guides: Cloud deployment] From d3347c24ebae3e305fd5a19e3be23ea424730a0a Mon Sep 17 00:00:00 2001 From: David Mueller Date: Mon, 9 Sep 2024 19:58:38 -0400 Subject: [PATCH 59/62] edits per reivew #7459 --- .../pages/custom-mptelemetry-metrics.adoc | 6 ++- .../ROOT/pages/microprofile-telemetry.adoc | 9 ++-- modules/ROOT/pages/microprofile.adoc | 2 +- modules/ROOT/pages/observability.adoc | 17 ++++-- modules/ROOT/pages/prepare-mptelemetry.adoc | 52 +++++++++++++++++++ modules/ROOT/pages/telemetry-trace.adoc | 35 ++----------- 6 files changed, 80 insertions(+), 41 deletions(-) create mode 100644 modules/ROOT/pages/prepare-mptelemetry.adoc diff --git a/modules/ROOT/pages/custom-mptelemetry-metrics.adoc b/modules/ROOT/pages/custom-mptelemetry-metrics.adoc index 371eaae7e1..1a89ca1511 100644 --- a/modules/ROOT/pages/custom-mptelemetry-metrics.adoc +++ b/modules/ROOT/pages/custom-mptelemetry-metrics.adoc @@ -14,7 +14,11 @@ You can use the OpenTelemetry metrics API to define custom metrics in your application code. When you enable the MicroProfile Telemetry feature 2.0 or later, you can then collect and emit these metrics to customize the observability of your application. -For more information about collecting and emitting metrics with MicroPRofile Telemetry, see xref:microprofile-telemetry.adoc#metrics[Configuring Open Liberty to use MicroProfile Telemetry to collect metrics]. +For more information about collecting and emitting metrics with MicroProfile Telemetry, see xref:microprofile-telemetry.adoc#metrics[Configuring Open Liberty to use MicroProfile Telemetry to collect metrics]. + +For more information about OpenTelemetry metrics, see the link:https://www.javadoc.io/doc/io.opentelemetry/opentelemetry-api/1.39.0/io/opentelemetry/api/metrics/package-summary.html[OpenTelemetry metrics API documentation]. + +Before you can use MicroProfile Telemetry to define custom metrics metrics, you must enable MicroProfile Telemetry in your development environment by editing your runtime configuration. You must also add the OpenTelemetry API and annotations as a dependency on your build path. For more information, see xref:prepare-mptelemetry-metrics.adoc[Prepare your development environment for MicroProfile Telemetry]. The following example defines a custom counter metric: diff --git a/modules/ROOT/pages/microprofile-telemetry.adoc b/modules/ROOT/pages/microprofile-telemetry.adoc index 9ed03a835c..11e54229c2 100644 --- a/modules/ROOT/pages/microprofile-telemetry.adoc +++ b/modules/ROOT/pages/microprofile-telemetry.adoc @@ -12,9 +12,8 @@ :page-type: general = Enable observability with MicroProfile Telemetry -MicroProfile Telemetry helps you collect and analyze data on the paths that application requests take through services. With MicroProfile Telemetry 2.0 and later, you can manage logs, metrics, and traces in a standardized way by using the OpenTelemetry protocol. - -MicroProfile Telemetry is based on the https://opentelemetry.io/[OpenTelemetry project], a collection of open source vendor-independent tools, APIs, and SDKs for creating and managing logs, metrics, and trace data. +With MicroProfile Telemetry 2.0 and later, you can manage logs, metrics, and traces in a standardized way by using the OpenTelemetry protocol. +MicroProfile Telemetry is based on the OpenTelemetry project, a collection of open source vendor-independent tools, APIs, and SDKs for creating and managing logs, metrics, and trace data. The following sections explain how to prepare your Open Liberty runtime and application code to use MicroProfile Telemetry. @@ -265,3 +264,7 @@ Different versions of the MicroProfile Telemetry feature are compatible with dif * feature:jakartaee-8.0[] and feature:microProfile-4.1[] * feature:jakartaee-9.1[] and feature:microProfile-5.0[] //// + +== See also + +https://opentelemetry.io/[OpenTelemetry project] diff --git a/modules/ROOT/pages/microprofile.adoc b/modules/ROOT/pages/microprofile.adoc index b2fe2ccdc1..d9ebe32816 100644 --- a/modules/ROOT/pages/microprofile.adoc +++ b/modules/ROOT/pages/microprofile.adoc @@ -122,7 +122,7 @@ For more information, see xref:microservice-observability-metrics.adoc[Microserv === Enable distributed tracing of your microservices link:https://projects.eclipse.org/projects/technology.microprofile/releases/microprofile-telemetry-1.0/plan%E2%80%A8%E2%80%A822.0.0.10-bet[MicroProfile Telemetry] is based on the https://opentelemetry.io/[OpenTelemetry project], which is a collection of open source vendor-agnostic tools, APIs, and SDKs for creating and managing trace data. -In Open Liberty, MicroPRofile Telemetry is supported by the feature:mpTelemetry[display=MicroProfile Telemetry] feature. You can export the data that this feature collects to tracing systems such as Jaeger or Zipkin. For more information, see xref:microprofile-telemetry.adoc[Enable observability with MicroProfile Telemetry]. +In Open Liberty, MicroProfile Telemetry is supported by the feature:mpTelemetry[display=MicroProfile Telemetry] feature. You can export the data that this feature collects to tracing systems such as Jaeger or Zipkin. For more information, see xref:microprofile-telemetry.adoc[Enable observability with MicroProfile Telemetry]. MicroProfile Telemetry replaces MicroProfile OpenTracing. For information about migrating your applications from MicroProfile OpenTracing to MicroProfile Telemetry, see xref:reference:diff/mp-50-60-diff.adoc#telemetry[Differences between MicroProfile Telemetry 1.0 and MicroProfile OpenTracing 3.0]. diff --git a/modules/ROOT/pages/observability.adoc b/modules/ROOT/pages/observability.adoc index 276f63dc94..04e8bf74bf 100644 --- a/modules/ROOT/pages/observability.adoc +++ b/modules/ROOT/pages/observability.adoc @@ -19,6 +19,18 @@ With MicroProfile Telemetry 2.0 and later, you can manage logs, metrics, and tra The following topics provide information to help you develop observable applications with Open Liberty. +xref:prepare-mptelemetry-metrics.adoc[Prepare your development environment for MicroProfile Telemetry]:: +Before you manually instrument your code to use MicroProfile Telemetry to collect metrics and traces, you must enable MicroProfile Telemetry in your development environment by editing your runtime configuration. You must also add the OpenTelemetry API and annotations as a dependency on your build path. + +xref:custom-mptelemetry-metrics.adoc[Define custom MicroProfile Telemetry metrics]:: +You can use the OpenTelemetry metrics API to define custom metrics in your application code. When you enable the MicroProfile Telemetry feature 2.0 or later, you can then collect and emit these metrics to customize the observability of your application. + +xref:telemetry-trace.adoc[Code instrumentation for MicroProfile Telemetry tracing]:: +After you enable MicroProfile Telemetry and configure a trace server, you can instrument tracing in your application code. You can instrument your code automatically, manually, or by using the Java agent. + +xref:mptelemetry-logging.adoc[Write logs with MicroProfile Telemetry logging]:: +You can use MicroProfile Telemetry to mange your logs, along with traces and metrics. When you enable the MicroProfile Telemetry feature version 2.0 or later, MicroProfile Telemetry automatically collects any logs that are sent to the `java.util.logging` API. + xref:microservice-observability-metrics.adoc[Microservice observability with MicroProfile metrics]:: Open Liberty uses MicroProfile Metrics to provide metrics that describe the internal state of many Open Liberty components. MicroProfile Metrics provides a `/metrics` endpoint from which you can access all metrics that are emitted by an Open Liberty server and deployed applications. @@ -28,10 +40,5 @@ The Open Liberty MicroProfile Metrics 5.0 feature uses embedded Micrometer metri xref:health-check-microservices.adoc[Health checks for microservices]:: A health check is a special REST API implementation that you can use to validate the status of a microservice and its dependencies. With MicroProfile Health, microservices can self-check their health and publish their overall status to a defined endpoint. -xref:custom-mptelemetry-metrics.adoc[Define custom MicroProfile Telemetry metrics]:: -You can use the OpenTelemetry metrics API to define custom metrics in your application code. When you enable the MicroProfile Telemetry feature 2.0 or later, you can then collect and emit these metrics to customize the observability of your application. - -xref:telemetry-trace.adoc[Code instrumentation for MicroProfile Telemetry tracing]:: -After you enable MicroProfile Telemetry and configure a trace server, you can instrument tracing in your application code. You can instrument your code automatically, manually, or by using the Java agent. For hands-on tutorials on different observability configurations for Open Liberty, check out our link:/guides/#observability[Observability guides]. diff --git a/modules/ROOT/pages/prepare-mptelemetry.adoc b/modules/ROOT/pages/prepare-mptelemetry.adoc new file mode 100644 index 0000000000..208d89dae1 --- /dev/null +++ b/modules/ROOT/pages/prepare-mptelemetry.adoc @@ -0,0 +1,52 @@ +// Copyright (c) 2024 IBM Corporation and others. +// Licensed under Creative Commons Attribution-NoDerivatives +// 4.0 International (CC BY-ND 4.0) +// https://creativecommons.org/licenses/by-nd/4.0/ +// +// Contributors: +// IBM Corporation +// +:page-description: +:seo-description: +:page-layout: general-reference +:page-type: general += Prepare your development environment for MicroProfile Telemetry + +Before you manually instrument your code to use MicroProfile Telemetry to collect metrics and traces, you must enable MicroProfile Telemetry in your development environment by editing your runtime configuration. You must also add the OpenTelemetry API and annotations as a dependency on your build path. + +Before you xref:telemetry-trace.adoc[manually instrument your code to collect traces] or xref:custom-mptelemetry-metrics.adoc[define custom metrics], complete the following prerequisites. + +. Enable MicroProfile Telemetry, as described in xref:microprofile-telemetry.adoc#global[Enabling MicroProfile Telemetry for Open Liberty]. + +. xref:class-loader-library-config.adoc#3rd-party[Enable third-party APIs] for your application by adding the following code in your `server.xml` file: ++ +[source,xml] +---- + + + +---- + +. Add the `opentelemetry` API and OpenTelemetry instrumentation annotations as a provided dependency to your build path. For example, with Maven, add the following code to your `pom.xml` file. ++ +[source,xml] +---- + + io.opentelemetry + opentelemetry-api + 1.39.0 + provided + + + io.opentelemetry.instrumentation + opentelemetry-instrumentation-annotations + 1.39.0-alpha + provided + +---- + +After you complete those prerequisites, you're ready to instrument your code. + +== See also +- xref:telemetry-trace.adoc[Code instrumentation for MicroProfile Telemetry tracing] +- xref:custom-mptelemetry-metrics.adoc[Define custom MicroProfile Telemetry metrics] diff --git a/modules/ROOT/pages/telemetry-trace.adoc b/modules/ROOT/pages/telemetry-trace.adoc index 2a82a09924..e2f85874d3 100644 --- a/modules/ROOT/pages/telemetry-trace.adoc +++ b/modules/ROOT/pages/telemetry-trace.adoc @@ -1,4 +1,4 @@ -// Copyright (c) 2022 IBM Corporation and others. +// Copyright (c) 2024 IBM Corporation and others. // Licensed under Creative Commons Attribution-NoDerivatives // 4.0 International (CC BY-ND 4.0) // https://creativecommons.org/licenses/by-nd/4.0/ @@ -50,36 +50,9 @@ otel.exporter.zipkin.endpoint=http://localhost:9411/api/v2/spans [#manual] == Manual instrumentation -Automatic instrumentation is available only for JAX-RS and Jakarta RESTful web service applications. To create spans for other operations, such as database calls, you can add manual instrumentation to the source code for those operations by using the https://www.javadoc.io/doc/io.opentelemetry/opentelemetry-api/1.19.0/io/opentelemetry/api/trace/package-summary.html[OpenTelemetry API]. However, before you manually instrument your code, you must complete the following prerequisites. +Automatic instrumentation is available only for JAX-RS and Jakarta RESTful web service applications. To create spans for other operations, such as database calls, you can add manual instrumentation to the source code for those operations by using the https://www.javadoc.io/doc/io.opentelemetry/opentelemetry-api/1.39.0/io/opentelemetry/api/trace/package-summary.html[OpenTelemetry API]. -. Enable MicroProfile Telemetry, as described in <<#ol-config,Configuring Open Liberty to use MicroProfile Telemetry>>. - -. xref:class-loader-library-config.adoc#3rd-party[Enable third-party APIs] for your application by adding the following code in your `server.xml` file: -+ -[source,xml] ----- - - - ----- - -. Add the `opentelemetry` API and OpenTelemetry instrumentation annotations as a provided dependency to your build path. For example, with Maven, add the following code to your `pom.xml` file. -+ -[source,xml] ----- - - io.opentelemetry - opentelemetry-api - 1.19.0 - provided - - - io.opentelemetry.instrumentation - opentelemetry-instrumentation-annotations - 1.19.0-alpha - provided - ----- +However, before you manually instrument your code, you must enable MicroProfile Telemetry in your development environment by editing your runtime configuration. You must also add the OpenTelemetry API and annotations as a dependency on your build path. For more information, see xref:prepare-mptelemetry-metrics.adoc[Prepare your development environment for MicroProfile Telemetry]. After you complete those prerequisites, you're ready to instrument your code. The following examples show configuration options with the OpenTelemetry API. @@ -170,7 +143,7 @@ The OpenTelemetry Java agent is a tool that is provided by the OpenTelemetry pro ** Configuration is shared between all applications that are deployed to the server. ** Configuration properties are only read from system properties and environment variables. They are not read from MicroProfile Config configuration sources. ** Because the agent reads its configuration early in the startup process, system properties are not read from the `bootstrap.properties` file. Alternatively, you can set system properties in the `jvm.options` file by using the following syntax: `-Dname=value` - ** Implementations of https://www.javadoc.io/doc/io.opentelemetry/opentelemetry-sdk-extension-autoconfigure-spi/1.19.0/index.html[SPI extensions] within applications are ignored. For more information, see the https://opentelemetry.io/docs/instrumentation/java/automatic/agent-config/#extensions[agent documentation for providing SPI extensions]. + ** Implementations of https://www.javadoc.io/doc/io.opentelemetry/opentelemetry-sdk-extension-autoconfigure-spi/1.39.0/index.html[SPI extensions] within applications are ignored. For more information, see the https://opentelemetry.io/docs/instrumentation/java/automatic/agent-config/#extensions[agent documentation for providing SPI extensions]. * When you use the agent, it takes over the instrumentation of REST calls and methods that are annotated with `@WithSpan`. As a result, the created spans might be slightly different. * The agent is not compatible with https://www.ibm.com/docs/en/was-liberty/base?topic=security-java-2[Java 2 security]. * Open Liberty uses many open source libraries internally. Some of these libraries might be automatically instrumented by the agent. From 17e191c1bfac785868a428f352bee16e0f97996c Mon Sep 17 00:00:00 2001 From: David Mueller Date: Mon, 9 Sep 2024 20:02:20 -0400 Subject: [PATCH 60/62] Update nav.adoc --- modules/ROOT/nav.adoc | 1 + 1 file changed, 1 insertion(+) diff --git a/modules/ROOT/nav.adoc b/modules/ROOT/nav.adoc index 4e274aeabb..0387cea373 100644 --- a/modules/ROOT/nav.adoc +++ b/modules/ROOT/nav.adoc @@ -49,6 +49,7 @@ ** xref:distributed-session-caching.adoc[Distributed session caching] ** xref:configuring-infinispan-support.adoc[Configuring Infinispan as a JCache provider] * xref:observability.adoc[Observability] + ** xref:prepare-mptelemetry.adoc[Prepare your development environment for MicroProfile Telemetry] ** xref:custom-mptelemetry-metrics.adoc[Define custom MicroProfile Telemetry metrics] ** xref:telemetry-trace.adoc[Code instrumentation for MicroProfile Telemetry tracing] ** xref:mptelemetry-logging.adoc[Write logs with MicroProfile Telemetry logging] From 36c416275c5f1df98d3545c54fd61ed2a0fac029 Mon Sep 17 00:00:00 2001 From: David Mueller Date: Mon, 9 Sep 2024 20:09:38 -0400 Subject: [PATCH 61/62] Update telemetry-trace.adoc --- modules/ROOT/pages/telemetry-trace.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/ROOT/pages/telemetry-trace.adoc b/modules/ROOT/pages/telemetry-trace.adoc index e2f85874d3..50197cc134 100644 --- a/modules/ROOT/pages/telemetry-trace.adoc +++ b/modules/ROOT/pages/telemetry-trace.adoc @@ -144,6 +144,6 @@ The OpenTelemetry Java agent is a tool that is provided by the OpenTelemetry pro ** Configuration properties are only read from system properties and environment variables. They are not read from MicroProfile Config configuration sources. ** Because the agent reads its configuration early in the startup process, system properties are not read from the `bootstrap.properties` file. Alternatively, you can set system properties in the `jvm.options` file by using the following syntax: `-Dname=value` ** Implementations of https://www.javadoc.io/doc/io.opentelemetry/opentelemetry-sdk-extension-autoconfigure-spi/1.39.0/index.html[SPI extensions] within applications are ignored. For more information, see the https://opentelemetry.io/docs/instrumentation/java/automatic/agent-config/#extensions[agent documentation for providing SPI extensions]. -* When you use the agent, it takes over the instrumentation of REST calls and methods that are annotated with `@WithSpan`. As a result, the created spans might be slightly different. +* When you use the agent, it takes over the instrumentation of REST calls and methods that are annotated with the `@WithSpan` annotation. As a result, the created spans might be slightly different. * The agent is not compatible with https://www.ibm.com/docs/en/was-liberty/base?topic=security-java-2[Java 2 security]. * Open Liberty uses many open source libraries internally. Some of these libraries might be automatically instrumented by the agent. From 597764b36c7def803126298ed2bdecbf506b7bdd Mon Sep 17 00:00:00 2001 From: David Mueller Date: Tue, 10 Sep 2024 07:06:41 -0400 Subject: [PATCH 62/62] fix link --- modules/ROOT/pages/custom-mptelemetry-metrics.adoc | 2 +- modules/ROOT/pages/observability.adoc | 2 +- modules/ROOT/pages/telemetry-trace.adoc | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/modules/ROOT/pages/custom-mptelemetry-metrics.adoc b/modules/ROOT/pages/custom-mptelemetry-metrics.adoc index 1a89ca1511..d92a5bbf35 100644 --- a/modules/ROOT/pages/custom-mptelemetry-metrics.adoc +++ b/modules/ROOT/pages/custom-mptelemetry-metrics.adoc @@ -18,7 +18,7 @@ For more information about collecting and emitting metrics with MicroProfile Tel For more information about OpenTelemetry metrics, see the link:https://www.javadoc.io/doc/io.opentelemetry/opentelemetry-api/1.39.0/io/opentelemetry/api/metrics/package-summary.html[OpenTelemetry metrics API documentation]. -Before you can use MicroProfile Telemetry to define custom metrics metrics, you must enable MicroProfile Telemetry in your development environment by editing your runtime configuration. You must also add the OpenTelemetry API and annotations as a dependency on your build path. For more information, see xref:prepare-mptelemetry-metrics.adoc[Prepare your development environment for MicroProfile Telemetry]. +Before you can use MicroProfile Telemetry to define custom metrics metrics, you must enable MicroProfile Telemetry in your development environment by editing your runtime configuration. You must also add the OpenTelemetry API and annotations as a dependency on your build path. For more information, see xref:prepare-mptelemetry.adoc[Prepare your development environment for MicroProfile Telemetry]. The following example defines a custom counter metric: diff --git a/modules/ROOT/pages/observability.adoc b/modules/ROOT/pages/observability.adoc index 04e8bf74bf..fc71f777fb 100644 --- a/modules/ROOT/pages/observability.adoc +++ b/modules/ROOT/pages/observability.adoc @@ -19,7 +19,7 @@ With MicroProfile Telemetry 2.0 and later, you can manage logs, metrics, and tra The following topics provide information to help you develop observable applications with Open Liberty. -xref:prepare-mptelemetry-metrics.adoc[Prepare your development environment for MicroProfile Telemetry]:: +xref:prepare-mptelemetry.adoc[Prepare your development environment for MicroProfile Telemetry]:: Before you manually instrument your code to use MicroProfile Telemetry to collect metrics and traces, you must enable MicroProfile Telemetry in your development environment by editing your runtime configuration. You must also add the OpenTelemetry API and annotations as a dependency on your build path. xref:custom-mptelemetry-metrics.adoc[Define custom MicroProfile Telemetry metrics]:: diff --git a/modules/ROOT/pages/telemetry-trace.adoc b/modules/ROOT/pages/telemetry-trace.adoc index 50197cc134..c38b65c0cb 100644 --- a/modules/ROOT/pages/telemetry-trace.adoc +++ b/modules/ROOT/pages/telemetry-trace.adoc @@ -52,7 +52,7 @@ otel.exporter.zipkin.endpoint=http://localhost:9411/api/v2/spans Automatic instrumentation is available only for JAX-RS and Jakarta RESTful web service applications. To create spans for other operations, such as database calls, you can add manual instrumentation to the source code for those operations by using the https://www.javadoc.io/doc/io.opentelemetry/opentelemetry-api/1.39.0/io/opentelemetry/api/trace/package-summary.html[OpenTelemetry API]. -However, before you manually instrument your code, you must enable MicroProfile Telemetry in your development environment by editing your runtime configuration. You must also add the OpenTelemetry API and annotations as a dependency on your build path. For more information, see xref:prepare-mptelemetry-metrics.adoc[Prepare your development environment for MicroProfile Telemetry]. +However, before you manually instrument your code, you must enable MicroProfile Telemetry in your development environment by editing your runtime configuration. You must also add the OpenTelemetry API and annotations as a dependency on your build path. For more information, see xref:prepare-mptelemetry.adoc[Prepare your development environment for MicroProfile Telemetry]. After you complete those prerequisites, you're ready to instrument your code. The following examples show configuration options with the OpenTelemetry API.