Skip to content

Commit

Permalink
[One Discover] Display stacktrace in the logs overview tab (elastic#2…
Browse files Browse the repository at this point in the history
…04521)

## 📓 Summary
Adds a new section to the overview tab in the log details flyout in
Discover to display stacktrace information for logs and exceptions.

In a follow-up, the stacktrace could be moved to a new tab in the log
details flyout and actions can be added to the stacktrace (and quality)
icons in the document table to open the relevant sections in the flyout.

Closes elastic#190460

### APM - Log stacktrace (library frames)
<img width="1470" alt="image"
src="https://github.com/user-attachments/assets/8991f882-d329-4bc5-aa37-424576bcee72"
/>

### APM - Exception (with cause)
<img width="1476" alt="image"
src="https://github.com/user-attachments/assets/cfbf24a7-6f82-48f1-b275-5aac977411ac"
/>

### APM - Exception (simple stacktrace)
<img width="1474" alt="image"
src="https://github.com/user-attachments/assets/fc0306c4-5fcd-4b74-bb0d-c1784a48d677"
/>

### Apache Tomcat Integration (Catalina) - Stacktrace
<img width="1472" alt="image"
src="https://github.com/user-attachments/assets/281f1822-faea-4e2d-9515-c11a9ee12f50"
/>

## 📝 Notes for reviewers
- The `@kbn/apm-types` package was marked as platform / shared as it's
being used by the
[unified_doc_viewer](https://github.com/elastic/kibana/blob/main/src/plugins/unified_doc_viewer/kibana.jsonc)
- The code used to render stacktraces in APM was moved into a new
`@kbn/event-stacktrace` package as it is reused in the
`unified_doc_viewer`
- The code used to render metadata table in APM was moved into a new
`@kbn/key-value-metadata-table` package

## 🧪 Testing instructions
The deployed environments have sample logs that can be used (time range:
Jan 1, 2025 - now). For a local setup, please follow the instructions
below:

1. Ingest sample logs with stacktraces
([gist](https://gist.github.com/gbamparop/0da21ca7f65b24c4a9c071ce9e9b97b0)).
Please note that these are test data and some fields that are not used
by stacktraces might not be consistent
2. View relevant logs in Discover (Query: `service.name: "synth-node-0"
OR apache_tomcat :*`, Time range: Jan 1, 2025 - now)

---------

Co-authored-by: kibanamachine <[email protected]>
  • Loading branch information
2 people authored and viduni94 committed Jan 23, 2025
1 parent 3dc85e5 commit 0644f0a
Show file tree
Hide file tree
Showing 116 changed files with 772 additions and 111 deletions.
4 changes: 3 additions & 1 deletion .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -801,12 +801,15 @@ x-pack/platform/packages/shared/index-lifecycle-management/index_lifecycle_manag
x-pack/platform/packages/shared/index-management/index_management_shared_types @elastic/kibana-management
x-pack/platform/packages/shared/kbn-ai-assistant @elastic/search-kibana
x-pack/platform/packages/shared/kbn-alerting-comparators @elastic/response-ops
x-pack/platform/packages/shared/kbn-apm-types @elastic/obs-ux-infra_services-team
x-pack/platform/packages/shared/kbn-cloud-security-posture/common @elastic/kibana-cloud-security-posture
x-pack/platform/packages/shared/kbn-data-forge @elastic/obs-ux-management-team
x-pack/platform/packages/shared/kbn-elastic-assistant @elastic/security-generative-ai
x-pack/platform/packages/shared/kbn-elastic-assistant-common @elastic/security-generative-ai
x-pack/platform/packages/shared/kbn-entities-schema @elastic/obs-entities
x-pack/platform/packages/shared/kbn-event-stacktrace @elastic/obs-ux-infra_services-team @elastic/obs-ux-logs-team
x-pack/platform/packages/shared/kbn-inference-endpoint-ui-common @elastic/response-ops @elastic/appex-ai-infra @elastic/obs-ai-assistant @elastic/security-generative-ai
x-pack/platform/packages/shared/kbn-key-value-metadata-table @elastic/obs-ux-infra_services-team @elastic/obs-ux-logs-team
x-pack/platform/packages/shared/kbn-langchain @elastic/security-generative-ai
x-pack/platform/packages/shared/kbn-slo-schema @elastic/obs-ux-management-team
x-pack/platform/packages/shared/ml/aiops_common @elastic/ml-ui
Expand Down Expand Up @@ -907,7 +910,6 @@ x-pack/solutions/observability/packages/alert_details @elastic/obs-ux-management
x-pack/solutions/observability/packages/alerting_test_data @elastic/obs-ux-management-team
x-pack/solutions/observability/packages/get_padded_alert_time_range_util @elastic/obs-ux-management-team
x-pack/solutions/observability/packages/kbn-alerts-grouping @elastic/response-ops
x-pack/solutions/observability/packages/kbn-apm-types @elastic/obs-ux-infra_services-team
x-pack/solutions/observability/packages/kbn-custom-integrations @elastic/obs-ux-logs-team
x-pack/solutions/observability/packages/kbn-investigation-shared @elastic/obs-ux-management-team
x-pack/solutions/observability/packages/kbn-streams-schema @elastic/streams-program-team
Expand Down
4 changes: 3 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -188,7 +188,7 @@
"@kbn/apm-data-access-plugin": "link:x-pack/solutions/observability/plugins/apm_data_access",
"@kbn/apm-data-view": "link:src/platform/packages/shared/kbn-apm-data-view",
"@kbn/apm-plugin": "link:x-pack/solutions/observability/plugins/apm",
"@kbn/apm-types": "link:x-pack/solutions/observability/packages/kbn-apm-types",
"@kbn/apm-types": "link:x-pack/platform/packages/shared/kbn-apm-types",
"@kbn/apm-utils": "link:src/platform/packages/shared/kbn-apm-utils",
"@kbn/app-link-test-plugin": "link:test/plugin_functional/plugins/app_link_test",
"@kbn/application-usage-test-plugin": "link:x-pack/test/usage_collection/plugins/application_usage_test",
Expand Down Expand Up @@ -499,6 +499,7 @@
"@kbn/event-annotation-plugin": "link:src/platform/plugins/private/event_annotation",
"@kbn/event-log-fixture-plugin": "link:x-pack/test/plugin_api_integration/plugins/event_log",
"@kbn/event-log-plugin": "link:x-pack/platform/plugins/shared/event_log",
"@kbn/event-stacktrace": "link:x-pack/platform/packages/shared/kbn-event-stacktrace",
"@kbn/expandable-flyout": "link:x-pack/solutions/security/packages/expandable-flyout",
"@kbn/exploratory-view-example-plugin": "link:x-pack/examples/exploratory_view_example",
"@kbn/exploratory-view-plugin": "link:x-pack/solutions/observability/plugins/exploratory_view",
Expand Down Expand Up @@ -596,6 +597,7 @@
"@kbn/kbn-top-nav-plugin": "link:test/plugin_functional/plugins/kbn_top_nav",
"@kbn/kbn-tp-custom-visualizations-plugin": "link:test/plugin_functional/plugins/kbn_tp_custom_visualizations",
"@kbn/kbn-tp-run-pipeline-plugin": "link:test/interpreter_functional/plugins/kbn_tp_run_pipeline",
"@kbn/key-value-metadata-table": "link:x-pack/platform/packages/shared/kbn-key-value-metadata-table",
"@kbn/kibana-cors-test-plugin": "link:x-pack/test/functional_cors/plugins/kibana_cors_test",
"@kbn/kibana-overview-plugin": "link:src/platform/plugins/private/kibana_overview",
"@kbn/kibana-react-plugin": "link:src/platform/plugins/shared/kibana_react",
Expand Down
4 changes: 2 additions & 2 deletions packages/kbn-apm-synthtrace-client/src/lib/logs/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -58,8 +58,8 @@ export type LogDocument = Fields &
'cloud.project.id'?: string;
'cloud.instance.id'?: string;
'error.stack_trace'?: string;
'error.exception.stacktrace'?: string;
'error.log.stacktrace'?: string;
'error.exception'?: unknown;
'error.log'?: unknown;
'log.custom': Record<string, unknown>;
'host.geo.location': number[];
'host.ip': string;
Expand Down
6 changes: 3 additions & 3 deletions packages/kbn-apm-synthtrace/src/scenarios/simple_logs.ts
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,7 @@ const scenario: Scenario<LogDocument> = async (runOptions) => {
.defaults({
...commonLongEntryFields,
'error.message': message,
'error.exception.stacktrace': 'Error message in error.exception.stacktrace',
'error.stack_trace': 'Stacktrace',
})
.timestamp(timestamp);
});
Expand Down Expand Up @@ -174,7 +174,7 @@ const scenario: Scenario<LogDocument> = async (runOptions) => {
.defaults({
...commonLongEntryFields,
'event.original': message,
'error.log.stacktrace': 'Error message in error.log.stacktrace',
'error.stack_trace': 'Stacktrace',
'event.start': eventDate,
'event.end': moment(eventDate).add(1, 'm').toDate(),
})
Expand Down Expand Up @@ -203,7 +203,7 @@ const scenario: Scenario<LogDocument> = async (runOptions) => {
.setHostIp(getIpAddress())
.defaults({
...commonLongEntryFields,
'error.stack_trace': 'Error message in error.stack_trace',
'error.stack_trace': 'Stacktrace',
})
.timestamp(timestamp);
});
Expand Down
3 changes: 2 additions & 1 deletion packages/kbn-apm-synthtrace/src/scenarios/spiked_latency.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ import {
generateLongId,
generateShortId,
Instance,
LogDocument,
} from '@kbn/apm-synthtrace-client';
import { Scenario } from '../cli/scenario';
import { getSynthtraceEnvironment } from '../lib/utils/get_synthtrace_environment';
Expand All @@ -26,7 +27,7 @@ const ENVIRONMENT = getSynthtraceEnvironment(__filename);
const alwaysSpikeTransactionName = 'GET /always-spike';
const sometimesSpikeTransactionName = 'GET /sometimes-spike';

const scenario: Scenario<ApmFields> = async ({ logger, ...runOptions }) => {
const scenario: Scenario<LogDocument | ApmFields> = async ({ logger, ...runOptions }) => {
const { isLogsDb } = parseLogsScenarioOpts(runOptions.scenarioOpts);

return {
Expand Down
1 change: 1 addition & 0 deletions src/dev/storybook/aliases.ts
Original file line number Diff line number Diff line change
Expand Up @@ -77,4 +77,5 @@ export const storybookAliases = {
ui_actions_enhanced: 'src/platform/plugins/shared/ui_actions_enhanced/.storybook',
unified_search: 'src/platform/plugins/shared/unified_search/.storybook',
profiling: 'x-pack/solutions/observability/plugins/profiling/.storybook',
event_stacktrace: 'x-pack/platform/packages/shared/kbn-event-stacktrace/.storybook',
};
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,8 @@ export interface LogDocument extends DataTableRecord {
'data_stream.dataset': string;

'error.stack_trace'?: string;
'error.exception.stacktrace'?: string;
'error.log.stacktrace'?: string;
'error.exception.stacktrace.abs_path'?: string;
'error.log.stacktrace.abs_path'?: string;
};
}

Expand Down Expand Up @@ -83,8 +83,8 @@ export interface ResourceFields {

export interface StackTraceFields {
'error.stack_trace'?: string;
'error.exception.stacktrace'?: string;
'error.log.stacktrace'?: string;
'error.exception.stacktrace.abs_path'?: string;
'error.log.stacktrace.abs_path'?: string;
}

export interface SmartFieldGridColumnOptions {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ export const TRACE_ID_FIELD = 'trace.id';
export const LOG_FILE_PATH_FIELD = 'log.file.path';
export const DATASTREAM_NAMESPACE_FIELD = 'data_stream.namespace';
export const DATASTREAM_DATASET_FIELD = 'data_stream.dataset';
export const DATASTREAM_TYPE_FIELD = 'data_stream.type';

// Resource Fields
export const AGENT_NAME_FIELD = 'agent.name';
Expand All @@ -41,5 +42,5 @@ export const DEGRADED_DOCS_FIELDS = [IGNORED_FIELD, IGNORED_FIELD_VALUES_FIELD]

// Error Stacktrace
export const ERROR_STACK_TRACE = 'error.stack_trace';
export const ERROR_EXCEPTION_STACKTRACE = 'error.exception.stacktrace';
export const ERROR_LOG_STACKTRACE = 'error.log.stacktrace';
export const ERROR_EXCEPTION_STACKTRACE_ABS_PATH = 'error.exception.stacktrace.abs_path';
export const ERROR_LOG_STACKTRACE_ABS_PATH = 'error.log.stacktrace.abs_path';
Original file line number Diff line number Diff line change
Expand Up @@ -9,19 +9,19 @@

import { getFieldValue, LogDocument, StackTraceFields } from '..';
import {
ERROR_EXCEPTION_STACKTRACE,
ERROR_LOG_STACKTRACE,
ERROR_EXCEPTION_STACKTRACE_ABS_PATH,
ERROR_LOG_STACKTRACE_ABS_PATH,
ERROR_STACK_TRACE,
} from '../field_constants';

export const getStacktraceFields = (doc: LogDocument): StackTraceFields => {
const errorStackTrace = getFieldValue(doc, ERROR_STACK_TRACE);
const errorExceptionStackTrace = getFieldValue(doc, ERROR_EXCEPTION_STACKTRACE);
const errorLogStackTrace = getFieldValue(doc, ERROR_LOG_STACKTRACE);
const errorExceptionStackTrace = getFieldValue(doc, ERROR_EXCEPTION_STACKTRACE_ABS_PATH);
const errorLogStackTrace = getFieldValue(doc, ERROR_LOG_STACKTRACE_ABS_PATH);

return {
[ERROR_STACK_TRACE]: errorStackTrace,
[ERROR_EXCEPTION_STACKTRACE]: errorExceptionStackTrace,
[ERROR_LOG_STACKTRACE]: errorLogStackTrace,
[ERROR_EXCEPTION_STACKTRACE_ABS_PATH]: errorExceptionStackTrace,
[ERROR_LOG_STACKTRACE_ABS_PATH]: errorLogStackTrace,
};
};
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,4 @@
"kibanaUtils"
]
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,13 @@ import { DocViewRenderProps } from '@kbn/unified-doc-viewer/types';
import { getLogDocumentOverview } from '@kbn/discover-utils';
import { EuiHorizontalRule, EuiSpacer } from '@elastic/eui';
import { ObservabilityLogsAIAssistantFeatureRenderDeps } from '@kbn/discover-shared-plugin/public';
import { getStacktraceFields, LogDocument } from '@kbn/discover-utils/src';
import { LogsOverviewHeader } from './logs_overview_header';
import { LogsOverviewHighlights } from './logs_overview_highlights';
import { FieldActionsProvider } from '../../hooks/use_field_actions';
import { getUnifiedDocViewerServices } from '../../plugin';
import { LogsOverviewDegradedFields } from './logs_overview_degraded_fields';
import { LogsOverviewStacktraceSection } from './logs_overview_stacktrace_section';

export type LogsOverviewProps = DocViewRenderProps & {
renderAIAssistant?: (deps: ObservabilityLogsAIAssistantFeatureRenderDeps) => JSX.Element;
Expand All @@ -34,6 +36,8 @@ export function LogsOverview({
const { fieldFormats } = getUnifiedDocViewerServices();
const parsedDoc = getLogDocumentOverview(hit, { dataView, fieldFormats });
const LogsOverviewAIAssistant = renderAIAssistant;
const stacktraceFields = getStacktraceFields(hit as LogDocument);
const isStacktraceAvailable = Object.values(stacktraceFields).some(Boolean);

return (
<FieldActionsProvider
Expand All @@ -47,6 +51,7 @@ export function LogsOverview({
<EuiHorizontalRule margin="xs" />
<LogsOverviewHighlights formattedDoc={parsedDoc} flattenedDoc={hit.flattened} />
<LogsOverviewDegradedFields rawDoc={hit.raw} />
{isStacktraceAvailable && <LogsOverviewStacktraceSection hit={hit} dataView={dataView} />}
{LogsOverviewAIAssistant && <LogsOverviewAIAssistant doc={hit} />}
</FieldActionsProvider>
);
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the "Elastic License
* 2.0", the "GNU Affero General Public License v3.0 only", and the "Server Side
* Public License v 1"; you may not use this file except in compliance with, at
* your election, the "Elastic License 2.0", the "GNU Affero General Public
* License v3.0 only", or the "Server Side Public License, v 1".
*/
import { EuiAccordion, EuiHorizontalRule, EuiTitle, useGeneratedHtmlId } from '@elastic/eui';
import { DataTableRecord } from '@kbn/discover-utils';
import { i18n } from '@kbn/i18n';
import React from 'react';
import type { DataView } from '@kbn/data-views-plugin/public';
import { StacktraceContent } from './sub_components/stacktrace/stacktrace_content';

const stacktraceAccordionTitle = i18n.translate(
'unifiedDocViewer.docView.logsOverview.accordion.title.stacktrace',
{
defaultMessage: 'Stacktrace',
}
);

export function LogsOverviewStacktraceSection({
hit,
dataView,
}: {
hit: DataTableRecord;
dataView: DataView;
}) {
const accordionId = useGeneratedHtmlId({
prefix: stacktraceAccordionTitle,
});

return (
<>
<EuiAccordion
id={accordionId}
buttonContent={
<EuiTitle size="xs">
<p>{stacktraceAccordionTitle}</p>
</EuiTitle>
}
paddingSize="m"
initialIsOpen={false}
data-test-subj="unifiedDocViewLogsOverviewStacktraceAccordion"
>
<StacktraceContent hit={hit} dataView={dataView} />
</EuiAccordion>
<EuiHorizontalRule margin="xs" />
</>
);
}
Loading

0 comments on commit 0644f0a

Please sign in to comment.