Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DOC-964 New bloblang error functions #151

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

asimms41
Copy link
Collaborator

@asimms41 asimms41 commented Jan 21, 2025

Description

Resolves DOC-964
Review deadline: 22nd January

Page previews

Bloblang functions

Checks

  • New feature
  • Content gap
  • Support Follow-up
  • Small fix (typos, links, copyedits, etc)

Copy link

netlify bot commented Jan 21, 2025

Deploy Preview for redpanda-connect ready!

Name Link
🔨 Latest commit 563fc4e
🔍 Latest deploy log https://app.netlify.com/sites/redpanda-connect/deploys/678f75bb80481d00082bba1f
😎 Deploy Preview https://deploy-preview-151--redpanda-connect.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@asimms41 asimms41 requested a review from mihaitodor January 21, 2025 10:34
@asimms41 asimms41 marked this pull request as ready for review January 21, 2025 10:34
@asimms41 asimms41 requested a review from a team as a code owner January 21, 2025 10:34
Copy link
Collaborator

@mihaitodor mihaitodor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you! :shipit:

@asimms41 asimms41 requested a review from JakeSCahill January 21, 2025 15:30
Copy link
Contributor

@JakeSCahill JakeSCahill left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to add these functions with more examples of real outputs to xref:configuration:error_handling.adoc?

@mihaitodor
Copy link
Collaborator

@JakeSCahill Good point! I took a stab at it here:

diff --git a/modules/configuration/pages/error_handling.adoc b/modules/configuration/pages/error_handling.adoc
index 73907a5..84e279d 100644
--- a/modules/configuration/pages/error_handling.adoc
+++ b/modules/configuration/pages/error_handling.adoc
@@ -49,18 +49,21 @@ pipeline:
 
 == Logging errors
 
-When an error occurs there will occasionally be useful information stored within the error flag that can be exposed with the interpolation function xref:configuration:interpolation.adoc#bloblang-queries[`error`]. This allows you to expose the information with processors.
+When an error occurs there will occasionally be useful information stored within the error flag that can be exposed with the interpolation function xref:configuration:interpolation.adoc#bloblang-queries[`error`]. Additionally, you can use the xref:guides:bloblang/methods.adoc#error_source_label[`error_source_label`], xref:guides:bloblang/methods.adoc#error_source_name[`error_source_name`] and / or xref:guides:bloblang/methods.adoc#error_source_path[`error_source_path`] bloblang functions to get extra information about the processor which failed.
 
-For example, when catching failed processors you can xref:components:processors/log.adoc[`log`] the messages:
+For example, when xref:components:processors/catch.adoc[`catch`]ing failed processors, you can xref:components:processors/log.adoc[`log`] the error message along with the label of the processor which failed:
 
 [source,yaml]
 ----
 pipeline:
   processors:
-    - resource: foo # Processor that might fail
+    - try:
+      - resource: foo # Processor that might fail
+      - resource: bar # Processor that might fail
+      - resource: baz # Processor that might fail
     - catch:
       - log:
-          message: "Processing failed due to: ${!error()}"
+          message: "Processor ${!error_source_label()} failed due to: ${!error()}"
 ----
 
 Or perhaps augment the message payload with the error message:

You can experiment with this:

res.yaml

processor_resources:
  - label: foo
    mapping: |
      root = content().uppercase()

  - label: bar
    mapping: |
      root = throw("kaboom!")

  - label: baz
    mapping: |
      root = content() + "blobfish"

conf.yaml

pipeline:
  processors:
    - try:
      - resource: foo # Processor that might fail
      - resource: bar # Processor that might fail
      - resource: baz # Processor that might fail

    - catch:
      - log:
          message: "Processor ${!error_source_label()} failed due to: ${!error()}"
$ rpk connect run -r res.yaml conf.yaml

Type something and hit enter.

WDYT? (cc @asimms41)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants