Insights instrumentation (#591)

* Basic working Insights instrumentation

* Fill in more insights plugins & tweaks

* Put old filter_map back & remove comment

* Perhaps we should actually send the event

* Fix test & pass raw metadata to EventFilter

* Plug is doing the same thing as this.

* Also remove it from the list

* Update lib/honeybadger/insights/ecto.ex

Co-authored-by: Benjamin Curtis <[email protected]>

* Shuffled used module specific functions around

* Add Ecto obfuscation

* Apply suggestions from code review

Co-authored-by: Parker Selbert <[email protected]>

* Add missing end from PR feedback merge

* process_measurements/1 refactor

Thanks @sorentwo

* Add tests and class docs

This also includes the `full_url` options for tesla and finch libraries

* Update README doc

This also won't report an event if the `filter/3` returns nil.

* Load support files correctly

* Update comment for Finch module

Also ignore insights events for Tesla requests that use the Finch
adapter, as we will send 2 events.

* Fix test

* Use async

* Apply suggestions from code review

Co-authored-by: Parker Selbert <[email protected]>

* Use `String.to_existing_atom/1`

* Load the atoms earlier

* Stop using strings to register telemetry_events

My decision to use dotified strings in the configuration to map to
instrumented telemetry events kept rubbing the wrong way. It was at the
point where I was preloading atoms to make sure the the terms were
avaliable so `to_existing_atom` would not error that the pain was
enough to reevalute things.

The config now uses lists of atoms for instrumented events. I am also
using these lists in all the places where callbacks were using the
string event name.

* Fix nested notifications issue

Since we are now using the same adapter that we are instrumenting on,
when we send an event to insights we get a telemetry event about
sending that event, and then send another event! Recursion!

This does not instrument a request event if the user-agent comes from
this library.

I also had to remove my Finch stub, since the :req dependency adds
finch.

* Fix tests

---------

Co-authored-by: Benjamin Curtis <[email protected]>
Co-authored-by: Parker Selbert <[email protected]>

Author: 3 people

README.md

@@ -150,6 +150,118 @@ rescue
 end
 ```
 
+## Insights Automatic Instrumentation
+
+Honeybadger Insights allows you to automatically track various events in your
+application. To enable Insights automatic instrumentation, add the following to
+your configuration:
+
+```elixir
+config :honeybadger,
+  insights_enabled: true
+```
+
+### Supported Libraries
+
+Honeybadger automatically instruments the following libraries when they are available:
+
+* Absinthe: GraphQL query execution
+* Ecto: Database queries
+* Finch: HTTP client requests, often used by other libraries like Req
+* Tesla: HTTP client requests
+* LiveView: Phoenix LiveView lifecycle events
+* Oban: Background job processing
+* Plug/Phoenix: HTTP requests
+
+### Custom Configuration
+
+Each library can be configured individually using the `insights_config` option:
+
+```elixir
+config :honeybadger, insights_config: %{
+  # Ecto configuration
+  ecto: %{
+    # A list of strings or regex patterns of queries to exclude
+    excluded_queries: [~r/^(begin|commit)/i],
+
+    # A list of table/source names to exclude
+    excluded_sources: ["schema_migrations"]
+  },
+
+  # Finch/Tesla configuration
+  finch: %{
+    # Include full URLs instead of just hostnames
+    full_url: false
+  },
+
+  # Plug configuration
+  plug: %{
+    telemetry_events: [[:phoenix, :endpoint, :stop]]
+  },
+
+  # LiveView configuration
+  live_view: %{
+    telemetry_events: [
+      [:phoenix, :live_view, :mount, :stop],
+      [:phoenix, :live_view, :update, :stop]
+    ]
+  },
+
+  # Absinthe configuration
+  absinthe: %{
+    telemetry_events: [
+      [:absinthe, :execute, :operation, :start],
+      [:absinthe, :execute, :operation, :stop],
+      [:absinthe, :execute, :operation, :exception]
+    ]
+  },
+
+  # Oban configuration
+  oban: %{
+    telemetry_events: [
+      [:oban, :job, :stop],
+      [:oban, :job, :exception]
+    ]
+  }
+}
+```
+
+### Event Filtering
+
+You can filter out or customize automatic events sent to Honeybadger Insights
+by implementing the `Honeybadger.EventFilter` behaviour:
+
+```elixir
+defmodule MyApp.EventFilter do
+  @behaviour Honeybadger.EventFilter
+
+  @default_filter &Honeybadger.EventFilter.Default.filter/3
+
+  @impl Honeybadger.EventFilter
+  # Pattern match on the event type name
+  def filter_telemetry_event(data, raw, [:phoenix, :live_view, :update, :stop] = event) do
+    if data.duration < 100 do
+      data
+      |> Map.put(:slow, true)
+      # You might want to run the event through the default filter to remove
+      # any sensitive data
+      |> @default_filter.(raw, event)
+    else
+      # Ignore events that are fast
+      nil
+    end
+  end
+
+  def filter(data, raw, event), do: @default_filter.(data, raw, event)
+end
+```
+
+Then configure the filter in your application's configuration:
+
+```elixir
+config :honeybadger, event_filter: MyApp.EventFilter
+```
+
 ## Breadcrumbs
 
 Breadcrumbs allow you to record events along a processes execution path. If
@@ -311,26 +423,30 @@ Here are all of the options you can pass in the keyword list:
 | ------------------------ | --------------------------------------------------------------------------------------------- | ---------------------------------------- |
 | `app`                    | Name of your app's OTP Application as an atom                                                 | `Mix.Project.config[:app]`               |
 | `api_key`                | Your application's Honeybadger API key                                                        | `System.get_env("HONEYBADGER_API_KEY"))` |
-| `environment_name`       | (required) The name of the environment your app is running in.                                | `:prod`                                    |
-|`exclude_errors`          |Filters out errors from being sent to Honeybadger        | `[]`|
+| `environment_name`       | (required) The name of the environment your app is running in.                                | `:prod`                                  |
+| `exclude_errors`         | Filters out errors from being sent to Honeybadger                                             | `[]`                                     |
 | `exclude_envs`           | Environments that you want to disable Honeybadger notifications                               | `[:dev, :test]`                          |
 | `hostname`               | Hostname of the system your application is running on                                         | `:inet.gethostname`                      |
 | `origin`                 | URL for the Honeybadger API                                                                   | `"https://api.honeybadger.io"`           |
 | `project_root`           | Directory root for where your application is running                                          | `System.cwd/0`                           |
 | `revision`               | The project's git revision                                                                    | `nil`                                    |
 | `filter`                 | Module implementing `Honeybadger.Filter` to filter data before sending to Honeybadger.io      | `Honeybadger.Filter.Default`             |
-| `filter_keys`            | A list of keywords (atoms) to filter.  Only valid if `filter` is `Honeybadger.Filter.Default` | `[:password, :credit_card]`              |
+| `filter_keys`            | A list of keywords (atoms) to filter.  Only valid if `filter` is `Honeybadger.Filter.Default` | `[:password, :credit_card, :__changed__, :flash, :_csrf_token]` |
 | `filter_args`            | If true, will remove function arguments in backtraces                                         | `true`                                   |
 | `filter_disable_url`     | If true, will remove the request url                                                          | `false`                                  |
 | `filter_disable_session` | If true, will remove the request session                                                      | `false`                                  |
 | `filter_disable_params`  | If true, will remove the request params                                                       | `false`                                  |
+| `filter_disable_assigns` | If true, will remove the live_view event assigns                                              | `false`                                  |
 | `fingerprint_adapter`    | Implementation of FingerprintAdapter behaviour                                                |                                          |
 | `notice_filter`          | Module implementing `Honeybadger.NoticeFilter`. If `nil`, no filtering is done.               | `Honeybadger.NoticeFilter.Default`       |
 | `sasl_logging_only`      | If true, will notifiy for SASL errors but not Logger calls                                    | `true`                                   |
 | `use_logger`             | Enable the Honeybadger Logger for handling errors outside of web requests                     | `true`                                   |
 | `ignored_domains`        | Add domains to ignore Error events in `Honeybadger.Logger`.                                   | `[:cowboy]`                              |
 | `breadcrumbs_enabled`    | Enable breadcrumb event tracking                                                              | `false`                                  |
 | `ecto_repos`             | Modules with implemented Ecto.Repo behaviour for tracking SQL breadcrumb events               | `[]`                                     |
+| `event_filter`           | Module implementing `Honeybadger.EventFilter`. If `nil`, no filtering is done.                | `Honeybadger.EventFilter.Default`        |
+| `insights_enabled`       | Enable sending automatic events to Honeybadger Insights                                       | `false`                                  |
+| `insights_config`        | Specific library Configuration for Honeybadger Insights.                                      | `%{}`                                    |
 | `http_adapter`           | Module implementing `Honeybadger.HttpAdapter` to send data to Honeybadger.io                  | Any available adapter (`Req`, `hackney`) |
 
 ### HTTP Adapters

lib/honeybadger.ex

@@ -163,6 +163,8 @@ defmodule Honeybadger do
 
       config :honeybadger,
         ecto_repos: [MyApp.Repo]
+
+  #### Insights
   """
 
   use Application
@@ -206,6 +208,19 @@ defmodule Honeybadger do
       Honeybadger.Breadcrumbs.Telemetry.attach()
     end
 
+    if config[:insights_enabled] do
+      [
+        Honeybadger.Insights.Plug,
+        Honeybadger.Insights.Ecto,
+        Honeybadger.Insights.LiveView,
+        Honeybadger.Insights.Oban,
+        Honeybadger.Insights.Absinthe,
+        Honeybadger.Insights.Finch,
+        Honeybadger.Insights.Tesla
+      ]
+      |> Enum.each(& &1.attach())
+    end
+
     children = [{Client, [config]}, EventsWorker]
 
     Supervisor.start_link(children, strategy: :one_for_one)

lib/honeybadger/event_filter.ex

@@ -0,0 +1,28 @@
+defmodule Honeybadger.EventFilter do
+  @moduledoc """
+  Specification for filtering instrumented events.
+
+  Most users won't need this, but if you need complete control over
+  filtering, implement this behaviour and configure like:
+
+      config :honeybadger,
+        event_filter: MyApp.MyEventFilter
+  """
+
+  @doc """
+  Filters an instrumented telemetry event.
+
+  ## Parameters
+
+    * `data` - The current data for the event
+    * `raw_event` - The raw event metadata
+    * `event` - The telemetry event being processed, e.g. [:phoenix, :endpoint, :start]
+
+  ## Returns
+
+    The filtered metadata map that will be sent to Honeybadger or `nil` to skip
+    the event.
+  """
+  @callback filter_telemetry_event(data :: map(), raw_event :: map(), event :: [atom(), ...]) ::
+              map() | nil
+end

lib/honeybadger/event_filter/default.ex

@@ -0,0 +1,22 @@
+defmodule Honeybadger.EventFilter.Default do
+  @behaviour Honeybadger.EventFilter
+
+  alias Honeybadger.Utils
+
+  def filter_telemetry_event(data, _raw, _event) do
+    data
+    |> disable(:filter_disable_url, :url)
+    |> disable(:filter_disable_session, :session)
+    |> disable(:filter_disable_assigns, :assigns)
+    |> disable(:filter_disable_params, :params)
+    |> Utils.sanitize(remove_filtered: true)
+  end
+
+  defp disable(meta, config_key, map_key) do
+    if Honeybadger.get_env(config_key) do
+      Map.drop(meta, [map_key])
+    else
+      meta
+    end
+  end
+end

lib/honeybadger/insights/absinthe.ex

@@ -0,0 +1,125 @@
+defmodule Honeybadger.Insights.Absinthe do
+  @moduledoc """
+  Captures telemetry events from GraphQL operations executed via Absinthe.
+
+  ## Default Configuration
+
+  By default, this module listens for the following Absinthe telemetry events:
+
+      "absinthe.execute.operation.stop"
+      "absinthe.execute.operation.exception"
+
+  ## Custom Configuration
+
+  You can customize the telemetry events to listen for by updating the insights_config:
+
+      config :honeybadger, insights_config: %{
+        absinthe: %{
+          telemetry_events: [
+            [:absinthe, :execute, :operation, :stop],
+            [:absinthe, :execute, :operation, :exception],
+            [:absinthe, :resolve, :field, :stop]
+          ]
+        }
+      }
+
+  Note that adding field-level events like "absinthe.resolve.field.stop" can
+  significantly increase the number of telemetry events generated.
+  """
+
+  use Honeybadger.Insights.Base
+
+  @required_dependencies [Absinthe]
+
+  @telemetry_events [
+    [:absinthe, :execute, :operation, :stop],
+    [:absinthe, :execute, :operation, :exception]
+  ]
+
+  # This is not loaded by default since it can add a ton of events, but is here
+  # in case it is added to the insights_config.
+  def extract_metadata(%{resolution: resolution}, [:absinthe, :resolve, :field, :stop]) do
+    %{
+      field_name: resolution.definition.name,
+      parent_type: resolution.parent_type.name,
+      state: resolution.state
+    }
+  end
+
+  def extract_metadata(meta, _name) do
+    %{
+      operation_name: get_operation_name(meta),
+      operation_type: get_operation_type(meta),
+      selections: get_graphql_selections(meta),
+      schema: get_schema(meta),
+      errors: get_errors(meta)
+    }
+  end
+
+  defp get_schema(%{blueprint: blueprint}) when is_map(blueprint), do: Map.get(blueprint, :schema)
+  defp get_schema(_), do: nil
+
+  defp get_errors(%{blueprint: blueprint}) when is_map(blueprint) do
+    case Map.get(blueprint, :result) do
+      result when is_map(result) -> Map.get(result, :errors)
+      _ -> nil
+    end
+  end
+
+  defp get_errors(_), do: nil
+
+  defp get_graphql_selections(%{blueprint: blueprint}) when is_map(blueprint) do
+    operation = current_operation(blueprint)
+
+    case operation do
+      nil ->
+        []
+
+      operation ->
+        case Map.get(operation, :selections) do
+          selections when is_list(selections) ->
+            selections
+            |> Enum.map(fn selection -> Map.get(selection, :name) end)
+            |> Enum.uniq()
+
+          _ ->
+            []
+        end
+    end
+  end
+
+  defp get_graphql_selections(_), do: []
+
+  defp get_operation_type(%{blueprint: blueprint}) when is_map(blueprint) do
+    operation = current_operation(blueprint)
+
+    case operation do
+      nil -> nil
+      operation -> Map.get(operation, :type)
+    end
+  end
+
+  defp get_operation_type(_), do: nil
+
+  defp get_operation_name(%{blueprint: blueprint}) when is_map(blueprint) do
+    operation = current_operation(blueprint)
+
+    case operation do
+      nil -> nil
+      operation -> Map.get(operation, :name)
+    end
+  end
+
+  defp get_operation_name(_), do: nil
+
+  # Replace Absinthe.Blueprint.current_operation/1
+  defp current_operation(blueprint) do
+    case Map.get(blueprint, :operations) do
+      operations when is_list(operations) ->
+        Enum.find(operations, fn op -> Map.get(op, :current) == true end)
+
+      _ ->
+        nil
+    end
+  end
+end