enhancement(http provider): Add automatic bearer token acquisition in http-client [version 2]

[KowalczykBartek]

PR to solve a #20635 feature request.
This change brings new section in configuration for HttpSinkConfig :

pub struct HttpClientAuthorizationConfig {
    auth: HttpClientAuthorizationStrategy,
    tls: Option<TlsConfig>,
}

this allows to explicit define a oauth2 server which will be used to acquire bearer token (also, this request allows to bring mtls via TlsConfig),

sources:
  my_source_id:
    scrape_interval_secs: 1
    type: http_client
    endpoint: http://127.0.0.1:9898/logs

sinks:
  my_sink_id:
    http_client_authorization_strategy:
      auth:
        strategy: "o_auth2"
        client_id: "your.clientid"
        client_secret: "verysecretstring"
        token_endpoint: "https://your.provider.com/oauth/token"
    type: http
    encoding:
      codec: raw_message
    inputs:
      - my_source_id
    uri: http://127.0.0.1:9091/output

Some basic error handling is also on place, so invalid credentials will not crash a vector instance :) you can expect an event and error message with body from oauth server (unfortunately I over engineered this error handling a little bit, I have limited rust knowledge and was unable to fully understand all patterns - so, suggestions, how to improve this would be appreciated :) )

2024-10-22T21:51:18.256079Z  WARN sink{component_kind="sink" component_id=my_sink_id component_type=http}:request{request_id=1}:http: vector::internal_events::http_client: HTTP Auth extension error. error=Server error from authentication server: {"error":"invalid_client","error_description":"Bad credentials"} error_type="request_failed" stage="processing" internal_log_rate_limit=true

So, as discussed in #21023 (comment) - @pront I would be grateful for review :)

[KowalczykBartek]

I had no idea how to handle this

 Box<dyn Error>

to some vector friendly error better 😞

[KowalczykBartek]

I was able to improve it a little bit with snafu

auth_extension.modify_request(&mut request)
    .instrument(auth_span.clone().or_current())
    .await
    .inspect_err(|error| {
        // Emit the error into the internal events system.
        emit!(http_client::GotAuthExtensionError{error});
    })
    .context(ExtensionAuthenticationSnafu)?;

[pront]

Hi @KowalczykBartek thank you for this contribution!

I did a quick scan of the changes and left a few comments.

[KowalczykBartek]

I introduced grace_period property so, it can be adjusted - before it was hard coded for 1min. I also played around this feature more and found bug around grace calculation (overflow), so also fixed this.
I also verified following config works (so use case from original issue, oauth2 + mtls)

sinks:
  my_sink_id:
    http_client_authorization_strategy:
      auth:
        strategy: "o_auth2"
        client_id: "clientid"
        client_secret: "secret"
        grace_period: 300
        token_endpoint: "https://127.0.0.1:8091/oauth/token"
      tls:
        crt_path: /config/cert.pem
        key_file: /config/server.key

what I need still to do is test against more providers (I tested with UAA, its pretty common).

Hi @singhbaljit, may I ask you to check if that integration works in your env ? if there will be any required change I could make an adjustments.

[pront]

This is starting to look great. Left a few nits, these should improve config readability.

[KowalczykBartek]

after testing on real environments I found another issue, oauth2 + mtls should be strict to the rfc https://datatracker.ietf.org/doc/html/rfc8705, so, speaking in cURL
this is oauth2 + mtls extension

curl --cert cert.pem --key key.pem --request POST \
  --url https://something/oauth/token \
  --header 'Content-Type: application/x-www-form-urlencoded' \
  --data 'grant_type=client_credentials' \
  --data 'client_id=abcdabcdabcd'

this is oauth2

curl --request POST \
  --url https://something/oauth/token \
  --header 'Content-Type: application/x-www-form-urlencoded' \
  --data client_secret=abcdabcdabcd \
  --data grant_type=client_credentials \
  --data client_id=abcdabcdabcd

I also updated a comment around HttpClientAuthorizationStrategy. With those changes I think I can say: it works :)

[pront]

after testing on real environments I found another issue, oauth2 + mtls should be strict to the efc https://datatracker.ietf.org/doc/html/rfc8705, so, speaking in cURL this is oauth2 + mtls extension

curl --cert cert.pem --key key.pem --request POST \
  --url https://something/oauth/token \
  --header 'Content-Type: application/x-www-form-urlencoded' \
  --data 'grant_type=client_credentials' \
  --data 'client_id=abcdabcdabcd'

this is oauth2

curl --request POST \
  --url https://something/oauth/token \
  --header 'Content-Type: application/x-www-form-urlencoded' \
  --data client_secret=abcdabcdabcd \
  --data grant_type=client_credentials \
  --data client_id=abcdabcdabcd

I also updated a comment around HttpClientAuthorizationStrategy. With those changes I think I can say: it works :)

Thank you @KowalczykBartek, I will do another review tomorrow.

[pront]

👋 Left some comments. Most of them are nits but we should get rid of the unwraps.

Also, we lack tests here. We could mock the endpoint but we can also test some internal bits separately. Please see existing tests for inspiration.

Thank you again for this contribution!

[KowalczykBartek]

Also, we lack tests here. We could mock the endpoint but we can also test some internal bits separately. Please see existing tests for inspiration.

just FYI (because I see some jobs running :D ) - I will provide tests, I just need few days more :)

[pront]

Also, we lack tests here. We could mock the endpoint but we can also test some internal bits separately. Please see existing tests for inspiration.

just FYI (because I see some jobs running :D ) - i will provide tests, I just need few days more :)

Thank you @KowalczykBartek, much appreciated. Also, I wanted to share a few notes to help with the CI checks:

• we can add an http scope here
• also see our changelog guide
• cargo install -f --path vdev #cd to the repo root
• Run cargo vdev fmt
• Run cargo vdev check rust --clippy

[KowalczykBartek]

I was able to run this command,

# review any changes
# commit if they look good

but this this is tricky, I am not sure what this http.cue is :) It looks, good, but is it good ? not sure :)

btw, last time I saw ruby was like.. 6years ago when deploying database using bosh.io (not a drills company :)) :D

[KowalczykBartek]

Windows has some differences across error messages, so, I just removed error msg expectation for connection refused and internal server errors, its too much library and platform dependent

[singhbaljit]

Chiming in a bit late... this is highly appreciated! I'll certainly test this feature and report any issues I find. Thank you very much!

[tareksha]

many thanks @KowalczykBartek for the great work!

[tareksha]

hi @pront, @jszwedko, when is a version containing this change expected to be released?

[pront]

hi @pront, @jszwedko, when is a version containing this change expected to be released?

The next Vector release, probably v0.43 will include this enhancement.
Current version is https://vector.dev/releases/0.42.0/.

[tareksha]

The next Vector release, probably v0.43 will include this enhancement. Current version is https://vector.dev/releases/0.42.0/.

@pront when is v0.43 expected to be released?

[jszwedko]

The next Vector release, probably v0.43 will include this enhancement. Current version is https://vector.dev/releases/0.42.0/.

@pront when is v0.43 expected to be released?

Dec 2nd. You can see this via the milestone. Also calendar.vector.dev which is a public calendar with the releases on it (without version numbers).

[pront]

Hi all, unfortunately I found a UX issue with this PR which should be fixed before including in a Vector release. Apologies, I missed it when reviewing because I was focusing on other technical details.

Note how we already provide:

• https://vector.dev/docs/reference/configuration/sinks/http/#auth
• In code: https://github.com/vectordotdev/vector/blob/master/src/http.rs#L677-L710 (note how it is tagged with strategy)

I created the website locally:

There is a lot of redundancy in the newly added authorization_config. The newly added oath_2 could have simply been a new variant under Auth. This is backwards compatible as well since we are just adding a new strategy.

So we have two options here:

1. follow-up and fix the UX before the next release
2. revert the PR

Is option 1 something that you are willing to contribute?

[KowalczykBartek]

I missed it when reviewing because I was focusing on other technical details.

Forrest Gump used to say: 'Shit happens... sometimes' :)

Is option 1 something that you are willing to contribute?

maybe not willing, but reverting it back would be sad for me :)

I opened new PR, I cannot push here, #21857
as i understand, you opt for this

/// Configuration of the authentication strategy for HTTP requests.
///
/// HTTP authentication should be used with HTTPS only, as the authentication credentials are passed as an
/// HTTP header without any additional encryption beyond what is provided by the transport itself.
#[configurable_component]
#[derive(Clone, Debug, Eq, PartialEq)]
#[serde(deny_unknown_fields, rename_all = "snake_case", tag = "strategy")]
#[configurable(metadata(docs::enum_tag_description = "The authentication strategy to use."))]
pub enum Auth {
    /// Basic authentication.
    ///
    /// The username and password are concatenated and encoded via [base64][base64].
    ///
    /// [base64]: https://en.wikipedia.org/wiki/Base64
    Basic {
        /// The basic authentication username.
        #[configurable(metadata(docs::examples = "${USERNAME}"))]
        #[configurable(metadata(docs::examples = "username"))]
        user: String,

        /// The basic authentication password.
        #[configurable(metadata(docs::examples = "${PASSWORD}"))]
        #[configurable(metadata(docs::examples = "password"))]
        password: SensitiveString,
    },

    /// Bearer authentication.
    ///
    /// The bearer token value (OAuth2, JWT, etc.) is passed as-is.
    Bearer {
        /// The bearer authentication token.
        token: SensitiveString,
    },

    /// Authentication based on OAuth 2.0 protocol.
    ///
    /// This strategy allows to dynamically acquire and use token based on provided parameters.
    /// Both standard client_credentials and mTLS extension is supported, for standard client_credentials just provide both
    /// client_id and client_secret parameters:
    ///
    /// # Example
    ///
    /// ```yaml
    /// strategy:
    ///  strategy: "o_auth2"
    ///  client_id: "client.id"
    ///  client_secret: "secret-value"
    ///  token_endpoint: "https://yourendpoint.com/oauth/token"
    /// ```
    /// In case you want to use mTLS extension [rfc8705](https://datatracker.ietf.org/doc/html/rfc8705), provide desired key and certificate,
    /// together with client_id (with no client_secret parameter).
    ///
    /// # Example
    ///
    /// ```yaml
    /// strategy:
    ///  strategy: "o_auth2"
    ///  client_id: "client.id"
    ///  token_endpoint: "https://yourendpoint.com/oauth/token"
    /// tls:
    ///  crt_path: cert.pem
    ///  key_file: key.pem
    /// ```
    OAuth2 {
        /// Token endpoint location, required for token acquisition.
        #[configurable(metadata(docs::examples = "https://auth.provider/oauth/token"))]
        token_endpoint: String,

        /// The client id.
        #[configurable(metadata(docs::examples = "client_id"))]
        client_id: String,

        /// The sensitive client secret.
        #[configurable(metadata(docs::examples = "client_secret"))]
        client_secret: Option<SensitiveString>,

        /// The grace period configuration for a bearer token.
        /// To avoid random authorization failures caused by expired token exception,
        /// we will acquire new token, some time (grace period) before current token will be expired,
        /// because of that, we will always execute request with fresh enough token.
        #[serde(default = "default_oauth2_token_grace_period")]
        #[configurable(metadata(docs::examples = 300))]
        #[configurable(metadata(docs::type_unit = "seconds"))]
        #[configurable(metadata(docs::human_name = "Grace period for bearer token."))]
        grace_period: u32,
    },
}