Multiple Streams Auth Repeated

[freimer]

Current PR #4 requires multiple authentication information, per stream. It is often the case that the authentication information is the same for streams that are grouped together in a project/pipeline, as they often access the same service. It would be nice to have some flexibility to specify matches of various types for which authentication information is used for which streams. I propose the following scheme:

streams_auth:
  - match_type: exact
    name: stream1234
    headers:
      "X-apiKey": auth1234
  - match_type: prefix
    name: stream_abc_
    headers:
      "X-apiKey": auth_abc1234
  - match_type: suffix
    name: _accounts_
    headers:
      "X-apiKey": auth_acc1234
  - match_type: regex
    name: "^stream_[a-z]+_[0-9]+"
    headers:
      "X-apiKey": auth_re1234

The order of the auth info would matter; the first entry that matched is used, and if none match no authentication is assumed. This can be expanded later to include other authentication types, by adding an auth_type in addition or instead of headers and other appropriate keys.
Headers should also be allowed in both the main streams array, as well as streams_auth. Non-sensitive headers should be specified in streams, and sensitive headers, such as Authorization or X-APIKey, should be specified in streams_auth.

[jlloyd-widen]

@freimer I haven't accept the PR related to this issue because the way the tap is designed (as of right now) requires the same base url for all the streams in the tap. This is design assumes:

1. That you will only have 1 tap per api
2. That each stream operates using the same authorization scheme (because they are just distinct paths of the same base api).

Under this design if you want to add a stream from a different api you have to create a completely different tap with the new configuration (new base_url, new auth, etc.).

Having said that, this is a poor design if there are API's out in the wild that use different authorization mechanisms depending on which path you are trying to access. I am not aware of any. However, if this issue is just a round-a-bout way of wanting a design that allows you to access more than one API within the same tap, then that requires a broader discussion because that is not really a design I'm in favor of, but happy to hear use cases and continue the discussion.