feat: hf inference support for gated repos

[tstadel]

Related Issues

• this allows us to use meta-llama/Meta-Llama-3-8B-Instruct via HF inference API

Proposed Changes:

• pass hf token to DefaultPromptHandler for tokenizer
• chore: add stop param to HFInferenceInvocationLayer (for usage via model_kwargs)

How did you test it?

• ran locally, e.g.

  pn = PromptNode(
      model_name_or_path="meta-llama/Meta-Llama-3-8B-Instruct", 
      api_key="TOKEN",
      use_auth_token="TOKEN",
      model_kwargs={
          "stop": ["<|end_of_text|>", "<|eot_id|>"],
      }
  )
  pn("What is the capital of Germany?")

Notes for the reviewer

Checklist

• I have read the contributors guidelines and the code of conduct
• I have updated the related issue with new insights and changes
• I added unit tests and updated the docstrings
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.
• I documented my code
• I ran pre-commit hooks and fixed any issue

[julian-risch]

Looks good to me! 👍 Instead of just token, I'd find use_auth_token more consistent with the setting in PromptNode and in HFLocalInvocationLayer https://docs.haystack.deepset.ai/v1.26-unstable/reference/invocation-layers-api#hflocalinvocationlayer__init__

[tstadel]

Looks good to me! 👍 Instead of just token, I'd find use_auth_token more consistent with the setting in PromptNode and in HFLocalInvocationLayer https://docs.haystack.deepset.ai/v1.26-unstable/reference/invocation-layers-api#hflocalinvocationlayer__init__

Ok, it's a bit weird in general how we use use_auth_token and api_key for HF throughout invocation layers. Would you then also rather take the use_auth_token param instead of api_key. Both are expected to be a HF token in this case.