Skip to content

Commit

Permalink
RST formatting updates to restgenerator page (#912)
Browse files Browse the repository at this point in the history
  • Loading branch information
leondz authored Sep 13, 2024
2 parents b5d5391 + b405897 commit 1a17cf6
Showing 1 changed file with 30 additions and 38 deletions.
68 changes: 30 additions & 38 deletions docs/source/garak.generators.rest.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,28 +4,17 @@ garak.generators.rest
Flexible connector for REST-based APIs.

Uses the following options from ``_config.plugins.generators["rest.RestGenerator"]``:
* ``uri`` - (optional) the URI of the REST endpoint; this can also be passed
in --model_name
* ``uri`` - (optional) the URI of the REST endpoint; this can also be passed in --model_name
* ``name`` - a short name for this service; defaults to the uri
* ``key_env_var`` - (optional) the name of the environment variable holding an
API key, by default REST_API_KEY
* ``req_template`` - a string where the text ``$KEY`` is replaced by env
var REST_API_KEY and ``$INPUT`` is replaced by the prompt. Default is to
just send the input text.
* ``req_template_json_object`` - (optional) the request template as a Python
object, to be serialised as a JSON string before replacements
* ``method`` - a string describing the HTTP method, to be passed to the
requests module; default "post".
* ``key_env_var`` - (optional) the name of the environment variable holding an API key, by default ``REST_API_KEY``
* ``req_template`` - a string where the text ``$KEY`` is replaced by env var ``REST_API_KEY``'s value (or whatever's specified in ``key_env_var``) and ``$INPUT`` is replaced by the prompt. Default is to just send the input text.
* ``req_template_json_object`` - (optional) the request template as a Python object, to be serialised as a JSON string before replacements
* ``method`` - a string describing the HTTP method, to be passed to the requests module; default "post".
* ``headers`` - dict describing HTTP headers to be sent with the request
* ``response_json`` - Is the response in JSON format? (bool)
* ``response_json_field`` - (optional) Which field of the response JSON
should be used as the output string? Default ``text``. Can also
be a JSONPath value, and ``response_json_field`` is used as such
if it starts with ``$``.
* ``request_timeout`` - How many seconds should we wait before timing out?
Default 20
* ``ratelimit_codes`` - Which endpoint HTTP response codes should be caught
as indicative of rate limiting and retried? ``List[int]``, default ``[429]``
* ``response_json_field`` - (optional) Which field of the response JSON should be used as the output string? Default ``text``. Can also be a JSONPath value, and ``response_json_field`` is used as such if it starts with ``$``.
* ``request_timeout`` - How many seconds should we wait before timing out? Default 20
* ``ratelimit_codes`` - Which endpoint HTTP response codes should be caught as indicative of rate limiting and retried? ``List[int]``, default ``[429]``

Templates can be either a string or a JSON-serialisable Python object.
Instance of ``$INPUT`` here are replaced with the prompt; instances of ``$KEY``
Expand All @@ -37,26 +26,25 @@ The ``$INPUT`` and ``$KEY`` placeholders can also be specified in header values.
If we want to call an endpoint where the API key is defined in the value
of an ``X-Authorization`` header, sending and receiving JSON where the prompt
and response value are both under the ``text`` key, we'd define the service
using something like: ::
using something like:

.. code-block:: JSON
:linenos:
{
"rest": {
"RestGenerator": {
"name": "example service",
"uri": "https://example.ai/llm",
"method": "post",
"headers": {
"X-Authorization": "$KEY"
},
"req_template_json_object": {
"text": "$INPUT"
},
"response_json": true,
"response_json_field": "text"
}
"RestGenerator": {
"name": "example service",
"uri": "https://example.ai/llm",
"method": "post",
"headers": {
"X-Authorization": "$KEY"
},
"req_template_json_object": {
"text": "$INPUT"
},
"response_json": true,
"response_json_field": "text"
}
}
}
Expand All @@ -65,18 +53,22 @@ the target text is not in a top level field. It is treated as a JSONPath
when ``response_json_field`` starts with ``$``.

To use this specification with garak, you can either pass the JSON as a
strong option on the command line via --generator_options, or save the
strong option on the command line via ``--generator_options``, or save the
JSON definition into a file and pass the filename to
``--generator_option_file`` / ``-G``. For example, if we save the above
JSON into ``example_service.json``, we can invoke garak as: ::
JSON into ``example_service.json``, we can invoke garak as:

garak --model_type rest -G example_service.json
.. code-block::
garak --model_type rest -G example_service.json
This will load up the default ``RestGenerator`` and use the details in the
JSON file to connect to the LLM endpoint.

If you need something more flexible, add a new module or class and inherit
from RestGenerator :)
from RestGenerator.

----

.. automodule:: garak.generators.rest
:members:
Expand Down

0 comments on commit 1a17cf6

Please sign in to comment.