[Testing] How to assert on the requests served

[bmbouter]

We use proxy.py in our test suite, and it's really great.

While we do have requests flowing through the proxy, how can I have a record of the requests it serves so my test code can assert on them?

If there isn't some easy/official way to support this, perhaps I could be pointed to what attributes in the code I could pytest.monkeypatch onto to make my own record as calls to that attribute occur. What would you recommend?

We currently use it simply like this:

    proxypy_args = [
        "--num-workers",
        "4",
        "--hostname",
        host,
        "--port",
        str(port),
    ]

    with proxy.Proxy(input_args=proxypy_args):
        yield proxy_data

[abhinavsingh]

@bmbouter Here is one option that came to my mind.

CacheResponsesPlugin

One option is to use CacheResponsesPlugin. By default it will write upstream responses in a file per request. Recently, --cache-requests flag was also added. When used, plugin will also dump request packets in the same file written per request.

Next, you will need a way to access this cached file path. Luckily, it's available as cache_file_path within access log context dictionary. Just for reference, this log context is populated from here https://github.com/abhinavsingh/proxy.py/blob/develop/proxy/plugin/cache/cache_responses.py#L54

Finally, to make this all work in your test environment

1. Write a plugin, enable your plugin by adding ["--plugin", "....."].
2. You will also need to enable CacheResponsesPlugin plugin. Remember to enable it before your plugin in the flag order.
3. Your plugin only needs to provide on_access_log implementation
4. You implementation can return back the context as-is in the end (recommended). Because by doing so, we'll let default access log handler to invoke. Here (in your plugin) we just wanted to get access to the log context.
5. Now write your companion test within on_access_log implementation:
   • Read cache_file_path
   • Make sure expected HTTP request/response packet exists
6. Recently, --cache-by-content-type flag was also added. It's a WIP. But when enabled, it will extract the content out of the cache file and dump in a separate directory. E.g. imagine if the response is of type application/json, with this flag enabled, you will also see a upstream.domain-x-y-z.json file dumped on disk. Wanted to just put this feature out, may be all you want is to assert on that json file content :D

Monkey Patch

You can of-course monkey patch too. But:

1. That will be equivalent to testing proxy.py (more or less).
2. And frustratingly, your patches might keep breaking as proxy.py core refactors. Or you are at-least forced to stick to a single proxy.py version.

Let me know if it helps.

[abhinavsingh]

If your app is making HTTPS requests, you will also have to enable TLS interception. Remember that proxy.py cannot inspect and extract contents unless it can see within HTTPS traffic. Wanted to add this info, as chances are your app is making https:// requests.

[abhinavsingh]

I also realized you asked for assert on requests. If request data is all you want, your plugin alone can do it (no need of cache responses plugin). Your plugin can do it by implementing handle_client_request. See the proxy plugin base class to understand when this method is called. Also see if other lifecycle callback makes sense or are of use for your use-case. Best.

[bmbouter]

Thanks so much for this clear and great reply. I'll give this a try!