-
-
Notifications
You must be signed in to change notification settings - Fork 57
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
45 changed files
with
3,316 additions
and
1,886 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,7 +1,7 @@ | ||
--- | ||
order: 1 | ||
dir: | ||
order: 3 | ||
order: 4 | ||
--- | ||
|
||
# Available components | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,6 +1,6 @@ | ||
--- | ||
dir: | ||
order: 2 | ||
order: 3 | ||
--- | ||
|
||
# Extending taskiq | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
--- | ||
order: 1 | ||
dir: | ||
order: 2 | ||
--- | ||
|
||
# Framework integrations | ||
|
||
Taskiq is meant to be simple and adaptive. That's why we try to add different integrations to make development with taskiq and your favorite framework easy and fun! | ||
|
||
Integrations with frameworks add two things: | ||
1. Startup and Shutdown events; | ||
1. Dependencies to use in your handler. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,126 @@ | ||
--- | ||
order: 2 | ||
--- | ||
|
||
# Taskiq + AioHTTP | ||
|
||
AioHTTP is a framework for building robust applications. We created several libraries to make the experience with AioHTTP even better. | ||
|
||
# Dependency inecjeciton for AioHTTP | ||
|
||
We created a library [aiohttp-deps](https://pypi.org/project/aiohttp-deps/) to add FastAPI-like dependency injection in AioHTTP. | ||
|
||
To install it, simply run: | ||
|
||
```python | ||
pip install "aiohttp-deps" | ||
``` | ||
|
||
After the installation, please add startup event to your application to initialize dependencies context. | ||
|
||
```python | ||
from aiohttp import web | ||
import aiohttp_deps | ||
|
||
|
||
app = web.Application() | ||
|
||
# This startup event makes all the magic happen. | ||
# It parses current handlers and create dependency graphs for them. | ||
app.on_startup.append(aiohttp_deps.init) | ||
|
||
web.run_app(app) | ||
``` | ||
|
||
You can read more about dependency injection and available dependencies in the project's [README.md](https://github.com/taskiq-python/aiohttp-deps). | ||
|
||
|
||
|
||
## Adding taskiq integration | ||
|
||
We highly recommend using aiohttp with aiohttp-deps because it allows us to reuse the same dependencies for your handlers and tasks. First of all, you should install the [taskiq-aiohttp](https://pypi.org/project/taskiq-aiohttp/) library. | ||
|
||
```python | ||
pip install "taskiq-aiohttp" | ||
``` | ||
|
||
After the installation is complete, add an initialization function call to your broker's main file so it becomes something like this: | ||
|
||
```python | ||
import taskiq_aiohttp | ||
|
||
broker = MyBroker() | ||
|
||
# The second argument is a path to web.Application variable. | ||
# Also you can provide here a factory function that takes no | ||
# arguments and returns an application. This function can be async. | ||
taskiq_aiohttp.init(broker, "my_project.main:app") | ||
``` | ||
|
||
From this point, you'll be able to reuse the same dependencies as with `aiohttp-deps`. | ||
Let's take a look at this function: | ||
|
||
```python | ||
from aiohttp import web | ||
from taskiq import TaskiqDepends | ||
from my_project.tkq import broker | ||
|
||
@broker.task | ||
async def my_task(app: web.Application = TaskiqDepends()): | ||
... | ||
|
||
``` | ||
|
||
In this example, we depend on the current application. We can use its state in a current task or any other dependency. We can take db_pool from your application's state, which is the same pool, as the one you've created on AiohTTP's startup. | ||
But this application is only a mock of your application. It has correct types and all your variables that you filled on startup, but it doesn't handle any request. | ||
This integration adds two main dependencies: | ||
* web.Application - current application. | ||
* web.Request - mocked request. This request only exists to be able to use the same dependencies. | ||
|
||
You can find more detailed examples in the [examples repo](https://github.com/taskiq-python/examples). | ||
|
||
## Testing | ||
|
||
Writing tests for AioHTTP with taskiq is as easy as writing tests for the aiohttp application. The only difference is that, if you want to use InMemoryBroker, then you need to add context for dependency injection. It's easier to call `populate_context` when creating a `test_client` fixture. | ||
|
||
```python | ||
import taskiq_aiohttp | ||
|
||
@pytest.fixture | ||
async def test_client( | ||
app: web.Application, | ||
) -> AsyncGenerator[TestClient, None]: | ||
""" | ||
Create a test client. | ||
This function creates a TestServer | ||
and a test client for the application. | ||
Also this fixture populates context | ||
with needed variables. | ||
:param app: current application. | ||
:yield: ready to use client. | ||
""" | ||
loop = asyncio.get_running_loop() | ||
server = TestServer(app) | ||
client = TestClient(server, loop=loop) | ||
|
||
await client.start_server() | ||
|
||
# This is important part. | ||
# Since InMemoryBroker doesn't | ||
# run as a worker process, we have to populate | ||
# broker's context by hand. | ||
taskiq_aiohttp.populate_context( | ||
broker=broker, | ||
server=server.runner.server, | ||
app=app, | ||
loop=loop, | ||
) | ||
|
||
yield client | ||
|
||
broker.custom_dependency_context = {} | ||
await client.close() | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,118 @@ | ||
--- | ||
order: 1 | ||
--- | ||
|
||
# Taskiq + FastAPI | ||
|
||
FastAPI is a highly popular async web framework in Python. It has gained its popularity because of two things: | ||
1. It's easy to use; | ||
2. Cool dependency injection. | ||
|
||
In taskiq, we try to make our libraries easy to use, and We have a dependency injection too. So we have created the library "[taskiq-fastapi](https://github.com/taskiq-python/taskiq-fastapi)" to make integration with FastAPI as smooth as possible. | ||
|
||
Let's see what we got here. In this library, we provide users with only one public function called `init`. It takes a broker and a string path (as in uvicorn) to the fastapi application (or factory function). People should call this function in their main broker file. | ||
|
||
```python | ||
from taskiq import ZeroMQBroker | ||
import taskiq_fastapi | ||
|
||
broker = ZeroMQBroker() | ||
|
||
taskiq_fastapi.init(broker, "my_package.application:app") | ||
|
||
``` | ||
|
||
There are two rules to make everything work as you expect: | ||
1. Add `TaskiqDepends` as a default value for every parameter with `Request` or `HTTPConnection` types in base dependencies. | ||
2. Use only `TaskiqDepends` in tasks. | ||
|
||
|
||
::: tip Cool and important note! | ||
|
||
The `Request` or `HTTPConnection` that you'll get injected in your task is not the same request or connection you have had in your handler when you were sending the task! | ||
|
||
::: | ||
|
||
Many fastapi dependency functions depend on `fastapi.Request`. We provide a mocked request to such dependencies. But taskiq cannot resolve dependencies until you explicitly specify that this parameter must be injected. | ||
|
||
As an example. If you previously had a dependency like this: | ||
|
||
```python | ||
from fastapi import Request | ||
from typing import Any | ||
|
||
def get_redis_pool(request: Request) -> Any: | ||
return request.app.state.redis_pool | ||
|
||
``` | ||
|
||
To make it resolvable in taskiq, people should add `TaskiqDepends` as a default value for each parameter. Like this: | ||
```python | ||
from fastapi import Request | ||
from taskiq import TaskiqDepends | ||
|
||
|
||
async def get_redis_pool(request: Request = TaskiqDepends()): | ||
return request.app.state.redis_pool | ||
|
||
``` | ||
|
||
|
||
Also you want to call startup of your brokers somewhere. | ||
|
||
```python | ||
from fastapi import FastAPI | ||
from your_project.taskiq import broker | ||
|
||
app = FastAPI() | ||
|
||
|
||
@app.on_event("startup") | ||
async def app_startup(): | ||
if not broker.is_worker_process: | ||
await broker.startup() | ||
|
||
|
||
@app.on_event("shutdown") | ||
async def app_shutdown(): | ||
if not broker.is_worker_process: | ||
await broker.shutdown() | ||
|
||
``` | ||
|
||
And that's it. Now you can use your taskiq tasks with functions and classes that depend on FastAPI dependenices. You can find bigger examples in the [examples repo](https://github.com/taskiq-python/examples/). | ||
|
||
|
||
## Testing | ||
|
||
Testing is no different from general testing advice from articles about [testing](../guide/testing-taskiq.md). But if you use `InMemoryBroker` in your tests, you need to provide it with a custom dependency context because it doesn't run as a worker process. | ||
|
||
Let's imagine that you have a fixture of your application. It returns a new fastapi application to use in tests. | ||
```python | ||
|
||
@pytest.fixture | ||
def fastapi_app() -> FastAPI: | ||
return get_app() | ||
|
||
``` | ||
|
||
Right after this fixture, we define another one. | ||
|
||
```python | ||
import taskiq_fastapi | ||
|
||
|
||
@pytest.fixture(autouse=True) | ||
def init_taskiq_deps(fastapi_app: FastAPI): | ||
# This is important part. Here we add dependency context, | ||
# this thing helps in resolving dependencies for tasks | ||
# for inmemory broker. | ||
taskiq_fastapi.populate_dependency_context(broker, fastapi_app) | ||
|
||
yield | ||
|
||
broker.custom_dependency_context = {} | ||
|
||
``` | ||
|
||
This fixture has autouse flag, which means it would run on every test automatically. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.