-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DM-43699: Port templatebot to faststream/FastAPI #67
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This code is specific for aiohttp and using aiokafka. We'll reimplement this for FastAPI/FastStream. We're keeping the "service" code around, but it'll have to be heavily modified to fit the new architecture.
- Drop aiohttp-related dependencies - Add dependency set found in squarebot backends like unfurlbot
This was referenced Sep 18, 2024
jonathansick
force-pushed
the
tickets/DM-43699
branch
2 times, most recently
from
September 30, 2024 21:01
e7d15c7
to
1fd109c
Compare
This basic structure is ported from our unfurlbot backend.
Based on the SQuaRE FastAPI template.
In Phalanx we use Helm charts
Adopt shared ruff configurations from the SQuaRE FastAPI template.
Place the repo manager under "storage" since it's managing files for us.
We'll reference and possibly re-add this code during the migration to Faststream; but for the moment we want to quickly put the code base in a good start to start running with basic functionality.
This serves as a smoketest for start up the app.
This will allow us to trigger testcontainers via nox for running Kafka during tests.
We're not using Redis at this time in templatebot
This client can send POST and GET requests. We're also creating Pydantic models for the requests, as well as a starting a generic Block Kit Pydantic model package for composing BlockKit messages.
This service will handle Kafka messages from Squarebot and then trigger the internal services for performing template operations.
We want templatebot to just respond when talked to; either through an IM or when mentioned.
Each subscriber function needs a unique consumer group name.
The main hangup on faststream 0.5 was an apparent type issue with the aiokafka KafkaMessage record. We'll ignore that typing issue to see if its read. The other issue is a forward type reference issue with pydantic; this is solved by taking the future annotations out of the kafka handler module.
And refactor existing models to make it easier to continue building up the Block Kit models.
We can use the native Pydantic max_length validation for the plain text and markdown text objects by implementing a __len__ dunder method on the text objects. To do this, create a base class for text objects.
This creates a global singleton instance of the RepoManager. Added TEMPLATEBOT_TEMPLATE_REPO_URL and TEMPLATEBOT_TEMPLATE_CACHE_DIR environment variable configurations.
This service specifically handles functionality on a template repo as a whole. Initially it handles the task of presenting users with a Slack static menu selection of available templates. It will be called from the SlackMessage service that handles slack messages from users.
So the pattern will a top-layer of services that can parse and route the Slack payloads of certain types, and a lower layer of services that do the domain specific functionality.
This is a potential fix for the AIOKafkaUnclosedConsumer issue with Faststream<0.6.
This modal is derived from the base SlackModalView with a customized constructor classmethod that creates the modal based on a TemplateKit template. We're using the original templatebot recipe for creating fields based on the templatekit.yaml definition for a template. Modals provide a lot more types of inputs and for additional content in modals, so we can improve this over time. The private_metadata send with the modal is a pydantic model as well. It includes all the metadata needed to for templatebot to pick up the interaction and create a project from the template. These tests of the modal construction use a mock template repository. We're embedding it directly in this repo so we don't have a live dependency on a real templates repo like https://github.com/lsst/templates
This service carries out functionality on individual templates, like creating the modal for a user to specify template variables and for rendering out the template. This plums the block action triggered by a template selection menu through the TemplateService and into the TemplateVariablesModal to create the modal.
Handle the submission and pass the action off to TemplateService.
This logic is adapted from the orginal codebase. Perhaps the code could be relocated into the TemplateVariablesModal class? Or into a Template domain model?
The AuthorDb class comes from lsst-templatebot-aide, originally, and provies an interface to the authordb.yaml file in lsst-texmf.
If a template has a variable named author_id we automatically treat it as an ID in authordb.yaml, and expand it into a set of standardized variables about the author.
- Add configuration environment variables to configure a github app client. These are the same credentials that the Squarebot app uses. - Add a custom GitHubAppClientFactory that allows you to get the installation_id for an organization. This is handy for the squarebot use case of creating repositories. This will be upstreamed into Safir. - Add the GitHub app client factory to the factory.
This ports the original logic from lsst-sqre/lsst-templatebot-aide, but now uses the GitHubAppClientFactory.
Different templates have different methods for determining the GitHub repo name and organization. Initially we do this with a series of cases, but we might want to refactor this or make it more systematic to better allow new templates to reliably set their intended github repo without needing to modify this code.
- Use a new GitHubRepo storage class as an adapter to GitHub APIs that operate on a repository. - Call from the TemplateService when creating a project
- Add an LtdClient that can register projects on LSST the Docs - Add TEMPLATEBOT_LTD_USERNAME and TEMPLATEBOT_LTD_PASSWORD configurations - Integrate LTD registration with project creation
We're having some difficulty getting the GitHub App's email address right. It should be <appid>+<name>[bot]@... but this isn't working quite right.
I think that we can upgrade to cookiecutter>2.2 if we install jinja-time separately. Coordinate this with templatekit and the templates repo.
Mypy needed this after a rebase. Not entirely sure why, or why it wasn't necessary before.
jonathansick
force-pushed
the
tickets/DM-43699
branch
from
October 8, 2024 20:33
a27c484
to
0360982
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR begins the process of porting templatebot into a faststream/FastAPI app to work with the modern Squarebot.