Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-43699: Port templatebot to faststream/FastAPI #67

Merged
merged 67 commits into from
Oct 8, 2024

Conversation

jonathansick
Copy link
Member

This PR begins the process of porting templatebot into a faststream/FastAPI app to work with the modern Squarebot.

This code is specific for aiohttp and using aiokafka. We'll reimplement
this for FastAPI/FastStream. We're keeping the "service" code around,
but it'll have to be heavily modified to fit the new architecture.
- Drop aiohttp-related dependencies
- Add dependency set found in squarebot backends like unfurlbot
This basic structure is ported from our unfurlbot backend.
Based on the SQuaRE FastAPI template.
In Phalanx we use Helm charts
Adopt shared ruff configurations from the SQuaRE FastAPI template.
Place the repo manager under "storage" since it's managing files for us.
We'll reference and possibly re-add this code during the migration to
Faststream; but for the moment we want to quickly put the code base in a
good start to start running with basic functionality.
This serves as a smoketest for start up the app.
This will allow us to trigger testcontainers via nox for running Kafka
during tests.
We're not using Redis at this time in templatebot
This client can send POST and GET requests. We're also creating Pydantic
models for the requests, as well as a starting a generic Block Kit
Pydantic model package for composing BlockKit messages.
This service will handle Kafka messages from Squarebot and then trigger
the internal services for performing template operations.
We want templatebot to just respond when talked to; either through an IM
or when mentioned.
Each subscriber function needs a unique consumer group name.
The main hangup on faststream 0.5 was an apparent type issue with the
aiokafka KafkaMessage record. We'll ignore that typing issue to see if
its read.

The other issue is a forward type reference issue with pydantic; this is
solved by taking the future annotations out of the kafka handler module.
And refactor existing models to make it easier to continue building up
the Block Kit models.
We can use the native Pydantic max_length validation for the plain text
and markdown text objects by implementing a __len__ dunder method on the
text objects. To do this, create a base class for text objects.
This creates a global singleton instance of the RepoManager. Added
TEMPLATEBOT_TEMPLATE_REPO_URL and TEMPLATEBOT_TEMPLATE_CACHE_DIR
environment variable configurations.
This service specifically handles functionality on a template repo as a
whole. Initially it handles the task of presenting users with a Slack
static menu selection of available templates. It will be called from the
SlackMessage service that handles slack messages from users.
So the pattern will a top-layer of services that can parse and route the
Slack payloads of certain types, and a lower layer of services that do
the domain specific functionality.
This is a potential fix for the AIOKafkaUnclosedConsumer issue with
Faststream<0.6.
This modal is derived from the base SlackModalView with a customized
constructor classmethod that creates the modal based on a TemplateKit
template. We're using the original templatebot recipe for creating
fields based on the templatekit.yaml definition for a template. Modals
provide a lot more types of inputs and for additional content in modals,
so we can improve this over time.

The private_metadata send with the modal is a pydantic model as well. It
includes all the metadata needed to for templatebot to pick up the
interaction and create a project from the template.

These tests of the modal construction use a mock template repository.
We're embedding it directly in this repo so we don't have a live
dependency on a real templates repo like
https://github.com/lsst/templates
This service carries out functionality on individual templates, like
creating the modal for a user to specify template variables and for
rendering out the template.

This plums the block action triggered by a template selection menu
through the TemplateService and into the TemplateVariablesModal to
create the modal.
Handle the submission and pass the action off to TemplateService.
This logic is adapted from the orginal codebase. Perhaps the code could
be relocated into the TemplateVariablesModal class? Or into a Template
domain model?
The AuthorDb class comes from lsst-templatebot-aide, originally, and
provies an interface to the authordb.yaml file in lsst-texmf.
If a template has a variable named author_id we automatically treat it
as an ID in authordb.yaml, and expand it into a set of standardized
variables about the author.
- Add configuration environment variables to configure a github app
  client. These are the same credentials that the Squarebot app uses.

- Add a custom GitHubAppClientFactory that allows you to get the
  installation_id for an organization. This is handy for the
  squarebot use case of creating repositories. This will be upstreamed
  into Safir.

- Add the GitHub app client factory to the factory.
This ports the original logic from lsst-sqre/lsst-templatebot-aide, but
now uses the GitHubAppClientFactory.
Different templates have different methods for determining the GitHub
repo name and organization. Initially we do this with a series of cases,
but we might want to refactor this or make it more systematic to better
allow new templates to reliably set their intended github repo without
needing to modify this code.
- Use a new GitHubRepo storage class as an adapter to GitHub APIs that
  operate on a repository.
- Call from the TemplateService when creating a project
- Add an LtdClient that can register projects on LSST the Docs
- Add TEMPLATEBOT_LTD_USERNAME and TEMPLATEBOT_LTD_PASSWORD
  configurations
- Integrate LTD registration with project creation
We're having some difficulty getting the GitHub App's email address
right. It should be <appid>+<name>[bot]@... but this isn't working quite
right.
I think that we can upgrade to cookiecutter>2.2 if we install jinja-time
separately. Coordinate this with templatekit and the templates repo.
Mypy needed this after a rebase. Not entirely sure why, or why it wasn't
necessary before.
@jonathansick jonathansick marked this pull request as ready for review October 8, 2024 20:38
@jonathansick jonathansick merged commit a02b3ec into main Oct 8, 2024
3 checks passed
@jonathansick jonathansick deleted the tickets/DM-43699 branch October 8, 2024 20:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant