Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fetch and cache slack channels #338

Merged
merged 1 commit into from
Jun 20, 2019
Merged

Fetch and cache slack channels #338

merged 1 commit into from
Jun 20, 2019

Conversation

ovv
Copy link
Member

@ovv ovv commented Jun 20, 2019

Relevant Issue & Description

I like the idea behind #186 and #40 . This is the first step towards thoses

@ovv ovv requested a review from mattrasband June 20, 2019 15:52
@ovv ovv force-pushed the fetch-slack-channels branch 2 times, most recently from 65a0a6a to ccad1c6 Compare June 20, 2019 15:55
logger.exception("Error refreshing slack user cache")
raise
return
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't raise here because nothing else will catch it and asyncio will complains that we have an unhandled exception

Copy link
Contributor

@mattrasband mattrasband left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DRY up the slack pagination (or if you prefer I can do that later on tonight)

channels = []
while True:
params = {}
async with session.get(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably add a pyslackersweb/util/slack.py utility file that does the paging and just yields the pages to us to DRY this up a little.

At some point I'd like to pull the actual HTTP logic out of the tasks, this may be a good time to do it for the slack stuff.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't DRY it more than that.
I took a quick look at the slack client v2 but it would still requires a bunch of code around compared to slack-sansio

@mattrasband
Copy link
Contributor

We can pursue #186 - we just need to answer some questions of "what defines popularity", as showing #annoucements would be silly since it's mandatory whereas #django may be smaller but highly active.

@ovv ovv force-pushed the fetch-slack-channels branch 2 times, most recently from 28ae9e1 to 2e3d281 Compare June 20, 2019 16:55
Copy link
Contributor

@mattrasband mattrasband left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor thing to relocate the slack_client to an independent signal

@@ -29,4 +34,5 @@
async def client_session(app: web.Application) -> None:
async with ClientSession(raise_for_status=True) as session:
app["client_session"] = session
app["slack_client"] = SlackAPI(token=app["slack_token"], session=session)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be an independent context (or setup signal). It depends on this one, but shouldn't expand what the client_session cleanup_ctx does.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was afraid you where going to say that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't sure on the ordering of processing, but it looks like cleanup_ctx always is applied first on setup and cleanup:

https://github.com/aio-libs/aiohttp/blob/c8cbbcc3b979e13305c187929f4708c528c19084/aiohttp/web_app.py#L117-L119

@ovv ovv force-pushed the fetch-slack-channels branch 2 times, most recently from b7836d6 to 9fcc1d0 Compare June 20, 2019 17:11
@ovv ovv force-pushed the fetch-slack-channels branch from 9fcc1d0 to 250fad3 Compare June 20, 2019 17:11
@mattrasband mattrasband merged commit 59f7cfe into master Jun 20, 2019
@mattrasband mattrasband deleted the fetch-slack-channels branch June 20, 2019 17:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants