Sensitive content report moderation workflow

[obulat]

Start Date	Project Lead	Actual Ship Date
2023-02-13	@sarayourfriend	TBD

Description

This project creates the sensitive content reporting flow guidelines.

Documents

• Project Proposal: Sensitive content report moderation #1176
• Implementation Plan: Copy updates mature -> sensitive #1965
• Implementation Plan: Django admin access control and tool improvements #1966
• Implementation Plan: Bulk moderation actions #1967
• Implementation Plan: Computer vision metadata for content reports #1968
• Implementation Plan: Fine-grained API response cache management #1969
• Implementation Plan: Moderation queue metrification #1970

Milestones

• Copy changes: https://github.com/WordPress/openverse/milestone/18
• Implementation plan drafting: https://github.com/WordPress/openverse/milestone/8
• Django admin access control and basic functionality changes: https://github.com/WordPress/openverse/milestone/22
• Bulk moderation actions: https://github.com/WordPress/openverse/milestone/24
• Metrification: https://github.com/WordPress/openverse/milestone/27

Prior Art

[krysal]

Required work to enable the DAG for reported media pending review created at WordPress/openverse-catalog#513. I noticed it hadn't run yet.

[sarayourfriend]

@AetherUnbound has started on the first implementation plan for this project. We probably need to get the next implementation plan also started. The project proposal shows that the API cache management and the initial Django access control implementation plans can be started at any time. If any @WordPress/openverse-maintainers do not currently have an implementation plan they are working on or can take on a new one, please volunteer for one of the ones for this project.

I will start working on #1969 next week.

[openverse-bot]

Hi @sarayourfriend, this project has not received an update comment in 14 days. Please leave an update comment as soon as you can. See the documentation on project updates for more information.

[sarayourfriend]

#2126 is pending review from the requested reviewers, @dhruvkb and @krysal. @zackkrida given the delay, do you think it's appropriate for my review of the implementation plan to serve as one of the two approvals for go ahead? @dhruvkb and @krysal, can y'all share an ETA of when you'll be able to review the implementation plan? It's been in the decision round much longer than the expected time span.

I was going to start working on the #1969 IP, but that hasn't happened due to recent incidents and high priority retrospective items taking up lots of time followed by MSR. I hope to be able to start that implementation plan next week.

The recent Redis memory consumption issue has brought to attention that the fine-grained cache IP will need to consider Redis memory usage.

[sarayourfriend]

After starting to work on the fine-grained cache implementation plan, I opened a discussion about our approach to caching generally: #2453

In that discussion we've decided to explore significantly reducing our cache TTL, potentially low enough such that needing fine-grained cache invalidation on the search endpoint would no longer be necessary. Therefore, I will stall work on the implementation plan to further investigate how changes to the cache will effect performance, especially Elasticsearch and Postgres resource consumption. This is possible now that the thumbnails service is live and we have confirmed that API instability was due almost entirely to long-running thumbnails requests.

Other implementation plans may move forward, such as the Django admin access control and baseline changes, which would unblock several other implementation plans.

[openverse-bot]

Hi @sarayourfriend, this project has not received an update comment in 14 days. Please leave an update comment as soon as you can. See the documentation on project updates for more information.

[sarayourfriend]

Apologies for missing an update on this.

@dhruvkb's got a very cool PR for the "locking" of reports, so that moderators know when someone is reviewing a work or not: #4374

I also opened a related issue, that will sit outside the project, for us to consider different ways of logging into Django admin that could make user management and permissions a bit easier: #4342. That is out of scope for this project, and we do not realistically need, nor would benefit from all that complexity. The plan right now does not involved a large number of moderators, nor would they change often, so we don't benefit from making user management overall easier, if the underlying implementation is complicated.

I intended to start working on some issues for this project this week, but got pulled away to things related to an API incident. Hopefully I'll wrap those up early next week and be able to work on some of the issues from this milestone: https://github.com/WordPress/openverse/milestone/22

Krystle's picked up the computer vision labelling implementation plan. Looking forward to seeing that one, especially with the ongoing work in #431 in the works.

[sarayourfriend]

To reduce scope and help make more time/room in our schedule for projects important to search, we're cutting the computer vision and metrification aspects of this project. They were always considered "nice to have", and while I'm disappointed @dhruvkb's IP for the metrification part won't be implemented right away, the work is planned and available for us to work on later. It is less critical now that we know who will be doing the moderation. The visibility we'll have with the basic features after we've implemented bulk moderation will be more than sufficient to ensure reports are reviewed in a timely and consistent manner, as monitored by the moderators themselves (following self-accountability, if it needs to be more explicit). @zackkrida do you think we should close the issues in the milestone, or leave them as is?

The computer vision aspect would also be nice to have, and a continuation of #4040, but ultimately not necessary for MVP.

[zackkrida]

@zackkrida do you think we should close the issues in the milestone, or leave them as is?

I've just peeked at the milestones for the project. I closed the IP milestone, and the two remaining milestones look to contain the correct issues.

Django admin access control and basic functionality changes: WordPress/openverse/milestone/22
Bulk moderation actions: WordPress/openverse/milestone/24

Perhaps you've already made changes, @sarayourfriend? In any case, it all looks good to me.

[sarayourfriend]

I meant specifically about the metrification issues. There are only three, and they're arguably unnecessary with the current plan of who will do the moderation. I'll close them as won't do for now, and we can revisit our needs in the future once we actually need it.

[zackkrida]

Ah thanks! I saw that those weren't part of the project's milestones so figured they'd be acceptable to keep as low priority for a rainy day, but you're totally right given our set of moderators.

[openverse-bot]

Hi @sarayourfriend, this project has not received an update comment in 14 days. Please leave an update comment as soon as you can. See the documentation on project updates for more information.

[openverse-bot]

Hi @sarayourfriend, this project has not received an update comment in 14 days. Please leave an update comment as soon as you can. See the documentation on project updates for more information.

[sarayourfriend]

Oops! I thought I'd left an update here two weeks ago! Apologies.

A single issue remains in the initial milestone, #3640, assigned to @dhruvkb. That one does not block work on the bulk moderation issues, so we could start on those issues as soon as we like.

The backfill had one small issue with foreign keys, fixed in #4512. We ran it in production, and have now dropped the status column in production. The new model for report decision tracking is fully operational!

[openverse-bot]

Hi @sarayourfriend, this project has not received an update comment in 14 days. Please leave an update comment as soon as you can. See the documentation on project updates for more information.

[zackkrida]

@dhruvkb do you mind contributing this update while @sarayourfriend is AFK?

[dhruvkb]

Issue #3640 from milestone https://github.com/WordPress/openverse/milestone/22 has an associated PR #4603, that has been reviewed by @sarayourfriend and the changes that Sara requested have been addressed. Since Sara is AFK, I have requested another reviewer to be picked automatically from the WordPress/openverse-api team.

Also I have started working on issue #3840 from milestone https://github.com/WordPress/openverse/milestone/24 and a PR for that will be up very soon.

[openverse-bot]

Hi @sarayourfriend, this project has not received an update comment in 14 days. Please leave an update comment as soon as you can. See the documentation on project updates for more information.

[dhruvkb]

All milestones for this project have now been completed, so the project has been moved to the "🚢 Shipped" column.

[zackkrida]

@WordPress/openverse-maintainers this project will be marked a success after the tooling has actually been used in a two-week trial by the sub-group of maintainers who will perform moderation.

Feedback from this trial may inform further improvements or changes.

[sarayourfriend]

It would be great to split feedback between bugs with the functionality implemented for this project and with iterative improvements and additional features, just so we don't accidentally expand the scope of this project beyond its original goals for the foundational functionality.

[openverse-bot]

Hi @sarayourfriend, this project has not received an update comment in 14 days. Please leave an update comment as soon as you can. See the documentation on project updates for more information.

[sarayourfriend]

@zackkrida do you know when the two-week trial will start, and if so, can you share the details here? If not, how can I help get that moving forward?

Nothing to add in the way of an update beyond that, though.

[zackkrida]

@sarayourfriend given upcoming team AFK and travel, I think the trial should take place from October 14-25th.

Additionally, I think we should stop sending reminder pings for shipped projects:
#4912

[sarayourfriend]

See #383 (comment)

Also, I opened a PR to stop these pings on shipped projects as suggested by Zack: #4993.