Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🎉 Added incremental indexer command #558

Merged
merged 9 commits into from
Nov 8, 2024
Merged

Conversation

pushrbx
Copy link
Collaborator

@pushrbx pushrbx commented Nov 4, 2024

Currently none of the indexing artisan commands can do incremental indexing. So far we relied on the season and schedule indexer (or full indexing in reverse order), but in case of self-hosted setups that's insufficient, the new entries from MAL never get imported. Not to mention manga entries.
This PR adds a new artisan command: indexer:incremental. This command will compare the latest version of MAL ids from the mal_id_cache github repository and compares them with the downloaded ids from the previous run. If no ids found from the previous run, a full indexing session is started.

Usage:

php artisan indexer:incremental anime manga

The first argument is an array, and it's required. You can also just run it with one media type to index:

php artisan indexer:incremental anime

You can also resume with --resume flag if you cancelled it. The command handles the SIGTERM singal as well to gracefully stop.

@pushrbx pushrbx requested a review from a team as a code owner November 4, 2024 21:18
Copy link

codecov bot commented Nov 4, 2024

Codecov Report

Attention: Patch coverage is 0% with 101 lines in your changes missing coverage. Please review.

Project coverage is 56.58%. Comparing base (744bd69) to head (3ec81f2).
Report is 12 commits behind head on master.

Files with missing lines Patch % Lines
...pp/Console/Commands/Indexer/IncrementalIndexer.php 0.00% 101 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master     #558      +/-   ##
============================================
- Coverage     57.86%   56.58%   -1.28%     
- Complexity     1361     1390      +29     
============================================
  Files           339      340       +1     
  Lines          5273     5625     +352     
============================================
+ Hits           3051     3183     +132     
- Misses         2222     2442     +220     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

COMMANDS.MD Show resolved Hide resolved
COMMANDS.MD Show resolved Hide resolved
app/Console/Commands/Indexer/IncrementalIndexer.php Outdated Show resolved Hide resolved
app/Console/Commands/Indexer/IncrementalIndexer.php Outdated Show resolved Hide resolved
app/Console/Commands/Indexer/IncrementalIndexer.php Outdated Show resolved Hide resolved
@pushrbx pushrbx merged commit 52c030d into master Nov 8, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants