-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Support for SQLite backend #130
Open
pranavmodx
wants to merge
49
commits into
acaloiaro:main
Choose a base branch
from
pranavmodx:sqlite-backend
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
49 commits
Select commit
Hold shift + click to select a range
3fc305a
change mod init name
pranavmodx c3df22a
sqlite backend work so far
pranavmodx 4375dbb
migration works
pranavmodx 9bff228
init db
pranavmodx a1ba274
enqueuing works
pranavmodx 7cfd550
notification - related work
pranavmodx ab982c6
job executes but db gets locked so not commited
pranavmodx ed54b2a
pending jobs works
pranavmodx 3800287
db lock issue fixed, enqueuing works
pranavmodx 5042ed4
comment few logs
pranavmodx 66f6154
fix enqueue locking indefinitely issue
pranavmodx 5e0dcb0
retrying works
pranavmodx e2d5d04
fix job chaining sync issue
pranavmodx 3be1f0d
code cleanup
pranavmodx 8b2be88
remove queue monitor and related fields
pranavmodx 9fdb1cb
use ctx versions of sql apis
pranavmodx 59201dd
code cleanup
pranavmodx e56993a
update and fix max_retries entries in db
pranavmodx a35e870
fix sqlite3 db path
pranavmodx 82a13d9
make queueListenerChan buffer size configurable
pranavmodx 00b8429
fix order of scanned columns in query
pranavmodx 4dea09d
remove deadline from query
pranavmodx a575976
add in progress job status
pranavmodx fd7e5cc
add dbgs
pranavmodx 330893c
add dbgs
pranavmodx ddbc6b1
add dbgs
pranavmodx 4859e6e
close db when calling shutdown
pranavmodx 66a7f1e
implement initFutureJobs
pranavmodx 0ac8178
remove dbgs
pranavmodx faaa11d
create neoq_dead_jobs table; implement moveToDeadQueue
pranavmodx aeebbf2
implement StartCron
pranavmodx 31de8d5
add basic test layout
pranavmodx 9e7265d
add tests for multiple queues, cron
pranavmodx b710137
add tests job errors, future job scheduling
pranavmodx 85e7409
change author back to original
pranavmodx f2e60ba
cleanup postgres backend logs
pranavmodx a159d2b
code cleanup; carve out updateJobToInProgress
pranavmodx b1c10a3
fix futureJobs concurrency issue
pranavmodx 7857380
extract queueListenerChan for a given queue beforehand
pranavmodx bf6680e
add support for deadline, insert deadline and max_retries in enqueueJ…
pranavmodx f29e7b3
add tests - TestMultipleProcessors, TestMultipleCronNodes
pranavmodx d59a5f4
populate id and deadline to dead queue
pranavmodx a4fe135
support json payload; remove payload2
pranavmodx e2b4608
add test for duplicate job rejection
pranavmodx 1ce892a
code cleanup
pranavmodx 0691d74
extract db path by trimming prefix
pranavmodx 91e3609
remove id in query for moveToDeadQueue
pranavmodx b818224
edit comment for initializeDB
pranavmodx e4da1db
replace TrimLeft with TrimPrefix
pranavmodx File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
CREATE TABLE neoq_jobs ( | ||
id integer primary key not null, | ||
fingerprint text not null unique, | ||
queue text, | ||
status text default "new", | ||
payload text, | ||
retries integer default 0, | ||
max_retries integer default 3, | ||
run_after datetime default CURRENT_TIMESTAMP, | ||
deadline datetime, | ||
ran_at datetime, | ||
created_at datetime default CURRENT_TIMESTAMP, | ||
error text | ||
); | ||
|
||
CREATE TABLE neoq_dead_jobs ( | ||
id integer primary key not null, | ||
fingerprint text not null, | ||
queue text, | ||
status text default "failed", | ||
payload text, | ||
retries integer, | ||
max_retries integer, | ||
run_after datetime default CURRENT_TIMESTAMP, | ||
deadline datetime, | ||
ran_at datetime, | ||
created_at datetime default CURRENT_TIMESTAMP, | ||
error text | ||
); |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like your editor/autoformatter adds an extreanous indent at the start of these
CREATE TABLE
statements (same for theneoq_jobs
table).