Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: refactor queue module #314

Open
wants to merge 103 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
103 commits
Select commit Hold shift + click to select a range
043f128
refactor: queue module
devhaozi Oct 7, 2023
22621b8
chore: update mocks
devhaozi Oct 7, 2023
33a9c54
feat: update
devhaozi Oct 8, 2023
28b9344
Merge remote-tracking branch 'origin/haozi/queue' into haozi/queue
devhaozi Oct 8, 2023
ca16d6a
chore: update mocks
devhaozi Oct 8, 2023
745bc27
fix: type
devhaozi Oct 8, 2023
071cccf
chore: update mocks
devhaozi Oct 8, 2023
150f302
feat: update database job
devhaozi Oct 9, 2023
d6e6599
Merge remote-tracking branch 'origin/haozi/queue' into haozi/queue
devhaozi Oct 9, 2023
45598dc
chore: update mocks
devhaozi Oct 9, 2023
e5d5b6c
feat: update facades
devhaozi Oct 9, 2023
ea6ac5a
Merge remote-tracking branch 'origin/haozi/queue' into haozi/queue
devhaozi Oct 9, 2023
cb3214c
fix: import cycle
devhaozi Oct 9, 2023
870781f
fix: lint
devhaozi Oct 9, 2023
9a7904f
feat: optimize config
devhaozi Oct 9, 2023
657663d
feat: optimize migrations
devhaozi Oct 9, 2023
854fc60
feat: sync and async
devhaozi Oct 9, 2023
03d489a
Merge remote-tracking branch 'origin/master' into haozi/queue
devhaozi Oct 29, 2023
e40811b
Merge remote-tracking branch 'origin/master' into haozi/queue
devhaozi Nov 9, 2023
c7dbcec
feat: sync master
devhaozi Nov 9, 2023
21578e7
Merge remote-tracking branch 'origin/master' into haozi/queue
devhaozi Dec 3, 2023
fcdf373
chore: merge master and update
devhaozi Dec 3, 2023
433d0b8
fix: sync databases
devhaozi Dec 3, 2023
4c34582
feat: update database migrations
devhaozi Dec 3, 2023
6527ab8
feat: update async and database
devhaozi Dec 3, 2023
924ceba
refactor: use Payload instead Arg
devhaozi Dec 4, 2023
f5f057e
feat: update
devhaozi Dec 4, 2023
c6c226f
feat: update
devhaozi Dec 4, 2023
3bb00eb
feat: redis
devhaozi Dec 4, 2023
2b159b9
refactor: redis driver
devhaozi Dec 5, 2023
760c48f
Merge remote-tracking branch 'origin/master' into haozi/queue
devhaozi Dec 5, 2023
fa03dc5
chore: merge master
devhaozi Dec 5, 2023
5c13b36
chore: update mocks
devhaozi Dec 5, 2023
19a97d2
fix: event error
devhaozi Dec 5, 2023
96f5ea1
Merge remote-tracking branch 'origin/haozi/queue' into haozi/queue
devhaozi Dec 5, 2023
1590461
fix: tests
devhaozi Dec 5, 2023
31cf3b6
fix: lint
devhaozi Dec 5, 2023
6481f4e
fix: email orm mock
devhaozi Dec 5, 2023
34d9850
fix: email orm mock
devhaozi Dec 5, 2023
d706baa
refactor: use reflect to handle job
devhaozi Dec 5, 2023
8e8c003
chore: update mocks
devhaozi Dec 5, 2023
44fe027
feat: add test case for argsToValues
devhaozi Dec 5, 2023
4309dc4
Merge remote-tracking branch 'origin/haozi/queue' into haozi/queue
devhaozi Dec 5, 2023
4b66470
fix: convert error
devhaozi Dec 5, 2023
0069909
fix: optimize SQL
devhaozi Dec 10, 2023
36f8b21
feat: some optimize
devhaozi Dec 17, 2023
26db41f
Merge remote-tracking branch 'origin/master' into haozi/queue
devhaozi Dec 17, 2023
04aff6d
feat: merge master
devhaozi Dec 17, 2023
97bab31
chore: update mocks
devhaozi Dec 17, 2023
87ecd1a
feat: add test cases
devhaozi Dec 17, 2023
1aec5ef
fix: nil
devhaozi Dec 17, 2023
f8c9b63
fix: mock
devhaozi Dec 17, 2023
093d250
Merge remote-tracking branch 'origin/master' into haozi/queue
devhaozi May 26, 2024
51aa085
chore: merge master
devhaozi May 26, 2024
8a1e5d0
chore: update mocks
devhaozi May 26, 2024
efa0b5e
refactor: use new method
devhaozi May 26, 2024
3b51863
chore: update mocks
devhaozi May 26, 2024
f15a41c
refactor: remove unused methods
devhaozi May 26, 2024
f414760
Merge remote-tracking branch 'origin/haozi/queue' into haozi/queue
devhaozi May 26, 2024
54a0d05
chore: update mocks
devhaozi May 26, 2024
119bb4c
fix: lint
devhaozi May 26, 2024
7ac2501
Merge remote-tracking branch 'origin/haozi/queue' into haozi/queue
devhaozi May 26, 2024
8e9f953
fix: email test
devhaozi May 26, 2024
7a607fb
fix: email test
devhaozi May 26, 2024
cb49814
refactor: use any type
devhaozi May 26, 2024
e35cca2
chore: update mocks
devhaozi May 26, 2024
c027db0
fix: lint
devhaozi May 26, 2024
fcbe1a0
fix: test
devhaozi May 26, 2024
807bee4
fix: test
devhaozi May 26, 2024
f09f9b3
fix: test
devhaozi May 26, 2024
4345964
fix: test
devhaozi May 26, 2024
4f570bd
feat: support custom
devhaozi May 26, 2024
ae5ac8b
feat: refactor job struct
devhaozi May 26, 2024
ce8577b
fix: test
devhaozi May 26, 2024
9158c13
fix: test
devhaozi May 26, 2024
4fcadbb
feat: add migrate make
devhaozi May 26, 2024
94593ba
feat: add method to get job
devhaozi May 26, 2024
98edd33
chore: update mocks
devhaozi May 26, 2024
5408563
Merge remote-tracking branch 'origin/master' into haozi/queue
devhaozi May 29, 2024
b573ced
chore: merge master
devhaozi May 29, 2024
443e5ef
chore: update github.com/charmbracelet/huh
devhaozi May 29, 2024
bddcb8b
fix: remove signal listen
devhaozi May 29, 2024
530af3e
Merge remote-tracking branch 'refs/remotes/origin/master' into haozi/…
devhaozi Jun 8, 2024
72e9e36
chore: merge master
devhaozi Jun 8, 2024
13e3e57
Merge branch 'refs/heads/master' into haozi/queue
devhaozi Jun 18, 2024
fed0b61
chore: merge master
devhaozi Jun 18, 2024
731b8d9
fix: lint
devhaozi Jun 18, 2024
6906b86
Merge branch 'refs/heads/master' into haozi/queue
devhaozi Jun 27, 2024
91c3c46
chore: merge master
devhaozi Jun 27, 2024
25e1b1b
Merge remote-tracking branch 'refs/remotes/origin/master' into haozi/…
devhaozi Jul 10, 2024
ad7e517
chore: merge master
devhaozi Jul 10, 2024
4069a4e
Merge remote-tracking branch 'origin/master' into haozi/queue
devhaozi Jul 10, 2024
5067a9a
fix: add missing Lang method
devhaozi Jul 10, 2024
410aac7
Merge branch 'refs/heads/haozi/lang' into haozi/queue
devhaozi Jul 10, 2024
ebb185a
feat: optimize code
devhaozi Jul 10, 2024
f2fb5aa
fix: lint
devhaozi Jul 10, 2024
d704469
Merge branch 'refs/heads/haozi/lang' into haozi/queue
devhaozi Jul 10, 2024
202ac79
Merge remote-tracking branch 'refs/remotes/origin/master' into haozi/…
devhaozi Jul 12, 2024
8fee275
chore: merge master
devhaozi Jul 12, 2024
00d3fef
Merge remote-tracking branch 'origin/master' into haozi/queue
devhaozi Jul 12, 2024
f1b118b
feat: optimize test
devhaozi Jul 12, 2024
78d1d98
Merge branch 'refs/heads/master' into haozi/queue
devhaozi Jul 28, 2024
b71cfeb
chore: merge master
devhaozi Jul 28, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions contracts/queue/driver.go
devhaozi marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
package queue

type Driver interface {
// ConnectionName returns the connection name for the driver.
ConnectionName() string
devhaozi marked this conversation as resolved.
Show resolved Hide resolved
// Push pushes the job onto the queue.
Push(job Job, args []Arg, queue string) error
// Bulk pushes a slice of jobs onto the queue.
Bulk(jobs []Jobs, queue string) error
// Later pushes the job onto the queue after a delay.
Later(delay int, job Job, args []Arg, queue string) error
// Pop pops the next job off of the queue.
Pop(queue string) (Job, []Arg, error)
// Delete removes a job from the queue.
Delete(queue string, job Job) error
// Release releases a reserved job back onto the queue.
Release(queue string, job Job, delay int) error
// Clear clears all pending jobs in the queue.
Clear(queue string) error
// Size returns the size of the queue.
Size(queue string) (int64, error)
// Server starts a queue server.
Server(concurrent int, queue string)
}
6 changes: 3 additions & 3 deletions contracts/queue/queue.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ package queue
type Queue interface {
Worker(args *Args) Worker
// Register register jobs
Register(jobs []Job)
Register(jobs []Job) error
// GetJobs get all jobs
GetJobs() []Job
// Job add a job to queue
Expand All @@ -26,6 +26,6 @@ type Args struct {
}

type Arg struct {
Type string
Value any
Type string `json:"type"`
Value any `json:"value"`
}
6 changes: 3 additions & 3 deletions contracts/queue/task.go
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
package queue

import (
"time"
"github.com/goravel/framework/support/carbon"
)

type Task interface {
// Dispatch dispatches the task.
Dispatch() error
// DispatchSync dispatches the task synchronously.
DispatchSync() error
// Delay dispatches the task after the given delay.
Delay(time time.Time) Task
// Delay dispatches the task after the given time.
Delay(time carbon.Carbon) Task
devhaozi marked this conversation as resolved.
Show resolved Hide resolved
// OnConnection sets the connection of the task.
OnConnection(connection string) Task
// OnQueue sets the queue of the task.
Expand Down
5 changes: 4 additions & 1 deletion event/application.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,10 @@ func (app *Application) Register(events map[event.Event][]event.Listener) {
}
}

app.queue.Register(jobs)
err := app.queue.Register(jobs)
if err != nil {
panic(err.Error())
}
devhaozi marked this conversation as resolved.
Show resolved Hide resolved
}

func (app *Application) GetEvents() map[event.Event][]event.Listener {
Expand Down
55 changes: 9 additions & 46 deletions go.mod
devhaozi marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,11 @@ module github.com/goravel/framework
go 1.20

require (
github.com/RichardKnop/machinery/v2 v2.0.11
github.com/bytedance/sonic v1.10.1
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc
github.com/gabriel-vasile/mimetype v1.4.2
github.com/glebarez/go-sqlite v1.21.2
github.com/glebarez/sqlite v1.9.0
github.com/go-redis/redis/v8 v8.11.5
github.com/go-sql-driver/mysql v1.7.1
github.com/golang-jwt/jwt/v5 v5.0.0
github.com/golang-migrate/migrate/v4 v4.16.2
Expand All @@ -26,6 +24,7 @@ require (
github.com/ory/dockertest/v3 v3.10.0
github.com/patrickmn/go-cache v2.1.0+incompatible
github.com/pkg/errors v0.9.1
github.com/redis/go-redis/v9 v9.2.1
github.com/rifflock/lfshook v0.0.0-20180920164130-b9218ef580f5
github.com/robfig/cron/v3 v3.0.1
github.com/rotisserie/eris v0.5.4
Expand All @@ -36,7 +35,7 @@ require (
github.com/urfave/cli/v2 v2.25.7
go.uber.org/atomic v1.11.0
golang.org/x/crypto v0.14.0
golang.org/x/exp v0.0.0-20230905200255-921286631fa9
golang.org/x/exp v0.0.0-20231006140011-7918f672742d
google.golang.org/grpc v1.58.2
gorm.io/driver/mysql v1.5.1
gorm.io/driver/postgres v1.5.2
Expand All @@ -46,18 +45,6 @@ require (
)

require (
github.com/sagikazarmark/locafero v0.3.0 // indirect
github.com/sagikazarmark/slog-shim v0.1.0 // indirect
github.com/sourcegraph/conc v0.3.0 // indirect
go.uber.org/multierr v1.9.0 // indirect
)

require (
cloud.google.com/go v0.110.7 // indirect
cloud.google.com/go/compute v1.23.0 // indirect
cloud.google.com/go/compute/metadata v0.2.3 // indirect
cloud.google.com/go/iam v1.1.1 // indirect
cloud.google.com/go/pubsub v1.33.0 // indirect
github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161 // indirect
github.com/Azure/go-autorest v14.2.0+incompatible // indirect
github.com/Azure/go-autorest/autorest/adal v0.9.16 // indirect
Expand All @@ -66,8 +53,6 @@ require (
github.com/Azure/go-autorest/tracing v0.6.0 // indirect
github.com/Microsoft/go-winio v0.6.1 // indirect
github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5 // indirect
github.com/RichardKnop/logging v0.0.0-20190827224416-1a693bdd4fae // indirect
github.com/aws/aws-sdk-go v1.37.16 // indirect
github.com/brianvoe/gofakeit/v6 v6.23.2
github.com/cenkalti/backoff/v4 v4.2.0 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
Expand All @@ -82,20 +67,11 @@ require (
github.com/docker/go-units v0.5.0 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/fsnotify/fsnotify v1.6.0 // indirect
github.com/go-redsync/redsync/v4 v4.0.4 // indirect
github.com/go-stack/stack v1.8.0 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang-jwt/jwt/v4 v4.5.0 // indirect
github.com/golang-sql/civil v0.0.0-20220223132316-b832511892a9 // indirect
github.com/golang-sql/sqlexp v0.1.0 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/snappy v0.0.4 // indirect
github.com/gomodule/redigo v2.0.0+incompatible // indirect
github.com/google/go-cmp v0.5.9 // indirect
github.com/google/s2a-go v0.1.7 // indirect
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.1 // indirect
github.com/googleapis/gax-go/v2 v2.12.0 // indirect
github.com/gookit/filter v1.2.0 // indirect
github.com/gookit/goutil v0.6.12 // indirect
github.com/goravel/file-rotatelogs v0.0.0-20211215053220-2ab31dd9575c // indirect
Expand All @@ -107,9 +83,6 @@ require (
github.com/jackc/pgx/v5 v5.3.1 // indirect
github.com/jinzhu/inflection v1.0.0 // indirect
github.com/jinzhu/now v1.1.5 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
github.com/kelseyhightower/envconfig v1.4.0 // indirect
github.com/klauspost/compress v1.17.0 // indirect
github.com/klauspost/cpuid/v2 v2.2.5 // indirect
github.com/lestrrat-go/strftime v1.0.5 // indirect
github.com/lib/pq v1.10.2 // indirect
Expand All @@ -121,40 +94,30 @@ require (
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.0.2 // indirect
github.com/opencontainers/runc v1.1.5 // indirect
github.com/opentracing/opentracing-go v1.2.0 // indirect
github.com/pelletier/go-toml/v2 v2.1.0 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/russross/blackfriday/v2 v2.1.0 // indirect
github.com/sagikazarmark/locafero v0.3.0 // indirect
github.com/sagikazarmark/slog-shim v0.1.0 // indirect
github.com/sourcegraph/conc v0.3.0 // indirect
github.com/spf13/afero v1.10.0 // indirect
github.com/spf13/pflag v1.0.5 // indirect
github.com/streadway/amqp v1.0.0 // indirect
github.com/stretchr/objx v0.5.0 // indirect
github.com/subosito/gotenv v1.6.0 // indirect
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
github.com/xdg-go/pbkdf2 v1.0.0 // indirect
github.com/xdg-go/scram v1.1.1 // indirect
github.com/xdg-go/stringprep v1.0.3 // indirect
github.com/xeipuuv/gojsonpointer v0.0.0-20190905194746-02993c407bfb // indirect
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 // indirect
github.com/xeipuuv/gojsonschema v1.2.0 // indirect
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect
github.com/xrash/smetrics v0.0.0-20201216005158-039620a65673 // indirect
github.com/youmark/pkcs8 v0.0.0-20181117223130-1be2e3e5546d // indirect
go.mongodb.org/mongo-driver v1.7.5 // indirect
go.opencensus.io v0.24.0 // indirect
go.uber.org/multierr v1.9.0 // indirect
golang.org/x/arch v0.3.0 // indirect
golang.org/x/mod v0.12.0 // indirect
golang.org/x/net v0.15.0 // indirect
golang.org/x/oauth2 v0.12.0 // indirect
golang.org/x/sync v0.3.0 // indirect
golang.org/x/mod v0.13.0 // indirect
golang.org/x/net v0.16.0 // indirect
golang.org/x/sys v0.13.0 // indirect
golang.org/x/text v0.13.0
golang.org/x/tools v0.13.0 // indirect
google.golang.org/api v0.143.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20230913181813-007df8e322eb // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20230913181813-007df8e322eb // indirect
golang.org/x/tools v0.14.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20230920204549-e6e6cdab5c13 // indirect
google.golang.org/protobuf v1.31.0 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
Expand Down
Loading
Loading