go-scheduler

A go task scheduler / runner library with various backends and triggers
Explore the docs »

Report Bug · Request Feature

Table of Contents

1. About The Project
   • Built With
2. Getting Started
   • Prerequisites
   • Installation
3. Roadmap
4. Contributing
5. License

About The Project

There are a handful of golang task schedulers but none of them suited our needs so we created one. Our initial need was for reminders (texts/emails/sms) with various triggers like cron, execute once, or natural language such as "every other tuesday"

Reasons we rolled our own scheduler:

• We wanted a trigger interface so that new triggers can easily be created
• We wanted a storage interface so that the scheduler can be run in different modes for different scales

Distinct Features:

• Trigger interface (this isn't unique to this project)
• Support for any storage backend you want to create
• Horizontal scalability
• Configurable execution window to control resource usage

Implemented Backends:

• Cockroachdb - To allow horizontal scalability, useful for real-world production scenarios

Planned Backends:

• Memory - An in-memory backend, useful for toys/testing

Implemented Triggers:

• ExecuteOnce - Executes once at the specified time, respects retries and expiration
• Cron - Executes on a cron schedule, respects retries and expiration

Planned Triggers:

• Natural language - Execute every other tuesday at 9:34am

FAQ:

• What do you mean by "execution window"
  • The execution window is a time.Duration that you pass to the NewScheduler() function. This defines how often to fetch tasks from the storage backend. This is configurable mostly to control resources. For example if set to 30 seconds, then every 30 seconds the scheduler will get the tasks scheduled to execute in the next 30 seconds and hold them in memory, executing your handler at the task's scheduled time. You can tune this to control memory usage.
• What does the ExpireAfter field on a task do?
  • This setting is used for fault tolerance. The backend store tracks when a task is in progress. If a task's scheduled time is in the past, the store will re-schedule the task if the ExpireAfter has passed. This would happen if there was some failure to update the task in the store, or if the handler hung, or something like that so that the task doesn't just get dropped. This lets you have handler functions that run longer than the execution window without executing multiple times.

Design: The scheduler runs 3 goroutines on tickers that each have distinct concerns but don't care about each other. This design is intended to ease troubleshooting, implementation, and maintenance by avoiding complex logic through separation of concerns.

1. Scheduler routine queries for task definitions that should run in the next window, and creates task instances accordingly. When a task instance is created, the scheduler also updates the task definition's next_fire_time based on the definition's trigger
2. Task runner routine queries for task instances that should run in the next window and haven't run yet, or are expired, and runs them at their specified time. The runner also sets the task instance's started_at time when the handler is called, and sets the completed_at time when the handler is finished, if the handler is successful
3. Cleanup routine deletes completed task instances and task definitions if appropriate.

(back to top)

Built With

(back to top)

Getting Started

This example uses the cockroachdb store, ran via gnomock, you can use any store that exists or PR your own.

// start a gnomock cockroach container
dbName := "test"
cockroachdbContainer, err := gnomock.Start(cockroachdb.Preset(cockroachdb.WithDatabase(dbName)))
uri := fmt.Sprintf("host=%s port=%d dbname=%s user=%s password=%s sslmode=disable",cockroachdbContainer.Host, cockroachdbContainer.DefaultPort(), dbName, "root", "")
// instantiate a new store
cockroachdbStore := cockroachdb_store.NewCockroachdbStore(uri, nil)
// define a handler function
handler := func(task pkg.Task) error {
  fmt.Println("handling a task")
  return nil
}
// instantiate a scheduler instance, defining the window to get scheduled tasks
scheduler, err := pkg.NewScheduler(1*time.Second, handler, store)
// start the scheduler in a goroutine
go scheduler.Run()
// define a task
id := uuid.New()
metaData := map[string]string{"some metadata": "about the task, this can be whatever you want"}
executeAt := time.Now().Add(5 * time.Second)
expireAfter := 10 * time.Second
task := pkg.Task{
  Id:                 &id,
  Metadata:           metaData,
  ExpireAfter: &expireAfter,
  ExecuteOnceTrigger: pkg.NewExecuteOnceTrigger(executeAt),
}
// schedule the task
err := scheduler.ScheduleTask(task)
// your handler will now be called at the time specified in the trigger, and the task you defined will be pased to your handler

Prerequisites

There are no prerequisites, it's an importable go library. If you're using a backend that requires a database, you have to deploy that database somewhere.

Installation

go get github.com/catalystcommunity/go-scheduler

(back to top)

Roadmap

• Add Cockroachdb backend support
• Add Natural language trigger

See the open issues for a full list of proposed features (and known issues).

(back to top)

Contributing

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

(back to top)

License

Distributed under the APACHE 2.0 License. See LICENSE.txt for more information.

(back to top)