Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Scheduler Feature for Running Cron Jobs #735

Merged
merged 21 commits into from
Sep 12, 2024
Merged

Add Scheduler Feature for Running Cron Jobs #735

merged 21 commits into from
Sep 12, 2024

Conversation

kaplanelad
Copy link
Contributor

@kaplanelad kaplanelad commented Sep 5, 2024

This PR introduces the ability to schedule and run cron jobs within the Loco framework. The new scheduler functionality includes:

  • Support for executing shell scripts as cron jobs.
  • Ability to run Loco tasks as cron jobs.
  • Propagation of environment variables to scheduled jobs for seamless execution.
  • Configurable job-level output control, allowing dynamic output settings for each job (e.g., stdout, silent).
  • Flexibility to run jobs by their name or group multiple jobs with the same tag.
  • Feature testing.
  • Documentation.

@@ -96,6 +96,13 @@ jobs:
REDIS_URL: redis://localhost:${{job.services.redis.ports[6379]}}
DATABASE_URL: postgres://postgres:postgres@localhost:5432/postgres_test

- name: scheduler
run: cargo run -- generate scheduler
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i would add some more value for example

$ cargo run -- generate job send_emails --cron '* * * * ..'

Added `send_emails` to scheduler jobs
Total 12 jobs in `config/development.yaml` under `scheduler:`.

This will upsert a schedule record.

You can also experiment with english-to-cron libraries

$ cargo run -- generate job send_emails --at 'every sunday at 1pm'

Cron schedule: 0 * * * 1 *
Added `send_emails` to jobs
Total 12 jobs in `config/development.yaml` under `scheduler:`.

Notice we generate a job
If we say scheduler it is the thing that runs jobs

Its also a good opportunity to centralize and sharpen our concepts and terminology:

  • task is a one time, you run it any way you like
  • job is a repeat on a schedule, run by the scheduler
  • worker is something that performs a background job from a queue

# A list of jobs to be scheduled.
jobs:
# The name of the job.
- name: "Run command"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use a hashmap instead of vec, id instead of name

jobs:
   send_emails:
         ...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree.
changed: 19d2a0f

# The name of the job.
- name: "Run command"
# The type of job.
shell:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should be just:

run: "...command.."
shell: true | false

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done: f85601b

# The cron expression defining the job's schedule.
cron: "*/1 * * * * *"
output: silent
tags:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we do tags: ['base', 'infra'] in YAML example?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. 667ce38

```console
$ LOCO_ENV=test blo-cli scheduler --config ./config/scheduler.yaml --list
# job_name cron tags kind
1 add text */1 * * * * * base, infra Shell { command: "echo loco >> ./scheduler.txt" }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there are some libraries that do "english to cron" and "cron to english"
might be good to add a "cron to english" description here, based on the cron expression that's saved in the configuration

@kaplanelad kaplanelad merged commit 52e4f82 into master Sep 12, 2024
16 of 17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants