Who this is for. Developers who build jobs that run on the clock: a nightly export, an hourly sync, a weekly clean-up. After this page you know what the scheduler does with a job's schedule, where a scheduled run goes once it is due, and the one rule that makes a run that fails halfway harmless.
Work on the clock needs its own checks: nobody is waiting for it to answer, it handles many records so a failure halfway leaves some done and some not, and it runs again tomorrow so a mistake repeats. How a schedule is set on a job (the cron fields, Paused until, Recurring until and one-off executions) is on Web events and schedules. This page is about what happens after that.
Scheduled tasks
The platform has one scheduler, and everything that runs on the clock is a scheduler task (Scheduler\Entities\Task). A task has:
- a name;
- a five-field cron expression (minute, hour, day of month, month, day of week), evaluated in the client's timezone, the Globals.timezone setting, which is UTC unless it has been changed;
- an optional date before which it does not start (Paused until on a job) and one after which it stops (Recurring until);
- what to put on the job queue when it comes due: a queue key and its parameters.
There are two kinds of task.
| Platform task | Scheduled job | |
|---|---|---|
| Defined by | The platform | You: a job with a schedule |
| Where it is registered | The platform's own task list, shipped with each release | On the job: its schedule creates a task whose parameters name the job |
| Examples | Publishing records whose publish date has arrived (every minute), detecting abandoned carts (every five minutes), rebuilding the sitemap (hourly), deleting old job runs (02:10 daily) | A nightly export, an hourly stock sync, a Monday report |
| What it runs | Platform code | The job's actions, as a run like any other |
| Travels with an export | Not applicable | No: set the schedule and the job's site again after an import |
A site builder does not add platform tasks; they come with the platform. Giving a job a schedule creates a scheduler task that points at the job, and changing the schedule updates that task. Each time the task comes due, the scheduler puts a queue job on the job queue; a background worker picks it up, creates a run of the job and dispatches it. The run's event is the task, not a record: $event->getTask() (Twig event.getTask()) returns the scheduler task that came due, and there is no record to read.
Two facts about due times are covered where the schedule is set, and matter here. A recurring task is due only in the exact minute its cron expression names, so a minute the scheduler missed is not caught up (Missed runs). And the scheduler does not look at the previous run, so a run that takes longer than its interval overlaps the next (When the previous run has not finished). Both are reasons for the rule below.
The job queue, and what a half-failed job leaves
A scheduled run is dispatched exactly like any other asynchronous run. Dispatching sets the run's status to waiting and puts it on the job queue; if putting it on the queue fails, the status is error and nothing runs. A background worker then picks the run up and works through its actions in order, and a parallel loop inside it adds one more queue job per element. None of this is transactional. When an action fails halfway through a run:
- every action before it has done its work, and none of it is undone: records written, requests sent, emails queued;
- the run stops with status
error; the actions after the failure never ran; - in a parallel loop, the elements that finished are finished and the failed ones are marked as errors, each with its message;
- the run's custom data is kept, and so are any files it created.
Re-running the run from Batch Logs does not resume it. It deletes the run's action results and output files and runs the job again from the first action, always in the background. So the rule is not "make the job resumable from where it stopped" but:
A scheduled job must be safe to re-run from the start. Every run selects the work that is still to do, and doing a piece of work twice leaves the same result as doing it once. Then a re-run, an overlap and tomorrow's run are the same thing: they finish what is left. The two tools are a watermark, for selecting what changed, and the external-id pattern, for writes to other systems.
Watermarks
A job that handles only what changed since its last run keeps a watermark: where the last run stopped, stored in site data with its timezone offset, and moved only after the run succeeded. The pattern, the template that selects the next records, and keeping the watermark consistent across timezones so it does not reprocess the same records on every run, are on Watermarks and chunking.
Chunking
A run should handle one chunk of the work and finish well inside its interval, with the watermark committing progress between chunks, so a backlog drains over several runs. How to choose the chunk size from a measurement is on Watermarks and chunking.
Monitoring
A scheduled job needs its own alert: nothing built in reports a stopped or missed run. The heartbeat and health-check patterns that catch it are on Finding out a scheduled job stopped.