Queues and scheduled work

This guide covers the two things that happen outside a web request: jobs that wait in a queue for a worker, and tasks the scheduler runs on a clock. You need it when a job is not running, when a deploy did not take effect in the background, or when you add work of your own.

The queue connection

config/queue.php reads QUEUE_CONNECTION, default database: jobs are rows in the jobs table, which the kit's migrations create, and failed jobs go to failed_jobs (driver database-uuids). No extra service is needed. The connections and their variables:

QUEUE_FAILED_DRIVER (default database-uuids) chooses where failed jobs are kept. Job batches use the job_batches table.

What the kit queues

What is not queued, so you know where to look when it is slow: notifications built on App\Notifications\Notification are sent during the request, database row, email and push included, because the base class uses Queueable without ShouldQueue; the three ticket notices above are the exception. The email verification code is sent synchronously with Mail::mailer(...)->send(), and the SMS code goes out synchronously too. Add implements ShouldQueue to a notification of your own when it fans out to many people; the announcement job is the pattern for doing that in bulk.

To create a queued job use the project stub, which already implements ShouldQueue:

php artisan make:job Orders/ArchiveOldOrders

Run the worker locally

composer dev starts four processes in one terminal: php artisan serve, php artisan queue:listen --tries=1, php artisan pail --timeout=0 (the log tail) and npm run dev. A job you dispatch runs a moment later and its output appears in the same terminal. queue:listen reloads the code on every job, so you do not restart it after editing a job class; that is why it is used here and not in production.

If you would rather not run a worker, set QUEUE_CONNECTION=sync in your local .env. Everything runs inline and exceptions surface in the request that caused them.

Run the worker in production

The worker is a long-lived process:

php artisan queue:work --sleep=3 --tries=3 --max-time=3600

queue:work loads the application once and keeps it in memory, which is what makes it fast and what makes it stale after a deploy. Keep it alive with Supervisor; the complete program block is in /help/deploy-your-project. --sleep=3 is the pause when the queue is empty, --tries=3 the attempts before a job is marked failed, and --max-time=3600 the lifetime after which the worker exits so Supervisor restarts it.

Add --queue=high,default to work several named queues in priority order, and --timeout=60 to kill a single job that runs longer than a minute (keep it below the connection's retry_after).

Restart it on every deploy

php artisan queue:restart

The command writes a timestamp into the cache; each worker checks it after every job and exits when it is newer than its own start. Supervisor then starts a new worker, which loads the new code. Two consequences: the cache store has to be one the worker and the web process share (database or redis, never array), and a job that is running when you call it finishes first. Without this command the worker keeps executing the previous release: a notification whose text you changed keeps arriving with the old text until the worker happens to die.

Failed jobs

A job that throws on every one of its --tries attempts lands in failed_jobs with its exception. The exception is also reported through the normal channels: the log, and Slack when SLACK_BOT_TOKEN is set.

php artisan queue:failed            # list them, with id, connection, queue and date
php artisan queue:retry 9f1c-...    # push one back onto the queue by its id
php artisan queue:retry all         # push all of them back
php artisan queue:forget 9f1c-...   # delete one without retrying
php artisan queue:flush             # delete them all
php artisan queue:prune-failed --hours=168   # delete the ones older than a week

Retry after fixing the cause and restarting the worker; retrying a job into the old code fails it again.

Scheduled tasks

The scheduler runs everything registered with Schedule:: in routes/console.php and in the service providers of the packages. Among the tasks that ship with the kit:

Times are in the application timezone, UTC in config/app.php. php artisan schedule:list prints the table with the next run of each.

Nothing runs unless cron calls the scheduler every minute:

* * * * * cd /var/www/your-project && php artisan schedule:run >> /dev/null 2>&1

schedule:run looks at the clock, runs whatever is due and exits. Locally, php artisan schedule:work does the same in the foreground once a minute, and php artisan schedule:test lets you pick a task and run it now.

The grace period after a failed payment

A renewal the gateway could not charge does not take the plan away at once. When Stripe reports the failed invoice, the subscription is marked with the date of the failure and the amount owed, and its owner receives the failed-payment notification. A usage period that billing:close-meter-periods could not charge on a running subscription starts the same clock. From then on the subscription keeps its plan, its included quota and its dates, the billing page shows Payment failed with the date the grace period ends and a Retry payment button, and the gateway keeps retrying.

How long that lasts is set in config/billing-core.php under past_due:

Every day at 03:00 billing:enforce-past-due goes through the subscriptions that still have a failed payment:

1. One whose grace period ends within BILLING_PAST_DUE_WARNING_DAYS and that was not warned yet: its owner receives the notice that the subscription will be cancelled for non-payment. The warning goes out once.
2. One whose grace period ran out: its open invoices are voided in Stripe, the subscription is cancelled on the spot (when Stripe cannot be reached it is marked cancelled locally, and the error goes to the log), the failure is cleared, and the amount owed becomes the outstanding balance of the account, which the user settles in one payment. Its owner is told about that balance.

A payment that goes through before then clears the clock, and the subscription is not touched. The command prints how many subscriptions it warned and how many it cancelled, and running it by hand is safe.

Configure Stripe to keep retrying a failed payment without cancelling the subscription on its own: if Stripe cancels it first, it ends before this grace period does.

Add your own

A job goes in app/Jobs, a command in app/Console/Commands (registered automatically), and its schedule in routes/console.php:

Schedule::command('orders:archive-old')->dailyAt('03:00');
Schedule::job(new ArchiveOldOrders)->weeklyOn(1, '04:00');

A scheduled command that takes long, or that must not overlap with itself, gets ->withoutOverlapping(); one that should run on one server only when you have several gets ->onOneServer(), which needs a shared cache store.