Scheduled Jobs for Web Apps: Cron Patterns and the Pitfalls That Bite

TL;DR: The simplest reliable way to run scheduled work for a web app is to expose a protected endpoint and have something call it on a cron schedule. Make every job idempotent, keep it short, log what it did, and decide up front what should happen if it runs twice, runs late, or doesn’t run at all.

Almost every web app grows a list of things that should happen “every night” or “every five minutes”: send a digest email, expire old sessions, refresh a cache, sync with a third-party API, clean up temporary files. On your laptop, you’d reach for crontab -e. On a hosted platform, the container your app runs in can be replaced on any deploy, so a crontab inside it is fragile. This guide covers the patterns that hold up and the mistakes that show up again and again.

What is a cron schedule, and how do you read one?

A cron expression is five fields: minute, hour, day of month, month and day of week. A star means “every”.

*/5 * * * * runs every five minutes.
0 3 * * * runs at 3:00 every day.
30 8 * * 1-5 runs at 8:30 on weekdays.
0 0 1 * * runs at midnight on the first of each month.

Two details trip people up. First, when both day of month and day of week are set, classic cron runs the job when either matches, not both. Second, the schedule is evaluated in some timezone, and it’s often UTC on servers. “Every day at 3 AM” in UTC is a different local hour in winter and summer for most of the world. If the wall-clock time matters to your users, write it down next to the schedule and check which zone your scheduler uses.

Why trigger jobs over HTTP instead of a crontab in the container?

There are two common ways to run scheduled work for a hosted app:

A long-running process with its own scheduler, such as a library that keeps a timer loop inside your app or a separate background process.
An external scheduler that calls an endpoint in your app, like POST /jobs/send-digest, on a schedule.

The HTTP approach has real advantages for small apps. The job runs inside your normal web process, so it uses the same code, configuration, database connection and logging as everything else. There’s no second process to size, deploy or keep alive. And the schedule lives outside the container, so a redeploy doesn’t wipe it.

The tradeoff is that the job has to fit within a request. Anything that takes more than a short while should be split into smaller chunks or handed to a queue, which we’ll come back to below.

This is the model Candor Deploy uses for its scheduled jobs: you add a cron schedule and a path, and the platform calls that path in your app. The portal shows run history and a run-now button, so you can see what fired and when (scheduled jobs changelog).

How do you stop strangers from triggering your job endpoint?

A job endpoint is a URL, and URLs get found. If /jobs/send-digest is reachable and unprotected, anyone can make your app send a digest whenever they like.

Protect it in layers:

Require a shared secret. The scheduler sends a header with a secret value, and your handler rejects any request without the right one. Store the secret as an environment variable, never in the repository, and compare it with a constant-time comparison function.
Keep the call off the public internet where you can. If the scheduler can reach your app on an internal address, the endpoint doesn’t need to be exposed publicly at all.
Accept only the method you expect. Use POST, so a crawler following links can’t trigger it with a GET.

On Candor Deploy, scheduled job requests carry a header secret so only the platform can trigger them, and they go to your app’s internal address rather than through the public internet (scheduled jobs changelog). Your handler should still check the header.

Why does every scheduled job need to be idempotent?

Because sooner or later it will run twice. A retry after a timeout, someone pressing a run-now button during testing, or two overlapping runs all lead to the same place. An idempotent job produces the same result whether it runs once or three times.

Practical ways to get there:

Record what you’ve done. For a daily digest, store a row like (user_id, digest_date) with a unique constraint. Before sending, insert the row; if the insert fails because it already exists, skip that user.
Work from state, not from “since last run”. Instead of “process everything since the last time this ran”, select rows where processed_at IS NULL and mark each one as you finish it. If a run dies halfway, the next run picks up where it left off.
Make deletes conditional. “Delete sessions where expires_at < now()” is safe to repeat. “Delete the oldest 1,000 sessions” is not.

What happens if a job runs long or two runs overlap?

If a job scheduled every five minutes sometimes takes seven, you’ll eventually have two copies running at once, both touching the same rows.

The usual fixes:

Take a lock. In Postgres, pg_try_advisory_lock(<job id>) at the start of the job lets the second run notice the first is still going and exit immediately. Release it when you’re done. A row in a job_locks table with an expiry time does the same job on other databases.
Bound the work per run. Process at most a fixed batch, say 500 rows, then return. If there’s more, the next scheduled run handles it. This keeps each request short and predictable.
Push heavy work to a queue. If a job genuinely takes minutes, let the scheduled endpoint enqueue tasks and return quickly, then have a background worker drain the queue. Redis is a common choice for the queue in small apps.

How do you know a scheduled job actually ran?

The worst scheduled-job bug is the one nobody notices: the job silently stopped weeks ago, and the first sign is a customer asking why they never got their invoice.

Build in visibility from day one:

Log a start line and an end line with the job name, how many items it handled, and how long it took. One line each is enough. If you see a start with no matching end, the job crashed or timed out.
Return a meaningful status code. Return a 2xx only when the job really succeeded, and a 5xx when it didn’t, so the scheduler’s run history tells the truth.
Store the last successful run time in your database and show it on an internal admin page. If it’s older than it should be, something is wrong.
Check the scheduler’s own history when you suspect a problem. A run history that shows each call and its result is much faster than reading through application logs.

Candor Deploy keeps that run history in the portal next to the run-now button, and it publishes how many scheduled jobs you get on the same tab, so you’re not left to discover the limit when a job stops (scheduled jobs changelog).

What are the most common scheduled-job mistakes?

A short checklist of the ones that come up most:

Timezone surprises. The job runs an hour early or late after a daylight saving change. Schedule in UTC and convert, or confirm your scheduler’s zone.
No idempotency. A retry sends every customer two emails.
Unbounded work. The job is fine with 100 rows and times out with 100,000.
Secrets in the URL. A token in the query string ends up in access logs. Put it in a header.
Silent failure. The handler catches every exception and returns 200, so the run history always looks green.
Testing in production by accident. Pressing run-now on a job that emails real users. Give jobs a dry-run flag, or test against a staging copy first.

The short version

Expose a protected POST endpoint, call it on a cron schedule, and design the job so running it twice is harmless. Keep each run short, lock against overlap, and log enough that you’d notice within a day if it stopped. If your app also needs a database for job bookkeeping, the databases guide covers attaching one, and the Candor Deploy quickstart gets your app running first.