How to monitor Ruby and Rails scheduled jobs

To monitor a Ruby or Rails scheduled job, ping a monitoring URL when the job starts and when it finishes, and send /fail with the error if it raises. Cronomonitor alerts you on a failure, a hang, or a run that never happens.

Why monitor it

Ruby jobs are scheduled with cron (often through the whenever gem), sidekiq-cron, GoodJob or Solid Queue recurring tasks, or a platform scheduler running a rake task. The same pattern works in all of them.

1. Create a check

Sign up for Cronomonitor, create a project and add a check named after the job. Give it the job's schedule, here the cron expression 0 3 * * * (every day at 3am, UTC), and a grace period that covers how long the job normally takes.

The check gives you a unique ping URL. Every snippet below is also on the check's page in the dashboard, already filled in with that URL and schedule.

2. Add the ping to your job

Example

require "net/http"

PING_URL = "https://cronomonitor.com/api/ping/<your-check-id>"

# Never let monitoring break the job.
def ping(path = "", body = nil)
  uri = URI(PING_URL + path)
  Net::HTTP.start(uri.host, uri.port, use_ssl: uri.scheme == "https", open_timeout: 10, read_timeout: 10) do |http|
    http.post(uri.path, body.to_s)
  end
rescue StandardError
  nil
end

ping("/start")
begin
  run_job # your code here, e.g. in a Rails job's perform
  ping
rescue Exception => e
  ping("/fail", e.full_message(highlight: false)[-10_000..] || e.full_message(highlight: false))
  raise
end

How it works

The ping method uses Net::HTTP from the standard library and rescues network errors, so monitoring can never break the job. If the job raises, the end of the error message and backtrace goes with the /fail ping, then the exception is re-raised.

In a Rails job, put the pings around the body of perform; in a rake task, around the task's block.

Background job schedulers

With sidekiq-cron, GoodJob or Solid Queue, the scheduler enqueues the job and a worker runs it. Pinging from inside perform covers both: if the scheduler stops enqueueing or the workers stop, the pings stop.

3. Test it

Run the job once by hand, or use Send a test ping on the check. The check turns green as soon as the first ping arrives, and from then on Cronomonitor expects the next one on schedule.

To check where alerts go, use Send test alert on the check. To see a real failure alert, make the job exit with an error once.

If the check says the job didn't run

Run the task by hand in the same environment, e.g. RAILS_ENV=production bin/rails runner, to catch load errors.

Read the check's ping log: a /start ping with no finish means the job hung or was killed.