Naijacloud
Docs

Background workers

A background worker builds from your repository like a web service and stays running, but it serves no traffic. Use it for queue consumers such as Celery, Sidekiq, RQ, BullMQ and Laravel queue workers.

4 min read

When to use one

Most Django, Rails and Laravel apps hand slow work to a queue: sending email, resizing images, calling a payment provider. The web service puts a job on the queue and answers at once, and a worker takes the job off the queue and does it.

You needUse
A process that answers HTTP requestsWeb service
A process that runs all the time and takes no requestsBackground worker
A command that runs on a schedule and exitsCron job

A worker has no port, no health check and no public URL. You do not need a fake port listener to keep a deploy from failing.

Creating a worker

Choose New service → Background worker in the environment that holds your web service and its database. The form is the web service form without a URL:

  1. Name. Shown in the dashboard. A worker has no address, so the name is only a label.
  2. Source. A GitHub repository, a public git URL or a public Docker image. Usually the same repository and branch as your web service, so both run the same code.
  3. Start command. Required for a repository. This is the command that runs your worker. We do not guess it: the command we detect for a repository is the web server's.
  4. Environment variables, region and instance type, as for a web service.
Start commands
bash
celery -A mysite worker --loglevel=info bundle exec sidekiq rq worker --url $REDIS_URL php artisan queue:work --tries=3 node dist/worker.js

For a Docker image the start command is optional. Leave it blank to run the image's own command.

Reaching your database and Redis

A worker joins its environment's private network, the same one your web service uses. Set the same DATABASE_URL and REDIS_URL on the worker as on the web service, using each data service's Internal URL. Nothing goes over the public internet.

Variables belong to one service. When you add a variable to the web service, add it to the worker too if the worker needs it.

Deploys

A worker deploys like a web service: on create, on Redeploy, and on a push to its branch (always for a GitHub repository, and for a public git URL when auto-deploy on push is on). The build is the same as a web service's.

A deploy goes live once the new container has stayed running for about 12 seconds. Then the old container is sent SIGTERM and given 10 seconds to finish before it is stopped. For those few seconds the old and the new worker both run, so a job can be picked up by either. Most queue libraries handle this, but a process that must only ever run once, such as Celery beat, should run in a single place and finish its current work cleanly on SIGTERM.

If the new container exits before then, the deploy fails and the old version keeps running.

Crashes and restarts

If a running worker exits or crashes, it is restarted automatically. When it keeps crashing, the service shows Crash-looping and the workspace is told by email. Read Logs to see why.

Logs and metrics

A worker's output is in Logs on its Argus tab (called Observability in some workspaces), with CPU and memory under Metrics. There is no Requests view, because a worker takes no requests. Argus error tracking works the same as for a web service.

Sizes and billing

Workers use the web service sizes and prices, and like any paid service they never sleep:

SizeResourcesMonthly price
Starter0.5 CPU · 512 MB₦3,500
Pro1 CPU · 1 GB₦7,000
Pro Max2 CPU · 2 GB₦14,000

A worker in af-west · Port Harcourt costs 1.5 times these prices, the same as a web service there. Billing is by the hour while the worker runs. A paused worker is not billed.

There is no Free worker. Free services sleep when nobody visits them, and a worker has no visitors.