Celery: Distributed Task Queue

Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well.

The execution units, called tasks, are executed concurrently on a single or more worker servers using multiprocessing, Eventlet, or gevent. Tasks can execute asynchronously (in the background) or synchronously (wait until ready).

Celery is used in production systems to process millions of tasks a day.

Read all news

Getting started

  1. Install celery by download or pip install -U Celery
  2. Set up RabbitMQ, Redis or one of the other supported brokers
  3. Select one of the following guides:
    1. First steps with Python
    2. First steps with Django

Easy to integrate

Celery is easy to integrate with web frameworks, some of which even have integration packages.

Celery is written in Python, but the protocol can be implemented in any language. It can also operate with other languages using webhooks.

Multi broker support

The recommended message broker is RabbitMQ, but support for Redis, Beanstalk, MongoDB, CouchDB, and databases (using SQLAlchemy or the Django ORM) is also available.

Related Projects

Flower

Real-time monitor and web admin for Celery

- Real-time monitoring using Celery Events
- Remote Control
- Broker monitoring
- HTTP API
- Basic Auth and Google OpenID authentication

Jobtastic

A Celery library that makes your user-responsive long-running jobs totally awesomer.

Jobtastic is a python library that adds useful features to your Celery tasks. Specifically, these are features you probably want if the results of your jobs are expensive or if your users need to wait while they compute their results.

Who is using celery