# Table of Contents - [What is BullMQ | BullMQ](#what-is-bullmq-bullmq) - [Quick Start | BullMQ](#quick-start-bullmq) - [Introduction | BullMQ](#introduction-bullmq) - [Queues | BullMQ](#queues-bullmq) - [Connections | BullMQ](#connections-bullmq) - [v2 | BullMQ](#v2-bullmq) - [v4 | BullMQ](#v4-bullmq) - [Meta | BullMQ](#meta-bullmq) - [Adding jobs in bulk | BullMQ](#adding-jobs-in-bulk-bullmq) - [Auto-removal of jobs | BullMQ](#auto-removal-of-jobs-bullmq) - [Concurrency | BullMQ](#concurrency-bullmq) - [Stalled Jobs | BullMQ](#stalled-jobs-bullmq) - [Global Rate Limit | BullMQ](#global-rate-limit-bullmq) - [Global Concurrency | BullMQ](#global-concurrency-bullmq) - [Removing Jobs | BullMQ](#removing-jobs-bullmq) - [Auto-removal of jobs | BullMQ](#auto-removal-of-jobs-bullmq) - [Manage Job Schedulers | BullMQ](#manage-job-schedulers-bullmq) - [Jobs | BullMQ](#jobs-bullmq) - [Adding flows in bulk | BullMQ](#adding-flows-in-bulk-bullmq) - [v3 | BullMQ](#v3-bullmq) - [Job Schedulers | BullMQ](#job-schedulers-bullmq) - [Dragonfly | BullMQ](#dragonfly-bullmq) - [LIFO | BullMQ](#lifo-bullmq) - [Telemetry | BullMQ](#telemetry-bullmq) - [Repeat Strategies | BullMQ](#repeat-strategies-bullmq) - [FIFO | BullMQ](#fifo-bullmq) - [Job Data | BullMQ](#job-data-bullmq) - [Graceful shutdown | BullMQ](#graceful-shutdown-bullmq) - [Parallelism and Concurrency | BullMQ](#parallelism-and-concurrency-bullmq) - [Fail Parent | BullMQ](#fail-parent-bullmq) - [Retrying failing jobs | BullMQ](#retrying-failing-jobs-bullmq) - [Create Custom Events | BullMQ](#create-custom-events-bullmq) - [Prometheus | BullMQ](#prometheus-bullmq) - [Returning job data | BullMQ](#returning-job-data-bullmq) - [Metrics | BullMQ](#metrics-bullmq) - [Job Ids | BullMQ](#job-ids-bullmq) - [Remove Dependency | BullMQ](#remove-dependency-bullmq) - [Pausing queues | BullMQ](#pausing-queues-bullmq) - [Bull to BullMQ | BullMQ](#bull-to-bullmq-bullmq) - [Named Processor | BullMQ](#named-processor-bullmq) - [Deduplication | BullMQ](#deduplication-bullmq) - [Running Jaeger | BullMQ](#running-jaeger-bullmq) - [Events | BullMQ](#events-bullmq) - [Stalled | BullMQ](#stalled-bullmq) - [Prioritized | BullMQ](#prioritized-bullmq) - [Get Flow Tree | BullMQ](#get-flow-tree-bullmq) - [NestJs | BullMQ](#nestjs-bullmq) - [Redis™ Compatibility | BullMQ](#redis-compatibility-bullmq) - [Repeat options | BullMQ](#repeat-options-bullmq) - [Producers | BullMQ](#producers-bullmq) - [Delayed | BullMQ](#delayed-bullmq) - [Architecture | BullMQ](#architecture-bullmq) - [Flows | BullMQ](#flows-bullmq) - [Getters | BullMQ](#getters-bullmq) - [Getting started | BullMQ](#getting-started-bullmq) - [Remove Child Dependency | BullMQ](#remove-child-dependency-bullmq) - [Migrations | BullMQ](#migrations-bullmq) - [Adding jobs in bulk across different queues | BullMQ](#adding-jobs-in-bulk-across-different-queues-bullmq) - [Failing fast when Redis is down | BullMQ](#failing-fast-when-redis-is-down-bullmq) - [Timeout jobs | BullMQ](#timeout-jobs-bullmq) - [Manual retrying | BullMQ](#manual-retrying-bullmq) - [AWS Elasticache | BullMQ](#aws-elasticache-bullmq) - [QueueScheduler | BullMQ](#queuescheduler-bullmq) - [Idempotent jobs | BullMQ](#idempotent-jobs-bullmq) - [Stop retrying jobs | BullMQ](#stop-retrying-jobs-bullmq) - [Queue Events Listeners | BullMQ](#queue-events-listeners-bullmq) - [Troubleshooting | BullMQ](#troubleshooting-bullmq) - [Redis™ hosting | BullMQ](#redis-hosting-bullmq) - [Running a simple example | BullMQ](#running-a-simple-example-bullmq) - [Redis Cluster | BullMQ](#redis-cluster-bullmq) - [Timeout for Sandboxed processors | BullMQ](#timeout-for-sandboxed-processors-bullmq) - [Throttle jobs | BullMQ](#throttle-jobs-bullmq) - [Going to production | BullMQ](#going-to-production-bullmq) - [Install | BullMQ](#install-bullmq) - [Manually processing jobs | BullMQ](#manually-processing-jobs-bullmq) - [Observables | BullMQ](#observables-bullmq) - [AWS MemoryDB | BullMQ](#aws-memorydb-bullmq) - [Telemetry | BullMQ](#telemetry-bullmq) - [New Releases | BullMQ](#new-releases-bullmq) - [Introduction | BullMQ](#introduction-bullmq) - [Local group rate limit | BullMQ](#local-group-rate-limit-bullmq) - [Getters | BullMQ](#getters-bullmq) - [Cancelation | BullMQ](#cancelation-bullmq) - [Concurrency | BullMQ](#concurrency-bullmq) - [NestJs | BullMQ](#nestjs-bullmq) - [Sandboxes for groups | BullMQ](#sandboxes-for-groups-bullmq) - [Deduplication | BullMQ](#deduplication-bullmq) - [Rate limiting | BullMQ](#rate-limiting-bullmq) - [v3 | BullMQ](#v3-bullmq) - [Queue Events Listeners | BullMQ](#queue-events-listeners-bullmq) - [Batches | BullMQ](#batches-bullmq) - [Introduction | BullMQ](#introduction-bullmq) - [Important Notes | BullMQ](#important-notes-bullmq) - [Install | BullMQ](#install-bullmq) - [Persistent connections | BullMQ](#persistent-connections-bullmq) - [Patterns | BullMQ](#patterns-bullmq) - [Prioritized intra-groups | BullMQ](#prioritized-intra-groups-bullmq) - [Groups | BullMQ](#groups-bullmq) - [Max group size | BullMQ](#max-group-size-bullmq) - [Process Step Jobs | BullMQ](#process-step-jobs-bullmq) - [Support | BullMQ](#support-bullmq) - [Debugging | BullMQ](#debugging-bullmq) - [Reusing Redis Connections | BullMQ](#reusing-redis-connections-bullmq) - [Message queue | BullMQ](#message-queue-bullmq) - [Changelog | BullMQ](#changelog-bullmq) - [Returning Job Completions | BullMQ](#returning-job-completions-bullmq) - [Redis cluster | BullMQ](#redis-cluster-bullmq) - [Pausing groups | BullMQ](#pausing-groups-bullmq) - [Producers | BullMQ](#producers-bullmq) - [v0 | BullMQ](#v0-bullmq) - [Custom backoff strategy | BullMQ](#custom-backoff-strategy-bullmq) - [v4 | BullMQ](#v4-bullmq) - [Local group concurrency | BullMQ](#local-group-concurrency-bullmq) - [Introduction | BullMQ](#introduction-bullmq) - [Manually fetching jobs | BullMQ](#manually-fetching-jobs-bullmq) - [v1 | BullMQ](#v1-bullmq) - [Quick Guide | BullMQ](#quick-guide-bullmq) - [v1 | BullMQ](#v1-bullmq) - [v5 | BullMQ](#v5-bullmq) - [v6 | BullMQ](#v6-bullmq) - [v2 | BullMQ](#v2-bullmq) - [Changelogs | BullMQ](#changelogs-bullmq) - [Email Protection | Cloudflare](#email-protection-cloudflare) --- # What is BullMQ | BullMQ BullMQ is a [Node.js](https://nodejs.org/) library that implements a fast and robust queue system built on top of [Redis](https://redis.io/) that helps in resolving many modern age micro-services architectures. The library is designed so that it will fulfill the following goals: * Exactly once queue semantics, i.e., attempts to deliver every message exactly one time, but it will deliver at least once in the worst case scenario\*. * Easy to scale horizontally. Add more workers for processing jobs in parallel. * Consistent. * High performant. Try to get the highest possible throughput from Redis by combining efficient .lua scripts and pipelining. View the repository, see open issues, and contribute back [on GitHub](https://github.com/taskforcesh/bullmq) ! [](https://docs.bullmq.io/#features) **Features** ------------------------------------------------------ If you are new to Message Queues, you may wonder why they are needed after all. Queues can solve many different problems in an elegant way, from smoothing out processing peaks to creating robust communication channels between micro-services or offloading heavy work from one server to many smaller workers, and many other use cases. Check the [Patterns](https://docs.bullmq.io/patterns/adding-bulks) section for getting some inspiration and information about best practices. * **Minimal CPU usage due to a polling-free design** * **Distributed job execution based on Redis** * **LIFO and FIFO jobs** * **Priorities** * **Delayed jobs** * **Scheduled and repeatable jobs according to cron specifications** * **Retries of failed jobs** * **Concurrency setting per worker** * **Threaded (sandboxed) processing functions** * **Automatic recovery from process crashes** * **Parent-Child dependencies** ### [](https://docs.bullmq.io/#used-by) Used by BullMQ is used by many organizations big and small, here are some notable examples: ![](https://docs.bullmq.io/~gitbook/image?url=https%3A%2F%2F1340146492-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-LUuDmt_xXMfG66Rn1GA%252Fuploads%252Fgit-blob-7eabf625a2932b111c2a5b7a651e2eaae6ad1b7a%252Fclipart1565701.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=88d0c630&sv=2) ![](https://docs.bullmq.io/~gitbook/image?url=https%3A%2F%2F1340146492-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-LUuDmt_xXMfG66Rn1GA%252Fuploads%252Fgit-blob-7ffcd73605e901be045c7cf43cb61eb9a7a00ee4%252Fwordmark-logo.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=b64bd75a&sv=2) ![](https://docs.bullmq.io/~gitbook/image?url=https%3A%2F%2F1340146492-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-LUuDmt_xXMfG66Rn1GA%252Fuploads%252Fgit-blob-33c0c65bd9d6936b8ac33f4be7cbba2814b83ead%252Fdatawrapper-logo.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=8ff2143f&sv=2) ![](https://docs.bullmq.io/~gitbook/image?url=https%3A%2F%2F1340146492-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-LUuDmt_xXMfG66Rn1GA%252Fuploads%252Fgit-blob-2038c9549e2aa39e1e8ce1a1017074c98953a7f4%252Fcurri-small.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=4934647b&sv=2) [NextQuick Start](https://docs.bullmq.io/readme-1) Last updated 1 year ago Was this helpful? --- # Quick Start | BullMQ [](https://docs.bullmq.io/readme-1#install) Install -------------------------------------------------------- Install using npm: Copy $ npm install bullmq Install using yarn: Copy $ yarn add bullmq BullMQ is written in TypeScript, and although it can be used in vanilla JavaScript, all examples in this guide will be written in TypeScript. Import into your project and add some jobs: Copy import { Queue } from 'bullmq'; const myQueue = new Queue('foo'); async function addJobs() { await myQueue.add('myJobName', { foo: 'bar' }); await myQueue.add('myJobName', { qux: 'baz' }); } await addJobs(); You need to have a Redis service running in your local computer to run these examples successfully. You can read more about Redis connections [here](https://docs.bullmq.io/guide/connections) . Jobs are added to the queue and can be processed at any time, with at least one Node.js process running a worker: Copy import { Worker } from 'bullmq'; import IORedis from 'ioredis'; const connection = new IORedis({ maxRetriesPerRequest: null }); const worker = new Worker( 'foo', async job => { // Will print { foo: 'bar'} for the first job // and { qux: 'baz' } for the second. console.log(job.data); }, { connection }, ); You can have as many worker processes as you want, BullMQ will distribute the jobs across your workers in a round robin fashion. You can listen to completed (or failed) jobs by attaching listeners to the workers: Copy worker.on('completed', job => { console.log(`${job.id} has completed!`); }); worker.on('failed', (job, err) => { console.log(`${job.id} has failed with ${err.message}`); }); There are many other events available, check the [Guide](https://github.com/taskforcesh/bullmq/blob/master/docs/gitbook/guide/events.md) or the [API reference](https://api.docs.bullmq.io/) for more information. Sometimes you need to listen to all the workers events in a given place, for this you need to use a special class [`QueueEvents`](https://api.docs.bullmq.io/classes/v5.QueueEvents.html) : Copy import { QueueEvents } from 'bullmq'; const queueEvents = new QueueEvents("my-queue-name"); queueEvents.on('waiting', ({ jobId }) => { console.log(`A job with ID ${jobId} is waiting`); }); queueEvents.on('active', ({ jobId, prev }) => { console.log(`Job ${jobId} is now active; previous status was ${prev}`); }); queueEvents.on('completed', ({ jobId, returnvalue }) => { console.log(`${jobId} has completed and returned ${returnvalue}`); }); queueEvents.on('failed', ({ jobId, failedReason }) => { console.log(`${jobId} has failed with reason ${failedReason}`); }); You may also access the timestamp of the event, which looks like "1580456039332-0". Copy import { QueueEvents } from 'bullmq'; const queueEvents = new QueueEvents("my-queue-name"); queueEvents.on('progress', ({ jobId, data }, timestamp) => { console.log(`${jobId} reported progress ${data} at ${timestamp}`); }); For performance reasons, the events emitted by a `QueueEvents` instance do not contain the `Job` instance, only the `jobId`. Use the [`Job.fromId`](https://api.docs.bullmq.io/classes/v5.Job.html#fromid) method if you need the `Job` instance. [PreviousWhat is BullMQ](https://docs.bullmq.io/) [NextChangelogs](https://docs.bullmq.io/changelog) Last updated 3 months ago Was this helpful? --- # Introduction | BullMQ BullMQ is based in 4 classes that together can be used to resolve many different problems. These classes are [_**Queue**_](https://api.docs.bullmq.io/classes/v5.Queue.html) , [_**Worker**_](https://api.docs.bullmq.io/classes/v5.Worker.html) , [_**QueueEvents**_](https://api.docs.bullmq.io/classes/v5.QueueEvents.html) and [_**FlowProducer**_](https://api.docs.bullmq.io/classes/v5.FlowProducer.html) . The first class you should know about is the _Queue_ class. This class represents a queue and can be used for adding _**jobs**_ to the queue as well as some other basic manipulation such as pausing, cleaning or getting data from the queue. Jobs in BullMQ are basically a user created data structure that can be stored in the queue. Jobs are processed by _**workers**_. A _Worker_ is the second class you should be aware about. Workers are instances capable of processing jobs. You can have many workers, either running in the same Node.js process, or in separate processes as well as in different machines. They will all consume jobs from the queue and mark the jobs as completed or failed. [Previousv1](https://docs.bullmq.io/changelog/changelog-v1) [NextConnections](https://docs.bullmq.io/guide/connections) Last updated 1 year ago Was this helpful? --- # Queues | BullMQ A Queue is nothing more than a list of jobs waiting to be processed. The jobs can be small, message like, so that the queue can be used as a message broker, or they can be larger long running jobs. Queues are controlled with the `Queue` class. As all classes in BullMQ, this is a lightweight class with a handful of methods that gives you control over the queue: Copy const queue = new Queue('Cars'); See [Connections](https://docs.bullmq.io/guide/connections) for details on how to pass Redis details to use by the queue. When you instantiate a Queue, BullMQ will just _upsert_ a small "meta-key", so if the queue existed before it will just pick it up and you can continue adding jobs to it. The most important method is probably the [_**add**_](https://api.docs.bullmq.io/classes/v5.Queue.html#add) method. This method allows you to add jobs to the queue in different fashions: Copy await queue.add('paint', { color: 'red' }); The code above will add a job named _paint_ to the queue, with payload `{ color: 'red' }`. This job will now be stored in Redis in a list waiting for some worker to pick it up and process it. Workers may not be running when you add the job, however as soon as one worker is connected to the queue it will pick the job and process it. When adding a job you can also specify an options object. This options object can dramatically change the behaviour of the added jobs. For example you can add a job that is delayed: Copy await queue.add('paint', { color: 'blue' }, { delay: 5000 }); The job will now wait **at** **least** 5 seconds before it is processed. Prior to BullMQ 2.0, in order for delay jobs to work you need to have at least one `QueueScheduler` somewhere in your infrastructure. Read more [here](https://docs.bullmq.io/guide/queuescheduler) . From BullMQ 2.0 and onwards, the `QueueScheduler` is not needed anymore. There are many other options available such as priorities, backoff settings, lifo behaviour, remove-on-complete policies, etc. Please check the remainder of this guide for more information regarding these options. [](https://docs.bullmq.io/guide/queues#read-more) Read more: ----------------------------------------------------------------- * 💡 [Queue API Reference](https://api.docs.bullmq.io/classes/v5.Queue.html) [PreviousConnections](https://docs.bullmq.io/guide/connections) [NextAuto-removal of jobs](https://docs.bullmq.io/guide/queues/auto-removal-of-jobs) Last updated 10 months ago Was this helpful? --- # Connections | BullMQ In order to start working with a Queue, a connection to a Redis instance is necessary. BullMQ uses the node module [ioredis](https://github.com/luin/ioredis) , and the options you pass to BullMQ are just passed to the constructor of ioredis. If you do not provide any options, it will default to port 6379 and localhost. Every class will consume at least one Redis connection, but it is also possible to reuse connections in some situations. For example, the _Queue_ and _Worker_ classes can accept an existing ioredis instance, and by that reusing that connection, however _QueueScheduler_ and _QueueEvents_ cannot do that because they require blocking connections to Redis, which makes it impossible to reuse them. Some examples: Copy import { Queue, Worker } from 'bullmq'; // Create a new connection in every instance const myQueue = new Queue('myqueue', { connection: { host: 'myredis.taskforce.run', port: 32856, }, }); const myWorker = new Worker('myqueue', async job => {}, { connection: { host: 'myredis.taskforce.run', port: 32856, }, }); Copy import { Queue } from 'bullmq'; import IORedis from 'ioredis'; const connection = new IORedis(); // Reuse the ioredis instance in 2 different producers const myFirstQueue = new Queue('myFirstQueue', { connection }); const mySecondQueue = new Queue('mySecondQueue', { connection }); Copy import { Worker } from 'bullmq'; import IORedis from 'ioredis'; const connection = new IORedis({ maxRetriesPerRequest: null }); // Reuse the ioredis instance in 2 different consumers const myFirstWorker = new Worker('myFirstWorker', async job => {}, { connection, }); const mySecondWorker = new Worker('mySecondWorker', async job => {}, { connection, }); Note that in the third example, even though the ioredis instance is being reused, the worker will create a duplicated connection that it needs internally to make blocking connections. Consult the [ioredis](https://github.com/luin/ioredis/blob/master/API.md) documentation to learn how to properly create an instance of `IORedis`. #### [](https://docs.bullmq.io/guide/connections#maxretriesperrequest) `maxRetriesPerRequest` This setting tells the ioredis client how many times to try a command that fails before throwing an error. So even though Redis is not reachable or offline, the command will be retried until this situation changes or the maximum number of attempts is reached. This guarantees that the workers will keep processing forever as long as there is a working connection. If you create a Redis client manually, BullMQ will throw an exception if this setting is not set to null when it is passed into worker instances. ### [](https://docs.bullmq.io/guide/connections#queue) Queue Also note that simple Queue instance used for managing the queue such as adding jobs, pausing, using getters, etc. usually has different requirements from the worker. For example, say that you are adding jobs to a queue as the result of a call to an HTTP endpoint - producer service. The caller of this endpoint cannot wait forever if the connection to Redis happens to be down when this call is made. Therefore the `maxRetriesPerRequest` setting should either be left at its default (which currently is 20) or set it to another value, maybe 1 so that the user gets an error quickly and can retry later. On the other hand, if you are adding jobs inside a Worker processor, this process is expected to happen in the background - consumer service. In this case you can share the same connection. For more details, refer to the [persistent connections](https://docs.bullmq.io/bull/patterns/persistent-connections) page. When using ioredis connections, be careful not to use the "keyPrefix" option in [ioredis](https://redis.github.io/ioredis/interfaces/CommonRedisOptions.html#keyPrefix) as this option is not compatible with BullMQ, which provides its own key prefixing mechanism by using [prefix](https://api.docs.bullmq.io/interfaces/v5.QueueOptions.html#prefix) option. If you can afford many connections, by all means just use them. Redis connections have quite low overhead, so you should not need to care about reusing connections unless your service provider imposes hard limitations. Make sure that your redis instance has the setting `maxmemory-policy=noeviction` in order to avoid automatic removal of keys which would cause unexpected errors in BullMQ [PreviousIntroduction](https://docs.bullmq.io/guide/introduction) [NextQueues](https://docs.bullmq.io/guide/queues) Last updated 10 months ago Was this helpful? --- # v2 | BullMQ [](https://docs.bullmq.io/changelog/changelog-v2#id-2.4.0-2022-10-24) [2.4.0](https://github.com/taskforcesh/bullmq/compare/v2.3.2...v2.4.0) (2022-10-24) --------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v2#features) Features * **flows:** allow parent on root jobs in addBulk method ([#1488](https://github.com/taskforcesh/bullmq/issues/1488) ) ref [#1480](https://github.com/taskforcesh/bullmq/issues/1480) ([92308e5](https://github.com/taskforcesh/bullmq/commit/92308e53acf14e0ce108d94ecd616633ac93e35d) ) ### [](https://docs.bullmq.io/changelog/changelog-v2#id-2.3.2-2022-10-18) [2.3.2](https://github.com/taskforcesh/bullmq/compare/v2.3.1...v2.3.2) (2022-10-18) #### [](https://docs.bullmq.io/changelog/changelog-v2#bug-fixes) Bug Fixes * **job:** send failed event when failParentOnFailure ([#1481](https://github.com/taskforcesh/bullmq/issues/1481) ) fixes [#1469](https://github.com/taskforcesh/bullmq/issues/1469) ([b20eb6f](https://github.com/taskforcesh/bullmq/commit/b20eb6f65c7e2c4593d5f9f4d4b940f780bf26d2) ) ### [](https://docs.bullmq.io/changelog/changelog-v2#id-2.3.1-2022-10-13) [2.3.1](https://github.com/taskforcesh/bullmq/compare/v2.3.0...v2.3.1) (2022-10-13) #### [](https://docs.bullmq.io/changelog/changelog-v2#bug-fixes-1) Bug Fixes * **redis:** replace throw exception by console.error ([fafa2f8](https://github.com/taskforcesh/bullmq/commit/fafa2f89e796796f950e6c4abbdda4d3d71ad1b0) ) [](https://docs.bullmq.io/changelog/changelog-v2#id-2.3.0-2022-10-13) [2.3.0](https://github.com/taskforcesh/bullmq/compare/v2.2.1...v2.3.0) (2022-10-13) --------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v2#features-1) Features * **redis-connection:** allow providing scripts for extension ([#1472](https://github.com/taskforcesh/bullmq/issues/1472) ) ([f193cfb](https://github.com/taskforcesh/bullmq/commit/f193cfb1830e127f9fd47a969baad30011a0e3a4) ) ### [](https://docs.bullmq.io/changelog/changelog-v2#id-2.2.1-2022-10-11) [2.2.1](https://github.com/taskforcesh/bullmq/compare/v2.2.0...v2.2.1) (2022-10-11) #### [](https://docs.bullmq.io/changelog/changelog-v2#performance-improvements) Performance Improvements * **scripts:** pre-build scripts ([#1441](https://github.com/taskforcesh/bullmq/issues/1441) ) ([7f72603](https://github.com/taskforcesh/bullmq/commit/7f72603d463f705d0617898cb221f832c49a4aa3) ) [](https://docs.bullmq.io/changelog/changelog-v2#id-2.2.0-2022-10-10) [2.2.0](https://github.com/taskforcesh/bullmq/compare/v2.1.3...v2.2.0) (2022-10-10) --------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v2#bug-fixes-2) Bug Fixes * **connection:** validate array of strings in Cluster ([#1468](https://github.com/taskforcesh/bullmq/issues/1468) ) fixes [#1467](https://github.com/taskforcesh/bullmq/issues/1467) ([8355182](https://github.com/taskforcesh/bullmq/commit/8355182a372b68ec62e9c3953bacbd69e0abfc74) ) #### [](https://docs.bullmq.io/changelog/changelog-v2#features-2) Features * **flow-producer:** allow parent opts in root job when adding a flow ([#1110](https://github.com/taskforcesh/bullmq/issues/1110) ) ref [#1097](https://github.com/taskforcesh/bullmq/issues/1097) ([3c3ac71](https://github.com/taskforcesh/bullmq/commit/3c3ac718ad84f6bd0cc1575013c948e767b46f38) ) ### [](https://docs.bullmq.io/changelog/changelog-v2#id-2.1.3-2022-09-30) [2.1.3](https://github.com/taskforcesh/bullmq/compare/v2.1.2...v2.1.3) (2022-09-30) #### [](https://docs.bullmq.io/changelog/changelog-v2#bug-fixes-3) Bug Fixes * **worker:** clear stalled jobs timer when closing worker ([1567a0d](https://github.com/taskforcesh/bullmq/commit/1567a0df0ca3c8d43a18990fe488888f4ff68040) ) ### [](https://docs.bullmq.io/changelog/changelog-v2#id-2.1.2-2022-09-29) [2.1.2](https://github.com/taskforcesh/bullmq/compare/v2.1.1...v2.1.2) (2022-09-29) #### [](https://docs.bullmq.io/changelog/changelog-v2#bug-fixes-4) Bug Fixes * **getters:** fix return type of getJobLogs ([d452927](https://github.com/taskforcesh/bullmq/commit/d4529278c59b2c94eee604c7d4455acc490679e9) ) ### [](https://docs.bullmq.io/changelog/changelog-v2#id-2.1.1-2022-09-28) [2.1.1](https://github.com/taskforcesh/bullmq/compare/v2.1.0...v2.1.1) (2022-09-28) #### [](https://docs.bullmq.io/changelog/changelog-v2#bug-fixes-5) Bug Fixes * **sandbox:** get open port using built-in module instead of get-port ([#1446](https://github.com/taskforcesh/bullmq/issues/1446) ) ([6db6288](https://github.com/taskforcesh/bullmq/commit/6db628868a9d64c5a3e47d1c9201017e6d05c1ae) ) [](https://docs.bullmq.io/changelog/changelog-v2#id-2.1.0-2022-09-23) [2.1.0](https://github.com/taskforcesh/bullmq/compare/v2.0.2...v2.1.0) (2022-09-23) --------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v2#features-3) Features * **job-options:** add failParentOnFailure option ([#1339](https://github.com/taskforcesh/bullmq/issues/1339) ) ([65e5c36](https://github.com/taskforcesh/bullmq/commit/65e5c3678771f26555c9128bdb908dd62e3584f9) ) ### [](https://docs.bullmq.io/changelog/changelog-v2#id-2.0.2-2022-09-22) [2.0.2](https://github.com/taskforcesh/bullmq/compare/v2.0.1...v2.0.2) (2022-09-22) #### [](https://docs.bullmq.io/changelog/changelog-v2#bug-fixes-6) Bug Fixes * **job:** update delay value when moving to wait ([#1436](https://github.com/taskforcesh/bullmq/issues/1436) ) ([9560915](https://github.com/taskforcesh/bullmq/commit/95609158c1800cf661f22ad7995541fb9474826a) ) ### [](https://docs.bullmq.io/changelog/changelog-v2#id-2.0.1-2022-09-21) [2.0.1](https://github.com/taskforcesh/bullmq/compare/v2.0.0...v2.0.1) (2022-09-21) #### [](https://docs.bullmq.io/changelog/changelog-v2#bug-fixes-7) Bug Fixes * **connection:** throw error when no noeviction policy ([3468390](https://github.com/taskforcesh/bullmq/commit/3468390dd6331291f4cf71a54c32028a06d1d99e) ) #### [](https://docs.bullmq.io/changelog/changelog-v2#performance-improvements-1) Performance Improvements * **events:** remove data and opts from added event ([e13d4b8](https://github.com/taskforcesh/bullmq/commit/e13d4b8e0c4f99203f4249ccc86e369d124ff483) ) [](https://docs.bullmq.io/changelog/changelog-v2#id-2.0.0-2022-09-21) [2.0.0](https://github.com/taskforcesh/bullmq/compare/v1.91.1...v2.0.0) (2022-09-21) ---------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v2#bug-fixes-8) Bug Fixes * **compat:** remove Queue3 class ([#1421](https://github.com/taskforcesh/bullmq/issues/1421) ) ([fc797f7](https://github.com/taskforcesh/bullmq/commit/fc797f7cd334c19a95cb1290ddb6611cd3417179) ) * **delayed:** promote delayed jobs instead of picking one by one ([1b938af](https://github.com/taskforcesh/bullmq/commit/1b938af75069d69772ddf2b03f95db7f53eada68) ) * **delayed:** remove marker when promoting delayed job ([1aea0dc](https://github.com/taskforcesh/bullmq/commit/1aea0dcc5fb29086cef3d0c432c387d6f8261963) ) * **getters:** compensate for "mark" job id ([231b9aa](https://github.com/taskforcesh/bullmq/commit/231b9aa0f4781e4493d3ea272c33b27c0b7dc0ab) ) * **sandbox:** remove progress method ([b43267b](https://github.com/taskforcesh/bullmq/commit/b43267be50f9eade8233500d189d46940a01cc29) ) * **stalled-jobs:** handle job id 0 ([829e6e0](https://github.com/taskforcesh/bullmq/commit/829e6e0252e78bf2cbc55ab1d3bd153faa0cee4c) ) * **worker:** do not allow stalledInterval to be less than zero ([831ffc5](https://github.com/taskforcesh/bullmq/commit/831ffc520ccd3c6ea63af6b04ddddc9f7829c667) ) * **workers:** use connection closing to determine closing status ([fe1d173](https://github.com/taskforcesh/bullmq/commit/fe1d17321f1eb49bd872c52965392add22729941) ) #### [](https://docs.bullmq.io/changelog/changelog-v2#features-4) Features * improve delayed jobs and remove QueueScheduler ([1f66e5a](https://github.com/taskforcesh/bullmq/commit/1f66e5a6c891d52e0671e58a685dbca511e45e7e) ) * move stalled jobs check and handling to Worker class from QueueScheduler ([13769cb](https://github.com/taskforcesh/bullmq/commit/13769cbe38ba22793cbc66e9706a6be28a7f1512) ) #### [](https://docs.bullmq.io/changelog/changelog-v2#breaking-changes) BREAKING CHANGES * **compat:** The compatibility class for Bullv3 is no longer available. * The QueueScheduler class is removed since it is not necessary anymore. Delayed jobs are now handled in a much simpler and robust way, without the need of a separate process. * Failed and stalled events are now produced by the Worker class instead of by the QueueScheduler. * The minimum Redis recommended version is 6.2.0. [Previousv3](https://docs.bullmq.io/changelog/changelog-v3) [Nextv1](https://docs.bullmq.io/changelog/changelog-v1) Last updated 1 year ago Was this helpful? --- # v4 | BullMQ ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.18.3-2025-07-19) [4.18.3](https://github.com/taskforcesh/bullmq/compare/v4.18.2...v4.18.3) (2025-07-19) #### [](https://docs.bullmq.io/changelog/changelog-v4#performance-improvements) Performance Improvements * **delayed:** add marker once when promoting delayed jobs ([#3312](https://github.com/taskforcesh/bullmq/issues/3312) ) ([c4a4ada](https://github.com/taskforcesh/bullmq/commit/c4a4adaf5db970d042efe0853145974c62aabdfe) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.18.2-2024-10-24) [4.18.2](https://github.com/taskforcesh/bullmq/compare/v4.18.1...v4.18.2) (2024-10-24) #### [](https://docs.bullmq.io/changelog/changelog-v4#bug-fixes) Bug Fixes * proper way to get version ([1a433d2](https://github.com/taskforcesh/bullmq/commit/1a433d2a17b58ba8d30f0ecf52d9091f8377978f) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.18.1-2024-10-19) [4.18.1](https://github.com/taskforcesh/bullmq/compare/v4.18.0...v4.18.1) (2024-10-19) #### [](https://docs.bullmq.io/changelog/changelog-v4#bug-fixes-1) Bug Fixes * use versions for lua commands ([b0a216d](https://github.com/taskforcesh/bullmq/commit/b0a216deacb0295e77b039ea67c100e57b045037) ) [](https://docs.bullmq.io/changelog/changelog-v4#id-4.18.0-2024-10-14) [4.18.0](https://github.com/taskforcesh/bullmq/compare/v4.17.0...v4.18.0) (2024-10-14) ------------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v4#features) Features * **queue:** add version support ([a600463](https://github.com/taskforcesh/bullmq/commit/a6004639612118ce5adc2a87f0402386ba8e1c2f) ) [](https://docs.bullmq.io/changelog/changelog-v4#id-4.17.0-2023-12-21) [4.17.0](https://github.com/taskforcesh/bullmq/compare/v4.16.0...v4.17.0) (2023-12-21) ------------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v4#features-1) Features * **sandbox:** support URL (local files) as processor file ([7eea670](https://github.com/taskforcesh/bullmq/commit/7eea6700b33bfd7f36b030b647b819a4c5fd9606) ) [](https://docs.bullmq.io/changelog/changelog-v4#id-4.16.0-2023-12-18) [4.16.0](https://github.com/taskforcesh/bullmq/compare/v4.15.4...v4.16.0) (2023-12-18) ------------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v4#features-2) Features * **queue:** add a paginated getDependencies ([#2327](https://github.com/taskforcesh/bullmq/issues/2327) ) ([c5b8ba3](https://github.com/taskforcesh/bullmq/commit/c5b8ba318b12a84a3a6a928345377fa0eaa08ee3) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.15.4-2023-12-14) [4.15.4](https://github.com/taskforcesh/bullmq/compare/v4.15.3...v4.15.4) (2023-12-14) #### [](https://docs.bullmq.io/changelog/changelog-v4#bug-fixes-2) Bug Fixes * **flows:** update constructor and methods to match queue base ([#2324](https://github.com/taskforcesh/bullmq/issues/2324) ) ([d6c2064](https://github.com/taskforcesh/bullmq/commit/d6c2064b1fdd88bd4cc61e049ce055ff620b0062) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.15.3-2023-12-13) [4.15.3](https://github.com/taskforcesh/bullmq/compare/v4.15.2...v4.15.3) (2023-12-13) #### [](https://docs.bullmq.io/changelog/changelog-v4#bug-fixes-3) Bug Fixes * **sandboxed:** better compatibility with esbuild ([8eaf955](https://github.com/taskforcesh/bullmq/commit/8eaf9550fe8b322df624893c507c55d2cce34b11) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.15.2-2023-12-07) [4.15.2](https://github.com/taskforcesh/bullmq/compare/v4.15.1...v4.15.2) (2023-12-07) #### [](https://docs.bullmq.io/changelog/changelog-v4#bug-fixes-4) Bug Fixes * **child-processor:** preserve dynamic imports in commonjs ([d97a5e0](https://github.com/taskforcesh/bullmq/commit/d97a5e06816cff04d86facdb8d32b512f29c6fb9) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.15.1-2023-12-06) [4.15.1](https://github.com/taskforcesh/bullmq/compare/v4.15.0...v4.15.1) (2023-12-06) #### [](https://docs.bullmq.io/changelog/changelog-v4#bug-fixes-5) Bug Fixes * **flows:** add meta key to queues created with flows ([272ec69](https://github.com/taskforcesh/bullmq/commit/272ec69557f601a138e1aaba739f7e7878d5344b) ) [](https://docs.bullmq.io/changelog/changelog-v4#id-4.15.0-2023-12-05) [4.15.0](https://github.com/taskforcesh/bullmq/compare/v4.14.4...v4.15.0) (2023-12-05) ------------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v4#features-3) Features * **sandboxes:** use the more compatible dynamic import instead of require ([6d2fe6e](https://github.com/taskforcesh/bullmq/commit/6d2fe6e7c0473b75aeb9a6d3080b0676f9521065) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.14.4-2023-11-28) [4.14.4](https://github.com/taskforcesh/bullmq/compare/v4.14.3...v4.14.4) (2023-11-28) #### [](https://docs.bullmq.io/changelog/changelog-v4#bug-fixes-6) Bug Fixes * **repeat-strategy:** add missing Promise return type ([#2301](https://github.com/taskforcesh/bullmq/issues/2301) ) ([6f8f534](https://github.com/taskforcesh/bullmq/commit/6f8f5342cc8aa03f596d9ed5b8831f96a1d4c736) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.14.3-2023-11-27) [4.14.3](https://github.com/taskforcesh/bullmq/compare/v4.14.2...v4.14.3) (2023-11-27) #### [](https://docs.bullmq.io/changelog/changelog-v4#bug-fixes-7) Bug Fixes * **update-progress:** remove old updateProgress script to prevent conflict ([#2298](https://github.com/taskforcesh/bullmq/issues/2298) ) (python) ([e65b819](https://github.com/taskforcesh/bullmq/commit/e65b819101f8e0e8fdef8c51cfdf9a52f5e73f13) ) * **worker:** get dirname by using module.filename ([#2296](https://github.com/taskforcesh/bullmq/issues/2296) ) fixes [#2288](https://github.com/taskforcesh/bullmq/issues/2288) ([6e4db5a](https://github.com/taskforcesh/bullmq/commit/6e4db5a3f3648c6a7e10991f2e18f3dab96fb1d7) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.14.2-2023-11-24) [4.14.2](https://github.com/taskforcesh/bullmq/compare/v4.14.1...v4.14.2) (2023-11-24) #### [](https://docs.bullmq.io/changelog/changelog-v4#bug-fixes-8) Bug Fixes * **worker:** should cap update progress events ([2cab9e9](https://github.com/taskforcesh/bullmq/commit/2cab9e94f65c7bdd053e3fb5944bcda6e3ebaa39) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.14.1-2023-11-23) [4.14.1](https://github.com/taskforcesh/bullmq/compare/v4.14.0...v4.14.1) (2023-11-23) #### [](https://docs.bullmq.io/changelog/changelog-v4#bug-fixes-9) Bug Fixes * **worker:** do not wait for slow jobs fixes [#2290](https://github.com/taskforcesh/bullmq/issues/2290) ([568d758](https://github.com/taskforcesh/bullmq/commit/568d7585edb1f2ef15991d4ae4a2425e6834046a) ) [](https://docs.bullmq.io/changelog/changelog-v4#id-4.14.0-2023-11-18) [4.14.0](https://github.com/taskforcesh/bullmq/compare/v4.13.3...v4.14.0) (2023-11-18) ------------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v4#features-4) Features * **worker:** better handling of concurrency when fetching jobs ([#2242](https://github.com/taskforcesh/bullmq/issues/2242) ) ([d2e2035](https://github.com/taskforcesh/bullmq/commit/d2e203588878ee64cb21e67141f73b32867dfb40) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.13.3-2023-11-16) [4.13.3](https://github.com/taskforcesh/bullmq/compare/v4.13.2...v4.13.3) (2023-11-16) #### [](https://docs.bullmq.io/changelog/changelog-v4#bug-fixes-10) Bug Fixes * **utils:** use EventEmitter as a type instead of a namespace ([#2283](https://github.com/taskforcesh/bullmq/issues/2283) ) ([41c9d1d](https://github.com/taskforcesh/bullmq/commit/41c9d1d05eedc7351272708e667e8d65eb6773fc) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.13.2-2023-11-09) [4.13.2](https://github.com/taskforcesh/bullmq/compare/v4.13.1...v4.13.2) (2023-11-09) #### [](https://docs.bullmq.io/changelog/changelog-v4#bug-fixes-11) Bug Fixes * **job:** set delay value on current job instance when it is retried ([#2266](https://github.com/taskforcesh/bullmq/issues/2266) ) (python) ([76e075f](https://github.com/taskforcesh/bullmq/commit/76e075f54d5745b6cec3cb11305bf3110d963eae) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.13.1-2023-11-08) [4.13.1](https://github.com/taskforcesh/bullmq/compare/v4.13.0...v4.13.1) (2023-11-08) #### [](https://docs.bullmq.io/changelog/changelog-v4#bug-fixes-12) Bug Fixes * **connection:** better handling of attached listeners ([02474ad](https://github.com/taskforcesh/bullmq/commit/02474ad59a7b340d7bb2a7415ae7a88e14200398) ) * **connection:** move redis instance check to queue base ([13a339a](https://github.com/taskforcesh/bullmq/commit/13a339a730f46ff22acdd4a046e0d9c4b7d88679) ) [](https://docs.bullmq.io/changelog/changelog-v4#id-4.13.0-2023-11-05) [4.13.0](https://github.com/taskforcesh/bullmq/compare/v4.12.10...v4.13.0) (2023-11-05) -------------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v4#features-5) Features * **queue:** improve clean to work iteratively ([#2260](https://github.com/taskforcesh/bullmq/issues/2260) ) ([0cfa66f](https://github.com/taskforcesh/bullmq/commit/0cfa66fd0fa0dba9b3941f183cf6f06d8a4f281d) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.12.10-2023-11-05) [4.12.10](https://github.com/taskforcesh/bullmq/compare/v4.12.9...v4.12.10) (2023-11-05) #### [](https://docs.bullmq.io/changelog/changelog-v4#bug-fixes-13) Bug Fixes * update delay job property when moving to delayed set ([#2261](https://github.com/taskforcesh/bullmq/issues/2261) ) ([69ece08](https://github.com/taskforcesh/bullmq/commit/69ece08babd7716c14c38c3dd50630b44c7c1897) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.12.9-2023-11-05) [4.12.9](https://github.com/taskforcesh/bullmq/compare/v4.12.8...v4.12.9) (2023-11-05) #### [](https://docs.bullmq.io/changelog/changelog-v4#bug-fixes-14) Bug Fixes * **add-job:** trim events when waiting-children event is published ([#2262](https://github.com/taskforcesh/bullmq/issues/2262) ) (python) ([198bf05](https://github.com/taskforcesh/bullmq/commit/198bf05fa5a4e1ce50081296033a2e0f26ece498) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.12.8-2023-11-03) [4.12.8](https://github.com/taskforcesh/bullmq/compare/v4.12.7...v4.12.8) (2023-11-03) #### [](https://docs.bullmq.io/changelog/changelog-v4#bug-fixes-15) Bug Fixes * **worker:** keep extending locks while closing workers ([#2259](https://github.com/taskforcesh/bullmq/issues/2259) ) ([c4d12ea](https://github.com/taskforcesh/bullmq/commit/c4d12ea3a9837ffd7f58e2134796137c4181c3de) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.12.7-2023-10-29) [4.12.7](https://github.com/taskforcesh/bullmq/compare/v4.12.6...v4.12.7) (2023-10-29) #### [](https://docs.bullmq.io/changelog/changelog-v4#performance-improvements-1) Performance Improvements * **redis-connection:** check redis version greater or equal than v6 only once ([#2252](https://github.com/taskforcesh/bullmq/issues/2252) ) ([a09b15a](https://github.com/taskforcesh/bullmq/commit/a09b15af0d5dedfa83bce7130ee9094f3fb69e10) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.12.6-2023-10-26) [4.12.6](https://github.com/taskforcesh/bullmq/compare/v4.12.5...v4.12.6) (2023-10-26) #### [](https://docs.bullmq.io/changelog/changelog-v4#bug-fixes-16) Bug Fixes * **sandbox:** do not return empty object result when it is undefined ([#2247](https://github.com/taskforcesh/bullmq/issues/2247) ) ([308db7f](https://github.com/taskforcesh/bullmq/commit/308db7f58758a72b8abb272da8e92509813a2178) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.12.5-2023-10-18) [4.12.5](https://github.com/taskforcesh/bullmq/compare/v4.12.4...v4.12.5) (2023-10-18) #### [](https://docs.bullmq.io/changelog/changelog-v4#performance-improvements-2) Performance Improvements * **events:** trim events when removing jobs ([#2235](https://github.com/taskforcesh/bullmq/issues/2235) ) (python) ([889815c](https://github.com/taskforcesh/bullmq/commit/889815c412666e5fad8f32d2e3a2d41cf650f001) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.12.4-2023-10-13) [4.12.4](https://github.com/taskforcesh/bullmq/compare/v4.12.3...v4.12.4) (2023-10-13) #### [](https://docs.bullmq.io/changelog/changelog-v4#bug-fixes-17) Bug Fixes * **events:** do not publish removed event on non-existent jobs ([#2227](https://github.com/taskforcesh/bullmq/issues/2227) ) ([c134606](https://github.com/taskforcesh/bullmq/commit/c1346064c6cd9f93c59b184f150eac11d51c91b4) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.12.3-2023-10-10) [4.12.3](https://github.com/taskforcesh/bullmq/compare/v4.12.2...v4.12.3) (2023-10-10) #### [](https://docs.bullmq.io/changelog/changelog-v4#bug-fixes-18) Bug Fixes * **events:** trim events when retrying a job ([#2224](https://github.com/taskforcesh/bullmq/issues/2224) ) ([1986b05](https://github.com/taskforcesh/bullmq/commit/1986b05ac03fe4ee48861aa60caadcc9df8170a6) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.12.2-2023-10-05) [4.12.2](https://github.com/taskforcesh/bullmq/compare/v4.12.1...v4.12.2) (2023-10-05) #### [](https://docs.bullmq.io/changelog/changelog-v4#bug-fixes-19) Bug Fixes * **sandbox:** update progress value on job instance ([#2214](https://github.com/taskforcesh/bullmq/issues/2214) ) fixes [#2213](https://github.com/taskforcesh/bullmq/issues/2213) ([3d0f36a](https://github.com/taskforcesh/bullmq/commit/3d0f36a134b7f5c6b6de26967c9d71bcfb346e72) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.12.1-2023-10-04) [4.12.1](https://github.com/taskforcesh/bullmq/compare/v4.12.0...v4.12.1) (2023-10-04) #### [](https://docs.bullmq.io/changelog/changelog-v4#bug-fixes-20) Bug Fixes * **delayed:** trim events when moving jobs to delayed (python) ([#2211](https://github.com/taskforcesh/bullmq/issues/2211) ) ([eca8c2d](https://github.com/taskforcesh/bullmq/commit/eca8c2d4dfeafbd8ac36a49764dbd4897303628c) ) [](https://docs.bullmq.io/changelog/changelog-v4#id-4.12.0-2023-09-29) [4.12.0](https://github.com/taskforcesh/bullmq/compare/v4.11.4...v4.12.0) (2023-09-29) ------------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v4#features-6) Features * expose addJobLog and updateJobProgress to the Queue instance ([#2202](https://github.com/taskforcesh/bullmq/issues/2202) ) ([2056939](https://github.com/taskforcesh/bullmq/commit/205693907a4d6c2da9bd0690fb552b1d1e369c08) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.11.4-2023-09-22) [4.11.4](https://github.com/taskforcesh/bullmq/compare/v4.11.3...v4.11.4) (2023-09-22) #### [](https://docs.bullmq.io/changelog/changelog-v4#bug-fixes-21) Bug Fixes * **queue:** batched unpack now uses range ([#2188](https://github.com/taskforcesh/bullmq/issues/2188) ) ([b5e97f4](https://github.com/taskforcesh/bullmq/commit/b5e97f420bc0c4bc82772f3e87883ee522be43d9) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.11.3-2023-09-22) [4.11.3](https://github.com/taskforcesh/bullmq/compare/v4.11.2...v4.11.3) (2023-09-22) #### [](https://docs.bullmq.io/changelog/changelog-v4#bug-fixes-22) Bug Fixes * **worker:** forward skipVersionCheck to blockingConnection ([#2189](https://github.com/taskforcesh/bullmq/issues/2189) ) ref [#2149](https://github.com/taskforcesh/bullmq/issues/2149) ([c8aa9a3](https://github.com/taskforcesh/bullmq/commit/c8aa9a36224cba8ecb19af1bf652f4f1c4c20d40) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.11.2-2023-09-20) [4.11.2](https://github.com/taskforcesh/bullmq/compare/v4.11.1...v4.11.2) (2023-09-20) #### [](https://docs.bullmq.io/changelog/changelog-v4#bug-fixes-23) Bug Fixes * **worker:** throw exception with NaN as concurrency ([#2184](https://github.com/taskforcesh/bullmq/issues/2184) ) ([f36ac8b](https://github.com/taskforcesh/bullmq/commit/f36ac8b61dcd4bb3d9e283278310cd50cfc83fae) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.11.1-2023-09-20) [4.11.1](https://github.com/taskforcesh/bullmq/compare/v4.11.0...v4.11.1) (2023-09-20) #### [](https://docs.bullmq.io/changelog/changelog-v4#bug-fixes-24) Bug Fixes * **queue:** differentiate score purpose per state in clean method ([#2133](https://github.com/taskforcesh/bullmq/issues/2133) ) fixes [#2124](https://github.com/taskforcesh/bullmq/issues/2124) ([862f10b](https://github.com/taskforcesh/bullmq/commit/862f10b586276314d9bffff2a5e6caf939399f7e) ) [](https://docs.bullmq.io/changelog/changelog-v4#id-4.11.0-2023-09-16) [4.11.0](https://github.com/taskforcesh/bullmq/compare/v4.10.0...v4.11.0) (2023-09-16) ------------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v4#features-7) Features * **sandbox:** convert wrapJob method as protected for extension ([#2182](https://github.com/taskforcesh/bullmq/issues/2182) ) ([1494b55](https://github.com/taskforcesh/bullmq/commit/1494b5566573356e0248b4a5cab48ae21d82f1da) ) [](https://docs.bullmq.io/changelog/changelog-v4#id-4.10.0-2023-09-12) [4.10.0](https://github.com/taskforcesh/bullmq/compare/v4.9.0...v4.10.0) (2023-09-12) ------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/changelog/changelog-v4#bug-fixes-25) Bug Fixes * **move-to-finished:** consider addition of prioritized jobs when processing last active job ([#2176](https://github.com/taskforcesh/bullmq/issues/2176) ) (python) ([4b01f35](https://github.com/taskforcesh/bullmq/commit/4b01f359c290cfc62ea74ff3ab0b43ccc6956a02) ) * **remove:** change error message when job is locked (python) ([#2175](https://github.com/taskforcesh/bullmq/issues/2175) ) ([2f5628f](https://github.com/taskforcesh/bullmq/commit/2f5628feffab66cdcc78abf4d7bb608bdcaa65bb) ) [](https://docs.bullmq.io/changelog/changelog-v4#id-4.9.0-2023-09-05) [4.9.0](https://github.com/taskforcesh/bullmq/compare/v4.8.0...v4.9.0) (2023-09-05) --------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v4#features-8) Features * **connection:** provide skipVersionCheck option for shared connections ([#2149](https://github.com/taskforcesh/bullmq/issues/2149) ) ref [#2148](https://github.com/taskforcesh/bullmq/issues/2148) ([914820f](https://github.com/taskforcesh/bullmq/commit/914820f720cbc48b49f4bd1c46d148eb2bb5b79c) ) [](https://docs.bullmq.io/changelog/changelog-v4#id-4.8.0-2023-08-20) [4.8.0](https://github.com/taskforcesh/bullmq/compare/v4.7.4...v4.8.0) (2023-08-20) --------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v4#features-9) Features * **sandbox:** emulate moveToDelayed method ([#2122](https://github.com/taskforcesh/bullmq/issues/2122) ) ref [#2118](https://github.com/taskforcesh/bullmq/issues/2118) ([4c4559b](https://github.com/taskforcesh/bullmq/commit/4c4559b3c678313b3727c9781a6d3f963bcfda4e) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.7.4-2023-08-19) [4.7.4](https://github.com/taskforcesh/bullmq/compare/v4.7.3...v4.7.4) (2023-08-19) #### [](https://docs.bullmq.io/changelog/changelog-v4#bug-fixes-26) Bug Fixes * **sandbox:** ignore extra params on processor ([#2142](https://github.com/taskforcesh/bullmq/issues/2142) ) ([3602c20](https://github.com/taskforcesh/bullmq/commit/3602c20ab80cbe0a0d3de66210a01ad119e1090b) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.7.3-2023-08-17) [4.7.3](https://github.com/taskforcesh/bullmq/compare/v4.7.2...v4.7.3) (2023-08-17) #### [](https://docs.bullmq.io/changelog/changelog-v4#bug-fixes-27) Bug Fixes * **worker:** abort rate-limit delay when closing worker ([264a81c](https://github.com/taskforcesh/bullmq/commit/264a81ca5f4e4f88c361d507312324b5f6c3225c) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.7.2-2023-08-12) [4.7.2](https://github.com/taskforcesh/bullmq/compare/v4.7.1...v4.7.2) (2023-08-12) #### [](https://docs.bullmq.io/changelog/changelog-v4#bug-fixes-28) Bug Fixes * **queue:** throw error when name is not provided ([#2123](https://github.com/taskforcesh/bullmq/issues/2123) ) ([78fb0e2](https://github.com/taskforcesh/bullmq/commit/78fb0e2a93cfa59a43a0fb337f857e78f1c6fcf4) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.7.1-2023-08-10) [4.7.1](https://github.com/taskforcesh/bullmq/compare/v4.7.0...v4.7.1) (2023-08-10) #### [](https://docs.bullmq.io/changelog/changelog-v4#performance-improvements-3) Performance Improvements * **rate-limit:** get pttl only if needed ([#2129](https://github.com/taskforcesh/bullmq/issues/2129) ) ([12ce2f3](https://github.com/taskforcesh/bullmq/commit/12ce2f3746626a81ea961961bb1a629077eed68a) ) [](https://docs.bullmq.io/changelog/changelog-v4#id-4.7.0-2023-08-03) [4.7.0](https://github.com/taskforcesh/bullmq/compare/v4.6.3...v4.7.0) (2023-08-03) --------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v4#features-10) Features * **queue:** add getRateLimitTtl method ([#2105](https://github.com/taskforcesh/bullmq/issues/2105) ) ([7426c64](https://github.com/taskforcesh/bullmq/commit/7426c64b109f1beacf742d57a987282597385469) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.6.3-2023-07-28) [4.6.3](https://github.com/taskforcesh/bullmq/compare/v4.6.2...v4.6.3) (2023-07-28) #### [](https://docs.bullmq.io/changelog/changelog-v4#performance-improvements-4) Performance Improvements * **job:** generate priority limit constant once ([#2102](https://github.com/taskforcesh/bullmq/issues/2102) ) ([8880f9f](https://github.com/taskforcesh/bullmq/commit/8880f9f2983282d343d603a89abe5e1e6bff78e5) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.6.2-2023-07-26) [4.6.2](https://github.com/taskforcesh/bullmq/compare/v4.6.0...v4.6.2) (2023-07-26) #### [](https://docs.bullmq.io/changelog/changelog-v4#performance-improvements-5) Performance Improvements * **retry:** compare prev state instead of regex expression ([#2099](https://github.com/taskforcesh/bullmq/issues/2099) ) ([c141283](https://github.com/taskforcesh/bullmq/commit/c1412831903d1fae0955af097e0be049024839fe) ) [](https://docs.bullmq.io/changelog/changelog-v4#id-4.6.0-2023-07-19) [4.6.0](https://github.com/taskforcesh/bullmq/compare/v4.5.0...v4.6.0) (2023-07-19) --------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v4#features-11) Features * **queue:** add promoteJobs to promote all delayed jobs ([6074592](https://github.com/taskforcesh/bullmq/commit/6074592574256ec4b1c340126288e803e56b1a64) ) [](https://docs.bullmq.io/changelog/changelog-v4#id-4.5.0-2023-07-18) [4.5.0](https://github.com/taskforcesh/bullmq/compare/v4.4.0...v4.5.0) (2023-07-18) --------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v4#features-12) Features * **job:** add option for removing children in remove method (python) ([#2064](https://github.com/taskforcesh/bullmq/issues/2064) ) ([841dc87](https://github.com/taskforcesh/bullmq/commit/841dc87a689897df81438ad1f43e45a4da77c388) ) [](https://docs.bullmq.io/changelog/changelog-v4#id-4.4.0-2023-07-17) [4.4.0](https://github.com/taskforcesh/bullmq/compare/v4.3.0...v4.4.0) (2023-07-17) --------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v4#features-13) Features * **job:** add removeDependencyOnFailure option ([#1953](https://github.com/taskforcesh/bullmq/issues/1953) ) ([ffd49e2](https://github.com/taskforcesh/bullmq/commit/ffd49e289c57252487200d47b92193228ae7451f) ) [](https://docs.bullmq.io/changelog/changelog-v4#id-4.3.0-2023-07-14) [4.3.0](https://github.com/taskforcesh/bullmq/compare/v4.2.1...v4.3.0) (2023-07-14) --------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v4#features-14) Features * **worker:** add id as part of token ([#2061](https://github.com/taskforcesh/bullmq/issues/2061) ) ([e255356](https://github.com/taskforcesh/bullmq/commit/e2553562271e1e4143a8fef616349bb30de4899d) ) ### [](https://docs.bullmq.io/changelog/changelog-v4#id-4.2.1-2023-07-10) [4.2.1](https://github.com/taskforcesh/bullmq/compare/v4.2.0...v4.2.1) (2023-07-10) #### [](https://docs.bullmq.io/changelog/changelog-v4#bug-fixes-29) Bug Fixes * **flow:** emit delayed event when parent is moved to delayed ([#2055](https://github.com/taskforcesh/bullmq/issues/2055) ) ([f419ff1](https://github.com/taskforcesh/bullmq/commit/f419ff1ec5cb34986fe4b79402c727a6487e949c) ) [](https://docs.bullmq.io/changelog/changelog-v4#id-4.2.0-2023-07-03) [4.2.0](https://github.com/taskforcesh/bullmq/compare/v4.1.0...v4.2.0) (2023-07-03) --------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v4#features-15) Features * **common:** add option to change repeatable jobs redis key hash algorithm ([#2023](https://github.com/taskforcesh/bullmq/issues/2023) ) ([ca17364](https://github.com/taskforcesh/bullmq/commit/ca17364cc2a52f6577fb66f09ec3168bbf9f1e07) ) [](https://docs.bullmq.io/changelog/changelog-v4#id-4.1.0-2023-06-23) [4.1.0](https://github.com/taskforcesh/bullmq/compare/v4.0.0...v4.1.0) (2023-06-23) --------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v4#features-16) Features * **queue:** add getPrioritized and getPrioritizedCount methods ([#2005](https://github.com/taskforcesh/bullmq/issues/2005) ) ([7363abe](https://github.com/taskforcesh/bullmq/commit/7363abebce6e3bcf067fc7c220d845807ebb1489) ) [](https://docs.bullmq.io/changelog/changelog-v4#id-4.0.0-2023-06-21) [4.0.0](https://github.com/taskforcesh/bullmq/compare/v3.15.8...v4.0.0) (2023-06-21) ---------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v4#features-17) Features * **queue:** add removeDeprecatedPriorityKey method #### [](https://docs.bullmq.io/changelog/changelog-v4#performance-improvements-6) Performance Improvements * **priority:** add prioritized as a new state ([#1984](https://github.com/taskforcesh/bullmq/issues/1984) ) (python) ([42a890a](https://github.com/taskforcesh/bullmq/commit/42a890a2bfe45b29348030f886766400f5d41aa3) ) #### [](https://docs.bullmq.io/changelog/changelog-v4#breaking-changes) BREAKING CHANGES * **priority:** priority is separeted in its own zset, no duplication needed * **job:** change job method name update to updateData ref [faster priority jobs](https://bullmq.io/news/062123/faster-priority-jobs/) [PreviousChangelogs](https://docs.bullmq.io/changelog) [Nextv3](https://docs.bullmq.io/changelog/changelog-v3) Last updated 3 months ago Was this helpful? --- # Meta | BullMQ The meta data of any queue can be retrieved in the following way: Copy import { Queue } from 'bullmq'; const { concurrency, max, duration, maxLenEvents, paused, version } = await queue.getMeta(); [](https://docs.bullmq.io/guide/queues/meta#read-more) Read more: ---------------------------------------------------------------------- * 💡 [Get Meta API Reference](https://api.docs.bullmq.io/classes/v5.Queue.html#getmeta) * 💡 [Global Concurrency](https://docs.bullmq.io/guide/queues/global-concurrency) * 💡 [Global Rate Limit](https://docs.bullmq.io/guide/queues/global-rate-limit) [PreviousGlobal Rate Limit](https://docs.bullmq.io/guide/queues/global-rate-limit) [NextRemoving Jobs](https://docs.bullmq.io/guide/queues/removing-jobs) Last updated 27 days ago Was this helpful? --- # Adding jobs in bulk | BullMQ Sometimes it is necessary to add many jobs atomically. For example, there could be a requirement that all the jobs must be placed in the queue or none of them. Also, adding jobs in bulk can be faster since it reduces the number of roundtrips to Redis: TypeScript Python Copy import { Queue } from 'bullmq'; const queue = new Queue('paint'); const name = 'jobName'; const jobs = await queue.addBulk([\ { name, data: { paint: 'car' } },\ { name, data: { paint: 'house' } },\ { name, data: { paint: 'boat' } },\ ]); Copy from bullmq import Queue queue = Queue("paint") jobs = await queue.addBulk([\ { "name": "jobName", "data": { "paint": "car" } },\ { "name": "jobName", "data": { "paint": "house" } },\ { "name": "jobName", "data": { "paint": "boat" } }\ ]) This call can only succeed or fail, and all or none of the jobs will be added. [](https://docs.bullmq.io/guide/queues/adding-bulks#read-more) Read more: ------------------------------------------------------------------------------ * 💡 [Add Bulk API Reference](https://api.docs.bullmq.io/classes/v5.Queue.html#addbulk) [PreviousAuto-removal of jobs](https://docs.bullmq.io/guide/queues/auto-removal-of-jobs) [NextGlobal Concurrency](https://docs.bullmq.io/guide/queues/global-concurrency) Last updated 3 months ago Was this helpful? --- # Auto-removal of jobs | BullMQ By default, when your queue jobs are completed (or failed), they are stored in two special sets, the "completed" and the "failed" set. This is useful so that you can examine the results of your jobs, particularly in the early stages of development. However, as the solution reaches a production-grade level, we usually need to restrict the number of finished jobs to be kept, so that we do not fill Redis with data that is not particularly useful. BullMQ supports different strategies for auto-removing finalized jobs. These strategies are configured on the Job's options [`removeOnComplete`](https://api.docs.bullmq.io/interfaces/v5.BaseJobOptions.html#removeoncomplete) and [`removeOnFail`](https://api.docs.bullmq.io/interfaces/v5.BaseJobOptions.html#removeonfail) . ### [](https://docs.bullmq.io/guide/queues/auto-removal-of-jobs#remove-all-finalized-jobs) Remove all finalized jobs The simplest option is to set `removeOnComplete`/`removeOnFail` to `true`, in this case, all jobs will be removed automatically as soon as they are finalized: Copy await myQueue.add( 'test', { foo: 'bar' }, { removeOnComplete: true, removeOnFail: true }, ); Jobs will be deleted regardless of their names. ### [](https://docs.bullmq.io/guide/queues/auto-removal-of-jobs#keep-a-certain-number-of-jobs) Keep a certain number of jobs It is also possible to specify a maximum number of jobs to keep. A good practice is to keep a handful of completed jobs and a much larger value of failed jobs: Copy await myQueue.add( 'test', { foo: 'bar' }, { removeOnComplete: 1000, removeOnFail: 5000 }, ); ### [](https://docs.bullmq.io/guide/queues/auto-removal-of-jobs#keep-jobs-based-on-their-age) Keep jobs based on their age Another possibility is to keep jobs up to a certain age. The `removeOn` option accepts a [`KeepJobs`](https://api.docs.bullmq.io/interfaces/v5.KeepJobs.html) object, that includes an `age` and a `count` fields. The `age` is used to specify how old jobs to keep (in seconds), and the `count` can be used to limit the total amount to keep. The `count` option is useful in cases we get an unexpected amount of jobs in a very short time, in this case we may just want to limit to a certain amount to avoid running out of memory. Copy await myQueue.add( 'test', { foo: 'bar' }, { removeOnComplete: { age: 3600, // keep up to 1 hour count: 1000, // keep up to 1000 jobs }, removeOnFail: { age: 24 * 3600, // keep up to 24 hours }, }, ); The auto removal of jobs works lazily. This means that jobs are not removed unless a new job completes or fails, since that is when the auto-removal takes place. ### [](https://docs.bullmq.io/guide/queues/auto-removal-of-jobs#what-about-idempotence) What about idempotence? One of the strategies to implement idempotence with BullMQ is to use unique job ids. When you add a job with an id that exists already in the queue, the new job is ignored and a **duplicated** event is triggered. It is important to keep this in mind when activating auto removal of jobs, since a job that has been removed will not be considered part of the queue anymore, and will not affect any future jobs that could have the same Id. [](https://docs.bullmq.io/guide/queues/auto-removal-of-jobs#read-more) Read more: -------------------------------------------------------------------------------------- * 💡 [Duplicated Event Reference](https://api.docs.bullmq.io/interfaces/v5.QueueEventsListener.html#duplicated) [PreviousQueues](https://docs.bullmq.io/guide/queues) [NextAdding jobs in bulk](https://docs.bullmq.io/guide/queues/adding-bulks) Last updated 3 months ago Was this helpful? --- # Concurrency | BullMQ There are basically two ways to achieve concurrency with BullMQ using Worker instances. You can run a worker with a concurrency factor larger than 1 (which is the default value), or you can run several workers in different node processes. #### [](https://docs.bullmq.io/guide/workers/concurrency#local-concurrency-factor) Local Concurrency factor The local concurrency factor is a worker option that determines how many jobs are allowed to be processed concurrently (or in parallel if using sandboxed processors) for that instance. This means that the same worker is able to process several jobs at the same time and still provide guarantees such as "at-least-once" and order of processing. Copy import { Worker, Job } from 'bullmq'; const worker = new Worker( queueName, async (job: Job) => { // Do something with job return 'some value'; }, { concurrency: 50 }, ); Note that the concurrency is only possible when workers perform asynchronous operations such as a call to a database or a external HTTP service, as this is how node supports concurrency natively. If your workers are very CPU intensive it is better to use [Sandboxed processors](https://docs.bullmq.io/guide/workers/sandboxed-processors) . In addition, you can update the concurrency value as you need while your worker is running: Copy worker.concurrency = 5; #### [](https://docs.bullmq.io/guide/workers/concurrency#multiple-workers) Multiple workers The other way to achieve concurrency is to provide multiple workers. This is the recommended way to setup BullMQ as besides providing concurrency it also provides higher availability for your workers. You can easily launch a fleet of workers running in many different machines in order to execute the jobs in parallel in a predictable and robust way. If you need to achieve a global concurrency of at most 1 job at a time, refer to [Global concurrency](https://docs.bullmq.io/guide/queues/global-concurrency) . You can still (and it is a perfectly good practice to) choose a high concurrency factor for every worker, so that the resources of every machine where the worker is running are used more efficiently. [PreviousAuto-removal of jobs](https://docs.bullmq.io/guide/workers/auto-removal-of-jobs) [NextGraceful shutdown](https://docs.bullmq.io/guide/workers/graceful-shutdown) Last updated 7 months ago Was this helpful? --- # Stalled Jobs | BullMQ Due to the nature of NodeJS, which is (in general) single threaded and consists of an event loop to handle the asynchronous operations, the process function needs to be written carefully so that the CPU is not occupied for a long time. When a job reaches a worker and starts to be processed, BullMQ will place a lock on this job to protect the job from being modified by any other client or worker. At the same time, the worker needs to periodically notify BullMQ that it is still working on the job. This period is configured with the `stalledInterval` setting, which normally you should not need to modify. However if the CPU is very busy (due to the process being very CPU intensive), the worker may not have time to renew the lock and tell the queue that it is still working on the job, which is likely to result in the job being marked as _stalled_. A stalled job is moved back to the waiting status and will be processed again by another worker, or if it has reached its maximum number of stalls, it will be moved to the _failed_ set. Therefore, it is very important to make sure the workers return control to the NodeJS event loop often enough to avoid this kind of problem. [PreviousCancelling jobs](https://docs.bullmq.io/guide/workers/cancelling-jobs) [NextSandboxed processors](https://docs.bullmq.io/guide/workers/sandboxed-processors) Last updated 1 year ago Was this helpful? --- # Global Rate Limit | BullMQ The global rate limit config is a queue option that determines how many jobs are allowed to be processed in a specific period of time. Copy import { Queue } from 'bullmq'; // 1 job per second await queue.setGlobalRateLimit(1, 1000); In order to get these values: Copy const { max, duration } = await queue.getGlobalRateLimit(); And in order to get current ttl: Copy const ttl = await queue.getRateLimitTtl(); Note that if you choose a rate limit level in your workers, it won't override the global one. ### [](https://docs.bullmq.io/guide/queues/global-rate-limit#remove-global-rate-limit) Remove Global Rate Limit It can be done using the following method: Copy await queue.removeGlobalRateLimit(); [](https://docs.bullmq.io/guide/queues/global-rate-limit#read-more) Read more: ----------------------------------------------------------------------------------- * 💡 [Set Global Rate Limit API Reference](https://api.docs.bullmq.io/classes/v5.Queue.html#setglobalratelimit) * 💡 [Get Global Rate Limit API Reference](https://api.docs.bullmq.io/classes/v5.Queue.html#getglobalratelimit) * 💡 [Get Rate Limit Ttl API Reference](https://api.docs.bullmq.io/classes/v5.Queue.html#getratelimitttl) * 💡 [Remove Global Rate Limit API Reference](https://api.docs.bullmq.io/classes/v5.Queue.html#removeglobalratelimit) [PreviousGlobal Concurrency](https://docs.bullmq.io/guide/queues/global-concurrency) [NextMeta](https://docs.bullmq.io/guide/queues/meta) Last updated 1 month ago Was this helpful? --- # Global Concurrency | BullMQ The global concurrency factor is a queue option that determines how many jobs are allowed to be processed in parallel across all your worker instances. Copy import { Queue } from 'bullmq'; await queue.setGlobalConcurrency(4); And in order to get this value: Copy const globalConcurrency = await queue.getGlobalConcurrency(); Note that if you choose a concurrency level in your workers, it will not override the global one, it will just be the maximum jobs a given worker can process in parallel but never more than the global one. ### [](https://docs.bullmq.io/guide/queues/global-concurrency#remove-global-concurrency) Remove Global Concurrency It can be done using the following method: Copy await queue.removeGlobalConcurrency(); [](https://docs.bullmq.io/guide/queues/global-concurrency#read-more) Read more: ------------------------------------------------------------------------------------ * 💡 [Set Global Concurrency API Reference](https://api.docs.bullmq.io/classes/v5.Queue.html#setglobalconcurrency) * 💡 [Get Global Concurrency API Reference](https://api.docs.bullmq.io/classes/v5.Queue.html#getglobalconcurrency) * 💡 [Remove Global Concurrency API Reference](https://api.docs.bullmq.io/classes/v5.Queue.html#removeglobalconcurrency) [PreviousAdding jobs in bulk](https://docs.bullmq.io/guide/queues/adding-bulks) [NextGlobal Rate Limit](https://docs.bullmq.io/guide/queues/global-rate-limit) Last updated 1 month ago Was this helpful? --- # Removing Jobs | BullMQ Currently we have 3 available methods in queue class: [](https://docs.bullmq.io/guide/queues/removing-jobs#drain) Drain ---------------------------------------------------------------------- Removes all jobs that are waiting or delayed, but not active, waiting-children, completed or failed. Copy import { Queue } from 'bullmq'; const queue = new Queue('paint'); await queue.drain(); Parent jobs that belong to the queue being drained will be kept in **waiting-children** status if they have pending children, but if they do not have any pending children they will just be removed. Parent jobs in queues different from the one being drained will either stay in **waiting-children** if they have pending children in other queues, or just moved to wait. [](https://docs.bullmq.io/guide/queues/removing-jobs#clean) Clean ---------------------------------------------------------------------- Removes jobs in a specific state, but keeps jobs within a certain grace period. Copy import { Queue } from 'bullmq'; const queue = new Queue('paint'); const deletedJobIds = await queue.clean( 60000, // 1 minute 1000, // max number of jobs to clean 'paused', ); [](https://docs.bullmq.io/guide/queues/removing-jobs#obliterate) Obliterate -------------------------------------------------------------------------------- Completely obliterates a queue and all of its contents. Copy import { Queue } from 'bullmq'; const queue = new Queue('paint'); await queue.obliterate(); Parent jobs in queues different from the one being obliterated will either stay in **waiting-children** if they have pending children in other queues, or just moved to wait. ### [](https://docs.bullmq.io/guide/queues/removing-jobs#read-more) Read more: * 💡 [Drain API Reference](https://api.docs.bullmq.io/classes/v5.Queue.html#drain) * 💡 [Clean API Reference](https://api.docs.bullmq.io/classes/v5.Queue.html#clean) * 💡 [Obliterate API Reference](https://api.docs.bullmq.io/classes/v5.Queue.html#obliterate) [PreviousMeta](https://docs.bullmq.io/guide/queues/meta) [NextWorkers](https://docs.bullmq.io/guide/workers) Last updated 1 year ago Was this helpful? --- # Auto-removal of jobs | BullMQ By default, when your queue jobs are completed (or failed), they are stored in two special sets, the "completed" and the "failed" set. This is useful so that you can examine the results of your jobs, particularly in the early stages of development. However, as the solution reaches a production-grade level, we usually need to restrict the number of finished jobs to be kept, so that we do not fill Redis with data that is not particularly useful. BullMQ supports different strategies for auto-removing finalized jobs. These strategies are configured on the Worker's options [`removeOnComplete`](https://api.docs.bullmq.io/interfaces/v5.WorkerOptions.html#removeoncomplete) and [`removeOnFail`](https://api.docs.bullmq.io/interfaces/v5.WorkerOptions.html#removeonfail) . ### [](https://docs.bullmq.io/guide/workers/auto-removal-of-jobs#remove-all-finalized-jobs) Remove all finalized jobs The simplest option is to set `removeOnComplete`/`removeOnFail` to `{count: 0}`, in this case, all jobs will be removed automatically as soon as they are finalized: Copy const myWorker = new Worker( 'myQueueName', async job => { // do some work }, { connection, removeOnFail: { count: 0 } }, ); Jobs will be deleted regardless of their names. ### [](https://docs.bullmq.io/guide/workers/auto-removal-of-jobs#keep-a-certain-number-of-jobs) Keep a certain number of jobs It is also possible to specify a maximum number of jobs to keep. A good practice is to keep a handful of completed jobs and a much larger value of failed jobs: Copy const myWorker = new Worker( 'myQueueName', async job => { // do some work }, { connection, removeOnComplete: { count: 1000 }, removeOnFail: { count: 5000 } }, ); ### [](https://docs.bullmq.io/guide/workers/auto-removal-of-jobs#keep-jobs-based-on-their-age) Keep jobs based on their age Another possibility is to keep jobs up to a certain age. The `removeOn` option accepts a [`KeepJobs`](https://api.docs.bullmq.io/interfaces/v5.KeepJobs.html) object, that includes an `age` and a `count` fields. The `age` is used to specify how old jobs to keep (in seconds), and the `count` can be used to limit the total amount to keep. The `count` option is useful in cases we get an unexpected amount of jobs in a very short time, in this case we may just want to limit to a certain amount to avoid running out of memory. Copy const myWorker = new Worker( 'myQueueName', async job => { // do some work }, { connection, removeOnComplete: { age: 3600, // keep up to 1 hour count: 1000, // keep up to 1000 jobs }, removeOnFail: { age: 24 * 3600, // keep up to 24 hours } }, ); The auto removal of jobs works lazily. This means that jobs are not removed unless a new job completes or fails, since that is when the auto-removal takes place. [PreviousWorkers](https://docs.bullmq.io/guide/workers) [NextConcurrency](https://docs.bullmq.io/guide/workers/concurrency) Last updated 3 months ago Was this helpful? --- # Manage Job Schedulers | BullMQ In BullMQ, managing the lifecycle and inventory of job schedulers is crucial for maintaining efficient and organized background tasks. In addition to the upsertJobScheduler method—which allows for the addition and updating of job schedulers—two other methods play essential roles: removeJobScheduler and getJobSchedulers. These functions enable the removal of schedulers and retrieval of all existing schedulers, respectively, providing comprehensive control over your job scheduling environment. #### [](https://docs.bullmq.io/guide/job-schedulers/manage-job-schedulers#remove-job-scheduler) Remove job scheduler The **removeJobScheduler** method is designed to delete a specific job scheduler from the queue. This is particularly useful when a scheduled task is no longer needed or if you wish to clean up inactive or obsolete schedulers to optimize resource usage. Copy // Remove a job scheduler with ID 'scheduler-123' const result = await queue.removeJobScheduler('scheduler-123'); console.log( result ? 'Scheduler removed successfully' : 'Missing Job Scheduler', ); The method will return true if there was a Job Scheduler to remove with the given ID, or false if there wasn't any. #### [](https://docs.bullmq.io/guide/job-schedulers/manage-job-schedulers#get-job-schedulers) Get Job Schedulers The **getJobSchedulers** method retrieves a list of all configured job schedulers within a specified range. This is invaluable for monitoring and managing multiple job schedulers, especially in systems where jobs are dynamically scheduled and require frequent reviews or adjustments. Copy // Retrieve the first 10 job schedulers in ascending order of their next execution time const schedulers = await queue.getJobSchedulers(0, 9, true); console.log('Current job schedulers:', schedulers); This method can be particularly useful for generating reports or dashboards that provide insights into when jobs are scheduled to run, aiding in system monitoring and troubleshooting. #### [](https://docs.bullmq.io/guide/job-schedulers/manage-job-schedulers#get-job-scheduler) Get Job Scheduler The **getJobScheduler** method retrieves a job scheduler by id. This is invaluable for inspecting dedicated configurations. Copy const scheduler = await queue.getJobScheduler('test'); console.log('Current job scheduler:', scheduler); [](https://docs.bullmq.io/guide/job-schedulers/manage-job-schedulers#read-more) Read more: ----------------------------------------------------------------------------------------------- * 💡 [Remove Job Scheduler API Reference](https://api.docs.bullmq.io/classes/v5.Queue.html#removejobscheduler) * 💡 [Get Job Schedulers API Reference](https://api.docs.bullmq.io/classes/v5.Queue.html#getjobschedulers) * 💡 [Get Job Scheduler API Reference](https://api.docs.bullmq.io/classes/v5.Queue.html#getjobscheduler) [PreviousRepeat options](https://docs.bullmq.io/guide/job-schedulers/repeat-options) [NextFlows](https://docs.bullmq.io/guide/flows) Last updated 3 months ago Was this helpful? --- # Jobs | BullMQ Queues can hold different types of jobs, which determine how and when they are processed. In this section, we will describe them in detail. An important thing to consider is that you can mix the different job types in the same queue, so you can add FIFO jobs, and at any moment add a LIFO or a delayed job. [](https://docs.bullmq.io/guide/jobs#read-more) Read more: --------------------------------------------------------------- * 💡 [Job API Reference](https://api.docs.bullmq.io/classes/v5.Job.html) [PreviousPausing queues](https://docs.bullmq.io/guide/workers/pausing-queues) [NextFIFO](https://docs.bullmq.io/guide/jobs/fifo) Last updated 1 year ago Was this helpful? --- # Adding flows in bulk | BullMQ Sometimes it is necessary to atomically add flows in bulk. For example, there could be a requirement that all the flows must be created or none of them. Also, adding flows in bulk can be faster since it reduces the number of roundtrips to Redis: Copy import { FlowProducer } from 'bullmq'; const flow = new FlowProducer({ connection }); const trees = await flow.addBulk([\ {\ name: 'root-job-1',\ queueName: 'rootQueueName-1',\ data: {},\ children: [\ {\ name,\ data: { idx: 0, foo: 'bar' },\ queueName: 'childrenQueueName-1',\ },\ ],\ },\ {\ name: 'root-job-2',\ queueName: 'rootQueueName-2',\ data: {},\ children: [\ {\ name,\ data: { idx: 1, foo: 'baz' },\ queueName: 'childrenQueueName-2',\ },\ ],\ },\ ]); This call can only succeed or fail, and all or none of the jobs will be added. [](https://docs.bullmq.io/guide/flows/adding-bulks#read-more) Read more: ----------------------------------------------------------------------------- * 💡 [Add Bulk API Reference](https://api.docs.bullmq.io/classes/v5.FlowProducer.html#addbulk) [PreviousFlows](https://docs.bullmq.io/guide/flows) [NextGet Flow Tree](https://docs.bullmq.io/guide/flows/get-flow-tree) Last updated 3 months ago Was this helpful? --- # v3 | BullMQ ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.15.8-2023-06-16) [3.15.8](https://github.com/taskforcesh/bullmq/compare/v3.15.7...v3.15.8) (2023-06-16) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes) Bug Fixes * **rate-limit:** keep priority fifo order ([#1991](https://github.com/taskforcesh/bullmq/issues/1991) ) fixes [#1929](https://github.com/taskforcesh/bullmq/issues/1929) (python) ([56bd7ad](https://github.com/taskforcesh/bullmq/commit/56bd7ad8c4daffcfb1f9f199abfc5d6495eb291e) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.15.7-2023-06-16) [3.15.7](https://github.com/taskforcesh/bullmq/compare/v3.15.6...v3.15.7) (2023-06-16) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-1) Bug Fixes * **worker:** set redis version always in initialization ([#1989](https://github.com/taskforcesh/bullmq/issues/1989) ) fixes [#1988](https://github.com/taskforcesh/bullmq/issues/1988) ([a1544a8](https://github.com/taskforcesh/bullmq/commit/a1544a8c0f29522cd33772b14f559969db852d1d) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.15.6-2023-06-13) [3.15.6](https://github.com/taskforcesh/bullmq/compare/v3.15.5...v3.15.6) (2023-06-13) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-2) Bug Fixes * **worker:** use timeout as integer for redis lower than v6.0.0 (python) ([#1981](https://github.com/taskforcesh/bullmq/issues/1981) ) ([0df6afa](https://github.com/taskforcesh/bullmq/commit/0df6afad5e71a693b721ba52ffa6be733ee45ccb) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.15.5-2023-06-11) [3.15.5](https://github.com/taskforcesh/bullmq/compare/v3.15.4...v3.15.5) (2023-06-11) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-3) Bug Fixes * **retry-job:** consider priority when moving job to wait (python) ([#1969](https://github.com/taskforcesh/bullmq/issues/1969) ) ([e753855](https://github.com/taskforcesh/bullmq/commit/e753855eef248da73a5e9f6b18f4b79319dc2f86) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.15.4-2023-06-08) [3.15.4](https://github.com/taskforcesh/bullmq/compare/v3.15.3...v3.15.4) (2023-06-08) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-4) Bug Fixes * **job:** import right reference of QueueEvents ([#1964](https://github.com/taskforcesh/bullmq/issues/1964) ) ([689c845](https://github.com/taskforcesh/bullmq/commit/689c84567f3a9fea51f349ca93b3008d5c187f62) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.15.3-2023-06-08) [3.15.3](https://github.com/taskforcesh/bullmq/compare/v3.15.2...v3.15.3) (2023-06-08) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-5) Bug Fixes * **job:** use QueueEvents type for waitUntilFinished ([#1958](https://github.com/taskforcesh/bullmq/issues/1958) ) ([881848c](https://github.com/taskforcesh/bullmq/commit/881848c1ee3835dac24daf6807b1f35da967f68b) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.15.2-2023-06-06) [3.15.2](https://github.com/taskforcesh/bullmq/compare/v3.15.1...v3.15.2) (2023-06-06) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-6) Bug Fixes * **worker:** better worker client naming ([c5f63af](https://github.com/taskforcesh/bullmq/commit/c5f63affe72f7b6616f4c5f3aafde858dcc0b200) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.15.1-2023-06-05) [3.15.1](https://github.com/taskforcesh/bullmq/compare/v3.15.0...v3.15.1) (2023-06-05) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-7) Bug Fixes * **rate-limit:** consider paused queue ([#1931](https://github.com/taskforcesh/bullmq/issues/1931) ) ([d97864a](https://github.com/taskforcesh/bullmq/commit/d97864a550992aeb8673557c7d8f186ab4ccb5bf) ) [](https://docs.bullmq.io/changelog/changelog-v3#id-3.15.0-2023-05-31) [3.15.0](https://github.com/taskforcesh/bullmq/compare/v3.14.2...v3.15.0) (2023-05-31) ------------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v3#features) Features * **job:** add changePriority method ([#1901](https://github.com/taskforcesh/bullmq/issues/1901) ) ref [#1899](https://github.com/taskforcesh/bullmq/issues/1899) ([9485ad5](https://github.com/taskforcesh/bullmq/commit/9485ad567e2d8c78d601cc9eb2b7dd37f96d00c9) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.14.2-2023-05-30) [3.14.2](https://github.com/taskforcesh/bullmq/compare/v3.14.1...v3.14.2) (2023-05-30) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-8) Bug Fixes * **rate-limit:** take in count priority ([#1919](https://github.com/taskforcesh/bullmq/issues/1919) ) fixes [#1915](https://github.com/taskforcesh/bullmq/issues/1915) ([b8157a3](https://github.com/taskforcesh/bullmq/commit/b8157a3424ceb60e662e80a3b0db918241b87ecc) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.14.1-2023-05-27) [3.14.1](https://github.com/taskforcesh/bullmq/compare/v3.14.0...v3.14.1) (2023-05-27) #### [](https://docs.bullmq.io/changelog/changelog-v3#performance-improvements) Performance Improvements * **retry-job:** get target queue list once ([#1921](https://github.com/taskforcesh/bullmq/issues/1921) ) ([8a7a9dd](https://github.com/taskforcesh/bullmq/commit/8a7a9ddd793161a8591485ed18a191ece37026a8) ) [](https://docs.bullmq.io/changelog/changelog-v3#id-3.14.0-2023-05-22) [3.14.0](https://github.com/taskforcesh/bullmq/compare/v3.13.4...v3.14.0) (2023-05-22) ------------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v3#features-1) Features * **worker:** make extendLocks overridable ([7b1386b](https://github.com/taskforcesh/bullmq/commit/7b1386bb823562d9666a1ad6e206e1deb63e57ec) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.13.4-2023-05-11) [3.13.4](https://github.com/taskforcesh/bullmq/compare/v3.13.3...v3.13.4) (2023-05-11) #### [](https://docs.bullmq.io/changelog/changelog-v3#performance-improvements-1) Performance Improvements * **rate-limit:** call pttl in script moveJobFromActiveToWait ([#1889](https://github.com/taskforcesh/bullmq/issues/1889) ) ([e0d2992](https://github.com/taskforcesh/bullmq/commit/e0d2992eb757d437dede52054c049470d986ad44) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.13.3-2023-05-10) [3.13.3](https://github.com/taskforcesh/bullmq/compare/v3.13.2...v3.13.3) (2023-05-10) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-9) Bug Fixes * **child:** use named import for EventEmitter ([#1887](https://github.com/taskforcesh/bullmq/issues/1887) ) ([1db396d](https://github.com/taskforcesh/bullmq/commit/1db396d1f54154dc94c796ae8b570336fc341f02) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.13.2-2023-05-09) [3.13.2](https://github.com/taskforcesh/bullmq/compare/v3.13.1...v3.13.2) (2023-05-09) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-10) Bug Fixes * **rate-limit:** consider paused queue when dynamic rate limit ([#1884](https://github.com/taskforcesh/bullmq/issues/1884) ) ([a23f37e](https://github.com/taskforcesh/bullmq/commit/a23f37e4079d34c8589efc85e4d726a62244f0d2) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.13.1-2023-05-07) [3.13.1](https://github.com/taskforcesh/bullmq/compare/v3.13.0...v3.13.1) (2023-05-07) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-11) Bug Fixes * **retry:** consider when queue is paused ([#1880](https://github.com/taskforcesh/bullmq/issues/1880) ) ([01b621f](https://github.com/taskforcesh/bullmq/commit/01b621fea0cbdae602482ff61361c05646823223) ) [](https://docs.bullmq.io/changelog/changelog-v3#id-3.13.0-2023-05-06) [3.13.0](https://github.com/taskforcesh/bullmq/compare/v3.12.1...v3.13.0) (2023-05-06) ------------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v3#features-2) Features * **worker:** add worker threads support ([0820985](https://github.com/taskforcesh/bullmq/commit/0820985e073582fdf841affad38ecc7ab64691ec) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.12.1-2023-05-05) [3.12.1](https://github.com/taskforcesh/bullmq/compare/v3.12.0...v3.12.1) (2023-05-05) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-12) Bug Fixes * **worker:** close open handles after closing ([#1861](https://github.com/taskforcesh/bullmq/issues/1861) ) fixes [#1312](https://github.com/taskforcesh/bullmq/issues/1312) ([39286e8](https://github.com/taskforcesh/bullmq/commit/39286e87e8ffabf641f229cf2da3db4c280f4637) ) [](https://docs.bullmq.io/changelog/changelog-v3#id-3.12.0-2023-04-20) [3.12.0](https://github.com/taskforcesh/bullmq/compare/v3.11.0...v3.12.0) (2023-04-20) ------------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v3#features-3) Features * upgrade ioredis to 5.3.2 ([375b1be](https://github.com/taskforcesh/bullmq/commit/375b1be52035e93c5fef6024e0d06aa723f602a9) ) [](https://docs.bullmq.io/changelog/changelog-v3#id-3.11.0-2023-04-17) [3.11.0](https://github.com/taskforcesh/bullmq/compare/v3.10.4...v3.11.0) (2023-04-17) ------------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v3#features-4) Features * **upstash:** don't throw an error when detecting an upstash host ([2e06bca](https://github.com/taskforcesh/bullmq/commit/2e06bca3615aafecd725d093045a510a67053fed) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.10.4-2023-04-05) [3.10.4](https://github.com/taskforcesh/bullmq/compare/v3.10.3...v3.10.4) (2023-04-05) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-13) Bug Fixes * **flow:** do not remove completed children results ([#1788](https://github.com/taskforcesh/bullmq/issues/1788) ) fixes [#1778](https://github.com/taskforcesh/bullmq/issues/1778) ([04b547a](https://github.com/taskforcesh/bullmq/commit/04b547ad3df02cb94c499f7f26678e19c6797e7e) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.10.3-2023-03-30) [3.10.3](https://github.com/taskforcesh/bullmq/compare/v3.10.2...v3.10.3) (2023-03-30) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-14) Bug Fixes * **flow:** consider removing dependency on removeOnFail true ([#1753](https://github.com/taskforcesh/bullmq/issues/1753) ) ([de5a299](https://github.com/taskforcesh/bullmq/commit/de5a299f109834ab0235ae6fb6286fd94fcef961) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.10.2-2023-03-22) [3.10.2](https://github.com/taskforcesh/bullmq/compare/v3.10.1...v3.10.2) (2023-03-22) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-15) Bug Fixes * **job:** avoid error when job is moved when processing ([#1354](https://github.com/taskforcesh/bullmq/issues/1354) ) fixes [#1343](https://github.com/taskforcesh/bullmq/issues/1343) [#1602](https://github.com/taskforcesh/bullmq/issues/1602) ([78085e4](https://github.com/taskforcesh/bullmq/commit/78085e4304357dd3695df61057f91e706c3a52bf) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.10.1-2023-03-06) [3.10.1](https://github.com/taskforcesh/bullmq/compare/v3.10.0...v3.10.1) (2023-03-06) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-16) Bug Fixes * **worker:** throw error with invalid concurrency fixes [#1723](https://github.com/taskforcesh/bullmq/issues/1723) ([2a1cdbe](https://github.com/taskforcesh/bullmq/commit/2a1cdbe3e871309f460aadc14b4d632238c32aa9) ) [](https://docs.bullmq.io/changelog/changelog-v3#id-3.10.0-2023-03-02) [3.10.0](https://github.com/taskforcesh/bullmq/compare/v3.9.0...v3.10.0) (2023-03-02) ------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-17) Bug Fixes * **worker:** close lock extended timer ([7995f18](https://github.com/taskforcesh/bullmq/commit/7995f18bb7712bd50d0fa3d17c4ab565b16ab379) ) * **worker:** correct lock extender logic ([6aa3569](https://github.com/taskforcesh/bullmq/commit/6aa3569db0fe0137790e61a4b5982d2b35ee5646) ) * **worker:** start stalled check timer ([4763be0](https://github.com/taskforcesh/bullmq/commit/4763be028b0c7b0460fd0804d4569c446a06ef4a) ) #### [](https://docs.bullmq.io/changelog/changelog-v3#features-5) Features * **worker:** replace Promise.race with efficient an async fifo ([0d94e35](https://github.com/taskforcesh/bullmq/commit/0d94e35e805b09c3b4c7404b8a2eeb71a1aff5c4) ) * **worker:** simplify lock extension to one call independent of concurrency ([ebf1aeb](https://github.com/taskforcesh/bullmq/commit/ebf1aeb2400383d0ae90ab68aeb4822aea03ba44) ) #### [](https://docs.bullmq.io/changelog/changelog-v3#performance-improvements-2) Performance Improvements * **scripts:** reuse keys array to avoid allocations ([feac7b4](https://github.com/taskforcesh/bullmq/commit/feac7b4070a6a3720597af36c43d095e9ea37173) ) * **worker:** improve worker memory consumption ([4846cf1](https://github.com/taskforcesh/bullmq/commit/4846cf1fe3f9ea35f58a679c11706e1a7101c898) ) [](https://docs.bullmq.io/changelog/changelog-v3#id-3.9.0-2023-02-25) [3.9.0](https://github.com/taskforcesh/bullmq/compare/v3.8.0...v3.9.0) (2023-02-25) --------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v3#features-6) Features * **worker:** add remove on complete and fail options ([#1703](https://github.com/taskforcesh/bullmq/issues/1703) ) ([cf13494](https://github.com/taskforcesh/bullmq/commit/cf1349471dcbf0e43feea9972eaa71d2299d619f) ) [](https://docs.bullmq.io/changelog/changelog-v3#id-3.8.0-2023-02-23) [3.8.0](https://github.com/taskforcesh/bullmq/compare/v3.7.2...v3.8.0) (2023-02-23) --------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-18) Bug Fixes * **worker:** run stalled check directly first time ([f71ec03](https://github.com/taskforcesh/bullmq/commit/f71ec03111a22897cbf2fad39073185e4aeac6d6) ) #### [](https://docs.bullmq.io/changelog/changelog-v3#features-7) Features * **worker:** add a public method to run the stalled checker ([3159266](https://github.com/taskforcesh/bullmq/commit/3159266ccb002d4fc71b7ee7ac63c465c536dbd1) ) * **worker:** add support to disable stalled checks ([49e860c](https://github.com/taskforcesh/bullmq/commit/49e860c6675853971e992c2945b445660504e3b2) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.7.2-2023-02-23) [3.7.2](https://github.com/taskforcesh/bullmq/compare/v3.7.1...v3.7.2) (2023-02-23) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-19) Bug Fixes * **worker:** restore failed event job parameter typing ([#1707](https://github.com/taskforcesh/bullmq/issues/1707) ) ([44c2203](https://github.com/taskforcesh/bullmq/commit/44c2203ab65d406be9a913254600fe07c83e62d5) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.7.1-2023-02-22) [3.7.1](https://github.com/taskforcesh/bullmq/compare/v3.7.0...v3.7.1) (2023-02-22) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-20) Bug Fixes * **worker:** failed event receives an optional job parameter ([#1702](https://github.com/taskforcesh/bullmq/issues/1702) ) fixes [#1690](https://github.com/taskforcesh/bullmq/issues/1690) ([6009906](https://github.com/taskforcesh/bullmq/commit/6009906355765bf00cba5c1505e9e0c6bf8f14db) ) [](https://docs.bullmq.io/changelog/changelog-v3#id-3.7.0-2023-02-16) [3.7.0](https://github.com/taskforcesh/bullmq/compare/v3.6.6...v3.7.0) (2023-02-16) --------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v3#performance-improvements-3) Performance Improvements * **move-to-active:** remove deprecated limiter reference ([#1673](https://github.com/taskforcesh/bullmq/issues/1673) ) ([a97b22f](https://github.com/taskforcesh/bullmq/commit/a97b22f518a9f6c5d9c30a77bfd03cafdcbc57ff) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.6.6-2023-02-15) [3.6.6](https://github.com/taskforcesh/bullmq/compare/v3.6.5...v3.6.6) (2023-02-15) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-21) Bug Fixes * **job:** check jobKey when saving stacktrace ([#1681](https://github.com/taskforcesh/bullmq/issues/1681) ) fixes [#1676](https://github.com/taskforcesh/bullmq/issues/1676) ([1856c76](https://github.com/taskforcesh/bullmq/commit/1856c7684c377ca4fd36294cca8e128404be27b8) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.6.5-2023-02-11) [3.6.5](https://github.com/taskforcesh/bullmq/compare/v3.6.4...v3.6.5) (2023-02-11) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-22) Bug Fixes * infinite worker process spawned for invalid JS file ([a445ba8](https://github.com/taskforcesh/bullmq/commit/a445ba8b7a261b370dec7d88091ae5f5af8b2728) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.6.4-2023-02-09) [3.6.4](https://github.com/taskforcesh/bullmq/compare/v3.6.3...v3.6.4) (2023-02-09) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-23) Bug Fixes * **worker:** add a maximum block time ([1a2618b](https://github.com/taskforcesh/bullmq/commit/1a2618bc5473288a62dddb85e3cb78d6cdb4f39f) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.6.3-2023-02-07) [3.6.3](https://github.com/taskforcesh/bullmq/compare/v3.6.2...v3.6.3) (2023-02-07) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-24) Bug Fixes * **master:** copy type declaration ([23ade6e](https://github.com/taskforcesh/bullmq/commit/23ade6e3e45df14bd3fbc2c3e7be47307b642872) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.6.2-2023-02-03) [3.6.2](https://github.com/taskforcesh/bullmq/compare/v3.6.1...v3.6.2) (2023-02-03) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-25) Bug Fixes * **redis:** increase minimum default retry time ([d521531](https://github.com/taskforcesh/bullmq/commit/d521531e22ba9eda8ad8d6e8eddf450fdc3f50f4) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.6.1-2023-01-31) [3.6.1](https://github.com/taskforcesh/bullmq/compare/v3.6.0...v3.6.1) (2023-01-31) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-26) Bug Fixes * **connection:** apply console.warn in noeviction message ([95f171c](https://github.com/taskforcesh/bullmq/commit/95f171cbc8cdd7d04865618b715dd21229f36a4a) ) [](https://docs.bullmq.io/changelog/changelog-v3#id-3.6.0-2023-01-31) [3.6.0](https://github.com/taskforcesh/bullmq/compare/v3.5.11...v3.6.0) (2023-01-31) ---------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v3#features-8) Features * **job:** allow clearing job's log ([#1600](https://github.com/taskforcesh/bullmq/issues/1600) ) ([0ded2d7](https://github.com/taskforcesh/bullmq/commit/0ded2d7709322bf105e0decac44d801ece5615f2) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.5.11-2023-01-27) [3.5.11](https://github.com/taskforcesh/bullmq/compare/v3.5.10...v3.5.11) (2023-01-27) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-27) Bug Fixes * **error:** remove global prototype toJSON ([#1642](https://github.com/taskforcesh/bullmq/issues/1642) ) fixes [#1414](https://github.com/taskforcesh/bullmq/issues/1414) ([d4e7108](https://github.com/taskforcesh/bullmq/commit/d4e7108a37aeabdd3085a26c9daf09cea5976f3e) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.5.10-2023-01-24) [3.5.10](https://github.com/taskforcesh/bullmq/compare/v3.5.9...v3.5.10) (2023-01-24) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-28) Bug Fixes * **move-to-finished:** return correct delayUntil ([#1643](https://github.com/taskforcesh/bullmq/issues/1643) ) ([c4bf9fa](https://github.com/taskforcesh/bullmq/commit/c4bf9fa6563eda1630d8eb2189b16e9324b01c7f) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.5.9-2023-01-19) [3.5.9](https://github.com/taskforcesh/bullmq/compare/v3.5.8...v3.5.9) (2023-01-19) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-29) Bug Fixes * **worker:** fix delayed jobs with concurrency fixes [#1627](https://github.com/taskforcesh/bullmq/issues/1627) ([99a8e6d](https://github.com/taskforcesh/bullmq/commit/99a8e6d3a339be51fb46f69c8afac4ecdebff6d3) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.5.8-2023-01-18) [3.5.8](https://github.com/taskforcesh/bullmq/compare/v3.5.7...v3.5.8) (2023-01-18) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-30) Bug Fixes * **move-to-active:** delete marker when it is moved to active ([#1634](https://github.com/taskforcesh/bullmq/issues/1634) ) ([ad1fcea](https://github.com/taskforcesh/bullmq/commit/ad1fcea4500d4ceed51d5d5b0a03dbb5e1735a42) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.5.7-2023-01-17) [3.5.7](https://github.com/taskforcesh/bullmq/compare/v3.5.6...v3.5.7) (2023-01-17) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-31) Bug Fixes * **move-to-active:** validate next marker and return delayUntil ([#1630](https://github.com/taskforcesh/bullmq/issues/1630) ) ([3cd3305](https://github.com/taskforcesh/bullmq/commit/3cd33052fc711a9ba560c9a431630be5cdd02193) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.5.6-2023-01-13) [3.5.6](https://github.com/taskforcesh/bullmq/compare/v3.5.5...v3.5.6) (2023-01-13) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-32) Bug Fixes * **worker:** add max concurrency from the beginning ([#1597](https://github.com/taskforcesh/bullmq/issues/1597) ) fixes [#1589](https://github.com/taskforcesh/bullmq/issues/1589) ([6f49db3](https://github.com/taskforcesh/bullmq/commit/6f49db3fb15119d13f99cd83d49f2a7bdcb614cd) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.5.5-2023-01-10) [3.5.5](https://github.com/taskforcesh/bullmq/compare/v3.5.4...v3.5.5) (2023-01-10) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-33) Bug Fixes * circular references ([#1622](https://github.com/taskforcesh/bullmq/issues/1622) ) ([f607ec7](https://github.com/taskforcesh/bullmq/commit/f607ec7530fb4430e8cab7ed325583bd9d171ccf) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.5.4-2023-01-09) [3.5.4](https://github.com/taskforcesh/bullmq/compare/v3.5.3...v3.5.4) (2023-01-09) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-34) Bug Fixes * [#1603](https://github.com/taskforcesh/bullmq/issues/1603) performance issues in `remove()` ([#1607](https://github.com/taskforcesh/bullmq/issues/1607) ) ([2541215](https://github.com/taskforcesh/bullmq/commit/2541215bcf81dcd52eaefa02530c3812a5135fbf) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.5.3-2023-01-07) [3.5.3](https://github.com/taskforcesh/bullmq/compare/v3.5.2...v3.5.3) (2023-01-07) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-35) Bug Fixes * **delayed:** remove marker after being consumed ([#1620](https://github.com/taskforcesh/bullmq/issues/1620) ) fixes [#1615](https://github.com/taskforcesh/bullmq/issues/1615) ([9fce0f0](https://github.com/taskforcesh/bullmq/commit/9fce0f05e5acc1918a276b03e8cb9c16083cb509) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.5.2-2023-01-04) [3.5.2](https://github.com/taskforcesh/bullmq/compare/v3.5.1...v3.5.2) (2023-01-04) #### [](https://docs.bullmq.io/changelog/changelog-v3#performance-improvements-4) Performance Improvements * **get-dependencies:** replace slow object destructuring with single object ([#1612](https://github.com/taskforcesh/bullmq/issues/1612) ) ([621748e](https://github.com/taskforcesh/bullmq/commit/621748ec7727b46ce57eb9d2b46ef981874cdf4c) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.5.1-2022-12-23) [3.5.1](https://github.com/taskforcesh/bullmq/compare/v3.5.0...v3.5.1) (2022-12-23) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-36) Bug Fixes * **connection:** throw exception if using keyPrefix in ioredis ([eb6a130](https://github.com/taskforcesh/bullmq/commit/eb6a1305541547725e1717eefe2b678bc445f4d0) ) * **connection:** use includes to check for upstash more reliably ([12efb5c](https://github.com/taskforcesh/bullmq/commit/12efb5c539cb6f031ea6f3a80e4128d2e556e627) ) [](https://docs.bullmq.io/changelog/changelog-v3#id-3.5.0-2022-12-20) [3.5.0](https://github.com/taskforcesh/bullmq/compare/v3.4.2...v3.5.0) (2022-12-20) --------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-37) Bug Fixes * **job:** fetch parent before job moves to complete ([#1580](https://github.com/taskforcesh/bullmq/issues/1580) ) ([6a6c0dc](https://github.com/taskforcesh/bullmq/commit/6a6c0dca30bb0a2417e0c62d4c80202c750322dd) ) * **sandbox:** throw error when no exported function ([#1588](https://github.com/taskforcesh/bullmq/issues/1588) ) fixes [#1587](https://github.com/taskforcesh/bullmq/issues/1587) ([c031891](https://github.com/taskforcesh/bullmq/commit/c03189184c8eeeb324f005b86e93d114abbe2154) ) #### [](https://docs.bullmq.io/changelog/changelog-v3#features-9) Features * **queue:** add getJobState method ([#1593](https://github.com/taskforcesh/bullmq/issues/1593) ) ref [#1532](https://github.com/taskforcesh/bullmq/issues/1532) ([b741e84](https://github.com/taskforcesh/bullmq/commit/b741e8456f262b51aa7c68f571c76a3c54d02d37) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.4.2-2022-12-15) [3.4.2](https://github.com/taskforcesh/bullmq/compare/v3.4.1...v3.4.2) (2022-12-15) #### [](https://docs.bullmq.io/changelog/changelog-v3#performance-improvements-5) Performance Improvements * **counts:** delete delayed marker when needed ([#1583](https://github.com/taskforcesh/bullmq/issues/1583) ) ([cc26f1c](https://github.com/taskforcesh/bullmq/commit/cc26f1cd550de76c7588d3a98187b80ee78c40c4) ) * **get-children-values:** replace slow object destructuring with single object ([#1586](https://github.com/taskforcesh/bullmq/issues/1586) ) ([857d403](https://github.com/taskforcesh/bullmq/commit/857d40377a6eb2c0101e6d16d9085ecd4b52b016) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.4.1-2022-12-10) [3.4.1](https://github.com/taskforcesh/bullmq/compare/v3.4.0...v3.4.1) (2022-12-10) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-38) Bug Fixes * **exponential:** respect exponential backoff delay ([#1581](https://github.com/taskforcesh/bullmq/issues/1581) ) ([145dd32](https://github.com/taskforcesh/bullmq/commit/145dd329bb9f8254b404f4c5fbf7a50359202d37) ) * **get-jobs:** filter marker ([#1551](https://github.com/taskforcesh/bullmq/issues/1551) ) ([4add0ef](https://github.com/taskforcesh/bullmq/commit/4add0efa7857cc2f7b6d3c0c78a7f82cb7a46933) ) [](https://docs.bullmq.io/changelog/changelog-v3#id-3.4.0-2022-12-09) [3.4.0](https://github.com/taskforcesh/bullmq/compare/v3.3.5...v3.4.0) (2022-12-09) --------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v3#features-10) Features * **worker:** add ready event for blockingConnection ([#1577](https://github.com/taskforcesh/bullmq/issues/1577) ) ([992cc9e](https://github.com/taskforcesh/bullmq/commit/992cc9e9b3046185d3b67f2cc956f30337f458e1) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.3.5-2022-12-08) [3.3.5](https://github.com/taskforcesh/bullmq/compare/v3.3.4...v3.3.5) (2022-12-08) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-39) Bug Fixes * **worker:** add token postfix ([#1575](https://github.com/taskforcesh/bullmq/issues/1575) ) ([1d3e368](https://github.com/taskforcesh/bullmq/commit/1d3e368021041bb9861761c86fe3e04914b0c52f) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.3.4-2022-12-07) [3.3.4](https://github.com/taskforcesh/bullmq/compare/v3.3.3...v3.3.4) (2022-12-07) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-40) Bug Fixes * **worker:** try catch setname call ([#1576](https://github.com/taskforcesh/bullmq/issues/1576) ) fixes [#1574](https://github.com/taskforcesh/bullmq/issues/1574) ([0c42fd8](https://github.com/taskforcesh/bullmq/commit/0c42fd8c07dbac7ace81e97e45440af93fc622a5) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.3.3-2022-12-07) [3.3.3](https://github.com/taskforcesh/bullmq/compare/v3.3.2...v3.3.3) (2022-12-07) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-41) Bug Fixes * do not allow move from active to wait if not owner of the job ([dc1a307](https://github.com/taskforcesh/bullmq/commit/dc1a3077d1521c5dc99824a7fc05d17da03906bc) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.3.2-2022-12-05) [3.3.2](https://github.com/taskforcesh/bullmq/compare/v3.3.1...v3.3.2) (2022-12-05) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-42) Bug Fixes * floor pexpire to integer ([1d5de42](https://github.com/taskforcesh/bullmq/commit/1d5de425a19ebf879a8f9a7e0543d87a4d358be1) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.3.1-2022-12-05) [3.3.1](https://github.com/taskforcesh/bullmq/compare/v3.3.0...v3.3.1) (2022-12-05) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-43) Bug Fixes * **get-workers:** set name when ready event in connection ([#1564](https://github.com/taskforcesh/bullmq/issues/1564) ) ([de93c17](https://github.com/taskforcesh/bullmq/commit/de93c172901650e1666c48423a39076f2c7b9c7b) ) * **job:** console warn custom job ids when they represent integers ([#1569](https://github.com/taskforcesh/bullmq/issues/1569) ) ([6e677d2](https://github.com/taskforcesh/bullmq/commit/6e677d2800957b368bef4247b8e4328c5758f262) ) [](https://docs.bullmq.io/changelog/changelog-v3#id-3.3.0-2022-12-04) [3.3.0](https://github.com/taskforcesh/bullmq/compare/v3.2.5...v3.3.0) (2022-12-04) --------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v3#features-11) Features * **queue-events:** support duplicated event ([#1549](https://github.com/taskforcesh/bullmq/issues/1549) ) ([18bc4eb](https://github.com/taskforcesh/bullmq/commit/18bc4eb50432f8aa27f2395750a7617317b66ca1) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.2.5-2022-12-04) [3.2.5](https://github.com/taskforcesh/bullmq/compare/v3.2.4...v3.2.5) (2022-12-04) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-44) Bug Fixes * **add-job:** throw error when jobId represents an integer ([#1556](https://github.com/taskforcesh/bullmq/issues/1556) ) ([db617d7](https://github.com/taskforcesh/bullmq/commit/db617d79e8f55b5c9e0df4b6bfd4247612016da1) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.2.4-2022-11-29) [3.2.4](https://github.com/taskforcesh/bullmq/compare/v3.2.3...v3.2.4) (2022-11-29) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-45) Bug Fixes * **add-job:** do not update job that already exist ([#1550](https://github.com/taskforcesh/bullmq/issues/1550) ) ([26f6311](https://github.com/taskforcesh/bullmq/commit/26f6311cd0d2b936e404d0abebca9637f314a209) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.2.3-2022-11-29) [3.2.3](https://github.com/taskforcesh/bullmq/compare/v3.2.2...v3.2.3) (2022-11-29) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-46) Bug Fixes * **rate-limit:** delete rateLimiterKey when 0 ([#1553](https://github.com/taskforcesh/bullmq/issues/1553) ) ([0b88e5b](https://github.com/taskforcesh/bullmq/commit/0b88e5b94b4a0dc0d4000f7fd4b327f402248ad2) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.2.2-2022-11-15) [3.2.2](https://github.com/taskforcesh/bullmq/compare/v3.2.1...v3.2.2) (2022-11-15) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-47) Bug Fixes * **rate-limit:** check job is active before moving to wait ([9502167](https://github.com/taskforcesh/bullmq/commit/9502167bb0d9008fc8811ff7980dc8126fbc5ac2) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.2.1-2022-11-15) [3.2.1](https://github.com/taskforcesh/bullmq/compare/v3.2.0...v3.2.1) (2022-11-15) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-48) Bug Fixes * **worker:** consider removed jobs in failed event ([#1500](https://github.com/taskforcesh/bullmq/issues/1500) ) ([8704b9a](https://github.com/taskforcesh/bullmq/commit/8704b9a10575fd7df738296f7156057123592b86) ) [](https://docs.bullmq.io/changelog/changelog-v3#id-3.2.0-2022-11-09) [3.2.0](https://github.com/taskforcesh/bullmq/compare/v3.1.3...v3.2.0) (2022-11-09) --------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v3#features-12) Features * **flow:** move parent to delayed when delay option is provided ([#1501](https://github.com/taskforcesh/bullmq/issues/1501) ) ([2f3e5d5](https://github.com/taskforcesh/bullmq/commit/2f3e5d54f0797bf0d1adf14dbb2b51ad9f9183ca) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.1.3-2022-11-04) [3.1.3](https://github.com/taskforcesh/bullmq/compare/v3.1.2...v3.1.3) (2022-11-04) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-49) Bug Fixes * **delayed:** better handling of marker id ([816376e](https://github.com/taskforcesh/bullmq/commit/816376e7880ae0eafe85a1f9a5aef9fdfe3031a9) ) * **delayed:** notify workers a delayed job is closer in time fixes [#1505](https://github.com/taskforcesh/bullmq/issues/1505) ([6ced4d0](https://github.com/taskforcesh/bullmq/commit/6ced4d06c5c9c8342c9e4f7920a21826871eac1b) ) * **job:** better error message in moveToFailed ([4e9f5bb](https://github.com/taskforcesh/bullmq/commit/4e9f5bb90f87c66eca959ffc9b7a09e05908c2d9) ) * **moveToFinish:** always promote delayed jobs ([7610cc3](https://github.com/taskforcesh/bullmq/commit/7610cc37d4695a885043c251990e153d4ce4440f) ) * **moveToFinished:** revert move promoteDelayedJobs ([7d780db](https://github.com/taskforcesh/bullmq/commit/7d780dbc1d7728ab7b762a5578871b31f27ff80c) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.1.2-2022-11-04) [3.1.2](https://github.com/taskforcesh/bullmq/compare/v3.1.1...v3.1.2) (2022-11-04) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-50) Bug Fixes * **repeat:** allow easy migration from bullmq <3 to >=3 ([e17b886](https://github.com/taskforcesh/bullmq/commit/e17b886d3e2978e25f23f1a99b88562537a08576) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.1.1-2022-11-03) [3.1.1](https://github.com/taskforcesh/bullmq/compare/v3.1.0...v3.1.1) (2022-11-03) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-51) Bug Fixes * **change-delay:** remove delayed stream ([#1509](https://github.com/taskforcesh/bullmq/issues/1509) ) ([6e4809e](https://github.com/taskforcesh/bullmq/commit/6e4809e5d8f7ef35bc0871d21bfcdcb0f1f316c6) ) * **worker:** restore dynamic concurrency change ([#1515](https://github.com/taskforcesh/bullmq/issues/1515) ) ([fdac5c2](https://github.com/taskforcesh/bullmq/commit/fdac5c27607dfaaaad1c1256c47f2ae448efcd21) ) [](https://docs.bullmq.io/changelog/changelog-v3#id-3.1.0-2022-11-02) [3.1.0](https://github.com/taskforcesh/bullmq/compare/v3.0.1...v3.1.0) (2022-11-02) --------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v3#features-13) Features * **workers:** better error message for missing lock ([bf1d086](https://github.com/taskforcesh/bullmq/commit/bf1d0860c70bcc2b604d02ca47e5db64f962d71d) ) ### [](https://docs.bullmq.io/changelog/changelog-v3#id-3.0.1-2022-11-02) [3.0.1](https://github.com/taskforcesh/bullmq/compare/v3.0.0...v3.0.1) (2022-11-02) #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-52) Bug Fixes * **move-to-delayed:** consider promoting delayed jobs ([#1493](https://github.com/taskforcesh/bullmq/issues/1493) ) ([909da2b](https://github.com/taskforcesh/bullmq/commit/909da2bc2718a588379b3fdd9791bc8e51ad1dad) ) * **retry-job:** consider promoting delayed jobs ([#1508](https://github.com/taskforcesh/bullmq/issues/1508) ) ([d0b3412](https://github.com/taskforcesh/bullmq/commit/d0b3412d222449c24ab36068a791d08ea19ed922) ) [](https://docs.bullmq.io/changelog/changelog-v3#id-3.0.0-2022-10-25) [3.0.0](https://github.com/taskforcesh/bullmq/compare/v2.4.0...v3.0.0) (2022-10-25) --------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/changelog/changelog-v3#bug-fixes-53) Bug Fixes * **backoff:** handle backoff strategy as function ([#1463](https://github.com/taskforcesh/bullmq/issues/1463) ) ([3640269](https://github.com/taskforcesh/bullmq/commit/36402691a3c7fa500f07e2e11a28318099bdb909) ) * **repeat:** remove cron in favor of pattern option ([#1456](https://github.com/taskforcesh/bullmq/issues/1456) ) ([3cc150e](https://github.com/taskforcesh/bullmq/commit/3cc150e32cb5971ad4ba6ff91246aaf75296c165) ) #### [](https://docs.bullmq.io/changelog/changelog-v3#features-14) Features * add support for dynamic rate limiting ([2d51d2b](https://github.com/taskforcesh/bullmq/commit/2d51d2b33ef49059503e1bca7a582c71f6861ef4) ) * **rate-limit:** remove group key support and improve global rate limit ([81f780a](https://github.com/taskforcesh/bullmq/commit/81f780aeed81e670107d01d01265d407a30e2a62) ) #### [](https://docs.bullmq.io/changelog/changelog-v3#breaking-changes) BREAKING CHANGES * **rate-limit:** limit by group keys has been removed in favor of a much simpler and efficent rate-limit implementation. * **backoff:** object mapping is replaced by single function [Previousv4](https://docs.bullmq.io/changelog/changelog-v4) [Nextv2](https://docs.bullmq.io/changelog/changelog-v2) Last updated 1 year ago Was this helpful? --- # Job Schedulers | BullMQ A Job Scheduler acts as a factory , producing jobs based on specified "repeat" settings. The Job Scheduler is highly flexible, accommodating various scenarios, including jobs produced at fixed intervals, according to cron expressions, or based on custom requirements. For historical reasons, jobs produced by the Job Scheduler are often referred to as ‘Repeatable Jobs’. To create a scheduler, simply use the "upsertJobScheduler" method as demonstrated in the following example: Copy // Creates a new Job Scheduler that generates a job every 1000 milliseconds (1 second) const firstJob = await queue.upsertJobScheduler('my-scheduler-id', { every: 1000, }); This example will create a new Job Scheduler that will produce a new job every second. It will also return the first job created for this Job Scheduler, which will be in "delayed" status waiting to be processed after 1 second. Now there are also a few important considerations that need to be explained here.: * **Upsert vs. Add:** the 'upsert' is used instead of 'add' to simplify management of recurring jobs, especially in production deployments. It ensures the scheduler is updated or created without duplications. * **Job Production Rate:** The scheduler will only generate new jobs when the last job begins processing. Therefore, if your queue is very busy, or if you do not have enough workers or concurrency, it is possible that you will get the jobs less frequently than the specified repetition interval. * **Job Status:** As long as a Job Scheduler is producing jobs, there will be always one job associated to the scheduler in the "Delayed" status. ### [](https://docs.bullmq.io/guide/job-schedulers#using-job-templates) Using Job Templates You can also define a template with standard names, data, and options for jobs added to a queue. This ensures that all jobs produced by the Job Scheduler inherit these settings: Copy // Create jobs every day at 3:15 (am) const firstJob = await queue.upsertJobScheduler( 'my-scheduler-id', { pattern: '0 15 3 * * *' }, { name: 'my-job-name', data: { foo: 'bar' }, opts: { backoff: 3, attempts: 5, removeOnFail: 1000, }, }, ); All jobs produced by this scheduler will use the given settings. Note that in the future you could call "upsertJobScheduler" again with the given "my-scheduler-id" in order to update any settings of this particular job scheduler, such as the repeat options or/and the job's template settings. Since jobs produced by the Job Scheduler will get a special job ID in order to guarantee that jobs will never be created more often than the given repeat settings, you cannot choose a custom job id. However you can use the job's name if you need to discriminate these jobs from other jobs. [](https://docs.bullmq.io/guide/job-schedulers#read-more) Read more: ------------------------------------------------------------------------- * 💡 [Upsert Job Scheduler API Reference](https://api.docs.bullmq.io/classes/v5.Queue.html#upsertjobscheduler) [PreviousGetters](https://docs.bullmq.io/guide/jobs/getters) [NextRepeat Strategies](https://docs.bullmq.io/guide/job-schedulers/repeat-strategies) Last updated 3 months ago Was this helpful? --- # Dragonfly | BullMQ [Dragonfly](https://www.dragonflydb.io/) offers a drop-in replacement for Redis™, boasting a much faster and more memory-efficient implementation of several data structures used by BullMQ. It also enables the utilization of all available cores in your CPUs. Check [this article](https://bullmq.io/news/101023/dragonfly-compatibility/) for some performance results. To fully leverage Dragonfly's capabilities, specific steps are necessary. Primarily, you should name your queues using curly braces. This naming convention allows Dragonfly to assign a thread to each queue. For instance, if your queue is named `myqueue,`rename it to `{myqueue}`. If you manage multiple queues, this approach enables you to allocate different CPU cores to each queue, significantly enhancing performance. Even with a single queue, you can still exploit multi-core advantages in some cases. Consider splitting your queue into multiple ones, like`{myqueue-1}`, `{myqueue-2}`, etc., and distribute jobs randomly or using a round-robin method. Be aware that certain features like priorities and rate-limiting might not function across multiple queues. Your specific requirements will determine whether you can divide a single queue in this manner. For comprehensive instructions and the necessary flags to optimize your Dragonfly instance for BullMQ, please consult the [official integration guide](https://www.dragonflydb.io/docs/integrations/bullmq) . [PreviousRedis™ Compatibility](https://docs.bullmq.io/guide/redis-tm-compatibility) [NextRedis™ hosting](https://docs.bullmq.io/guide/redis-tm-hosting) Last updated 2 years ago Was this helpful? --- # LIFO | BullMQ In some cases, it is useful to process jobs in a LIFO (_Last-in, First-Out_) fashion. This means that the newest jobs added to the queue will be processed **before** the older ones. Copy import { Queue } from 'bullmq'; const myQueue = new Queue('Paint'); // Add a job that will be processed before all others await myQueue.add('wall', { color: 'pink' }, { lifo: true }); [PreviousFIFO](https://docs.bullmq.io/guide/jobs/fifo) [NextJob Ids](https://docs.bullmq.io/guide/jobs/job-ids) Last updated 1 year ago Was this helpful? --- # Telemetry | BullMQ BullMQ provides a Telemetry interface that can be used to integrate it with any external telemetry backends. Currently we support the [OpenTelemetry](https://opentelemetry.io/) specification, which is the new de-facto standard for telemetry purposes, however the interface if flexible enough to support any other backends in the future. Telemetry is very useful for large applications where you want to get a detailed and general overview of the system. For BullMQ it helps to gain insight in the different statuses a job may be during its complete lifecycle. In a large application it helpts tracking the source of the jobs and all the interactions the jobs or messages may perform with other parts of the system. [PreviousCreate Custom Events](https://docs.bullmq.io/guide/events/create-custom-events) [NextGetting started](https://docs.bullmq.io/guide/telemetry/getting-started) Last updated 1 year ago Was this helpful? --- # Repeat Strategies | BullMQ BullMQ comes with two predefined strategies for creating repeatable jobs. The ‘every’ strategy is straightforward, allowing you to schedule jobs to repeat at specific intervals, measured in seconds. The more complex ‘cron’ strategy uses cron expressions, as defined by the [cron-parser](https://www.npmjs.com/package/cron-parser) to schedule jobs in intricate patterns. Additionally, BullMQ lets you create custom strategies, giving you the flexibility to define your own logic for setting job intervals. ### [](https://docs.bullmq.io/guide/job-schedulers/repeat-strategies#every-strategy) "Every" strategy The every strategy is used when we simply want to produce repeatable jobs at specific intervals: Copy const { Queue, Worker } = require('bullmq'); const connection = { host: 'localhost', port: 6379, }; const myQueue = new Queue('my-repeatable-jobs', { connection }); // Upserting a repeatable job in the queue await myQueue.upsertJobScheduler( 'repeat-every-10s', { every: 10000, // Job will repeat every 10000 milliseconds (10 seconds) }, { name: 'every-job', data: { jobData: 'data' }, opts: {}, // Optional additional job options }, ); // Worker to process the jobs const worker = new Worker( 'my-repeatable-jobs', async job => { console.log(`Processing job ${job.id} with data: ${job.data.jobData}`); }, { connection }, ); ### [](https://docs.bullmq.io/guide/job-schedulers/repeat-strategies#cron-strategy) "Cron" strategy The “cron” strategy in BullMQ leverages the [cron-parser](https://www.npmjs.com/package/cron-parser) library to use cron expressions for scheduling jobs with greater specificity. This approach is ideal for jobs requiring execution at precise times or intervals, such as automated reports or maintenance tasks. Below is the supported format for cron expressions in cron-parser: Copy * * * * * * ┬ ┬ ┬ ┬ ┬ ┬ │ │ │ │ │ │ │ │ │ │ │ └ day of week (0 - 7, 1L - 7L, where 0 or 7 is Sunday) │ │ │ │ └───── month (1 - 12) │ │ │ └────────── day of month (1 - 31, L for the last day of the month) │ │ └─────────────── hour (0 - 23) │ └──────────────────── minute (0 - 59) └───────────────────────── second (0 - 59, optional) This format includes the optional second field, which is not typically available in standard cron schedules, allowing for even more precise scheduling. Cron expressions are quite powerful as in they support seemless handling timezone differences and daylight saving time transitions, crucial for tasks that depend on local times. And also because of the use of special characters to denote specific days or things like the last day of the month, providing flexibility for monthly and weekly tasks. If you are new to Cron expressions, [Wikipedia](https://en.wikipedia.org/wiki/Cron) is an excelent starting point to learn how to use them. Here follows an example that sets up a job to execute at 9:00 AM from Monday to Friday: Copy const { Queue, Worker } = require('bullmq'); const connection = { host: 'localhost', port: 6379, }; const myQueue = new Queue('my-cron-jobs', { connection }); // Upserting a job with a cron expression await myQueue.upsertJobScheduler( 'weekday-morning-job', { pattern: '0 0 9 * * 1-5', // Runs at 9:00 AM every Monday to Friday }, { name: 'cron-job', data: { jobData: 'morning data' }, opts: {}, // Optional additional job options }, ); // Worker to process the jobs const worker = new Worker( 'my-cron-jobs', async job => { console.log( `Processing job ${job.id} at ${new Date()} with data: ${ job.data.jobData }`, ); }, { connection }, ); ### [](https://docs.bullmq.io/guide/job-schedulers/repeat-strategies#custom-strategy) Custom Strategy It is possible to define a different strategy to schedule repeatable jobs. The idea is that the repeat strategy, based on a pattern and the latest job's milliseconds, return the next desired timestamp. Although not used in the following example, you could have different behaviours on your repeat strategies based on the current job's name if you want to. However note that **only** **one** repeatStrategy can be defined for a given queue. For example we can create a custom one for [RRULE](https://jkbrzt.github.io/rrule/) like this: Copy import { Queue, Worker } from 'bullmq'; import { rrulestr } from 'rrule'; const settings = { repeatStrategy: (millis: number, opts: RepeatOptions, _jobName: string) => { const currentDate = opts.startDate && new Date(opts.startDate) > new Date(millis) ? new Date(opts.startDate) : new Date(millis); const rrule = rrulestr(opts.pattern); if (rrule.origOptions.count && !rrule.origOptions.dtstart) { throw new Error('DTSTART must be defined to use COUNT with rrule'); } const next_occurrence = rrule.after(currentDate, false); return next_occurrence?.getTime(); }, }; const myQueue = new Queue('Paint', { settings }); // Repeat job every 10 seconds await myQueue.upsertJobScheduler( 'collibris', { pattern: 'RRULE:FREQ=SECONDLY;INTERVAL=10;WKST=MO', }, { data: { color: 'green' }, }, ); // Repeat job every 20 seconds await myQueue.upsertJobScheduler( 'pingeons', { pattern: 'RRULE:FREQ=SECONDLY;INTERVAL=20;WKST=MO', }, { data: { color: 'gray' }, }, ); const worker = new Worker( 'Paint', async () => { doSomething(); }, { settings }, ); As you may have noticed, the repeat strategy setting should be provided in **both** the Queue and Worker classes. The reason we need it in both places is that when we first add the job to the Queue, we need to calculate when the next iteration will occur. After that, the Worker takes over, and we use the settings configured in the Worker. [PreviousJob Schedulers](https://docs.bullmq.io/guide/job-schedulers) [NextRepeat options](https://docs.bullmq.io/guide/job-schedulers/repeat-options) Last updated 22 days ago Was this helpful? --- # FIFO | BullMQ The first type of job we are going to describe is the FIFO (_First-In, First-Out_) type. This is the standard type when adding jobs to a queue. The jobs are processed in the same order as they are inserted into the queue. This order is preserved independently on the number of processors you have; however, if you have more than one worker or a concurrency factor larger than 1, even though the workers will start the jobs in order, they may be completed in a slightly different order, since some jobs may take more time to complete than others. Copy import { Queue } from 'bullmq'; const myQueue = new Queue('Paint'); // Add a job that will be processed after all others await myQueue.add('wall', { color: 'pink' }); When you add jobs to the queue there are several options that you can use. For example, you can specify how many jobs you want to keep when the jobs are completed or failed: Copy await myQueue.add( 'wall', { color: 'pink' }, { removeOnComplete: true, removeOnFail: 1000 }, ); In the example above all completed jobs will be removed automatically and the last 1000 failed will be kept in the queue. [](https://docs.bullmq.io/guide/jobs/fifo#default-job-options) Default job options --------------------------------------------------------------------------------------- Quite often, you will want to provide the same job options to all the jobs that you add to the queue. In this case, you can use the `defaultJobOptions` option when instantiating the `Queue` class: Copy const queue = new Queue('Paint', { defaultJobOptions: { removeOnComplete: true, removeOnFail: 1000 }); [](https://docs.bullmq.io/guide/jobs/fifo#read-more) Read more: -------------------------------------------------------------------- * 💡 [Add Job API Reference](https://api.docs.bullmq.io/classes/v5.Queue.html#add) [PreviousJobs](https://docs.bullmq.io/guide/jobs) [NextLIFO](https://docs.bullmq.io/guide/jobs/lifo) Last updated 1 year ago Was this helpful? --- # Job Data | BullMQ Every job can have its own custom data. The data is stored in the `**data**` attribute of the job: TypeScript Python Copy import { Queue } from 'bullmq'; const myQueue = new Queue('paint'); const job = await myQueue.add('wall', { color: 'red' }); job.data; // { color: 'red' } Copy from bullmq import Queue queue = Queue('paint') job = await queue.add('wall', {'color': 'red'}) job.data # { color: 'red' } [](https://docs.bullmq.io/guide/jobs/job-data#update-data) Update data --------------------------------------------------------------------------- If you want to change the data after inserting a job, just use the `**updateData**` method. For example: TypeScript Python Copy const job = await Job.create(queue, 'wall', { color: 'red' }); await job.updateData({ color: 'blue', }); job.data; // { color: 'blue' } Copy from bullmq import Queue queue = Queue('paint') job = await queue.add('wall', {'color': 'red'}) await job.updateData({'color': 'blue'}) job.data # { color: 'blue' } [](https://docs.bullmq.io/guide/jobs/job-data#read-more) Read more: ------------------------------------------------------------------------ * 💡 [Update Data API Reference](https://api.docs.bullmq.io/classes/v5.Job.html#updatedata) [PreviousJob Ids](https://docs.bullmq.io/guide/jobs/job-ids) [NextDeduplication](https://docs.bullmq.io/guide/jobs/deduplication) Last updated 3 months ago Was this helpful? --- # Graceful shutdown | BullMQ BullMQ supports graceful shutdowns of workers. This is important so that we can minimize stalled jobs when a worker for some reason must be shutdown. But note that even in the event of a "ungraceful shutdown", the stalled mechanism in BullMQ allows for new workers to pick up stalled jobs and continue working on them. Prior to BullMQ 2.0, in order for stalled jobs to be picked up by other workers you need to have a [`QueueScheduler`](https://docs.bullmq.io/guide/queuescheduler) class running in the system. From BullMQ 2.0 and onwards, the `QueueScheduler` is not needed anymore, so the information above is only valid for older versions. In order to perform a shutdown just call the `_**close**_` method: Copy await worker.close(); The above call will mark the worker as _closing_ so it will not pick up new jobs, and at the same time it will wait for all the current jobs to be processed (or failed). This call will not timeout by itself, so you should make sure that your jobs finalize in a timely manner. If this call fails for some reason or it is not able to complete, the pending jobs will be marked as stalled and processed by other workers (if correct stalled options are configured on the [`QueueScheduler`](https://api.docs.bullmq.io/interfaces/v1.QueueSchedulerOptions.html) ). [PreviousConcurrency](https://docs.bullmq.io/guide/workers/concurrency) [NextCancelling jobs](https://docs.bullmq.io/guide/workers/cancelling-jobs) Last updated 1 year ago Was this helpful? --- # Parallelism and Concurrency | BullMQ In this chapter we would like to clean up some misconceptions regarding parallel execution and concurrency, how these two terms are applied to BullMQ queues, and how they can be used to maximize throughput depending on the kind of jobs that you need to process. [](https://docs.bullmq.io/guide/parallelism-and-concurrency#parallelism) Parallelism ----------------------------------------------------------------------------------------- Parallelism is the simplest of these two concepts as it means basically what your intuition will tell you: that two or more tasks can be run in parallel, independently of each other. This is the case when you have a multi-core processor, or if you have several machines running at the same time. There is a chance that tasks are running in parallel. However, running in parallel does not guarantee that the CPU time is maximized, mainly because most software is continually being blocked by slow IO operations such as reading from the network, writing to disk, sending and receiving data via peripherals, and so on. If tasks are very CPU intensive though, then being able to run them 100% in parallel will give you the biggest performance, as there will be very little overhead, but this is the exception rather than the norm, thats why in modern computers most tasks are instead run concurrently. [](https://docs.bullmq.io/guide/parallelism-and-concurrency#concurrency) Concurrency ----------------------------------------------------------------------------------------- In the context of computer science, concurrency refers to the ability of different tasks to run at the same time by dividing the available CPU into small slices so that all the tasks can advance in their processing, giving the impression that they are being executed independently in parallel. So when we say that we have 100 tasks running concurrently, it means that they are all advancing their processing but never at exactly the same time. #### [](https://docs.bullmq.io/guide/parallelism-and-concurrency#nodejs-event-loop) NodeJS Event Loop One of the features that makes NodeJS very efficient at dispatching requests in an HTTP server is the fact that it has one single loop and is capable of running a huge amount of microtasks concurrently by exploiting the async nature of IO calls. So for example, if a call is performed to a database for querying some data, that call will not block the entire NodeJS, instead it will go and execute some other piece of code and then at the end of the current event loop, check if any of the async calls have completed so that they can continue running in the next iteration. This gives the effect of parallel execution, however, it is not the case, and the only reason it is efficient is that the code is IO heavy thus we can better utilize the CPU time by executing code instead of just being idle waiting for an asynchronous call to finish. #### [](https://docs.bullmq.io/guide/parallelism-and-concurrency#bullmq-concurrency) BullMQ concurrency BullMQ allows you to set a concurrency setting on your workers. This setting is local for the worker and exploits the NodeJS event loop so that jobs that are IO-heavy will benefit from higher throughput, as the worker does not need to wait for a job to complete before it can pick the next one. But it is very important to understand that if the jobs are not IO-heavy but instead are CPU intensive, increasing concurrency will decrease the overall throughput as we will just be adding overhead when there is not a lot that can be done to run concurrently. #### [](https://docs.bullmq.io/guide/parallelism-and-concurrency#what-about-threading) What about threading? Threads are a mechanism used by the operating system to provide concurrent (and parallel) execution, by pre-empting a given thread by another one (in the case of threads running in the same CPU), or by running threads in parallel if there are several CPU cores available. In practice however, a modern OS runs hundreds if not thousands of threads at any given time, so actually no guarantee running code in several threads will indeed run in parallel, there is just a chance they will, and this depends a lot on how the given OS has implemented its scheduler, this can be quite complex but as a rule of thumb we can say that if there are 2 threads that consume a lot of CPU, and there are at least 2 cores, then most likely these two threads will run each on its dedicated CPU. NodeJS has thread support, but it is important to note that these threads are pretty heavy and almost consume the same amount of memory as if they were running in two different OS processes. The reason for this is that every NodeJS thread requires a complete V8 VM, with a lot of built-in libraries, which will add up to several dozens of megabytes. [](https://docs.bullmq.io/guide/parallelism-and-concurrency#how-to-best-use-bullmqs-concurrency-then) How to best use BullMQ's concurrency then? ----------------------------------------------------------------------------------------------------------------------------------------------------- You have 2 ways to increase parallelism and concurrency with BullMQ: you can specify the concurrency factor per worker, and you can have several workers running in parallel. The concurrency factor will just take advantage of NodeJS's event loop so that the worker can process several jobs concurrently while the jobs are doing IO operations. If your jobs require IO operations, then you can increase this number quite a lot. Something between 100 and 300 is a quite standard setting, the only way to finetune this value is by observing how the worker processes the production workload. If the jobs are very CPU intensive without IO calls, then there is no point in having a large concurrency number as it will just add overhead. Still, since BullMQ itself also performs IO operations (when updating Redis and fetching new jobs), there is a chance that a slight concurrency factor may even improve the throughput of CPU-intensive jobs. Secondly, you can run as many workers as you want. Every worker will run in parallel if it has a CPU at its disposal. You can run several workers in a given machine if the machine has more than one core, but you can also run workers in totally different machines. The jobs running on different workers will be running in parallel, so even if the job is CPU-intensive you will be able to increase the throughput which will normally scale linearly with the number of workers. [PreviousRate limiting](https://docs.bullmq.io/guide/rate-limiting) [NextRetrying failing jobs](https://docs.bullmq.io/guide/retrying-failing-jobs) Last updated 2 months ago Was this helpful? --- # Fail Parent | BullMQ In certain workflows, you may need a parent job to fail immediately if any of its child jobs fail. The `failParentOnFailure` option allows you to achieve this behaviour. When set to true on a child job, it ensures that if the child fails, its parent job is also marked as failed. This effect can propagate recursively up the job hierarchy, potentially causing grandparents or higher-level ancestors to fail as well, depending on the configuration. ### [](https://docs.bullmq.io/guide/flows/fail-parent#key-points) Key Points * Selective Application: Only child jobs with failParentOnFailure: true will trigger the failure of their parent job upon failing. Child jobs without this option will not affect the parent's state if they fail. * Recursive Behavior: If a child with this option fails, and its parent also has failParentOnFailure: true, the failure propagates upward through the job tree, potentially affecting grandparents and beyond. * Immediate Effect: As soon as a qualifying child job fails, the parent job is moved to the failed state. ### [](https://docs.bullmq.io/guide/flows/fail-parent#example) Example Copy import { FlowProducer } from 'bullmq'; const flow = new FlowProducer({ connection }); const originalTree = await flow.add({ name: 'root-job', queueName: 'topQueueName', data: {}, children: [\ {\ name,\ data: { idx: 0, foo: 'bar' },\ queueName: 'childrenQueueName',\ // This child will fail its parent if it fails\ opts: { failParentOnFailure: true },\ children: [\ {\ name,\ data: { idx: 1, foo: 'bah' },\ queueName: 'grandChildrenQueueName',\ // This grandchild will fail its parent if it fails\ opts: { failParentOnFailure: true },\ },\ {\ name,\ data: { idx: 2, foo: 'baz' },\ queueName: 'grandChildrenQueueName',\ // No failParentOnFailure; its failure won't affect the parent\ },\ ],\ },\ {\ name,\ data: { idx: 3, foo: 'foo' },\ queueName: 'childrenQueueName',\ // No failParentOnFailure; its failure won't affect the parent\ },\ ], }); As soon as a _child_ with this option fails, the parent job will be marked as failed lazily. A worker must process the parent job before it transitions to the failed state. The failure will result in an _UnrecoverableError_ with the message **child {childKey} failed**. Additionally, this option will be validated recursively, meaning a grandparent or higher-level ancestor could also fail depending on the configuration. ### [](https://docs.bullmq.io/guide/flows/fail-parent#how-it-works) How it Works * If grandchild-job-1 fails, its parent (child-job-1) will fail because of failParentOnFailure: true. Since child-job-1 also has failParentOnFailure: true, the root job (root-job) will fail as well. * If grandchild-job-2 fails, its parent (child-job-1) will not fail because failParentOnFailure is not set on this grandchild. * Similarly, if child-job-2 fails, the root job will remain unaffected since failParentOnFailure is not enabled for that child. ### [](https://docs.bullmq.io/guide/flows/fail-parent#use-case) Use Case This option is particularly useful in workflows where the success of a parent job depends critically on specific child jobs, allowing you to enforce strict dependencies and fail fast when necessary. [](https://docs.bullmq.io/guide/flows/fail-parent#read-more) Read more: ---------------------------------------------------------------------------- * 💡 [Add Flow API Reference](https://api.docs.bullmq.io/classes/v5.FlowProducer.html#add) [PreviousGet Flow Tree](https://docs.bullmq.io/guide/flows/get-flow-tree) [NextContinue Parent](https://docs.bullmq.io/guide/flows/continue-parent) Last updated 4 months ago Was this helpful? --- # Retrying failing jobs | BullMQ As your queues process jobs, it is inevitable that over time some of these jobs will fail. In BullMQ, a job is considered failed in the following scenarios: * The processor function defined in your [`Worker`](https://docs.bullmq.io/guide/workers) has thrown an exception. * The job has become [_stalled_](https://docs.bullmq.io/guide/jobs/stalled) and it has consumed the "max stalled count" setting. The exceptions thrown in a processor must be an [`Error`](https://nodejs.org/api/errors.html#class-error) object for BullMQ to work correctly. In general, as a best practice, it is better to always throw `Error` objects. There is even an [ESLint rule](https://eslint.org/docs/latest/rules/no-throw-literal) if you want to enforce it. [](https://docs.bullmq.io/guide/retrying-failing-jobs#retrying-failing-jobs) Retrying failing jobs ------------------------------------------------------------------------------------------------------- When a processor throws an exception, the worker will catch it and move the job to the failed set. Depending on your [Queue settings](https://docs.bullmq.io/guide/queues/auto-removal-of-jobs) , the job may stay in the failed set forever, or it could be automatically removed. Often it is desirable to automatically retry failed jobs so that we do not give up until a certain amount of retries have failed. In order to activate automatic job retries you should use the [`attempts`](https://api.docs.bullmq.io/interfaces/v5.BaseJobOptions.html#attempts) setting with a value larger than 1 (see the examples below). BullMQ supports retries of failed jobs using back-off functions. It is possible to use the **built-in** backoff functions or provide **custom** ones. If you do not specify a back-off function, the jobs will be retried without delay as soon as they fail. Retried jobs will respect their priority when they are moved back to waiting state. ### [](https://docs.bullmq.io/guide/retrying-failing-jobs#built-in-backoff-strategies) Built-in backoff strategies The current built-in backoff functions are **fixed** and **exponential**. #### [](https://docs.bullmq.io/guide/retrying-failing-jobs#fixed) Fixed With a fixed backoff, it will retry after `delay` milliseconds, so with a delay of 3000 milliseconds, it will retry _every_ attempt 3000 milliseconds after the previous attempt. Copy import { Queue } from 'bullmq'; const myQueue = new Queue('foo'); await queue.add( 'test-retry', { foo: 'bar' }, { attempts: 3, backoff: { type: 'fixed', delay: 1000, }, }, ); You can also provide a [jitter](https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/) option, it will generate random delays between `delay` and 0 milliseconds depending on the percentage of jitter usage. For example, you can provide a jitter value of 0.5 value and a delay of 1000 milliseconds, it will generate a delay between `1000` milliseconds = 1 second and `1000 * 0.5` milliseconds = 500ms. Copy import { Queue } from 'bullmq'; const myQueue = new Queue('foo'); await queue.add( 'test-retry', { foo: 'bar' }, { attempts: 8, backoff: { type: 'fixed', delay: 1000, jitter: 0.5, }, }, ); #### [](https://docs.bullmq.io/guide/retrying-failing-jobs#exponential) Exponential With exponential backoff, it will retry after `2 ^ (attempts - 1) * delay` milliseconds. For example, with a delay of 3000 milliseconds, for the 7th attempt, it will retry `2^6 * 3000` milliseconds = 3.2 minutes after the previous attempt. The code below shows how to specify the built-in "exponential" backoff function with a 1-second delay as a seed value, so it will retry at most 2 times (after the first attempt, reaching a total 3 attempts) spaced after 1 second, 2 seconds, and 4 seconds respectively: Copy import { Queue } from 'bullmq'; const myQueue = new Queue('foo'); await queue.add( 'test-retry', { foo: 'bar' }, { attempts: 3, backoff: { type: 'exponential', delay: 1000, }, }, ); You can also provide a [jitter](https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/) option, it will generate random delays between `2 ^ (attempts - 1) * delay` and 0 milliseconds depending on the percentage of jitter usage. For example, you can provide a jitter value of 0.5 value and a delay of 3000 milliseconds, for the 7th attempt, it will generate a delay between `2^6 * 3000` milliseconds = 192000ms and `2^6 * 3000 * 0.5` milliseconds = 96000ms after the previous attempt. Copy import { Queue } from 'bullmq'; const myQueue = new Queue('foo'); await queue.add( 'test-retry', { foo: 'bar' }, { attempts: 8, backoff: { type: 'exponential', delay: 3000, jitter: 0.5, }, }, ); You can also define the back-off strategy in the queue's `defaultJobOptions`, and it will apply to all jobs added to the queue unless overridden when adding the job. For example: Copy import { Queue } from 'bullmq'; const myQueue = new Queue('foo', { defaultJobOptions: { attempts: 3, backoff: { type: 'exponential', delay: 1000, }, }, }); await queue.add('test-retry', { foo: 'bar' }); Jitter percentage option value must be between 0 and 1. 0 percentage means no randomness is applied (default behavior), while 1 means that random delays will be generated beween 0 and max generated value by any of our built-in strategies. ### [](https://docs.bullmq.io/guide/retrying-failing-jobs#custom-back-off-strategies) Custom back-off strategies If you want to define your custom backoff function, you need to define it in the worker settings: Copy import { Worker } from 'bullmq'; const worker = new Worker('foo', async job => doSomeProcessing(), { settings: { backoffStrategy: (attemptsMade: number) => { return attemptsMade * 1000; }, }, }); If your backoffStrategy returns 0, jobs will be moved at the end of our waiting list (priority 0) or moved back to prioritized state (priority > 0). If your backoffStrategy returns -1, jobs won't be retried, instead they will be moved to failed state. You can then use your custom strategy when adding jobs: Copy import { Queue } from 'bullmq'; const myQueue = new Queue('foo'); await queue.add( 'test-retry', { foo: 'bar' }, { attempts: 3, backoff: { type: 'custom', }, }, ); If you want to define multiple custom backoff types you need to define them like in the following example: Copy import { Worker } from 'bullmq'; const worker = new Worker('foo', async job => doSomeProcessing(), { settings: { backoffStrategy: ( attemptsMade: number, type: string, err: Error, job: Job, ) => { switch (type) { case 'custom1': { return attemptsMade * 1000; } case 'custom2': { return attemptsMade * 2000; } default: { throw new Error('invalid type'); } } }, }, }); [](https://docs.bullmq.io/guide/retrying-failing-jobs#read-more) Read more: -------------------------------------------------------------------------------- * 💡 [Stop Retrying Jobs](https://docs.bullmq.io/patterns/stop-retrying-jobs) [PreviousParallelism and Concurrency](https://docs.bullmq.io/guide/parallelism-and-concurrency) [NextReturning job data](https://docs.bullmq.io/guide/returning-job-data) Last updated 1 day ago Was this helpful? --- # Create Custom Events | BullMQ In BullMQ, creating a generic distributed realtime event emitter is possible by using our **QueueEventsProducer** class. Consumers must use **QueueEvents** class to subscribe to those events that they are interested in. Copy const queueName = 'customQueue'; const queueEventsProducer = new QueueEventsProducer(queueName, { connection, }); const queueEvents = new QueueEvents(queueName, { connection, }); interface CustomListener extends QueueEventsListener { example: (args: { custom: string }, id: string) => void; } queueEvents.on('example', async ({ custom }) => { // custom logic }); interface CustomEventPayload { eventName: string; custom: string; } await queueEventsProducer.publishEvent({ eventName: 'example', custom: 'value', }); Only eventName attribute is required. Some event names are reserved from [Queue Listener API Reference](https://api.docs.bullmq.io/interfaces/v5.QueueListener.html) . [](https://docs.bullmq.io/guide/events/create-custom-events#read-more) Read more: -------------------------------------------------------------------------------------- * 💡 [Queue Events API Reference](https://api.docs.bullmq.io/classes/v5.QueueEvents.html) * 💡 [Queue Events Listener API Reference](https://api.docs.bullmq.io/interfaces/v5.QueueEventsListener.html) * 💡 [Queue Events Producer API Reference](https://api.docs.bullmq.io/classes/v5.QueueEventsProducer.html) [PreviousEvents](https://docs.bullmq.io/guide/events) [NextTelemetry](https://docs.bullmq.io/guide/telemetry) Last updated 9 months ago Was this helpful? --- # Prometheus | BullMQ BullMQ provides a simple API to export metrics to Prometheus. To use it, create an endpoint in your web server that calls `exportPrometheusMetrics()`, and configure Prometheus to scrape metrics from this endpoint. #### [](https://docs.bullmq.io/guide/metrics/prometheus#basic-usage) Basic Usage Below is an example using vanilla Node.js: Copy import http from 'http'; import { Queue } from 'bullmq'; const queue = new Queue('my-queue'); const server = http.createServer( async (req: http.IncomingMessage, res: http.ServerResponse) => { try { if (req.url === '/metrics' && req.method === 'GET') { const metrics = await queue.exportPrometheusMetrics(); res.writeHead(200, { 'Content-Type': 'text/plain', 'Content-Length': Buffer.byteLength(metrics), }); res.end(metrics); } else { res.writeHead(404); res.end('Not Found'); } } catch (err: unknown) { res.writeHead(500); res.end(`Error: ${err instanceof Error ? err.message : 'Unknown error'}`); } }, ); const PORT = process.env.PORT || 3000; server.listen(PORT, () => { console.log(`Prometheus metrics server running on port ${PORT}`); console.log(`Metrics available at http://localhost:${PORT}/metrics`); }); Test the endpoint with: Copy curl http://localhost:3000/metrics This will return an output like: Copy HELP bullmq_job_count Number of jobs in the queue by state TYPE bullmq_job_count gauge bullmq_job_count{queue="my-queue", state="waiting"} 5 bullmq_job_count{queue="my-queue", state="active"} 3 bullmq_job_count{queue="my-queue", state="completed"} 12 bullmq_job_count{queue="my-queue", state="failed"} 2 For a simpler setup with Express.js: Copy import express from 'express'; import { Queue } from './src/queue'; const app = express(); const queue = new Queue('my-queue'); app.get('/metrics', async (req, res) => { try { const metrics = await queue.exportPrometheusMetrics(); res.set('Content-Type', 'text/plain'); res.send(metrics); } catch (err) { res.status(500).send(err.message); } }); const PORT = process.env.PORT || 3000; app.listen(PORT, () => { console.log(`Prometheus metrics server running on port ${PORT}`); console.log(`Metrics available at http://localhost:${PORT}/metrics`); }); #### [](https://docs.bullmq.io/guide/metrics/prometheus#advanced-usage-adding-global-variables-as-labels) Advanced Usage: Adding Global Variables as Labels The `exportPrometheusMetrics` function also supports an optional globalVariables parameter. This allows you to include additional labels (e.g., env, server) in your metrics, which is particularly useful when aggregating metrics from multiple environments (like production or staging) in tools like Grafana. The globalVariables parameter accepts a record of key-value pairs that are added as labels to each metric. #### [](https://docs.bullmq.io/guide/metrics/prometheus#example-with-global-variables) Example with Global Variables Here’s how to use this feature in vanilla Node.js: Copy import http from 'http'; import { Queue } from 'bullmq'; const queue = new Queue('my-queue'); const server = http.createServer( async (req: http.IncomingMessage, res: http.ServerResponse) => { try { if (req.url === '/metrics' && req.method === 'GET') { const globalVariables = { env: 'Production', server: '1' }; const metrics = await queue.exportPrometheusMetrics(globalVariables); res.writeHead(200, { 'Content-Type': 'text/plain', 'Content-Length': Buffer.byteLength(metrics), }); res.end(metrics); } else { res.writeHead(404); res.end('Not Found'); } } catch (err: unknown) { res.writeHead(500); res.end(`Error: ${err instanceof Error ? err.message : 'Unknown error'}`); } }, ); const PORT = process.env.PORT || 3000; server.listen(PORT, () => { console.log(`Prometheus metrics server running on port ${PORT}`); console.log(`Metrics available at http://localhost:${PORT}/metrics`); }); With globalVariables = { env: 'Production', server: '1' }, the output becomes: Copy # HELP bullmq_job_count Number of jobs in the queue by state # TYPE bullmq_job_count gauge bullmq_job_count{queue="my-queue", state="waiting", env="Production", server="1"} 5 bullmq_job_count{queue="my-queue", state="active", env="Production", server="1"} 3 bullmq_job_count{queue="my-queue", state="completed", env="Production", server="1"} 12 bullmq_job_count{queue="my-queue", state="failed", env="Production", server="1"} 2 These additional labels allow you to filter and group metrics in Prometheus or Grafana, making it easier to distinguish between different environments or servers. [](https://docs.bullmq.io/guide/metrics/prometheus#read-more) Read more: ----------------------------------------------------------------------------- * 💡 [Export Prometheus Metrics API Reference](https://api.docs.bullmq.io/classes/v5.Queue.html#exportprometheusmetrics) [PreviousMetrics](https://docs.bullmq.io/guide/metrics) [NextRate limiting](https://docs.bullmq.io/guide/rate-limiting) Last updated 3 months ago Was this helpful? --- # Returning job data | BullMQ When a worker is done processing, sometimes it is convenient to return some data. This data can then be accessed for example by listening to the `completed` event. This return data is available at the job's `returnvalue` property. Imagine a simple worker that performs some async processing: Copy import { Queue, Worker } from 'bullmq'; const myWorker = new Worker('AsyncProc', async job => { const result = await doSomeAsyncProcessing(); return result; }); Note, in the example above we could just directly return the result of `doSomeAsyncProcessing`, we just use a temporary variable to make the example more explicit. We can now listen to the completed event in order to get the result value: Copy import { Job, QueueEvents, Queue } from 'bullmq'; const queue = new Queue('AsyncProc'); const queueEvents = new QueueEvents('AsyncProc'); queueEvents.on('completed', async ({ jobId: string }) => { const job = await Job.fromId(queue, jobId); console.log(job.returnvalue); }); If you want to store the result of the processing function it is still much more robust to do it in the process function itself, as that will guarantee that if the job is completed the return value would be stored as well. Storing data on the completed event on the other hand could fail and still the job would complete without detecting the error. [](https://docs.bullmq.io/guide/returning-job-data#using-a-results-queue) Using a "results" Queue ------------------------------------------------------------------------------------------------------ Another common practice to send job results robustly is to have a special "results" queue to which results are sent. The worker for this "results" queue can reliably do something with the data such as storing it in a database. This approach is useful for designing robust micro-service architectures, where data is sent between services using queues. Even if the service that processes the result is down at the time the results queue receives the data, the result will still be processed as soon as the service come up online again. [PreviousRetrying failing jobs](https://docs.bullmq.io/guide/retrying-failing-jobs) [NextEvents](https://docs.bullmq.io/guide/events) Last updated 2 years ago Was this helpful? --- # Metrics | BullMQ BullMQ provides a simple metrics gathering functionality that allows you to track the performance of your queues. Workers can count the number of jobs they have processed **per minute** and store this data in a list inside Redis so that it can be queried later. You enable it on the worker settings by specifying how many data points you want to keep, which basically are counters of the number of jobs that have been processed either completed or failed during 1 minute intervals. As the metrics are aggregated in 1 minute intervals, using the recommended duration of 2 weeks of data should take a very small amount of total space, just around 120Kb of RAM per queue. The metrics will dispose older data points automatically so this RAM consumption will never increase after it reaches the maximum number of data points. Check in this example how to enable metrics on a worker: Copy import { Worker, MetricsTime } from 'bullmq'; const myWorker = new Worker('Paint', { connection, metrics: { maxDataPoints: MetricsTime.ONE_WEEK * 2, }, }); You need to use the same setting on all your workers to get consistent metrics. In order to query the metrics, use the `getMetrics` method on the `Queue` class. You can choose to gather the metrics for the _completed_ or _failed_ jobs: Copy import { Queue } from 'bullmq'; const myQueue = new Queue('Paint', { connection, }); const metrics = await queue.getMetrics('completed', 0, MetricsTime.ONE_WEEK * 2); /* Returns a Metrics object: { data: number[]; count: number; meta: { count: number; prevTS: number; prevCount: number; }; } */ Let's analyze what data we are getting back. First we have the `meta` field. The `prevTS` and `prevCount` subfields are used internally by the metrics system and should not be used, however you can use the`count` subfield to get a total number for all completed or failed jobs, this counter is not just the number of completed jobs in the given interval, but since the queue started processing jobs. The query also returns a `data` field which is an array where every position in the array represents 1 minute of time and has the total number of jobs that completed (or failed) in that minute. Note that the `getMetrics` method also accepts a `start` and `end` argument (`0` and `-1` by default), that you can use if you want to implement pagination. [](https://docs.bullmq.io/guide/metrics#read-more) Read more: ------------------------------------------------------------------ * 💡 [Get Metrics API Reference](https://api.docs.bullmq.io/classes/v5.Queue.html#getmetrics) [PreviousRemove Child Dependency](https://docs.bullmq.io/guide/flows/remove-child-dependency) [NextPrometheus](https://docs.bullmq.io/guide/metrics/prometheus) Last updated 3 months ago Was this helpful? --- # Job Ids | BullMQ All jobs in BullMQ need to have a unique job id. This id is used to construct a key to store the data in Redis, and as a pointer to the job as it is moved between the different states it can be in during its lifetime. By default, job ids are generated automatically as an increasing counter, however it is also possible to specify a _custom id_. The uniqueness requirement is scoped by queue, i.e. you can have the same job id in different queues without any issues. The counter for automatically generated ids is also scoped by queue. The main reason to be able to specify a custom id is in cases when you want to avoid duplicated jobs. Since ids must be unique, if you add a job with an existing id then that job will just be ignored and not added to the queue at all. Jobs that are removed from the queue (either manually, or when using settings such as `removeOnComplete`/`removeOnFailed`) will **not** be considered as duplicates, meaning that you can add the same job id many times over as long as the previous job has already been removed from the queue. In order to specify a custom job id, use the `jobId` option when adding jobs to the queue: Copy await myQueue.add( 'wall', { color: 'pink' }, { jobId: customJobId, }, ); Custom job ids must not contain the **:** separator as it will be translated in 2 different values, since we are also following Redis naming convention. So if you need to add a separator, use a different value, for example **\-**, **\_**. [](https://docs.bullmq.io/guide/jobs/job-ids#read-more) Read more: ----------------------------------------------------------------------- * 💡 [Duplicated Event Reference](https://api.docs.bullmq.io/interfaces/v5.QueueEventsListener.html#duplicated) [PreviousLIFO](https://docs.bullmq.io/guide/jobs/lifo) [NextJob Data](https://docs.bullmq.io/guide/jobs/job-data) Last updated 10 months ago Was this helpful? --- # Remove Dependency | BullMQ In some situations, you may have a parent job and need to remove the relationship when one of its children fail. The pattern to solve this requirement consists on using the **removeDependencyOnFailure** option. This option will make sure that when a job fails, the dependency is removed from the parent, so the parent will complete without waiting for the failed children. Copy const flow = new FlowProducer({ connection }); const originalTree = await flow.add({ name: 'root-job', queueName: 'topQueueName', data: {}, children: [\ {\ name,\ data: { idx: 0, foo: 'bar' },\ queueName: 'childrenQueueName',\ opts: { removeDependencyOnFailure: true },\ children: [\ {\ name,\ data: { idx: 1, foo: 'bah' },\ queueName: 'grandChildrenQueueName',\ },\ {\ name,\ data: { idx: 2, foo: 'baz' },\ queueName: 'grandChildrenQueueName',\ },\ ],\ },\ {\ name,\ data: { idx: 3, foo: 'foo' },\ queueName: 'childrenQueueName',\ },\ ], }); As soon as a **child** with this option fails, the parent job will be moved to a waiting state only if there are no more pending children. [](https://docs.bullmq.io/guide/flows/remove-dependency#read-more) Read more: ---------------------------------------------------------------------------------- * 💡 [Add Flow API Reference](https://api.docs.bullmq.io/classes/v5.FlowProducer.html#add) [PreviousContinue Parent](https://docs.bullmq.io/guide/flows/continue-parent) [NextIgnore Dependency](https://docs.bullmq.io/guide/flows/ignore-dependency) Last updated 1 year ago Was this helpful? --- # Pausing queues | BullMQ BullMQ supports pausing queues _globally_ or _locally_. When a queue is paused _globally_ no workers will pick up any jobs from the queue. When you pause a queue, the workers that are currently busy processing a job will continue working on that job until it completes (or fails), and then will keep idling until the queue is unpaused. Pausing a queue is performed by calling the `_**pause**_` method on a [queue](https://api.docs.bullmq.io/classes/v5.Queue.html) instance: Copy await myQueue.pause(); It is also possible to _locally_ pause a given worker instance. This pause works in a similar way as the global pause in the sense that the worker will conclude processing the jobs it has already started but will not process any new ones: Copy await myWorker.pause(); The call above will wait for all the jobs currently being processed by this worker to complete (or fail). If you do not want to wait for current jobs to complete before the call completes you can pass `true` to pause the worker **ignoring any running jobs**: Copy await myWorker.pause(true); [](https://docs.bullmq.io/guide/workers/pausing-queues#read-more) Read more: --------------------------------------------------------------------------------- * 💡 [Pause Queue API Reference](https://api.docs.bullmq.io/classes/v5.Queue.html#pause) * 💡 [Pause Worker API Reference](https://api.docs.bullmq.io/classes/v5.Worker.html#pause) [PreviousSandboxed processors](https://docs.bullmq.io/guide/workers/sandboxed-processors) [NextJobs](https://docs.bullmq.io/guide/jobs) Last updated 1 year ago Was this helpful? --- # Bull to BullMQ | BullMQ Bull and BullMQ have diverged too much now to actually give any guarantee of backwards compatibility. So the safest would be to use new queues for BullMQ and deprecate the old ones. New queues in this context mean using different queue names or passing a custom prefix option. You can have a period where bull and bullmq workers are running in parallel until all the jobs from the old queues are completed. Take into account that producers should add jobs in bullmq queues in this period. When all queues from bull are drained, you can deprecate them. You can use a dashboard tool such as https://taskforce.sh for monitoring this process and make sure the old queues are completely empty before removing them. [](https://docs.bullmq.io/guide/migration-to-newer-versions/bull-to-bullmq#read-more) Read more: ----------------------------------------------------------------------------------------------------- * 💡 [Worker Prefix Option Reference](https://api.docs.bullmq.io/interfaces/v5.WorkerOptions.html#prefix) * 💡 [Queue Prefix Option Reference](https://api.docs.bullmq.io/interfaces/v5.QueueOptions.html#prefix) [PreviousMigrations](https://docs.bullmq.io/guide/migration-to-newer-versions) [NextTroubleshooting](https://docs.bullmq.io/guide/troubleshooting) Last updated 2 months ago Was this helpful? --- # Named Processor | BullMQ When a Worker is instantiated, the most common usage is to specify a process function. Sometimes however, it is useful to be able to specify more than one function to process a job for a specific condition: Copy const worker = new Worker( 'queueName', async job => { switch (job.name) { case 'taskType1': { await doSomeLogic1(); break; } case 'taskType2': { await doSomeLogic2(); break; } } }, { connection }, ); You could use a simple switch case to differentiate your logic, in this example we are using the job name. This was a feature in the Bull package, but it creates a lot of confusion, so in order to provide an alternative, you can use this pattern. See [#297](https://github.com/taskforcesh/bullmq/issues/297) and [#69](https://github.com/taskforcesh/bullmq/issues/69) as reference [PreviousManually processing jobs](https://docs.bullmq.io/patterns/manually-fetching-jobs) [NextFlows](https://docs.bullmq.io/patterns/flows) Last updated 3 years ago Was this helpful? --- # Deduplication | BullMQ Deduplication in BullMQ is a process where job execution is delayed and deduplicated based on specific identifiers. It ensures that within a specified period, or until a specific job is completed or failed, no new jobs with the same identifier will be added to the queue. Instead, these attempts will trigger a deduplicated event. [](https://docs.bullmq.io/guide/jobs/deduplication#simple-mode) Simple Mode -------------------------------------------------------------------------------- The Simple Mode extends the deduplication duration until the job's completion or failure. This means that as long as the job remains in an incomplete state (neither succeeded nor failed), any subsequent job with the same deduplication ID will be ignored. Copy // Add a job that will be deduplicated as this record is not finished (completed or failed). await myQueue.add( 'house', { color: 'white' }, { deduplication: { id: 'customValue' } }, ); While this job is not moved to completed or failed state, next jobs added with same **deduplication id** will be ignored and a _deduplicated_ event will be triggered by our QueueEvent class. This mode is particularly useful for jobs that have a long running time or those that must not be duplicated until they are resolved, such as processing a file upload or performing a critical update that should not be repeated if the initial attempt is still in progress. [](https://docs.bullmq.io/guide/jobs/deduplication#throttle-mode) Throttle Mode ------------------------------------------------------------------------------------ In the Throttle Mode, deduplication works by assigning a delay (Time to Live, TTL) to a job upon its creation. If a similar job (identified by a unique deduplication ID) is added during this delay period, it is ignored. This prevents the queue from being overwhelmed with multiple instances of the same task, thus optimizing the processing time and resource utilization. Copy import { Queue } from 'bullmq'; const myQueue = new Queue('Paint'); // Add a job that will be deduplicated for 5 seconds. await myQueue.add( 'house', { color: 'white' }, { deduplication: { id: 'customValue', ttl: 5000 } }, ); In this example, after adding the house painting job with the deduplicated parameters (id and ttl), any subsequent job with the same deduplication ID customValue added within 5 seconds will be ignored. This is useful for scenarios where rapid, repetitive requests are made, such as multiple users or processes attempting to trigger the same job. [](https://docs.bullmq.io/guide/jobs/deduplication#debounce-mode) Debounce Mode ------------------------------------------------------------------------------------ Debounce Mode can be achieved by delaying a job upon creation while providing a matching TTL as well as having extend and replace options set as true. Debounce is achieved because if another job with the same deduplication ID is added during this delay (and TTL period) it will replace the previous job with the new one, as well as reseting the TTL, thus ensuring that only the most recent job is kept. This mechanism avoids flooding the queue with duplicates while maintaining the latest job's data. Copy import { Queue } from 'bullmq'; const myQueue = new Queue('Paint'); const worker = new Worker('Paint', async () => {}); worker.once('completed', job => { // only one instance is completed and // 9 additions were ignored console.log(job.data.color); // `white 10` }); // Add 10 jobs with deduplication option in debounce mode. for (let i = 1; i < 11; i++) { await myQueue.add( 'house1', { color: `white ${i}` }, { deduplication: { id: 'customValue', ttl: 5000, extend: true, replace: true, }, delay: 5000, }, ); } In this example, after adding the house painting job with the deduplicated parameters (id, ttl and replace) and 5 seconds as delay, any subsequent job with the same deduplication options added within 5 seconds will replace previous job information. This is useful for scenarios where rapid, repetitive requests are made, such as multiple users or processes attempting to trigger the same job but with different payloads, this way you will get the last updated data when processing a job. Note that you must provide a deduplication id that should represent your job. You can hash your entire job data or a subset of attributes for creating this identifier. Any manual deletion will disable the deduplication. For example, when calling _job.remove_ method. [](https://docs.bullmq.io/guide/jobs/deduplication#the-deduplicated-event) The Deduplicated Event ------------------------------------------------------------------------------------------------------ The **deduplicated** event is emitted whenever a job is ignored due to deduplication in either Simple Mode or Throttle Mode. This event allows you to monitor deduplication activity and take action if needed, such as logging the occurrence or notifying a user that their request was ignored. ### [](https://docs.bullmq.io/guide/jobs/deduplication#listening-for-the-deduplicated-event) Listening for the Deduplicated Event To listen for the **deduplicated** event, use the `QueueEvents` class from BullMQ: Copy import { QueueEvents } from 'bullmq'; const queueEvents = new QueueEvents('myQueue'); queueEvents.on( 'deduplicated', ({ jobId, deduplicationId, deduplicatedJobId }, id) => { console.log(`Job ${deduplicatedJobId} was deduplicated due to existing job ${jobId} with deduplication ID ${deduplicationId}`); }, ); In this example: * `jobId`: The Id of the existing job that triggered the deduplication. * `deduplicationId`: The deduplication Id that caused the job to be ignored. * `deduplicatedJobId`: The Id of the job that was ignored. [](https://docs.bullmq.io/guide/jobs/deduplication#get-deduplication-job-id) Get Deduplication Job Id ---------------------------------------------------------------------------------------------------------- If you need to know the id of the job that started the deduplicated state, you can call the **getDeduplicationJobId** method. Copy const jobId = await myQueue.getDeduplicationJobId('customValue'); [](https://docs.bullmq.io/guide/jobs/deduplication#remove-deduplication-key) Remove Deduplication Key ---------------------------------------------------------------------------------------------------------- If you need to stop deduplication before ttl finishes or before finishing a job, you can call the **queue.removeDeduplicationKey** method. Copy await myQueue.removeDeduplicationKey('customValue'); Or if you want to stop deduplication only if a specific job is the one that caused the deduplication Copy const isDeduplicatedKeyRemoved = await job.removeDeduplicationKey(); [](https://docs.bullmq.io/guide/jobs/deduplication#read-more) Read more: ----------------------------------------------------------------------------- * 💡 [Add Job API Reference](https://api.docs.bullmq.io/classes/v5.Queue.html#add) * 💡 [Queue Remove Deduplication Key API Reference](https://api.docs.bullmq.io/classes/v5.Queue.html#removededuplicationkey) * 💡 [Job Remove Deduplication Key API Reference](https://api.docs.bullmq.io/classes/v5.Job.html#removededuplicationkey) * 💡 [Deduplication Patterns](https://docs.bullmq.io/patterns/deduplication) [PreviousJob Data](https://docs.bullmq.io/guide/jobs/job-data) [NextDelayed](https://docs.bullmq.io/guide/jobs/delayed) Last updated 1 day ago Was this helpful? --- # Running Jaeger | BullMQ The easiest way to run Jaeger is by using Docker compose. If you have docker installed, it is a matter of running this docker-compose.yaml file: Copy services: jaeger: image: jaegertracing/all-in-one:latest container_name: BullMQ_with_opentelemetry_jaeger ports: - '4318:4318' - '16686:16686' Note that we need to expose 2 ports here, the first (4318) one is the endpoint to export our traces and the second one (16686) is our UI. You can now just run this service with: Copy docker-compose up In a few seconds the image will be up and running. You can verify that is working by opening a browser window and pointing it to [http://localhost:16686](http://localhost:16686/search) As no traces has been created yet you will get a quite empty dashboard: ![](https://docs.bullmq.io/~gitbook/image?url=https%3A%2F%2F1340146492-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-LUuDmt_xXMfG66Rn1GA%252Fuploads%252Fgit-blob-b6d84aedee561e5f859913f40391c6938c23cfab%252Fimage.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=df0501e8&sv=2) [PreviousGetting started](https://docs.bullmq.io/guide/telemetry/getting-started) [NextRunning a simple example](https://docs.bullmq.io/guide/telemetry/running-a-simple-example) Last updated 1 year ago Was this helpful? --- # Events | BullMQ All classes in BullMQ emit useful events that inform on the lifecycles of the jobs that are running in the queue. Every class is an `EventEmitter` and emits different events. Some examples: Copy import { Queue } from 'bullmq'; const myQueue = new Queue('Paint'); myQueue.on('waiting', (job: Job) => { // Job is waiting to be processed. }); Copy import { Worker } from 'bullmq'; const myWorker = new Worker('Paint'); myWorker.on('drained', () => { // Queue is drained, no more jobs left }); myWorker.on('completed', (job: Job) => { // job has completed }); myWorker.on('failed', (job: Job) => { // job has failed }); The events above are local for the workers that actually completed the jobs. However, in many situations you want to listen to all the events emitted by all the workers in one single place. For this you can use the [`QueueEvents`](https://api.docs.bullmq.io/classes/v5.QueueEvents.html) class: Copy import { QueueEvents } from 'bullmq'; const queueEvents = new QueueEvents('Paint'); queueEvents.on('completed', ({ jobId: string }) => { // Called every time a job is completed in any worker. }); queueEvents.on( 'progress', ({ jobId, data }: { jobId: string; data: number | object }) => { // jobId received a progress event }, ); The `QueueEvents` class is implemented using [Redis streams](https://redis.io/topics/streams-intro) . This has some nice properties, for example, it provides guarantees that the events are delivered and not lost during disconnections such as it would be the case with standard pub-sub. The event stream is auto-trimmed so that its size does not grow too much, by default it is ~10.000 events, but this can be configured with the `streams.events.maxLen` option. ### [](https://docs.bullmq.io/guide/events#manual-trim-events) Manual trim events In case you need to trim your events manually, you can use `**trimEvents**` method: TypeScript Python Copy import { Queue } from 'bullmq'; const queue = new Queue('paint'); await queue.trimEvents(10); // leaves 10 events Copy from bullmq import Queue queue = Queue('paint') await queue.trimEvents(10) # leaves 10 events [](https://docs.bullmq.io/guide/events#read-more) Read more: ----------------------------------------------------------------- * 💡 [Queue Events API Reference](https://api.docs.bullmq.io/classes/v5.QueueEvents.html) * 💡 [Queue Events Listener API Reference](https://api.docs.bullmq.io/interfaces/v5.QueueEventsListener.html) * 💡 [Queue Listener API Reference](https://api.docs.bullmq.io/interfaces/v5.QueueListener.html) * 💡 [Worker Listener API Reference](https://api.docs.bullmq.io/interfaces/v5.WorkerListener.html) [PreviousReturning job data](https://docs.bullmq.io/guide/returning-job-data) [NextCreate Custom Events](https://docs.bullmq.io/guide/events/create-custom-events) Last updated 1 year ago Was this helpful? --- # Stalled | BullMQ From BullMQ 2.0 and onwards, the `QueueScheduler` is not needed anymore. For manually fetching jobs check this [pattern](https://docs.bullmq.io/patterns/manually-fetching-jobs#checking-for-stalled-jobs) When a job is in an active state (i.e. it is being processed by a worker), it needs to continuously update the queue to notify that the worker is still working on the job. This mechanism prevents a worker that crashes or enters an endless loop from keeping a job in an active state forever. When a worker is not able to notify the queue that it is still working on a given job, that job is moved back to the waiting list, or to the failed set. We then say that the job has stalled and the queue will emit the 'stalled' event. There is not a 'stalled' state, only a 'stalled' event emitted when a job is automatically moved from _active_ to _waiting_ state. If a job stalls more than a predefined limit (see the [`maxStalledCount` option](https://api.docs.bullmq.io/interfaces/v5.WorkerOptions.html#maxstalledcount) ), the job will be failed permanently with the error "_job stalled more than allowable limit_". The default is 1, as stalled jobs should be a rare occurrence, but you can increase this number if needed. In order to avoid stalled jobs, make sure that your worker does not keep the Node.js event loop too busy. The default max stalled check duration is 30 seconds, so as long as you do not perform CPU operations exceeding that value you should not get stalled jobs. Another way to reduce the chance of stalled jobs is using so-called "sandboxed" processors. In this case, the workers will spawn new separate Node.js processes, running separately from the main process. main.ts Copy import { Worker } from 'bullmq'; const worker = new Worker('Paint', painter); painter.ts Copy export default = (job) => { // Paint something } [](https://docs.bullmq.io/guide/jobs/stalled#read-more) Read more: ----------------------------------------------------------------------- * 💡 [Queue Scheduler API Reference](https://api.docs.bullmq.io/classes/v1.QueueScheduler.html) [PreviousRemoving jobs](https://docs.bullmq.io/guide/jobs/removing-job) [NextGetters](https://docs.bullmq.io/guide/jobs/getters) Last updated 3 months ago Was this helpful? --- # Prioritized | BullMQ Jobs can also include a `priority` option. Using priorities, job processing order will be affected by the specified `priority` instead of following a FIFO or LIFO pattern. Adding prioritized jobs is a slower operation than the other types of jobs, with a complexity `O(log(n))` relative to the number of jobs in the prioritized set in the queue. Note that the priorities go from `1` to `2 097 152`, where a lower number is always a **higher** priority than higher numbers. Jobs without a `priority` assigned will get the highest priority, being processed before jobs with priorities assigned to them. Copy import { Queue } from 'bullmq'; const myQueue = new Queue('Paint'); await myQueue.add('wall', { color: 'pink' }, { priority: 10 }); await myQueue.add('wall', { color: 'brown' }, { priority: 5 }); await myQueue.add('wall', { color: 'blue' }, { priority: 7 }); // The wall will be painted first brown, then blue and // finally pink. If several jobs are added with the same priority value, then the jobs within that priority will be processed in [FIFO (_First in, first out_)](https://docs.bullmq.io/guide/jobs/fifo) fashion. [](https://docs.bullmq.io/guide/jobs/prioritized#change-priority) Change priority -------------------------------------------------------------------------------------- If you want to change the `priority` after inserting a job, use the `**changePriority**` method. For example, let's say that you want to change the `priority` from `16` to `1`: Copy const job = await Job.create(queue, 'test2', { foo: 'bar' }, { priority: 16 }); await job.changePriority({ priority: 1, }); or if you want to use the [LIFO (_Last In, First Out_)](https://docs.bullmq.io/guide/jobs/lifo) option: Copy const job = await Job.create(queue, 'test2', { foo: 'bar' }, { priority: 16 }); await job.changePriority({ lifo: true, }); [](https://docs.bullmq.io/guide/jobs/prioritized#get-prioritized-jobs) Get Prioritized jobs ------------------------------------------------------------------------------------------------ As prioritized is a new state. You must use `**getJobs**` or `**getPrioritized**` method as: Copy const jobs = await queue.getJobs(['prioritized']); const jobs2 = await queue.getPrioritized(); [](https://docs.bullmq.io/guide/jobs/prioritized#get-counts-per-priority) Get Counts per Priority ------------------------------------------------------------------------------------------------------ If you want to get the `count` of jobs in `prioritized` status (priorities higher than 0) or in `waiting` status (priority 0), use the `**getCountsPerPriority**` method. For example, let's say that you want to get counts for `priority` `1` and `0`: Copy const counts = await queue.getCountsPerPriority([1, 0]); /* { '1': 11, '0': 10 } */ [](https://docs.bullmq.io/guide/jobs/prioritized#read-more) Read more: --------------------------------------------------------------------------- * 📋 [Faster Priority jobs](https://bullmq.io/news/062123/faster-priority-jobs/) * 💡 [Change Priority API Reference](https://api.docs.bullmq.io/classes/v5.Job.html#changepriority) * 💡 [Get Prioritized API Reference](https://api.docs.bullmq.io/classes/v5.Queue.html#getprioritized) * 💡 [Get Counts per Priority API Reference](https://api.docs.bullmq.io/classes/v5.Queue.html#getcountsperpriority) [PreviousRepeatable](https://docs.bullmq.io/guide/jobs/repeatable) [NextRemoving jobs](https://docs.bullmq.io/guide/jobs/removing-job) Last updated 3 months ago Was this helpful? --- # Get Flow Tree | BullMQ In some situations, you need to get a job and all of its children, grandchildren, and so on. The pattern to solve this requirement consists of using the [`getFlow`](https://api.docs.bullmq.io/classes/v5.FlowProducer.html#getflow) method. Copy const flow = new FlowProducer({ connection }); const originalTree = await flow.add({ name: 'root-job', queueName: 'topQueueName', data: {}, children: [\ {\ name,\ data: { idx: 0, foo: 'bar' },\ queueName: 'childrenQueueName',\ children: [\ {\ name,\ data: { idx: 4, foo: 'baz' },\ queueName: 'grandchildrenQueueName',\ },\ ],\ },\ {\ name,\ data: { idx: 2, foo: 'foo' },\ queueName: 'childrenQueueName',\ },\ {\ name,\ data: { idx: 3, foo: 'bis' },\ queueName: 'childrenQueueName',\ },\ ], }); const { job: topJob } = originalTree; const tree = await flow.getFlow({ id: topJob.id, queueName: 'topQueueName', }); const { children, job } = tree; Each _child_ may have a `job` property and in the case they have children as well, they would have the `children` property You may also need a way to limit that information if you have many children for one of the job nodes. Copy const limitedTree = await flow.getFlow({ id: topJob.id, queueName: 'topQueueName', depth: 1, // get only the first level of children maxChildren: 2, // get only 2 children per node }); const { children, job } = limitedTree; [](https://docs.bullmq.io/guide/flows/get-flow-tree#read-more) Read more: ------------------------------------------------------------------------------ * 💡 [Get Flow API Reference](https://api.docs.bullmq.io/classes/v5.FlowProducer.html#getflow) [PreviousAdding flows in bulk](https://docs.bullmq.io/guide/flows/adding-bulks) [NextFail Parent](https://docs.bullmq.io/guide/flows/fail-parent) Last updated 3 months ago Was this helpful? --- # NestJs | BullMQ There is a compatible module to be used in [NestJs](https://github.com/nestjs/nest) . Copy npm i @nestjs/bullmq Once the installation process is complete, we can import the `**BullModule**` into the root `**AppModule**`. Copy import { Module } from '@nestjs/common'; import { BullModule } from '@nestjs/bullmq'; @Module({ imports: [\ BullModule.forRoot({\ connection: {\ host: 'localhost',\ port: 6379,\ },\ }),\ ], }) export class AppModule {} To register a queue, import the `**BullModule.registerQueue()**` dynamic module, as follows: Copy BullModule.registerQueue({ name: 'queueName', }); To register a flow producer, import the `**BullModule.registerFlowProducer()**` dynamic module, as follows: Copy BullModule.registerFlowProducer({ name: 'flowProducerName', }); [](https://docs.bullmq.io/guide/nestjs#processor) Processor ---------------------------------------------------------------- To register a processor, you may need to use the `**Processor**` decorator: Copy import { Processor, WorkerHost, OnWorkerEvent } from '@nestjs/bullmq'; import { Job } from 'bullmq'; @Processor('queueName') class TestProcessor extends WorkerHost { async process(job: Job): Promise { // do some stuff } @OnWorkerEvent('completed') onCompleted() { // do some stuff } } And then register it as a provider: Copy @Module({ imports: [\ BullModule.registerQueue({\ name: 'queueName',\ connection: {\ host: '0.0.0.0',\ port: 6380,\ },\ }),\ BullModule.registerFlowProducer({\ name: 'flowProducerName',\ connection: {\ host: '0.0.0.0',\ port: 6380,\ },\ }),\ ], providers: [TestProcessor], }) export class AppModule {} ### [](https://docs.bullmq.io/guide/nestjs#read-more) Read more: * 💡 [Queues Technique](https://docs.nestjs.com/techniques/queues) [PreviousArchitecture](https://docs.bullmq.io/guide/architecture) [NextProducers](https://docs.bullmq.io/guide/nestjs/producers) Last updated 2 years ago Was this helpful? --- # Redis™ Compatibility | BullMQ There are several alternatives for Redis and even though BullMQ is full Redis™ compliant with version 6.2.0 or newer, not all the alternatives are going to work properly. In this section we present the vendors that officially support BullMQ and that we regularly test to verify they keep staying compatible. [PreviousQueueScheduler](https://docs.bullmq.io/guide/queuescheduler) [NextDragonfly](https://docs.bullmq.io/guide/redis-tm-compatibility/dragonfly) Last updated 1 year ago Was this helpful? --- # Repeat options | BullMQ There are some options that can be used on all Job Schedulers, to control some aspects of the repetitions. Lets review them one by one: #### [](https://docs.bullmq.io/guide/job-schedulers/repeat-options#start-date) Start date This option sets a future date from which the job will start being scheduled. This can be useful for setting up jobs that should begin repeating on a specific day. Copy const { Queue } = require('bullmq'); const connection = { host: 'localhost', port: 6379 }; const myQueue = new Queue('my-dated-jobs', { connection }); await myQueue.upsertJobScheduler( 'start-later-job', { every: 60000, // every minute startDate: new Date('2024-10-15T00:00:00Z'), // start on October 15, 2024 }, { name: 'timed-start-job', data: { message: 'Starting later' }, }, ); #### [](https://docs.bullmq.io/guide/job-schedulers/repeat-options#end-date) End Date Use this to specify when the job should stop being scheduled, effectively setting an expiration date for the job repetitions. Copy await myQueue.upsertJobScheduler( 'end-soon-job', { every: 60000, // every minute endDate: new Date('2024-11-01T00:00:00Z'), // end on November 1, 2024 }, { name: 'timed-end-job', data: { message: 'Ending soon' }, }, ); #### [](https://docs.bullmq.io/guide/job-schedulers/repeat-options#limit) Limit This setting is used to limit the number of times a job will be repeated. When the count reaches this limit, no more jobs will be produced for the given job scheculer. Copy await myQueue.upsertJobScheduler( 'limited-job', { every: 10000, // every 10 seconds limit: 10, // limit to 10 executions }, { name: 'limited-execution-job', data: { message: 'Limited runs' }, }, ); #### [](https://docs.bullmq.io/guide/job-schedulers/repeat-options#immediately) immediately This setting forces the job to execute as soon as it is added, regardless of the schedule. This can help in situations where an immediate action is required before entering a regular cycle. When you use the every option in BullMQ, it schedules jobs based on fixed time intervals, which might seem a bit counterintuitive initially. For instance, if you set an interval of 2000ms, jobs will be triggered at every even second—such as 0, 2, 4, 6, 8 seconds, and so on. This means the scheduling aligns with the clock, regardless of when you actually added the job. If you need a job to begin processing immediately after you add a job scheduler, regardless of the interval’s alignment with the clock, you can use the immediately setting. This is especially crucial for long intervals. For example, if you set the job to repeat monthly, normally it would wait to start until the first second of the next month. If you add the job mid-month, it would not start until the beginning of the following month. Using immediately ensures the first instance of the job runs as soon as it’s added, bypassing the wait until the scheduled interval begins. Copy await myQueue.upsertJobScheduler( 'immediate-job', { every: 86400000, // once a day immediately: true, // execute the first one immediately }, { name: 'instant-job', data: { message: 'Immediate start' }, }, ); From version 5.19.0 and onwards the "immediately" option has been deprecated. The current behaviour is as if "immediately" was always true, in other words, the first repetition will always be immediately for a new inserted job scheduler, and then repeat according to the "every" setting. Subsequent calls to upsertJobScheduler for **existing** schedulers will not lead to immediate repetitions and will instead follow the "every" interval. [PreviousRepeat Strategies](https://docs.bullmq.io/guide/job-schedulers/repeat-strategies) [NextManage Job Schedulers](https://docs.bullmq.io/guide/job-schedulers/manage-job-schedulers) Last updated 9 months ago Was this helpful? --- # Producers | BullMQ Job producers add jobs to queues. Producers are typically application services (Nest providers). To add jobs to a queue, first inject the queue into the service as follows: Copy import { Injectable } from '@nestjs/common'; import { InjectQueue } from '@nestjs/bullmq'; import { Queue } from 'bullmq'; @Injectable() export class AudioService { constructor(@InjectQueue('audio') private audioQueue: Queue) {} } The `**@InjectQueue()**` decorator identifies the queue by its name, as provided in the `**registerQueue()**`. Now, add a job by calling the queue's add() method. Copy const job = await this.audioQueue.add('sample', { foo: 'bar', }); [](https://docs.bullmq.io/guide/nestjs/producers#flow-producers) Flow Producers ------------------------------------------------------------------------------------ To add flows, first inject the flow producer into the service as follows: Copy import { Injectable } from '@nestjs/common'; import { InjectFlowProducer } from '@nestjs/bullmq'; import { FlowProducer } from 'bullmq'; @Injectable() export class FlowService { constructor( @InjectFlowProducer('flow') private fooFlowProducer: FlowProducer, ) {} } The `**@InjectFlowProducer()**` decorator identifies the flow producer by its `name`, as provided in the `**registerFlowProducer()**`. Now, add a flow by calling the flow producer's \`add()\`\` method. Copy const job = await this.fooFlowProducer.add({ name: 'root-job', queueName: 'topQueueName', data: {}, children: [\ {\ name,\ data: { idx: 0, foo: 'bar' },\ queueName: 'childrenQueueName',\ },\ ], }); ### [](https://docs.bullmq.io/guide/nestjs/producers#read-more) Read more: * 💡 [Queues Technique](https://docs.nestjs.com/techniques/queues) [PreviousNestJs](https://docs.bullmq.io/guide/nestjs) [NextQueue Events Listeners](https://docs.bullmq.io/guide/nestjs/queue-events-listeners) Last updated 2 years ago Was this helpful? --- # Delayed | BullMQ Delayed jobs are a special type of job that is placed into a special "delayed set", instead of being processed as fast as possible. After the delay time has passed, the job is processed as a regular job. In order to add delayed jobs to the queue, use the `delay` option with the amount of time (in milliseconds) that you want to delay the job with. Note that it is not guaranteed that the job will be processed at the _exact_ delayed time specified, as it depends on how busy the workers are when the time has passed, and how many other delayed jobs are scheduled at that exact time. In practice, however, the delay time is quite accurate in most cases. This is an example of how to add delayed jobs to a queue: Copy import { Queue } from 'bullmq'; const myQueue = new Queue('Paint'); // Add a job that will be delayed by at least 5 seconds. await myQueue.add('house', { color: 'white' }, { delay: 5000 }); If you want to process the job after a specific point in time, just add the time remaining to that point in time. For example, let's say you want to process the job on the third of July 2035 at 10:30: Copy const targetTime = new Date('03-07-2035 10:30'); const delay = Number(targetTime) - Number(new Date()); await myQueue.add('house', { color: 'white' }, { delay }); [](https://docs.bullmq.io/guide/jobs/delayed#change-delay) Change delay ---------------------------------------------------------------------------- If you want to reschedule a delayed job _after_ inserting it, use the `**changeDelay**` method. This method reschedules the job to execute after the specified number of milliseconds from the current time, regardless of the original delay. For example: Copy const job = await Job.create(queue, 'test', { foo: 'bar' }, { delay: 2000 }); // Reschedule the job to execute 4000ms (4 seconds) from now await job.changeDelay(4000); Only jobs currently in the **delayed** state can have their delay changed. [](https://docs.bullmq.io/guide/jobs/delayed#read-more) Read more: ----------------------------------------------------------------------- * 💡 [Change Delay API Reference](https://api.docs.bullmq.io/classes/v5.Job.html#changedelay) [PreviousDeduplication](https://docs.bullmq.io/guide/jobs/deduplication) [NextRepeatable](https://docs.bullmq.io/guide/jobs/repeatable) Last updated 1 month ago Was this helpful? --- # Architecture | BullMQ In order to use the full potential of Bull queues, it is important to understand the lifecycle of a job. From the moment a producer calls the [`add`](https://api.docs.bullmq.io/classes/v5.Queue.html#add) method on a `Queue` instance, a job enters a lifecycle where it will be in different states, until its completion or failure (although technically a failed job could be retried and get a new lifecycle). ![](https://docs.bullmq.io/~gitbook/image?url=https%3A%2F%2F1340146492-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-LUuDmt_xXMfG66Rn1GA%252Fuploads%252Fgit-blob-1a63de3b90a0f66726ce0c6f9ada11d9ebd651c5%252Fmermaid-diagram-2023-06-22-093303.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=d5ee56fe&sv=2) Lifecycle of a job - Queue When a job is added to a queue it can be in one of three states: * **“wait”**: a waiting list, where all jobs must enter before they can be processed. * **“prioritized”**: implies that a job with higher priority will be processed first. * **“delayed”**: implies that the job is waiting for some timeout or to be promoted for being processed. These jobs are not processed directly, but instead are placed at the beginning of the waiting list, or in a prioritized set, and processed as soon as a worker is idle. Note that priorities go from `0` to `2^21`, where 0 is the highest priority. This follows a similar standard as processed in Unix (https://en.wikipedia.org/wiki/Nice\_(Unix), where a higher number means less priority). The next state for a job is the **“active”** state. The active state is represented by a set, and are jobs that are currently being processed (i.e. they are running in the `process` function explained in the previous chapter). A job can be in the active state for an unlimited amount of time until the process is completed or an exception is thrown so that the job will end in either the **“completed”** or the **“failed”** status. Another way to add a job is by the [`add`](https://api.docs.bullmq.io/classes/v5.FlowProducer.html#add) method on a flow producer instance. ![](https://docs.bullmq.io/~gitbook/image?url=https%3A%2F%2F1340146492-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-LUuDmt_xXMfG66Rn1GA%252Fuploads%252Fgit-blob-f3f8bd7c093d0f07d29d785667f09c9c673060ad%252Fmermaid-diagram-2023-06-22-095138.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=9df72de5&sv=2) Lifecycle of a job - Flow Producer When a job is added by a flow producer, it can be in one of three states, it can either be in the **“wait”** or **“prioritized“** or **“delayed“** status, when there aren't children, or it can be in a **“waiting-children”** status: a waiting-children status implies that the job is waiting for all its children to be completed, however, a waiting-children job will not be processed directly, instead it will be placed at the waiting list or at delayed set (if `delay` is provided) or at prioritized set (if `delay` is 0 and `priority` is greater than 0) as soon as the last child is marked as completed. [PreviousAWS Elasticache](https://docs.bullmq.io/guide/redis-tm-hosting/aws-elasticache) [NextNestJs](https://docs.bullmq.io/guide/nestjs) Last updated 1 year ago Was this helpful? --- # Flows | BullMQ The following pattern, although still useful, has been mostly super-seeded by the new [Flows](https://docs.bullmq.io/guide/flows) functionality In some situations, you may need to execute a flow of several actions, any of which could fail. For example, you may need to update a database, make calls to external services, or any other kind of asynchronous call. Sometimes it may not be possible to create an [idempotent job](https://docs.bullmq.io/patterns/idempotent-jobs) that can execute all these actions again in the case one of them failed for any reason. Instead, we may want to be able to only re-execute the action that failed and continue executing the rest of the actions that have not yet been executed. The pattern to solve this issue consists of dividing the flow of actions into one queue for every action. When the first action completes, it places the next action as a job in its corresponding queue. [PreviousNamed Processor](https://docs.bullmq.io/patterns/named-processor) [NextIdempotent jobs](https://docs.bullmq.io/patterns/idempotent-jobs) Last updated 2 years ago Was this helpful? --- # Getters | BullMQ When jobs are added to a queue, they will be in different statuses during their lifetime. BullMQ provides methods to retrieve information and jobs from the different statuses. ![](https://docs.bullmq.io/~gitbook/image?url=https%3A%2F%2F1340146492-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-LUuDmt_xXMfG66Rn1GA%252Fuploads%252Fgit-blob-fd5f0839e179555c5244c92e0ac2f2eb9199e22f%252Fcomplete-architecture.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=c49deb22&sv=2) Lifecycle of a job #### [](https://docs.bullmq.io/guide/jobs/getters#job-counts) Job Counts It is often necessary to know how many jobs are in a given status: TypeScript Python Copy import { Queue } from 'bullmq'; const myQueue = new Queue('Paint'); const counts = await myQueue.getJobCounts('wait', 'completed', 'failed'); // Returns an object like this { wait: number, completed: number, failed: number } Copy from bullmq import Queue myQueue = Queue('Paint') counts = await myQueue.getJobCounts('wait', 'completed', 'failed') # Returns an object like this { wait: number, completed: number, failed: number } The available status are: * _completed_, * _failed_, * _delayed_, * _active_, * _wait_, * _waiting-children_, * _prioritized_, * _paused_, and * _repeat_. #### [](https://docs.bullmq.io/guide/jobs/getters#get-jobs) Get Jobs It is also possible to retrieve the jobs with pagination style semantics. For example: TypeScript Python Copy const completed = await myQueue.getJobs(['completed'], 0, 100, true); // returns the oldest 100 jobs Copy completed = await myQueue.getJobs(['completed'], 0, 100, True) # returns the oldest 100 jobs [](https://docs.bullmq.io/guide/jobs/getters#read-more) Read more: ----------------------------------------------------------------------- * 💡 [Get Job Counts API Reference](https://api.docs.bullmq.io/classes/v5.Queue.html#getjobcounts) * 💡 [Get Jobs API Reference](https://api.docs.bullmq.io/classes/v5.Queue.html#getjobs) [PreviousStalled](https://docs.bullmq.io/guide/jobs/stalled) [NextJob Schedulers](https://docs.bullmq.io/guide/job-schedulers) Last updated 3 months ago Was this helpful? --- # Getting started | BullMQ In this guide we will show how to setup a local telemetry facility for BullMQ that should serve you as a good basis for how to integrate it in larger applications. As OpenTelemetry is a well supported standard there are many third party UIs for visualizing the traces and spans generated when running an application, for this guide we will use [Jaeger](https://www.jaegertracing.io/) . We assume that you have a working BullMQ project that you want to add telemetry to it, so lets start by adding the `bullmq-otel` package to the project: Copy npm add --save bullmq-otel This module provides a working implementation of BullMQ's telemetry interface for the OpenTelemetry standard. Adding it to your existing Queue's instances and Workers is quite straightforward: Copy import { Queue } from 'bullmq' import { BullMQOtel } from "bullmq-otel"; const queue = new Queue("myQueue", { connection: { host: "127.0.0.1", port: 6379, }, telemetry: new BullMQOtel("simple-guide"), }); Copy import { Worker } from "bullmq"; import { BullMQOtel } from "bullmq-otel"; const worker = new Worker( "myQueue", async (job) => { return 'some value' }, { name: "myWorker", connection: { host: "127.0.0.1", port: 6379, }, telemetry: new BullMQOtel("simple-guide"), } ); This is all that is needed in order to start producing traces and spans to observe your code. [PreviousTelemetry](https://docs.bullmq.io/guide/telemetry) [NextRunning Jaeger](https://docs.bullmq.io/guide/telemetry/running-jaeger) Last updated 1 year ago Was this helpful? --- # Remove Child Dependency | BullMQ In some situations, you may have a parent job and need to remove the dependency of one of its children. The pattern to solve this requirement consists on using the **removeChildDependency** method. It will make sure that if the job is the last pending child, to move its parent to _waiting_ and it won't be listed in unprocessed list of the parent. Copy const flow = new FlowProducer({ connection }); const originalTree = await flow.add({ name: 'root-job', queueName: 'topQueueName', data: {}, children: [\ {\ name,\ data: { idx: 0, foo: 'bar' },\ queueName: 'childrenQueueName',\ opts: {},\ },\ ], }); await originalTree.children[0].job.removeChildDependency(); As soon as a **child** calls this method, it will verify if it has an existing parent, if not, it'll throw an error. Failed or completed children using this option won't generate any removal as they won't be part of unprocessed list. [PreviousIgnore Dependency](https://docs.bullmq.io/guide/flows/ignore-dependency) [NextMetrics](https://docs.bullmq.io/guide/metrics) Last updated 18 days ago Was this helpful? --- # Migrations | BullMQ BullMQ's team regularly releases new versions packed with bug fixes, new features, or occasionally, breaking changes. As a user operating in a production environment, you might find upgrading to these new versions while maintaining service continuity challenging. This guide offers tips and advice to ease your transition. The strategies needed vary depending on the nature of the upgrade. Regardless, we urge you to read this guide thoroughly, irrespective of the upgrade you're pursuing. [](https://docs.bullmq.io/guide/migration-to-newer-versions#general-advice) General advice ----------------------------------------------------------------------------------------------- When upgrading BullMQ, always consult the _Changelog_. It helps determine the extent of the changes and flags crucial considerations before upgrading. Avoid making large leaps between versions. If you're currently on version 1.3.7, for instance, a jump to version 4.2.6 may not be advisable. Upgrade incrementally whenever possible. Start with as many bugfix releases as possible, then proceed to new features, and finally, the major releases that encompass breaking changes. [](https://docs.bullmq.io/guide/migration-to-newer-versions#bugfix-upgrade) Bugfix upgrade ----------------------------------------------------------------------------------------------- Bugfix releases increase only the micro version number according to [SemVer (_Semantic Versioning_)](https://semver.org/) (for instance, an upgrade from 3.14.4 to 3.14.7). Bugfix upgrades require no special strategies; simply update your instances to the latest version without changing your code or deployment. While it's not critical that all instances run on the same version, we recommend it for consistency. [](https://docs.bullmq.io/guide/migration-to-newer-versions#new-feature-upgrade) New feature upgrade --------------------------------------------------------------------------------------------------------- Following the SemVer specification, new features result in an increase in the minor version number (like going from 3.14.7 to 3.20.5). Generally, you can treat feature upgrades like bugfix upgrades — update all your instances to the latest version. However, if you're also upgrading your code to utilize a new feature, ensure it's backward compatible with the older BullMQ version. Otherwise, an older `Worker` might stop functioning if a new `Queue` adds jobs leveraging a feature the older `Worker` doesn't understand. The strategy here is to first upgrade all your instances to the version featuring the new functionality. After confirming all instances run the new version, proceed to deploy your code depending on those new features. [](https://docs.bullmq.io/guide/migration-to-newer-versions#breaking-changes) Breaking changes --------------------------------------------------------------------------------------------------- Occasionally, unavoidable changes incompatible with previous versions are made. We strive to minimize these, classifying them into two types: API-breaking changes and data structure-breaking changes. ### [](https://docs.bullmq.io/guide/migration-to-newer-versions#api-breaking-changes) API breaking changes API breaking changes could involve altered method parameters, removals, or different operational methods. These changes are usually straightforward to apply — you can run your BullMQ-dependent unit tests and address issues based on these changes. If you're using TypeScript, compilation errors will likely surface. Always read the [changelog](https://github.com/taskforcesh/bullmq/blob/master/docs/gitbook/guide/changelog.md) for essential information about these changes. ### [](https://docs.bullmq.io/guide/migration-to-newer-versions#data-structure-breaking-changes) Data structure breaking changes Data structure changes, which alter the queue's underlying structure, are more challenging. They can be either * **additive** (introducing new data structures that older BullMQ versions don't understand), or * **destructive** (changing or eradicating older data structures). For additive changes, you could simply upgrade all instances to the new version — they should apply the change and continue working without issues, akin to a [new feature upgrade](https://docs.bullmq.io/guide/migration-to-newer-versions#new-features-upgrade) . Destructive changes are the most demanding, as these fundamental alterations may make older versions unworkable, making rollback impossible if the upgrade fails. The [changelog](https://github.com/taskforcesh/bullmq/blob/master/docs/gitbook/guide/changelog.md) will provide crucial information to guide you through this type of upgrade. [](https://docs.bullmq.io/guide/migration-to-newer-versions#some-general-strategies) Some general strategies ----------------------------------------------------------------------------------------------------------------- For the most demanding upgrades, you might find these strategies useful: ### [](https://docs.bullmq.io/guide/migration-to-newer-versions#pause-upgrade-unpause) Pause/Upgrade/Unpause Since BullMQ supports global pause, one possible strategy, if suitable for your business case, is to pause the queue(s), wait until all current queued jobs have been processed, then perform the upgrade. Once all instances running BullMQ have been upgraded, you can unpause and let new jobs be processed by the new workers. Be aware this strategy is less useful if breaking changes affect `Queue` instances. Always consult the changelog for this type of information. ### [](https://docs.bullmq.io/guide/migration-to-newer-versions#use-new-queues-altogether) Use new queues altogether This drastic solution involves discontinuing use of older queues and creating new ones. You could rename older queues (e.g., "myQueueV2"), use a new Redis host, or maintain two versions of the service—one running an older BullMQ version with old queues, and a newer one with the latest BullMQ and a different set of queues. When the older version has no more jobs to process, it can be retired, leaving only the upgraded version. [PreviousGoing to production](https://docs.bullmq.io/guide/going-to-production) [NextBull to BullMQ](https://docs.bullmq.io/guide/migration-to-newer-versions/bull-to-bullmq) Last updated 2 years ago Was this helpful? --- # Adding jobs in bulk across different queues | BullMQ Sometimes it is necessary to atomically add jobs to different queues in bulk. For example, there could be a requirement that all the jobs must be created or none of them. Also, adding jobs in bulk can be faster, since it reduces the number of roundtrips to Redis: You may be think of [`queue.addBulk`](https://api.docs.bullmq.io/classes/v5.Queue.html#addbulk) , but this method only adds jobs to a single queue. Another option is [`flowProducer.addBulk`](https://api.docs.bullmq.io/classes/v5.FlowProducer.html#addbulk) , so let's see an example: Copy import { FlowProducer } from 'bullmq'; const flow = new FlowProducer({ connection }); const trees = await flow.addBulk([\ {\ name: 'job-1',\ queueName: 'queueName-1',\ data: {}\ },\ {\ name: 'job-2',\ queueName: 'queueName-2',\ data: {}\ },\ ]); It is possible to add individual jobs without children. This call can only succeed or fail, and all or none of the jobs will be added. [](https://docs.bullmq.io/patterns/adding-bulks#read-more) Read more: -------------------------------------------------------------------------- * 💡 [Add Bulk API Reference](https://api.docs.bullmq.io/classes/v5.FlowProducer.html#addbulk) [PreviousTroubleshooting](https://docs.bullmq.io/guide/troubleshooting) [NextDeduplication](https://docs.bullmq.io/patterns/deduplication) Last updated 3 months ago Was this helpful? --- # Failing fast when Redis is down | BullMQ By design, BullMQ reconnects to Redis automatically. If jobs are added to a queue while the queue instance is disconnected from Redis, the `add` command will not fail; instead, the call will keep waiting for a reconnection to occur until it can complete. This behavior is not always desirable; for example, if you have implemented a REST API that results in a call to `add`, you do not want to keep the HTTP call busy while `add` is waiting for the queue to reconnect to Redis. In this case, you can pass the option `enableOfflineQueue: false`, so that `ioredis` do not queue the commands and instead throws an exception: Copy const myQueue = new Queue("transcoding", { connection: { enableOfflineQueue: false, }, }); app.post("/jobs", async (req, res) => { try { const job = await myQueue.add("myjob", { req.body }); res.status(201).json(job.id); }catch(err){ res.status(503).send(err); } }) Using this approach, the caller can catch the exception and act upon it depending on its requirements (for example, retrying the call or giving up). Currently, there is a limitation in that the Redis instance must at least be online while the queue is being instantiated. [PreviousProcess Step Jobs](https://docs.bullmq.io/patterns/process-step-jobs) [NextStop retrying jobs](https://docs.bullmq.io/patterns/stop-retrying-jobs) Last updated 2 years ago Was this helpful? --- # Timeout jobs | BullMQ BullMQ does not provide a specific mechanism to timeout jobs, however this can be accomplished in many cases with a custom timeout code in the worker's process function. The basic concept is to set up a timeout callback that will abort the job processing, and throw an UnrecoverableError (to avoid retries, although this may not alway be the desired behaviour, if so just throw a normal Error). Note how we specified the timeout as a property of the job's data, in case we want to have different timeouts depending on the job, but we could also have a fixed constant timeout for all jobs if we wanted. Copy const worker = new Worker('foo', async job => { let controller = new AbortController(); const timer = setTimeout(() => controller.abort(), job.data.timeout); try { await doSomethingAbortable(controller.signal); } catch(err) { if (err.name == "AbortError") { throw new UnrecoverableError("Timeout"); } else { throw err; } } finally { clearTimeout(timer); } }); In this simple example we assume that doSomethingAbortable is an asynchronous function that can handle abort signals and abort itself gracefully. Now let's see another case when we want to timeout a fetch call, it would look like this: Copy const worker = new Worker("foo", async (job) => { let controller = new AbortController(); const timer = setTimeout(() => controller.abort(), job.data.timeout); try { let response = await fetch("/slowserver.com", { signal: controller.signal, }); const result = await response.text(); } catch (err) { if (err.name == "AbortError") { throw new UnrecoverableError("Timeout"); } else { throw err; } } finally { clearTimeout(timer) } }); In this example we are aborting the fetch call using [AbortController](https://developer.mozilla.org/en-US/docs/Web/API/AbortController) , which is the default mechanism provided by fetch to abort calls. Note that abort will even cause the async call to response.text() to also throw an Abort exception. In summary, while it is possible to implement timeout in your jobs, the mechanism to do it may vary depending on the type of asynchronous operations your jobs is performing, but in many cases using AbortController in combination with a setTimeout is more than enough. [PreviousStop retrying jobs](https://docs.bullmq.io/patterns/stop-retrying-jobs) [NextTimeout for Sandboxed processors](https://docs.bullmq.io/patterns/timeout-for-sandboxed-processors) Last updated 1 year ago Was this helpful? --- # Manual retrying | BullMQ There are situations when it is useful to retry a job right away when it is being processed. This can be handled using the `moveToWait` method. However, it is important to note that when a job is being processed by a worker, the worker keeps a lock on this job with a certain token value. For the `moveToWait` method to work, we need to pass said token so that it can unlock without error. Finally, we need to exit from the processor by throwing a special error (`WaitingError`) that will signal to the worker that the job has been retried so that it does not try to complete (or fail the job) instead. Copy import { WaitingError, Worker } from 'bullmq'; const worker = new Worker( 'queueName', async (job: Job, token?: string) => { try { await doSomething(); } catch (error) { await job.moveToWait(token); throw new WaitingError(); } }, { connection }, ); [](https://docs.bullmq.io/patterns/manual-retrying#read-more) Read more: ----------------------------------------------------------------------------- * 💡 [Move To Wait API Reference](https://api.docs.bullmq.io/classes/v5.Job.html#movetowait) [PreviousThrottle jobs](https://docs.bullmq.io/patterns/throttle-jobs) [NextProcess Step Jobs](https://docs.bullmq.io/patterns/process-step-jobs) Last updated 3 months ago Was this helpful? --- # AWS Elasticache | BullMQ Elasticache is a managed caching service offered by Amazon Web Services (AWS), and it can be a good option when using BullMQ within the AWS infrastructure. Here are some points to consider when using Elasticache with BullMQ within AWS: 1. Use the standard cache-nodes setup (i.e. not the serverless version, as serverless for the moment uses an incompatible maxmemory-policy) 2. In order to access your Elasticache instance you will need to create a security group that allows the instance to be accessible to the services running BullMQ instances. ![](https://docs.bullmq.io/~gitbook/image?url=https%3A%2F%2F1340146492-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-LUuDmt_xXMfG66Rn1GA%252Fuploads%252Fgit-blob-b5e688395c2061a69a970890d35e77fa217a460a%252Fimage.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=ebcd5fc2&sv=2) 4\. You will need to specify a VPC and an Inbound rule. Most likely just a custom TCP with port range 6379 and some suitable source (anywhere works well for testing and in some simpler cases), remember that the cluster will not be accessible outside from AWS in any case. 5. Attach the security group to your Elasticache cluster and to the services that need to access it. 6. Make sure that you are using maxmemory-policy: noeviction in your Redis parameters. As you cannot modify any default parameter group you will need to create a new one. Copy 1. Go to Elasticache > Parameter Groups and click on Create. 2. Fill name, description, and Family, at the time of writing redis7 is the newest and the recommended one.
7\. Click on "Create". Then find the parameter group in the list. Click on Edit parameter values and then search for maxmemory-policy: ![](https://docs.bullmq.io/~gitbook/image?url=https%3A%2F%2F1340146492-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-LUuDmt_xXMfG66Rn1GA%252Fuploads%252Fgit-blob-6305ad6bc21de4aa7e6e56ea7d8ee9772d895226%252Fimage.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=cc5d30d7&sv=2) 1. Change the value to "noeviction": ![](https://docs.bullmq.io/~gitbook/image?url=https%3A%2F%2F1340146492-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-LUuDmt_xXMfG66Rn1GA%252Fuploads%252Fgit-blob-556e6c2decabcb31969d8e0913077e2ba118142b%252Fimage.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=65cd0002&sv=2) 1. Save changes. Now you can go to your elasticache cluster and change the parameter group to your custom group. Just find your instance, click on modify and go to cluster settings where you can change the parameter group: ![](https://docs.bullmq.io/~gitbook/image?url=https%3A%2F%2F1340146492-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-LUuDmt_xXMfG66Rn1GA%252Fuploads%252Fgit-blob-837ae979da2ee9a9ed1405a576068d3dfbab2ea9%252Fimage.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=679de653&sv=2) 1. Preview changes and Modify. After this your cluster is ready to be used with BullMQ. [PreviousAWS MemoryDB](https://docs.bullmq.io/guide/redis-tm-hosting/aws-memorydb) [NextArchitecture](https://docs.bullmq.io/guide/architecture) Last updated 17 days ago Was this helpful? --- # QueueScheduler | BullMQ The `QueueScheduler` is deprecated from BullMQ 2.0 and onwards. The information below is only relevant for older versions. The `QueueScheduler` is a helper class used to manage stalled and delayed jobs for a given Queue. Copy import { QueueScheduler } from 'bullmq'; const queueScheduler = new QueueScheduler('test'); // Later, when shuting down gracefully await queueScheduler.close(); This class automatically moves delayed jobs back to the waiting queue when it is the right time to process them. It also automatically checks for stalled jobs (i.e. detects jobs that are active but where the worker has either crashed or stopped working properly). [Stalled jobs](https://docs.bullmq.io/guide/jobs/stalled) are moved back or failed depending on the settings selected when instantiating the class. You need at least one `QueueScheduler` running somewhere for a given queue if you require functionality such as delayed jobs, retries with backoff, and rate limiting. The reason for having this functionality in a separate class instead of in the workers (as in Bull 3.x) is because whereas you may want to have a large number of workers for parallel processing, for the scheduler you probably only want a couple of instances for each queue that requires delayed or stalled checks. One will be enough but you can have more just for redundancy. It is ok to have as many `QueueScheduler` instances as you want, just keep in mind that every instance will perform some bookkeeping so it may create some noticeable CPU and IO usage in your Redis instances. [](https://docs.bullmq.io/guide/queuescheduler#read-more) Read more: ------------------------------------------------------------------------- * 💡 [Queue Scheduler API Reference](https://api.docs.bullmq.io/classes/v1.QueueScheduler.html) [PreviousRunning a simple example](https://docs.bullmq.io/guide/telemetry/running-a-simple-example) [NextRedis™ Compatibility](https://docs.bullmq.io/guide/redis-tm-compatibility) Last updated 10 months ago Was this helpful? --- # Idempotent jobs | BullMQ In order to take advantage of [the ability to retry failed jobs](https://docs.bullmq.io/guide/retrying-failing-jobs) , your jobs should be designed with failure in mind. This means that it should not make a difference to the final state of the system if a job successfully completes on its first attempt, or if it fails initially and succeeds when retried. This is called _Idempotence_. To achieve this behaviour, your jobs should be as atomic and simple as possible. Performing many different actions (such as database updates, API calls, ...) at once makes it hard to keep track of the process flow and, if needed, rollback partial progress when an exception occurs. Simpler jobs also means simpler debugging, identifying bottlenecks, etc. If necessary, split complex jobs [as described in the flow pattern](https://docs.bullmq.io/patterns/flows) . [PreviousFlows](https://docs.bullmq.io/patterns/flows) [NextThrottle jobs](https://docs.bullmq.io/patterns/throttle-jobs) Last updated 2 years ago Was this helpful? --- # Stop retrying jobs | BullMQ When a processor throws an exception that is considered unrecoverable, you should use the `UnrecoverableError` class. In this case, BullMQ will just move the job to the failed set without performing any retries, overriding any `attempts` settings used when adding the job to the queue. Copy import { Worker, UnrecoverableError } from 'bullmq'; const worker = new Worker( 'foo', async job => { doSomeProcessing(); throw new UnrecoverableError('Unrecoverable'); }, { connection }, ); await queue.add( 'test-retry', { foo: 'bar' }, { attempts: 3, backoff: 1000, }, ); [](https://docs.bullmq.io/patterns/stop-retrying-jobs#fail-job-when-manual-rate-limit) Fail job when manual rate-limit --------------------------------------------------------------------------------------------------------------------------- When a job is rate limited using `RateLimitError` and tried again, the `attempts` check is ignored, as rate limiting is not considered a real error. However, if you want to manually check the attempts and avoid retrying the job, you can check `job.attemptsStarted` as following: Copy import { Worker, RateLimitError, UnrecoverableError } from 'bullmq'; const worker = new Worker( 'myQueue', async job => { const [isRateLimited, duration] = await doExternalCall(); if (isRateLimited) { await queue.rateLimit(duration); if (job.attemptsStarted >= job.opts.attempts) { throw new UnrecoverableError('Unrecoverable'); } // Do not forget to throw this special exception, // since we must differentiate this case from a failure // in order to move the job to wait again. throw new RateLimitError(); } }, { connection, limiter: { max: 1, duration: 500, }, }, ); `job.attemptsMade` is increased when any error different than `RateLimitError`, `DelayedError` or `WaitingChildrenError` is thrown. While `job.attemptsStarted` is increased every time that a job is moved to active. [](https://docs.bullmq.io/patterns/stop-retrying-jobs#read-more) Read more: -------------------------------------------------------------------------------- * 💡 [Rate Limit API Reference](https://api.docs.bullmq.io/classes/v5.Queue.html#ratelimit) [PreviousFailing fast when Redis is down](https://docs.bullmq.io/patterns/failing-fast-when-redis-is-down) [NextTimeout jobs](https://docs.bullmq.io/patterns/timeout-jobs) Last updated 3 months ago Was this helpful? --- # Queue Events Listeners | BullMQ To register a QueueEvents instance, you need to use the `**QueueEventsListener**` decorator: Copy import { QueueEventsListener, QueueEventsHost, OnQueueEvent, } from '@nestjs/bullmq'; @QueueEventsListener('queueName') export class TestQueueEvents extends QueueEventsHost { @OnQueueEvent('completed') onCompleted({ jobId, }: { jobId: string; returnvalue: string; prev?: string; }) { // do some stuff } } And then register it as a provider: Copy @Module({ imports: [\ BullModule.registerQueue({\ name: 'queueName',\ connection: {\ host: '0.0.0.0',\ port: 6380,\ },\ }),\ ], providers: [TestQueueEvents], }) export class AppModule {} [](https://docs.bullmq.io/guide/nestjs/queue-events-listeners#read-more) Read more: ---------------------------------------------------------------------------------------- * 💡 [Queues Technique](https://docs.nestjs.com/techniques/queues) * 💡 [Register Queue API Reference](https://nestjs.bullmq.pro/classes/BullModule.html#registerQueue) * 💡 [Queue Events Listener API Reference](https://api.docs.bullmq.io/interfaces/v5.QueueEventsListener.html) [PreviousProducers](https://docs.bullmq.io/guide/nestjs/producers) [NextGoing to production](https://docs.bullmq.io/guide/going-to-production) Last updated 1 year ago Was this helpful? --- # Troubleshooting | BullMQ In this section, you will be able to find hints and solutions for some common errors you might encounter when using BullMQ. ### [](https://docs.bullmq.io/guide/troubleshooting#missing-locks) Missing Locks An error that can be thrown by the workers has the following structure: “Missing lock for job 1234. moveToFinished.” This error occurs when a job being processed by a worker unexpectedly loses its “lock.” When a worker processes a job, it requires a special lock key to ensure that the job is currently “owned” by that worker, preventing other workers from picking up the same job. However, this lock can be deleted, and such a deletion may not be detected until the worker tries to move the job to a completed or failed status. A lock can be deleted for several reasons, the most common being: * The worker is consuming too much CPU and has no time to renew the lock every 30 seconds (which is the default expiration time for locks) * The worker has lost communication with Redis and cannot renew the lock in time. * The job has been forcefully removed using one of BullMQ's APIs to remove jobs (or by removing the entire queue). * The Redis instance has a wrong [maxmemory](https://docs.bullmq.io/guide/going-to-production#max-memory-policy) policy; it should be no-eviction to avoid Redis removing keys with the expiration date before hand. ### [](https://docs.bullmq.io/guide/troubleshooting#invalid-or-undefined-environment-variables) Invalid or Undefined Environment Variables If you rely on environment variables (e.g., for queue names or job data), a common pitfall is passing them directly to BullMQ methods when those environment variables are: * Undefined (not set at all) * Empty strings (i.e., "") * Non-string values (e.g., inadvertently passing objects or arrays) This can cause BullMQ’s internal Lua scripts to throw ERR Error running script ... Lua redis() command arguments must be strings or integers. It typically happens when a parameter passed into the Redis command ends up being something other than a valid string or number. **Best Practices to Avoid This Error** 1. Validate Environment Variables Early In your application’s initialization code, check all required environment variables: Copy const queueName = process.env.QUEUE_NAME; if (!queueName) { throw new Error("QUEUE_NAME is not defined or is empty."); } const queue = new Queue(queueName, { ... }); This ensures you fail fast if a variable isn’t set, instead of causing hidden Lua script errors. 1. Use TypeScript Strictness If you’re using TypeScript, enable strictNullChecks and explicitly type environment variables as string | undefined. That way, any code that attempts to use them without proper checks will cause a compile-time error. 1. Provide Defaults Where Appropriate In some cases, you may want a fallback value if an environment variable is missing: Copy const queueName = process.env.QUEUE_NAME ?? 'defaultQueue'; But be sure this fallback is actually valid in your production workflow. Following these guidelines helps prevent obscure Lua script errors in BullMQ that stem from passing undefined or invalid arguments into Redis commands. [PreviousBull to BullMQ](https://docs.bullmq.io/guide/migration-to-newer-versions/bull-to-bullmq) [NextAdding jobs in bulk across different queues](https://docs.bullmq.io/patterns/adding-bulks) Last updated 18 days ago Was this helpful? --- # Redis™ hosting | BullMQ [AWS MemoryDB](https://docs.bullmq.io/guide/redis-tm-hosting/aws-memorydb) [AWS Elasticache](https://docs.bullmq.io/guide/redis-tm-hosting/aws-elasticache) [PreviousDragonfly](https://docs.bullmq.io/guide/redis-tm-compatibility/dragonfly) [NextAWS MemoryDB](https://docs.bullmq.io/guide/redis-tm-hosting/aws-memorydb) Last updated 1 year ago Was this helpful? --- # Running a simple example | BullMQ ### [](https://docs.bullmq.io/guide/telemetry/running-a-simple-example#creating-a-producer) Creating a producer For this simple example we will create a producer that will add a couple of jobs, but it will add them in bulks instead of one by one, this will help us demonstrate how spans are linked between the consumers and the producers: producer.ts Copy import { Queue } from "bullmq"; import { BullMQOtel } from "bullmq-otel"; const queue = new Queue("myQueue", { connection: { host: "127.0.0.1", port: 6379, }, telemetry: new BullMQOtel("simple-guide"), }); const jobsBulk = Array.from({ length: 5 }, (_, i) => i); (async () => { for (let i = 0; i < 10; i++) { await queue.addBulk( jobsBulk.map((j) => ({ name: `myJob ${j}`, data: { i: j }, opts: { attempts: 2, backoff: 1000 }, })) ); } })(); ### [](https://docs.bullmq.io/guide/telemetry/running-a-simple-example#creating-a-consumer) Creating a consumer The consumer will be just a simple instance, we will use concurrency 10, so that jobs can be processed concurrently, and therefore create overlapping spans. We will also simulate jobs failures so that we can get retries, to show how spans are generated as the job gets failed, retried and finally completed: consumer.ts Copy import { Worker } from "bullmq"; import { BullMQOtel } from "bullmq-otel"; (async () => { const worker = new Worker( "myQueue", async (job) => { console.log("processing job", job.id, job.attemptsMade); await new Promise(async (res) => { setTimeout(() => res({}), 200); }); if (job.attemptsMade < 1) { throw new Error("This was an error"); } return "my result value"; }, { name: "myWorker", connection: { host: "127.0.0.1", port: 6379, }, telemetry: new BullMQOtel("simple-guide"), concurrency: 10, } ); })(); ### [](https://docs.bullmq.io/guide/telemetry/running-a-simple-example#creating-the-instrumentation-files) Creating the instrumentation files To test the telemetry functionality we can run a simple example. For that we also need to instantiate the OpenTelemetry SDK using a so called OpenTelemetry Protocol (OTLP) exporter. We must install the following modules that are part of the OpenTelemetry SDK: Copy npm install @opentelemetry/exporter-trace-otlp-proto \ @opentelemetry/exporter-metrics-otlp-proto And now we must create so called "instrumentation" files. We will create one for our "producer" service, which is the service actually taking care of producing jobs, it will look like this. Note that we use localhost (127.0.0.1) where our jaeger service is running: producer.inst.otlp.ts Copy import { NodeSDK } from '@opentelemetry/sdk-node'; import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-proto'; import { OTLPMetricExporter } from '@opentelemetry/exporter-metrics-otlp-proto'; import { PeriodicExportingMetricReader } from '@opentelemetry/sdk-metrics'; const sdk = new NodeSDK({ serviceName: 'producer', traceExporter: new OTLPTraceExporter({ url: 'http://127.0.0.1:4318/v1/traces' }), metricReader: new PeriodicExportingMetricReader({ exporter: new OTLPMetricExporter({ url: 'http://127.0.0.1:4318/v1/metrics' }), }), }); sdk.start(); Likewise we will create another instrumentation file for our "consumer" service, this is where the workers will run and consume the jobs produced by the "Queue" instance: consumer.inst.otlp.ts Copy import { NodeSDK } from '@opentelemetry/sdk-node'; import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-proto'; import { OTLPMetricExporter } from '@opentelemetry/exporter-metrics-otlp-proto'; import { PeriodicExportingMetricReader } from '@opentelemetry/sdk-metrics'; const sdk = new NodeSDK({ serviceName: 'consumer', traceExporter: new OTLPTraceExporter({ url: 'http://127.0.0.1:4318/v1/traces' }), metricReader: new PeriodicExportingMetricReader({ exporter: new OTLPMetricExporter({ url: 'http://127.0.0.1:4318/v1/metrics' }), }), }); sdk.start(); Both services looks basically the same, just the service name will differ in this case. ### [](https://docs.bullmq.io/guide/telemetry/running-a-simple-example#launching-the-services) Launching the services In order to guarantee that the OpenTelemetry instrumentation is run first, before everything else, and performs any required internal patching (even though BullMQ does not rely on patching other modules may do), we need to launch it like this (note that we use tsx in this example but Node runtime will do as well: Copy tsx --import producer.inst.otlp.ts producer.ts tsx --import consumer.inst.otlp.ts consumer.ts You can also use Node runtime directly if you are using javascript (or building from Typescript to javascript): `node --import producer.inst.otlp.js producer.js` As the services are launched we will see that the consumers starts processing the jobs and produce some logs on the console: Copy > tsx --import consumer.inst.otlp.ts consumer.ts processing job 1 0 processing job 2 0 processing job 3 0 processing job 4 0 processing job 5 0 processing job 6 0 ... processing job 43 1 processing job 44 1 processing job 45 1 processing job 46 1 processing job 47 1 processing job 48 1 processing job 49 1 processing job 50 1 These are just the logs that we wrote ourselves on the "process" function in our worker, so nothing special here. However if we go to Jaeger we will find the following: ![](https://docs.bullmq.io/~gitbook/image?url=https%3A%2F%2F1340146492-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-LUuDmt_xXMfG66Rn1GA%252Fuploads%252Fgit-blob-d7aa018eb4f0700b7d3a940fc7665693e46dd4a3%252Fimage.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=533ea417&sv=2) We have now 2 services to choose from, consumer and producer. If we search for traces in the producer we will be able to see all the traces where the producer is involved: ![](https://docs.bullmq.io/~gitbook/image?url=https%3A%2F%2F1340146492-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-LUuDmt_xXMfG66Rn1GA%252Fuploads%252Fgit-blob-c7c1340ba53ba9197045d49ae55b0c1a6af4f287%252Fimage.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=338d78d&sv=2) Here we can see as even though we are searching for the producer traces, we also get the consumer spans, and this is because jobs are linked between producers and consumers, so that we can trace all the way from the creation of a job to its final processing. If we look into the consumer spans for example, there are some interesting things to see: ![](https://docs.bullmq.io/~gitbook/image?url=https%3A%2F%2F1340146492-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-LUuDmt_xXMfG66Rn1GA%252Fuploads%252Fgit-blob-7d53b02677bd41f82b0e3e67d09b9c558a1232ec%252Fimage.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=b5fd5533&sv=2) First of all, note how the producer span "addBulk myQueue", is the root of this trace. Since this was an addBulk, it means that several jobs were added to the queue in one go, 5 in this case. So the spans created by the consumer are therefore linked to this one producer span. The consumer spans "process myQueue" are generated for every job that is being processed, and since we had a concurrency factor larger than 5, all 5 jobs are processed concurrently, which we can see in the spans all starting at the same time. But we also forced the jobs to fail 1 time, so that they would be retried with a small backoff (delay), which is why we can see a "delay myQueue" span and then a final "process myQueue" span. If we open the spans we can find other useful information: ![](https://docs.bullmq.io/~gitbook/image?url=https%3A%2F%2F1340146492-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-LUuDmt_xXMfG66Rn1GA%252Fuploads%252Fgit-blob-31a8a702b6c3ecbc6d167a54225837a08410be5e%252Fimage.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=4b52a5&sv=2) We have some useful tags related to this particular job, and also logs that shows events that happened during the span lifetime, for instance here we can see that the job failed with the given error message. If we go to the last span of the trace we can see that the job was finally completed after being delayed a bit before its last retry: ![](https://docs.bullmq.io/~gitbook/image?url=https%3A%2F%2F1340146492-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-LUuDmt_xXMfG66Rn1GA%252Fuploads%252Fgit-blob-656753225f7817cf8989d26e84a0ff30cfc7c819%252Fimage.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=e8ccbe1f&sv=2) [PreviousRunning Jaeger](https://docs.bullmq.io/guide/telemetry/running-jaeger) [NextQueueScheduler](https://docs.bullmq.io/guide/queuescheduler) Last updated 1 year ago Was this helpful? --- # Redis Cluster | BullMQ Bull internals require atomic operations that span different keys. This behavior breaks Redis's rules for cluster configurations. However, it is still possible to use a cluster environment by using the proper bull prefix option as a cluster "hash tag". Hash tags are used to guarantee that certain keys are placed in the same hash slot, read more about hash tags in the [redis cluster tutorial](https://redis.io/topics/cluster-tutorial) . A hash tag is defined with brackets. I.e. a key that has a substring inside brackets will use that substring to determine in which hash slot the key will be placed. In summary, to make bull compatible with Redis cluster, use a queue prefix inside brackets. For example: You can use two approaches in order to make the Queues compatible with Cluster. Either define a queue prefix: Copy const queue = new Queue('cluster', { prefix: '{myprefix}', }); or wrap the queue name itself: Copy const queue = new Queue('{cluster}'); Note that If you use several queues in the same cluster, you should use different prefixes so that the queues are evenly placed in the cluster nodes, potentially increasing performance and memory usage. [PreviousTimeout for Sandboxed processors](https://docs.bullmq.io/patterns/timeout-for-sandboxed-processors) [NextIntroduction](https://docs.bullmq.io/bullmq-pro/introduction) Last updated 1 year ago Was this helpful? --- # Timeout for Sandboxed processors | BullMQ When you are working with sandboxed processors, every job is run in a separate process. This opens the possibility to implement a time-to-live (TTL) mechanism, that kills the process if it has not been able to complete in a reasonable time. It is important to understand that killing a process can have unintented consecuences, for instance it could be killed in the middle of a writing transaction to a file, that would most likely result in a corrupt file. However, this is kind of the best that is possible to achieve in a runtime as NodeJS which based on asynchronous calls within an event loop. There is currently no known method to achieve this functionality in a more robust way. This pattern tries to be as possible, but please, keep in mind the trade-offs mentioned above. The pattern uses two timeouts so that it is possible to have a cleanup operation to minimize the effects of a hard kill of the process. Obviously if the cleanup itself hangs, or if the cleanup is not correctly implemented, we can still end killing some database connections or writing operantions right in the middle, with the potential negative outcomes. Copy // This processor will timeout in 30 seconds. const MAX_TTL = 30_000; // The processor will have a cleanup timeout of 5 seconds. const CLEANUP_TTL = 5_000; // We use a custom exit code to mark the TTL, but any would do in practice // as long as it is < 256 (Due to Unix limitation to 8 bits per exit code) const TTL_EXIT_CODE = 10; module.exports = async function (job) { let hasCompleted = false; const harKillTimeout = setTimeout(() => { if (!hasCompleted) { process.exit(TTL_EXIT_CODE); } }, MAX_TTL); const softKillTimeout = setTimeout(async () => { if (!hasCompleted) { await doCleanup(job); } }, CLEANUP_TTL); try { // If doAsyncWork is CPU intensive and blocks NodeJS loop forever, // the timeout will never be triggered either. await doAsyncWork(job); hasCompleted = true; } finally { // Important to clear the timeouts before returning as this process will be reused. clearTimeout(harKillTimeout); clearTimeout(softKillTimeout); } }; const doAsyncWork = async job => { // Simulate a long running operation. await new Promise(resolve => setTimeout(resolve, 10000)); }; const doCleanup = async job => { // Simulate a cleanup operation. await job.updateProgress(50); }; There are some very important points to consider with this pattern. * If the processor has hanged because there is an infinite loop that does not let the NodeJS event loop to run, the TTL timeouts will never be called. * We keep a `hasCompleted` flag so that we can cover the edge case where the async work has just completed at the same time the timeout is triggered. * When using this pattern it is very useful to put debug logs in strategic places to see where the job actually gets stuck when it is killed due to the TTL being exceeded. [PreviousTimeout jobs](https://docs.bullmq.io/patterns/timeout-jobs) [NextRedis Cluster](https://docs.bullmq.io/patterns/redis-cluster) Last updated 4 months ago Was this helpful? --- # Throttle jobs | BullMQ Sometimes, you may want to enqueue a job in reaction to a frequently occurring event, without running that job for _every_ event. For example, you may want to send an email to a user when they update their profile, but you don't want to send an email for every single update if they make many changes in rapid succession. You can achieve this by setting an identical `jobId` (using `JobsOptions.jobId?: string` to override the default unique integer) so **"identical" jobs are considered duplicates and not added to the queue**. If you use this option, it is up to you to ensure the `jobId` is unique. Hint: Be careful if using `removeOnComplete`/`removeOnFailed` options, since a removed job will not count as existing and a new job with the same job ID could be added to the queue without being detected as a duplicate. example: Copy import { Job, Queue, Worker } from 'bullmq'; const myQueue = new Queue('Paint'); const worker = new Worker('Paint', async (job: Job) => { console.log('Do something with job'); return 'some value'; }); worker.on('completed', (job: Job, returnvalue: any) => { console.log('worker done painting', new Date()); }); worker.on('failed', (job: Job, error: Error) => { console.error('worker fail painting', job, error, new Date()); }); // Add only one job that will be delayed at least 1 second. myQueue.add('house', { color: 'white' }, { delay: 1000, jobId: 'house' }); myQueue.add('house', { color: 'white' }, { delay: 1000, jobId: 'house' }); myQueue.add('house', { color: 'white' }, { delay: 1000, jobId: 'house' }); myQueue.add('house', { color: 'white' }, { delay: 1000, jobId: 'house' }); myQueue.add('house', { color: 'white' }, { delay: 1000, jobId: 'house' }); myQueue.add('house', { color: 'white' }, { delay: 1000, jobId: 'house' }); myQueue.add('house', { color: 'white' }, { delay: 1000, jobId: 'house' }); [PreviousIdempotent jobs](https://docs.bullmq.io/patterns/idempotent-jobs) [NextManual retrying](https://docs.bullmq.io/patterns/manual-retrying) Last updated 3 months ago Was this helpful? --- # Going to production | BullMQ In this chapter, we will offer crucial considerations and tips to help you achieve a robust solution when deploying your BullMQ-based application to production. ### [](https://docs.bullmq.io/guide/going-to-production#persistence) Persistence Since BullMQ is based on Redis, persistence needs to be configured manually. Many hosting solutions do not offer persistence by default; instead, it needs to be configured per instance. We recommend enabling [AOF (_Append Only File_)](https://redis.io/docs/management/persistence/#aof-advantages) , which provides a robust and fast solution. Usually, 1 second per write is enough for most applications. Even though persistence is very fast, it will have some effect on performance, so please make the proper benchmarks to know that it is not impacting your solution in a way that is not acceptable to you. ### [](https://docs.bullmq.io/guide/going-to-production#max-memory-policy) Max memory policy Redis is used quite often as a cache, meaning that it will remove keys according to some defined policy when it reaches several levels of memory consumption. BullMQ on the other hand cannot work properly if Redis evicts keys arbitrarily. **Therefore is very important to configure the** `**maxmemory-policy**` **setting to** `**noeviction**`**.** This is the **only** setting that guarantees the correct behavior of the queues. ### [](https://docs.bullmq.io/guide/going-to-production#automatic-reconnections) Automatic reconnections In a production setting, one of the things that are crucial for system robustness is to be able to recover automatically after connection issues. It is impossible to guarantee that a connection between BullMQ and Redis will always stay online. However, the important thing is that it recovers as fast as possible when the connection can be re-established without any human intervention. In order to understand how to properly handle disconnections it is important to understand some options provided by [IORedis](https://www.npmjs.com/package/ioredis#Auto-reconnect) . The ones interesting for us are: * `retryStrategy` * `maxRetriesPerRequest` * `enableOfflineQueue` It is also important to understand the difference in behavior that is often desired for `Queue` and `Worker` classes. Normally the operations performed using the `Queue` class should [fail quickly](https://docs.bullmq.io/patterns/failing-fast-when-redis-is-down) if there is a temporal disconnection, whereas for `Worker`s we want to wait indefinitely without raising any exception. #### [](https://docs.bullmq.io/guide/going-to-production#retrystrategy) `retryStrategy` This option is used to determine the function used to perform retries. The retries will continue forever until the reconnection has been accomplished. For IORedis connections created inside BullMQ we use the following strategy: Copy retryStrategy: function (times: number) { return Math.max(Math.min(Math.exp(times), 20000), 1000); } In other words, it will retry using exponential backoff, with a minimum 1-second retry time and max of 20 seconds. This `retryStrategy` can easily be overridden by passing a custom one defining custom IORedis options. #### [](https://docs.bullmq.io/guide/going-to-production#maxretriesperrequest) `maxRetriesPerRequest` This option sets a limit on the number of times a retry on a failed request will be performed. For `Worker`s, it is important to set this option to `**null**`. Otherwise, the exceptions raised by Redis when calling certain commands could break the worker functionality. When instantiating a `Worker` this option will always be set to `null` by default, but it could be overridden, either if passing an existing IORedis instance or by passing a different value for this option when instantiating the `Worker`. In both cases BullMQ will output a warning; please make sure to address this warning as it can have several unintended consequences. #### [](https://docs.bullmq.io/guide/going-to-production#enableofflinequeue) `enableOfflineQueue` IORedis provides a small offline queue that is used to queue commands while the connection is offline. You will probably want to disable this queue for the `Queue` instance, but leave it as is for `Worker` instances. That will make the `Queue` calls [fail quickly](https://docs.bullmq.io/patterns/failing-fast-when-redis-is-down) while leaving the `Worker`s to wait as needed until the connection has been re-established. ### [](https://docs.bullmq.io/guide/going-to-production#log-errors) Log errors It is really useful to attach a handler for the error event which will be triggered when there are connection issues. This will be helpful when debugging your queues and prevent "unhandled errors". Copy worker.on("error", (err) => { // Log your error. }) Copy queue.on("error", (err) => { // Log your error. }) ### [](https://docs.bullmq.io/guide/going-to-production#gracefully-shut-down-workers) Gracefully shut-down workers Since your workers will run on servers, it is unavoidable that these servers will need to be restarted from time to time. As your workers may be processing jobs when the server is about to restart, it is important to properly close the workers to minimize the risk of stalled jobs. If a worker is killed without waiting for their jobs to complete, these jobs will be marked as stalled and processed automatically when new workers come online (with a waiting time of about 30 seconds by default). However it is better to avoid having stalled jobs, and as mentioned this can be done by closing the workers when the server is going to be restarted. In a Node.js server, it is considered good practice to listen for both `SIGINT` and `SIGTERM` signals to close gracefully. Here's why: * `SIGINT` is typically sent when a user types Ctrl+C in the terminal to interrupt a process. Your server should listen to this signal during development or when it's running in the foreground, so you can shut it down properly when this key combination is pressed. * `SIGTERM` is the signal that is usually sent to a process to request its termination. Unlike `SIGKILL`, this signal can be caught by the process (which can then clean up resources and exit gracefully). This is the signal that system daemons, orchestration tools like Kubernetes, or process managers like PM2 typically use to stop a service. Here is an example on how you accomplish this: Copy const gracefulShutdown = async (signal) => { console.log(`Received ${signal}, closing server...`); await worker.close(); // Other asynchronous closings process.exit(0); } process.on('SIGINT', () => gracefulShutdown('SIGINT')); process.on('SIGTERM', () => gracefulShutdown('SIGTERM')); Keep in mind that the code above does not guarantee that the jobs will never end up being stalled, as the job may take longer time than the grace period for the server to restart. ### [](https://docs.bullmq.io/guide/going-to-production#auto-job-removal) Auto-job removal By default, all jobs processed by BullMQ will be either _completed_, or _failed_ and kept forever. This behavior is not usually the most desired, so you will likely want to configure a maximum number of jobs to keep. The most common configuration is to keep a handful of completed jobs, just to have some visibility of the latest completed, whereas you can keep either all of the failed jobs or a very large number in case you want to manually retry them or perform a deeper debugging study on the reason why the jobs failed. You can read more about how to configure auto removal [here](https://docs.bullmq.io/guide/queues/auto-removal-of-jobs) . ### [](https://docs.bullmq.io/guide/going-to-production#protecting-data) Protecting data Another important point to think about when deploying for production is the fact that the data field of the jobs is stored in clear text. **The best is to avoid storing sensitive data in the job altogether.** However, if this is not possible, then it is highly recommended to encrypt the part of the data that is sensitive before it is added to the queue. **Please do not take security lightly as it should be a major concern today, and the risks of losing data and economic damage to your business are real and very serious.** ### [](https://docs.bullmq.io/guide/going-to-production#unhandled-exceptions-and-rejections) Unhandled exceptions and rejections Another common issue, especially in production environments, is the fact that NodeJS by default will break if there are unhandled exceptions. This is not unique for BullMQ-based applications, but a general rule for all NodeJS applications. We recommend that somewhere in your service you make sure that you handle the unhandled exceptions gracefully, and so you can fix them when they arise without any risk of the application breaking when they happen: Copy process.on("uncaughtException", function (err) { // Handle the error safely logger.error(err, "Uncaught exception"); }); process.on("unhandledRejection", (reason, promise) => { // Handle the error safely logger.error({ promise, reason }, "Unhandled Rejection at: Promise"); }); [PreviousQueue Events Listeners](https://docs.bullmq.io/guide/nestjs/queue-events-listeners) [NextMigrations](https://docs.bullmq.io/guide/migration-to-newer-versions) Last updated 1 year ago Was this helpful? --- # Install | BullMQ In order to install BullMQ Pro you need to use a NPM token from [taskforce.sh](https://taskforce.sh/) . With the token at hand just update or create a `.npmrc` file in your app repository with the following contents: Copy @taskforcesh:registry=https://npm.taskforce.sh/ //npm.taskforce.sh/:_authToken=${NPM_TASKFORCESH_TOKEN} always-auth=true where `NPM_TASKFORCESH_TOKEN` is an environment variable pointing to your token. Then just install the `@taskforcesh/bullmq-pro` package as you would install any other package, with `npm`, `yarn` or `pnpm`: Copy yarn add @taskforcesh/bullmq-pro In order to use BullMQ Pro just import the _Pro_ versions of the classes. These classes are subclasses of the open source BullMQ library with new functionality: Copy import { QueuePro, WorkerPro } from '@taskforcesh/bullmq-pro'; const queue = new QueuePro('myQueue'); const worker = new WorkerPro('myQueue', async job => { // Process job }); ### [](https://docs.bullmq.io/bullmq-pro/install#using-docker) Using Docker If you use docker you must make sure that you also add the `.npmrc` file above in your `Dockerfile`: Copy WORKDIR /app ADD .npmrc /app/.npmrc [PreviousIntroduction](https://docs.bullmq.io/bullmq-pro/introduction) [NextObservables](https://docs.bullmq.io/bullmq-pro/observables) Last updated 6 months ago Was this helpful? --- # Manually processing jobs | BullMQ When a Worker is instantiated, the most common usage is to specify a process function so that the worker will automatically process the jobs that arrive to the queue. Sometimes however it is useful to be able to fetch the jobs manually. Just instantiate the worker without a processor and call getNextJob to fetch the next job: Copy const worker = new Worker('my-queue'); // Specify a unique token const token = 'my-token'; const job = (await worker.getNextJob(token)) as Job; // Access job.data and do something with the job // processJob(job.data) if (succeeded) { await job.moveToCompleted('some return value', token, false); } else { await job.moveToFailed(new Error('my error message'), token, false); } await worker.close(); There is an important consideration regarding job "locks" when processing manually. Locks prevent workers from fetching a job that is already being processed by another worker. The ownership of the lock is determined by the "token" that is sent when getting the job. the lock duration setting is called "visibility window" in other queue systems. Normally a job gets locked as soon as it is fetched from the queue with a max duration of the specified `lockDuration` worker option. The default is 30 seconds but can be changed to any value easily. For example, to change it to 60 seconds: Copy const worker = new Worker('my-queue', null, { lockDuration: 60000 }); When using standard worker processors, the lock is renewed automatically after half the lock duration time has passed. However, this mechanism does not exist when processing jobs manually, so to avoid the job being moved back to the waiting list of the queue, you need to make sure to process the job faster than the `lockDuration`, or manually extend the lock: Copy const job = (await worker.getNextJob(token)) as Job; // Extend the lock 30 more seconds await job.extendLock(token, 30000); ### [](https://docs.bullmq.io/patterns/manually-fetching-jobs#choosing-a-token) Choosing a token A token represents ownership by given worker currently working on a given job. If the worker dies unexpectedly, the job could be picked up by another worker when the lock expires. A good approach for generating tokens for jobs is simply to generate a UUID for every new job, but it all depends on your specific use case. [](https://docs.bullmq.io/patterns/manually-fetching-jobs#checking-for-stalled-jobs) Checking for stalled jobs ------------------------------------------------------------------------------------------------------------------- When processing jobs manually you may also want to start the stalled jobs checker. This checker is needed to move stalled jobs (whose lock has expired) back to the _wait_ status (or _failed_ if they have exhausted the maximum number of [stalled attempts](https://api.docs.bullmq.io/interfaces/v5.WorkerOptions.html#maxstalledcount) , which is 1 by default). Copy await worker.startStalledCheckTimer() The checker will run periodically (based on the [`stalledInterval`](https://api.docs.bullmq.io/interfaces/v5.WorkerOptions.html#stalledinterval) option) until the worker is closed. [](https://docs.bullmq.io/patterns/manually-fetching-jobs#looping-through-jobs) Looping through jobs --------------------------------------------------------------------------------------------------------- In many cases, you will have an "infinite" loop that processes jobs one by one like the following example. Note that the third parameter in `job.moveToCompleted`/`job.moveToFailed` is not used, signalling that the next job should be returned automatically. Copy const worker = new Worker('my-queue'); const token = 'my-token'; let job; while (1) { let jobData = null, jobId, success; if (job) { // Use job.data to process this particular job. // and set success variable if succeeded if (success) { [jobData, jobId] = await job.moveToCompleted('some return value', token); } else { await job.moveToFailed(new Error('some error message'), token); } if (jobData) { job = Job.fromJSON(worker, jobData, jobId); } else { job = null; } } else { if (!job) { job = await worker.getNextJob(token); } } } [](https://docs.bullmq.io/patterns/manually-fetching-jobs#rate-limiting) Rate Limiting ------------------------------------------------------------------------------------------- If you want to move a job back to wait because your queue is rate limited. Copy const worker = new Worker('my-queue', null, { connection, prefix }); const token = 'my-token'; await Job.create(queue, 'test', { foo: 'bar' }); const job = (await worker.getNextJob(token)) as Job; await queue.rateLimit(60000); await job.moveToWait(token); [](https://docs.bullmq.io/patterns/manually-fetching-jobs#read-more) Read more: ------------------------------------------------------------------------------------ * 💡 [Get Next Job API Reference](https://api.docs.bullmq.io/classes/v5.Worker.html#getnextjob) * 💡 [Move To Completed API Reference](https://api.docs.bullmq.io/classes/v5.Job.html#movetocompleted) * 💡 [Move To Failed API Reference](https://api.docs.bullmq.io/classes/v5.Job.html#movetofailed) * 💡 [Move To Wait API Reference](https://api.docs.bullmq.io/classes/v5.Job.html#movetowait) [PreviousDeduplication](https://docs.bullmq.io/patterns/deduplication) [NextNamed Processor](https://docs.bullmq.io/patterns/named-processor) Last updated 3 months ago Was this helpful? --- # Observables | BullMQ Instead of returning regular promises, your workers can also return an `Observable`, this allows for some more advanced uses cases: * It makes it possible to cleanly cancel a running job. * You can define a "Time to live" (TTL) so that jobs that take too long will be automatically canceled. * Since the last value returned by the observable is persisted, you could retry a job and continue where you left off, for example, if the job implements a state machine or similar. If you are new to `Observables` you may want to read this [introduction](https://www.learnrxjs.io/learn-rxjs/concepts/rxjs-primer) . The two biggest advantages that `Observables` have over `Promises` are that they can emit more than 1 value and that they are _cancelable_. Let's see a silly example of a worker making use of `Observables`: Copy import { WorkerPro } from "@taskforcesh/bullmq-pro" import { Observable } from "rxjs" const processor = async () => { return new Observable(subscriber => { subscriber.next(1); subscriber.next(2); subscriber.next(3); const intervalId = setTimeout(() => { subscriber.next(4); subscriber.complete(); }, 500); // Provide a way of canceling and disposing the interval resource return function unsubscribe() { clearInterval(intervalId); }; }); }; const worker = new WorkerPro(queueName, processor, { connection }); In the example above, the observable will emit 4 values, the first 3 directly and then a 4th after 500 ms. Also note that the `subscriber` returns a `unsubscribe` function. This is the function that will be called if the `Observable` is cancelled, so this is where you would do the necessary clean up. You may be asking what the use of returning several values from a worker is. One case that comes to mind is if you have a larger processor and you want to make sure that if the process crashes you can continue from the latest value. You could do this with a `switch` statement on the return value, something like this: Copy import { WorkerPro } from "@taskforcesh/bullmq-pro" import { Observable } from "rxjs" const processor = async (job) => { return new Observable(subscriber => { switch(job.returnvalue){ default: subscriber.next(1); case 1: subscriber.next(2); case 2: subscriber.next(3); case 3: subscriber.complete(); } }); }; const worker = new WorkerPro(queueName, processor, { connection }); [PreviousInstall](https://docs.bullmq.io/bullmq-pro/install) [NextCancelation](https://docs.bullmq.io/bullmq-pro/observables/cancelation) Last updated 2 years ago Was this helpful? --- # AWS MemoryDB | BullMQ AWS provides a Redis™ 7 compatible managed database that is easy to use and is fully compatible with BullMQ. There are some considerations to take care when using MemoryDB though. * MemoryDB only works in Cluster mode. So you need to use "hash tags" so that the queues get attached to a given cluster node ([read more here](https://docs.bullmq.io/bull/patterns/redis-cluster) ). * MemoryDB can only be accessed within an AWS VPC, so you cannot access the Redis™ cluster outside of AWS. The easiest way to use MemoryDB with BullMQ is to first instantiate a IORedis Cluster instance, and then use that connection as an option to your workers or queue instances, for example: Copy import { Cluster } from 'ioredis'; import { Worker } from 'bullmq'; const connection = new Cluster( [\ {\ host: 'clustercfg.xxx.amazonaws.com',\ port: 6379,\ },\ ], { tls: {}, }, ); const worker = new Worker( 'myqueue', async (job: Job) => { // Do some usefull stuff }, { connection }, ); // ... // Do not forget to close the connection as well as the worker when shutting down await worker.close(); await connection.quit(); [PreviousRedis™ hosting](https://docs.bullmq.io/guide/redis-tm-hosting) [NextAWS Elasticache](https://docs.bullmq.io/guide/redis-tm-hosting/aws-elasticache) Last updated 1 year ago Was this helpful? --- # Telemetry | BullMQ In the same fashion we support telemetry in BullMQ open source edition, we also support telemetry for BullMQ Pro. It works basically the same, in fact you can just the same integrations available for BullMQ in the Pro version. So in order to enable it you would do something like this: Copy import { QueuePro } from '@taskforcesh/bullmq-pro'; import { BullMQOtel } from 'bullmq-otel'; // Initialize a Pro queue using BullMQ-Otel const queue = new QueuePro('myProQueue', { connection, telemetry: new BullMQOtel('guide'), }); await queue.add( 'myJob', { data: 'myData' }, { attempts: 2, backoff: 1000, group: { id: 'myGroupId', }, }, ); For the Worker we will do it in a similar way: Copy import { WorkerPro } from '@taskforcesh/bullmq-pro'; import { BullMQOtel } from 'bullmq-otel'; const worker = new WorkerPro( 'myProQueue', async job => { console.log('processing job', job.id); }, { name: 'myWorker', connection, telemetry: new BullMQOtel('guide'), concurrency: 10, batch: { size: 10 }, }, ); For an introductury guide on how to integrate OpenTelemetry in you BullMQ applications take a look at this tutorial: [https://blog.taskforce.sh/how-to-integrate-bullmqs-telemetry-on-a-newsletters-subscription-application-2/](https://blog.taskforce.sh/how-to-integrate-bullmqs-telemetry-on-a-newsletters-subscription-application-2/) [PreviousSandboxes for groups](https://docs.bullmq.io/bullmq-pro/groups/sandboxes-for-groups) [NextBatches](https://docs.bullmq.io/bullmq-pro/batches) Last updated 11 months ago Was this helpful? --- # New Releases | BullMQ If you want to get notifications when we do a new release of BullMQ Pro, please enable notifications on this Github issue where we automatically create a new comment for every new release: [https://github.com/taskforcesh/bullmq-pro-support/issues/86](https://github.com/taskforcesh/bullmq-pro-support/issues/86) [Previousv1](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1) [NextSupport](https://docs.bullmq.io/bullmq-pro/support) Last updated 11 months ago Was this helpful? --- # Introduction | BullMQ BullMQ Pro, the commercial version of BullMQ, boasts advanced features and offers dedicated support from the library's authors. You can easily install this package as a drop-in replacement for the standard BullMQ to access these new features. The BullMQ Pro version continues to evolve, with more features being added regularly. You can check the [roadmap](https://github.com/taskforcesh/bullmq-pro-support/projects/1) section to gain insight into what to expect in the forthcoming months. ![](https://docs.bullmq.io/~gitbook/image?url=https%3A%2F%2F1340146492-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-LUuDmt_xXMfG66Rn1GA%252Fuploads%252Fgit-blob-20555d6090e111c93a562c459a1a35bb41f9b28f%252Fimage.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=6fba0a16&sv=2) ### [](https://docs.bullmq.io/bullmq-pro/introduction#commercial-license) Commercial License BullMQ Pro uses a per-organization licensing model, allowing unlimited use across all your projects. You can request a free trial token on [this page](https://taskforce.sh/account/bullmqpro) , allowing you to evaluate its value. If you find it beneficial, consider subscribing under the '[Subscriptions](https://taskforce.sh/account/subscriptions) ' tab. [PreviousRedis Cluster](https://docs.bullmq.io/patterns/redis-cluster) [NextInstall](https://docs.bullmq.io/bullmq-pro/install) Last updated 2 years ago Was this helpful? --- # Local group rate limit | BullMQ Sometimes it is required that different groups have different rate limits, this could be the case for example if a group represents a given user in the system, and depending on the user's quota or other factors we would like to have a different rate-limit for it. You can use a local group rate limit, which would be used only for the specific group that have the rate-limit setup. For example: Copy import { QueuePro, WorkerPro } from '@taskforcesh/bullmq-pro'; const queue = new QueuePro('myQueue', { connection }); const groupId = 'my group'; const maxJobsPerDuration = 100; const duration = 1000; // duration in ms. await queue.setGroupRateLimit(groupId, maxJobsPerDuration, duration); const worker = new WorkerPro( 'myQueue', async () => { // do something }, { group: { limit: { // default rate limit configuration max: 1000, duration: 1000, }, }, connection, }, ); This code would set a specific rate limit on the group "my group" of max 100 jobs per second. Note that you can still have a ["default" rate-limit](https://docs.bullmq.io/bullmq-pro/groups/rate-limiting) specified for the rest of the groups, the call to `setGroupRateLimit` will therefore allow you to override that rate-limit. You must specify a default rate limit by passing group.limit option in each Worker instance. In this way, workers are allowed to check if groups are rate limited or not. ### [](https://docs.bullmq.io/bullmq-pro/groups/local-group-rate-limit#read-more) Read more: * 💡 [Local Rate Limit Group API Reference](https://api.bullmq.pro/classes/v7.QueuePro.html#setgroupratelimit) [PreviousRate limiting](https://docs.bullmq.io/bullmq-pro/groups/rate-limiting) [NextConcurrency](https://docs.bullmq.io/bullmq-pro/groups/concurrency) Last updated 3 months ago Was this helpful? --- # Getters | BullMQ #### [](https://docs.bullmq.io/bullmq-pro/groups/getters#job-counts) Job Counts It is often necessary to know how many jobs are in any group: Copy import { QueuePro } from '@taskforcesh/bullmq-pro'; const queue = new QueuePro('myQueue', { connection }); const groupId = 'my group'; const count = await queue.getGroupsJobsCount(1000); // 1000 groups in each iteration This count value includes prioritized and non-prioritized jobs included groups. Or if you want to get active jobs count for an specific group Copy const activeCount = await queue.getGroupActiveCount(groupId); #### [](https://docs.bullmq.io/bullmq-pro/groups/getters#get-jobs) Get Jobs It is also possible to retrieve the jobs with pagination style semantics in a given group. For example: Copy const jobs = await queue.getGroupJobs(groupId, 0, 100); [](https://docs.bullmq.io/bullmq-pro/groups/getters#read-more) Read more: ------------------------------------------------------------------------------ * 💡 [Get Groups Jobs Count API Reference](https://api.bullmq.pro/classes/v7.QueuePro.html#getgroupsjobscount) * 💡 [Get Group Active Count API Reference](https://api.bullmq.pro/classes/v7.QueuePro.html#getgroupactivecount) * 💡 [Get Group Jobs API Reference](https://api.bullmq.pro/classes/v7.QueuePro.html#getgroupjobs) [PreviousGroups](https://docs.bullmq.io/bullmq-pro/groups) [NextRate limiting](https://docs.bullmq.io/bullmq-pro/groups/rate-limiting) Last updated 3 months ago Was this helpful? --- # Cancelation | BullMQ As mentioned, `Observables` allow for clean cancellation. Currently we support a TTL value that defines the maximum processing time before the job is finally cancelled: Copy import { WorkerPro } from '@taskforcesh/bullmq-pro'; const worker = new WorkerPro(queueName, processor, { ttl: 100, connection, }); This parameter allows to provide `ttl` values per job name too: Copy const worker = new WorkerPro(queueName, processor, { ttl: { test1: 100, test2: 200 }, connection, }); [PreviousObservables](https://docs.bullmq.io/bullmq-pro/observables) [NextGroups](https://docs.bullmq.io/bullmq-pro/groups) Last updated 2 years ago Was this helpful? --- # Concurrency | BullMQ By default, there is no limit on the number of jobs that workers can run in parallel for every group. Even using a rate limit, that would only limit the processing speed, but still you could have an unbounded number of jobs processed simultaneously in every group. It is possible to constrain how many jobs are allowed to be processed concurrently per group. For example, if you choose 3 as max concurrency factor, the workers will never work on more than 3 jobs at the same time for any given group. This limits only the group; you could have any number of concurrent jobs as long as they are not from the same group. The concurrency factor is configured as follows: Copy import { WorkerPro } from '@taskforcesh/bullmq-pro'; const worker = new WorkerPro('myQueue', processFn, { group: { concurrency: 3, // Limit to max 3 parallel jobs per group }, concurrency: 100, connection, }); The concurrency factor is global, so in the example above, independently of the concurrency factor per worker or the number of workers that you instantiate in your application, it will never process more than 3 jobs per group at any given time. [PreviousLocal group rate limit](https://docs.bullmq.io/bullmq-pro/groups/local-group-rate-limit) [NextLocal group concurrency](https://docs.bullmq.io/bullmq-pro/groups/local-group-concurrency) Last updated 1 year ago Was this helpful? --- # NestJs | BullMQ There is a compatible module to be used in [NestJs](https://github.com/nestjs/nest) based on [@nestjs/bullmq](https://www.npmjs.com/package/@nestjs/bullmq) . Copy yarn add @taskforcesh/nestjs-bullmq-pro BullMQ-Pro needs a token, as explained in the [install](https://docs.bullmq.io/bullmq-pro/install) section. Once the installation process is complete, we can import the `BullModule` into the root `AppModule`. Copy import { Module } from '@nestjs/common'; import { BullModule } from '@taskforcesh/nestjs-bullmq-pro'; @Module({ imports: [\ BullModule.forRoot({\ connection: {\ host: 'localhost',\ port: 6379,\ },\ }),\ ], }) export class AppModule {} To register a queue, import the `BullModule.registerQueue()` dynamic module, as follows: Copy BullModule.registerQueue({ name: 'queueName', }); To register a flow producer, import the `BullModule.registerFlowProducer()` dynamic module, as follows: Copy BullModule.registerFlowProducer({ name: 'flowProducerName', }); [](https://docs.bullmq.io/bullmq-pro/nestjs#processor) Processor --------------------------------------------------------------------- To register a processor, you may need to use `Processor` decorator: Copy import { Processor, WorkerHost, OnWorkerEvent, } from '@taskforcesh/nestjs-bullmq-pro'; import { JobPro } from 'taskforcesh/bullmq-pro'; @Processor('queueName') class TestProcessor extends WorkerHost { async process(job: JobPro): Promise { // do some stuff } @OnWorkerEvent('completed') onCompleted() { // do some stuff } } And then register it as a provider: Copy @Module({ imports: [\ BullModule.registerQueue({\ name: 'queueName',\ connection: {\ host: '0.0.0.0',\ port: 6380,\ },\ }),\ BullModule.registerFlowProducer({\ name: 'flowProducerName',\ connection: {\ host: '0.0.0.0',\ port: 6380,\ },\ }),\ ], providers: [TestProcessor], }) export class AppModule {} [](https://docs.bullmq.io/bullmq-pro/nestjs#example) Example ----------------------------------------------------------------- A working example is available [here](https://github.com/taskforcesh/nestjs-bullmq-pro-example) . ### [](https://docs.bullmq.io/bullmq-pro/nestjs#read-more) Read more: * 💡 [Queues Technique](https://docs.nestjs.com/techniques/queues) * 💡 [Register Queue API Reference](https://nestjs.bullmq.pro/classes/BullModule.html#registerQueue) * 💡 [Register Flow Producer API Reference](https://nestjs.bullmq.pro/classes/BullModule.html#registerFlowProducer) * 💡 [Worker Listener API Reference](https://api.docs.bullmq.io/interfaces/v5.WorkerListener.html) [PreviousBatches](https://docs.bullmq.io/bullmq-pro/batches) [NextProducers](https://docs.bullmq.io/bullmq-pro/nestjs/producers) Last updated 1 year ago Was this helpful? --- # Sandboxes for groups | BullMQ It is also possible to use [Sandboxes](https://docs.bullmq.io/guide/workers/sandboxed-processors) for processing groups. It works essentially the same as in standard BullMQ, but you gain access to the "gid" property in the job object sent to your processor, for example: Copy import { SandboxedJobPro } from '@taskforcesh/bullmq-pro'; module.exports = function (job: SandboxedJobPro) { expect(job).to.have.property('gid'); expect(job.opts).to.have.property('group'); expect(job.opts.group).to.have.property('id'); expect(job.opts.group.id).to.be.a('string'); expect(job.opts.group.id).to.equal(job.gid); }; Groups are the only Pro features supported by Sandboxed processors for now. [PreviousPrioritized intra-groups](https://docs.bullmq.io/bullmq-pro/groups/prioritized) [NextTelemetry](https://docs.bullmq.io/bullmq-pro/telemetry) Last updated 1 year ago Was this helpful? --- # Deduplication | BullMQ Sometimes, you may want to decide when you want to stop deduplicating jobs. [](https://docs.bullmq.io/patterns/deduplication#until-job-is-active) Until job is active ---------------------------------------------------------------------------------------------- As soon as job is moved to active, you must call **removeDeduplicationKey** method: Copy import { Job, Queue, Worker } from 'bullmq'; const myQueue = new Queue('Paint'); const worker = new Worker('Paint', async (job: Job) => { await job.removeDeduplicationKey(); console.log('Do something with job'); return 'some value'; }); myQueue.add('house', { color: 'white' }, { deduplication: { id: 'house' } }); Previous example uses [Simple Mode](https://docs.bullmq.io/guide/jobs/deduplication#simple-mode) but it can be combined with [Throttle Mode](https://docs.bullmq.io/guide/jobs/deduplication#throttle-mode) or [Debounce Mode](https://docs.bullmq.io/guide/jobs/deduplication#debounce-mode) . [](https://docs.bullmq.io/patterns/deduplication#using-job-schedulers) Using job schedulers ------------------------------------------------------------------------------------------------ Sometimes it is desired to deduplicate jobs that are generated by job schedulers to save resources and avoid unnecessary work: Deduplication options are not available in [`JobSchedulerTemplateOptions`](https://api.docs.bullmq.io/types/v5.JobSchedulerTemplateOptions.html) because they could interfere with job creation from scheduler templates. Since a new job is added as soon as the previous one moves to the active state, deduplication could disrupt this process by preventing the addition of this new record. However, there is an alternative to handle this. Let's look at an example: Copy import { Queue, Worker } from 'bullmq'; const myQueue = new Queue('Paint'); const worker = new Worker( 'Paint', async job => { if (job.name === 'paint-trigger') { // Add a job that will be deduplicated for 90 seconds. await myQueue.add( 'house', { color: 'white' }, { deduplication: { id: 'customValue', ttl: 90000 } }, ); } }, { connection }, ); await myQueue.upsertJobScheduler('repeat', { pattern: '* * * * *', // every minute template: { name: 'paint-trigger', data: {}, }, }); In this way, you can deduplicate a job when using job schedulers. [](https://docs.bullmq.io/patterns/deduplication#read-more) Read more: --------------------------------------------------------------------------- * 💡 [Add Job API Reference](https://api.docs.bullmq.io/classes/v5.Queue.html#add) * 💡 [Deduplication Reference](https://docs.bullmq.io/guide/jobs/deduplication) * 💡 [Remove Deduplication Key API Reference](https://api.docs.bullmq.io/classes/v5.Job.html#removededuplicationkey) * 💡 [Upsert Job Scheduler API Reference](https://api.docs.bullmq.io/classes/v5.Queue.html#upsertJobScheduler) [PreviousAdding jobs in bulk across different queues](https://docs.bullmq.io/patterns/adding-bulks) [NextManually processing jobs](https://docs.bullmq.io/patterns/manually-fetching-jobs) Last updated 1 day ago Was this helpful? --- # Rate limiting | BullMQ A useful feature when using groups is to be able to rate limit the groups independently of each other, so you can evenly process the jobs belonging to many groups and still limit how many jobs per group are allowed to be processed by unit of time. The way the rate limiting works is that when the jobs for a given group exceed the maximum amount of jobs per unit of time, that particular group gets rate limited. The jobs that belong to this particular group will not be processed until the rate limit expires. For example "group 2" is rate limited in the following chart: ![](https://docs.bullmq.io/~gitbook/image?url=https%3A%2F%2F1340146492-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-LUuDmt_xXMfG66Rn1GA%252Fuploads%252Fgit-blob-f99cb4797afb865d39a87c94a92595743d1ba93a%252Fimage.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=a09d5329&sv=2) Rate limited group While one or more groups are rate limited, the rest of the jobs belonging to non rate limited groups will continue to be consumed normally or until they also get rate limited. The rate limit is configured on the worker instances: Copy import { WorkerPro } from '@taskforcesh/bullmq-pro'; const worker = new WorkerPro('myQueue', processFn, { group: { limit: { max: 100, // Limit to 100 jobs per second per group duration: 1000, } }, connection }); ### [](https://docs.bullmq.io/bullmq-pro/groups/rate-limiting#manual-rate-limit) Manual rate-limit Sometimes it's useful to rate-limit a group manually instead of based on some static options. For example, if you have an API that returns `429 Too Many Requests`, and you want to rate-limit the group based on that response. For this purpose, you can use the worker method `rateLimitGroup` like this: Copy import { WorkerPro } from '@taskforcesh/bullmq-pro'; const worker = new WorkerPro( 'myQueue', async job => { const groupId = job.opts.group.id; const [isRateLimited, duration] = await doExternalCall(groupId); if (isRateLimited) { await worker.rateLimitGroup(job, duration); // Do not forget to throw this special exception, // since the job is no longer active after being rate limited. throw Worker.RateLimitError(); } }, { connection, }, ); ### [](https://docs.bullmq.io/bullmq-pro/groups/rate-limiting#get-group-rate-limit-ttl) Get Group Rate Limit Ttl Sometimes is useful to know if our group is rate limited. For this purpose, you can use the `**getGroupRateLimitTtl**` method like this: Copy import { QueuePro } from '@taskforcesh/bullmq-pro'; const queue = new QueuePro('myQueue', { connection }); const groupId = '0'; const maxJobs = 100; const ttl = await queue.getGroupRateLimitTtl(groupId, maxJobs); if (ttl > 0) { console.log('Group is rate limited'); } [](https://docs.bullmq.io/bullmq-pro/groups/rate-limiting#read-more) Read more: ------------------------------------------------------------------------------------ * 💡 [Rate Limit Group API Reference](https://api.bullmq.pro/classes/v7.QueuePro.html#ratelimitgroup) * 💡 [Get Group Rate Limit Ttl API Reference](https://api.bullmq.pro/classes/v7.QueuePro.html#getgroupratelimitttl) [PreviousGetters](https://docs.bullmq.io/bullmq-pro/groups/getters) [NextLocal group rate limit](https://docs.bullmq.io/bullmq-pro/groups/local-group-rate-limit) Last updated 3 months ago Was this helpful? --- # v3 | BullMQ ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v3#bug-fixes) Bug Fixes * **groups:** do not parse gid when deserializing jobs fixes [#25](https://github.com/taskforcesh/bullmq-pro/issues/25) ([b03a1e9](https://github.com/taskforcesh/bullmq-pro/commit/b03a1e9c637e62e7c1722a77b61d55e208983852) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v3#breaking-changes) BREAKING CHANGES * **groups:** Group ids must be strings. Numbers are not allowed anymore. Fixes https://github.com/taskforcesh/bullmq-pro-support/issues/25 [Previousv4](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v4) [Nextv2](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2) Last updated 3 months ago Was this helpful? --- # Queue Events Listeners | BullMQ To register a `QueueEvents` instance, you need to use `QueueEventsListener` decorator: Copy import { QueueEventsListener, QueueEventsHost, OnQueueEvent, } from '@taskforcesh/nestjs-bullmq-pro'; @QueueEventsListener('queueName') export class TestQueueEvents extends QueueEventsHost { @OnQueueEvent('completed') onCompleted({ jobId, }: { jobId: string; returnvalue: string; prev?: string; }) { // do some stuff } } And then register it as a provider: Copy @Module({ imports: [\ BullModule.registerQueue({\ name: 'queueName',\ connection: {\ host: '0.0.0.0',\ port: 6380,\ },\ }),\ ], providers: [TestQueueEvents], }) export class AppModule {} [](https://docs.bullmq.io/bullmq-pro/nestjs/queue-events-listeners#read-more) Read more: --------------------------------------------------------------------------------------------- * 💡 [Queues Technique](https://docs.nestjs.com/techniques/queues) * 💡 [Register Queue API Reference](https://nestjs.bullmq.pro/classes/BullModule.html#registerQueue) * 💡 [Queue Events Pro Listener API Reference](https://api.bullmq.pro/interfaces/v7.QueueEventsListenerPro.html) [PreviousProducers](https://docs.bullmq.io/bullmq-pro/nestjs/producers) [NextChangelog](https://docs.bullmq.io/bullmq-pro/nestjs/changelog) Last updated 3 months ago Was this helpful? --- # Batches | BullMQ It is possible to configure workers so that instead of processing one job at a time they can process up to a number of jobs (a so-called _batch_) in one go. Workers using batches have slightly different semantics and behavior than normal workers, so read carefully the following examples to avoid pitfalls. To enable batches, pass the `batch` option with a `size` property representing the maximum number of jobs per batch: Copy const worker = new WorkerPro( 'My Queue', async (job: JobPro) => { const batch = job.getBatch(); for (let i = 0; i < batch.length; i++) { const batchedJob = batch[i]; await doSomethingWithBatchedJob(batchedJob); } }, { connection, batch: { size: 10 } }, ); There is no strict maximum limit for the size of batches; however, keep in mind that larger batches introduce overhead proportional to their size, which could lead to performance issues. Typical batch sizes range between 10 and 50 jobs. ### [](https://docs.bullmq.io/bullmq-pro/batches#new-batch-options-minsize-and-timeout) New Batch Options: `minSize` and `timeout` In addition to the size option, two new options—`minSize` and `timeout`—provide greater control over batch processing: * `minSize`: Specifies the minimum number of jobs required before the worker processes a batch. The worker will wait until at least minSize jobs are available before fetching and processing them, up to the size limit. If fewer than minSize jobs are available, the worker waits indefinitely unless a timeout is also set. * `timeout`: Defines the maximum time (in milliseconds) the worker will wait for minSize jobs to accumulate. If the timeout expires before minSize is reached, the worker processes whatever jobs are available, up to the size limit. If minSize is not set the timeout option is effectively ignored, as the worker batches only avaialble jobs. Important: minSize and timeout are not compatible with groups. When groups are used, the worker ignores minSize and tries to batch avaialble jobs without waiting. Here’s an example configuration using both `minSize` and `timeout`: Copy const worker = new WorkerPro( 'My Queue', async (job: JobPro) => { const batch = job.getBatch(); for (let i = 0; i < batch.length; i++) { const batchedJob = batch[i]; await doSomethingWithBatchedJob(batchedJob); } }, { connection, batch: { size: 10, // Maximum jobs per batch minSize: 5, // Wait for at least 5 jobs timeout: 30_000 // Wait up to 30 seconds }, }, ); In this example: * The worker waits for at least 5 jobs to become available, up to a maximum of 10 jobs per batch. * If 5 or more jobs are available within 30 seconds, it processes the batch (up to 10 jobs). * If fewer than 5 jobs are available after 30 seconds, it processes whatever jobs are present, even if below `minSize`. ### [](https://docs.bullmq.io/bullmq-pro/batches#failing-jobs) Failing jobs When using batches, the default is that if the processor throws an exception, **all jobs in the batch will fail.** To fail specific jobs instead, use the `setAsFailed` method on individual jobs within the batch: Copy const worker = new WorkerPro( 'My Queue', async (job: JobPro) => { const batch = job.getBatch(); for (let i = 0; i < batch.length; i++) { const batchedJob = batch[i]; try { await doSomethingWithBatchedJob(batchedJob); } catch (err) { batchedJob.setAsFailed(err); } } }, { connection, batch: { size: 10 } }, ); Only jobs explicitly marked with `setAsFailed` will fail; the remaining jobs in the batch will complete succesfully once the processor finishes. ### [](https://docs.bullmq.io/bullmq-pro/batches#handling-events) Handling events Batches are managed by wrapping all jobs in a batch into a dummy job that holds the jobs in an internal array. This simplifies batch processing but affects event handling. For example, worker-level event listeners (e.g., `worker.on('completed', ...)`) report events for the dummy batch job, not the individual jobs within it. To retrieve the jobs in a batch from an event handler, use the `getBatch` method: Copy worker.on('completed', job => { const batch = job.getBatch(); // ... }); Using a global event listener you can listen to individual job events even though they may be processed in a batch: Copy import { QueueEventsPro } from '@taskforcesh/bullmq-pro'; const queueEvents = new QueueEventsPro(queueName, { connection }); queueEvents.on('completed', (jobId, err) => { // ... }); ### [](https://docs.bullmq.io/bullmq-pro/batches#limitations) Limitations Currently, all worker options can be used with batches, however, there are some unsupported features that may be implemented in the future: * [Dynamic rate limit](https://docs.bullmq.io/guide/rate-limiting#manual-rate-limit) * [Manually processing jobs](https://docs.bullmq.io/patterns/manually-fetching-jobs) * [Dynamically delay jobs](https://docs.bullmq.io/patterns/process-step-jobs#delaying) . [PreviousTelemetry](https://docs.bullmq.io/bullmq-pro/telemetry) [NextNestJs](https://docs.bullmq.io/bullmq-pro/nestjs) Last updated 9 months ago Was this helpful? --- # Introduction | BullMQ Bull is the legacy version of BullMQ. As it is still heavily used today, it is also maintained for bugs but not for new major features. If you want to use a battle-tested queue library and you do not need a better typescript integration or the latest features you are good to go using this library for years to come. ### [](https://docs.bullmq.io/bull/introduction#used-by) Used by Bull has been a part of the NodeJS ecosystem for a long time and is used by many organizations both in commercial and open-source projects. A few special mentions: ![](https://docs.bullmq.io/~gitbook/image?url=https%3A%2F%2F1340146492-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-LUuDmt_xXMfG66Rn1GA%252Fuploads%252Fgit-blob-3740cd90575f03d7dadae45d200a3dd1d8deeb50%252FScreenshot%25202022-02-15%2520at%252011.32.39.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=4f8e6fd4&sv=2) ![](https://docs.bullmq.io/~gitbook/image?url=https%3A%2F%2F1340146492-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-LUuDmt_xXMfG66Rn1GA%252Fuploads%252Fgit-blob-9e5d7d3fff2f351a8addcccc56e4171a5bca72a7%252Fmozilla-logo-bw-rgb.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=e725b10a&sv=2) ![](https://docs.bullmq.io/~gitbook/image?url=https%3A%2F%2F1340146492-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-LUuDmt_xXMfG66Rn1GA%252Fuploads%252Fgit-blob-191801ea830187195aa09eeb819b2f36b176eeb4%252Fautodesk-logo-white.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=9fbe8b38&sv=2) ![](https://docs.bullmq.io/~gitbook/image?url=https%3A%2F%2F1340146492-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-LUuDmt_xXMfG66Rn1GA%252Fuploads%252Fgit-blob-968e68098732529d79d6b439a62eaf882b11dc34%252FAtlassian-horizontal-blue-rgb.webp%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=9b5def75&sv=2) ![](https://docs.bullmq.io/~gitbook/image?url=https%3A%2F%2F1340146492-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-LUuDmt_xXMfG66Rn1GA%252Fuploads%252Fgit-blob-6bd0987e7b9dd48eef7a43a0ade1d515699e29eb%252Fmidwayjs-logo.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=3baa4806&sv=2) ![](https://docs.bullmq.io/~gitbook/image?url=https%3A%2F%2F1340146492-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-LUuDmt_xXMfG66Rn1GA%252Fuploads%252Fgit-blob-a91bddf5e2fdd7f08d644eecad6feda5329f80e6%252Fsalesforce-logo.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=9368f3a3&sv=2) ![](https://docs.bullmq.io/~gitbook/image?url=https%3A%2F%2F1340146492-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-LUuDmt_xXMfG66Rn1GA%252Fuploads%252Fgit-blob-80406c82b62236727e67c63d9b1ee0385a691ea0%252Fentethalliance-logo.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=79281f7e&sv=2) ![](https://docs.bullmq.io/~gitbook/image?url=https%3A%2F%2F1340146492-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-LUuDmt_xXMfG66Rn1GA%252Fuploads%252Fgit-blob-6118716cc14fd52811a1db6b9ae2b7c129282f2e%252Fkisspng-logo-retail-target-corporation-advertising-5ae5ef43944c89.3404142515250184356074.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=bb6c0b93&sv=2) [PreviousSupport](https://docs.bullmq.io/bullmq-pro/support) [NextInstall](https://docs.bullmq.io/bull/install) Last updated 3 years ago Was this helpful? --- # Important Notes | BullMQ The queue aims for an "at least once" working strategy. This means that in some situations, a job could be processed more than once. This mostly happens when a worker fails to keep a lock for a given job during the total duration of the processing. When a worker is processing a job it will keep the job "locked" so other workers can't process it. It's important to understand how locking works to prevent your jobs from losing their lock - becoming _stalled_ - and being restarted as a result. Locking is implemented internally by creating a lock for `lockDuration` on interval `lockRenewTime` (which is usually half `lockDuration`). If `lockDuration` elapses before the lock can be renewed, the job will be considered stalled and is automatically restarted; it will be **double processed**. This can happen when: 1. The Node process running your job processor unexpectedly terminates. 2. Your job processor was too CPU-intensive and stalled the Node event loop, and as a result, Bull couldn't renew the job lock (see [#488](https://github.com/OptimalBits/bull/issues/488) for how we might better detect this). You can fix this by breaking your job processor into smaller parts so that no single part can block the Node event loop. Alternatively, you can pass a larger value for the `lockDuration` setting (with the tradeoff being that it will take longer to recognize a real stalled job). As such, you should always listen for the `stalled` event and log this to your error monitoring system, as this means your jobs are likely getting double-processed. As a safeguard so problematic jobs won't get restarted indefinitely (e.g. if the job processor always crashes its Node process), jobs will be recovered from a stalled state a maximum of `maxStalledCount` times (default: `1`). [PreviousQuick Guide](https://docs.bullmq.io/bull/quick-guide) [NextPatterns](https://docs.bullmq.io/bull/patterns) Last updated 4 years ago Was this helpful? --- # Install | BullMQ Install with **Npm**: Copy npm install bull --save or Yarn: Copy yarn add bull In order to work with Bull, you also need to have a Redis server running. For local development you can easily install it using [docker](https://hub.docker.com/_/redis/) . Bull will by default try to connect to a Redis server running on `localhost:6379` _Bull requires a Redis version greater than or equal to_ `_2.8.18_`_._ ### [](https://docs.bullmq.io/bull/install#typescript-definitions) Typescript Definitions Copy npm install @types/bull --save-dev Copy yarn add --dev @types/bull Definitions are currently maintained in the [DefinitelyTyped](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/master/types/bull) repo. [PreviousIntroduction](https://docs.bullmq.io/bull/introduction) [NextQuick Guide](https://docs.bullmq.io/bull/quick-guide) Last updated 1 year ago Was this helpful? --- # Persistent connections | BullMQ A crucial feature for a subsystem in a microservice architecture is that it should automatically handle disconnections from other services and keep these connections alive for as long as the service is running. For example, if your service has a connection to a database, and the connection to said database breaks, you would like that service to handle this disconnection as gracefully as possible and as soon as the database is back online continue to work without human intervention. Since Bull relies on **ioredis** for accessing Redis, the default is auto-reconnect forever. This behaviour can be customized but most likely the default is the best setting currently: [https://github.com/luin/ioredis#auto-reconnect](https://github.com/luin/ioredis#auto-reconnect) In the context of Bull, we have normally two different cases that are handled differently. ### [](https://docs.bullmq.io/bull/patterns/persistent-connections#workers) Workers A worker is consuming jobs from the queue as fast as it can. If it loses the connection to Redis we want the worker to "wait" until Redis is available again. For this to work we need to understand an important setting in our Redis options (which are handled by ioredis): #### [](https://docs.bullmq.io/bull/patterns/persistent-connections#maxretriesperrequest) `maxRetriesPerRequest` This setting tells the ioredis client how many times to try a command that fails before throwing an error. So even though Redis is not reachable or offline, the command will be retried until this situation changes or the maximum number of attempts is reached. In Bull we set this setting to null both for the "bclient" and "eclient" connections, which are used for the workers and events respectively. This guarantees that the workers will keep processing forever as long as there is a working connection. If you create a Redis client manually, Bull will throw an exception if this setting is not set to null. ### [](https://docs.bullmq.io/bull/patterns/persistent-connections#queue) Queue A simple Queue instance used for managing the queue such as adding jobs, pausing, using getters, etc. usually has different requirements from the worker. For example, say that you are adding jobs to a queue as the result of a call to an HTTP endpoint. The caller of this endpoint cannot wait forever if the connection to Redis happens to be down when this call is made. Therefore the `maxRetriesPerRequest` setting should either be left at its default (which currently is 20) or set it to another value, maybe 1 so that the user gets an error quickly and can retry later. [PreviousPatterns](https://docs.bullmq.io/bull/patterns) [NextMessage queue](https://docs.bullmq.io/bull/patterns/message-queue) Last updated 10 months ago Was this helpful? --- # Patterns | BullMQ Here are a few examples of useful patterns that are often implemented with Bull: * [Persistent connections](https://docs.bullmq.io/bull/patterns/persistent-connections) * [Message Queue](https://docs.bullmq.io/bull/patterns/message-queue) * [Returning Job Completions](https://docs.bullmq.io/bull/patterns/returning-job-completions) * [Reusing Redis Connections](https://docs.bullmq.io/bull/patterns/reusing-redis-connections) * [Redis Cluster](https://docs.bullmq.io/bull/patterns/redis-cluster) * [Debugging](https://docs.bullmq.io/bull/patterns/debugging) * [Custom backoff strategy](https://docs.bullmq.io/bull/patterns/custom-backoff-strategy) * [Manually fetching jobs](https://docs.bullmq.io/patterns/manually-fetching-jobs) If you have any other common patterns you want to add, pull request them! [PreviousImportant Notes](https://docs.bullmq.io/bull/important-notes) [NextPersistent connections](https://docs.bullmq.io/bull/patterns/persistent-connections) Last updated 1 year ago Was this helpful? --- # Prioritized intra-groups | BullMQ BullMQ Pro supports priorities per group. A job is prioritized in a group when group and priority options are provided _together_. Copy await myQueue.add( 'paint', { foo: 'bar' }, { group: { id: 'groupId', priority: 10, }, }, ); The priorities go from 0 to 2097151, where a higher number means lower priority (as in Unix [processes](https://en.wikipedia.org/wiki/Nice_(Unix)) ). Thus, jobs without any explicit priority will have the highest priority. [](https://docs.bullmq.io/bullmq-pro/groups/prioritized#get-counts-per-priority-for-group) Get Counts per Priority for Group --------------------------------------------------------------------------------------------------------------------------------- If you want to get the `count` of jobs in `prioritized` status (priorities higher than 0) or in `waiting` status (priority 0) for specific group, use the `**getCountsPerPriorityForGroup**` method. For example, let's say that you want to get counts for `priority` `1` and `0`: Copy const counts = await queue.getCountsPerPriorityForGroup('groupId', [1, 0]); /* { '1': 11, '0': 10 } */ [](https://docs.bullmq.io/bullmq-pro/groups/prioritized#read-more) Read more: ---------------------------------------------------------------------------------- * 💡 [Add Job API Reference](https://api.bullmq.pro/classes/v7.QueuePro.html#add) * 💡 [Get Counts per Priority for Group API Reference](https://api.bullmq.pro/classes/v7.QueuePro.html#getcountsperpriorityforgroup) [PreviousPausing groups](https://docs.bullmq.io/bullmq-pro/groups/pausing-groups) [NextSandboxes for groups](https://docs.bullmq.io/bullmq-pro/groups/sandboxes-for-groups) Last updated 3 months ago Was this helpful? --- # Groups | BullMQ Groups allows you to use a single queue while distributing the jobs among groups so that the jobs are processed one by one relative to the group they belong to. For example, imagine that you have 1 queue for processing video transcoding for all your users, you may have thousands of users in your application. You need to offload the transcoding operation since it is lengthy and CPU consuming. If you have many users that want to transcode many files, then in a non-grouped queue one user could fill the queue with jobs and the rest of the users will need to wait for that user to complete all its jobs before their jobs get processed. Groups resolves this problem since jobs will be processed in a "[round-robin](https://en.wikipedia.org/wiki/Round-robin_item_allocation) " fashion among all the users. ![](https://docs.bullmq.io/~gitbook/image?url=https%3A%2F%2F1340146492-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-LUuDmt_xXMfG66Rn1GA%252Fuploads%252Fgit-blob-b189f3837a876d50e4cf025f25bc64c7025fb1c8%252Fimage.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=3d76c505&sv=2) If you have several workers or a concurrency factor larger than one, jobs will be processed in parallel, but they will be picked up from the groups as mentioned before following a round-robin ordering. Of course you can have as many workers as you want and also scale up/down the amount of workers depending on how many jobs you have waiting in the queue. If you only use grouped jobs in a queue, the waiting jobs list will not grow, instead it will just keep the next job to be processed if any. But you can add non-grouped jobs to the same queue, and they will get precedence from the jobs waiting in their respective groups. There is no hard limit on the amount of groups that you can have, nor do they have any impact on performance. When a group is empty, the group itself does not consume any resources in Redis. Another way to see groups is like "virtual" queues. So instead of having one queue per "user", you have a "virtual" queue per user so that all users get their jobs processed in a more predictable way. In order to use the group functionality, use the group property in the job options when adding a job: Copy import { QueuePro } from '@taskforcesh/bullmq-pro'; const queue = new QueuePro(); const job1 = await queue.add( 'test', { foo: 'bar1' }, { group: { id: 1, }, }, ); const job2 = await queue.add( 'test', { foo: 'bar2' }, { group: { id: 2, }, }, ); In order to process the jobs, use a pro worker as you normally do with standard workers: Copy import { WorkerPro } from '@taskforcesh/bullmq-pro'; const worker = new WorkerPro('test', async job => { // Do something usefull. // You can also do something different depending on the group await doSomethingSpecialForMyGroup(job.opts.group); }); [PreviousCancelation](https://docs.bullmq.io/bullmq-pro/observables/cancelation) [NextGetters](https://docs.bullmq.io/bullmq-pro/groups/getters) Last updated 9 months ago Was this helpful? --- # Max group size | BullMQ It is possible to set a maximum group size. This can be useful if you want to keep the number of jobs within some limits and you can afford to discard new jobs. When a group has reached the defined max size, adding new jobs to that group will result in an exception being thrown that you can catch and ignore if you do not care about it. You can use the `maxSize` option when adding jobs to a group like this: Copy import { QueuePro, GroupMaxSizeExceededError } from '@taskforcesh/bullmq-pro'; const queue = new QueuePro('myQueue', { connection }); const groupId = 'my group'; try { await queue.add('paint', { foo: 'bar' }, { group: { id: groupId, maxSize: 7, }, }); } catch (err) { if (err instanceof GroupMaxSizeExceededError){ console.log(`Job discarded for group ${groupId}`) } else { throw err; } } The `maxSize` option is not yet available for `addBulk`. [PreviousLocal group concurrency](https://docs.bullmq.io/bullmq-pro/groups/local-group-concurrency) [NextPausing groups](https://docs.bullmq.io/bullmq-pro/groups/pausing-groups) Last updated 2 years ago Was this helpful? --- # Process Step Jobs | BullMQ Sometimes, it is useful to break processor functions into small pieces that will be processed depending on the previous executed step. One way to handle this kind of logic is by using switch statements: TypeScript Python Copy enum Step { Initial, Second, Finish, } const worker = new Worker( 'queueName', async job => { let step = job.data.step; while (step !== Step.Finish) { switch (step) { case Step.Initial: { await doInitialStepStuff(); await job.updateData({ step: Step.Second, }); step = Step.Second; break; } case Step.Second: { await doSecondStepStuff(); await job.updateData({ step: Step.Finish, }); step = Step.Finish; return Step.Finish; } default: { throw new Error('invalid step'); } } } }, { connection }, ); Copy class Step(int, Enum): Initial = 1 Second = 2 Finish = 3 async def process(job: Job, token: str): step = job.data.get("step") while step != Step.Finish: if step == Step.Initial: await doInitialStepStuff() await job.updateData({ "step": Step.Second }) step = Step.Second elif step == Step.Second: await doSecondStepStuff() await job.updateData({ "step": Step.Finish }) step = Step.Finish else: raise Exception("invalid step") worker = Worker("queueName", process, {"connection": connection}) By saving the next step value every time we complete the previous step (here, saving it in the job's data), we can ensure that if the job errors and retries, it does so starting from the correct step. [](https://docs.bullmq.io/patterns/process-step-jobs#delaying) Delaying ---------------------------------------------------------------------------- There are situations when it is useful to delay a job when it is being processed. This can be handled using the `moveToDelayed` method. However, it is important to note that when a job is being processed by a worker, the worker keeps a lock on this job with a certain token value. For the `moveToDelayed` method to work, we need to pass said token so that it can unlock without error. Finally, we need to exit from the processor by throwing a special error (`DelayedError`) that will signal to the worker that the job has been delayed so that it does not try to complete (or fail the job) instead. Copy import { DelayedError, Worker } from 'bullmq'; enum Step { Initial, Second, Finish, } const worker = new Worker( 'queueName', async (job: Job, token?: string) => { let step = job.data.step; while (step !== Step.Finish) { switch (step) { case Step.Initial: { await doInitialStepStuff(); await job.moveToDelayed(Date.now() + 200, token); await job.updateData({ step: Step.Second, }); throw new DelayedError(); } case Step.Second: { await doSecondStepStuff(); await job.updateData({ step: Step.Finish, }); step = Step.Finish; } default: { throw new Error('invalid step'); } } } }, { connection }, ); [](https://docs.bullmq.io/patterns/process-step-jobs#waiting-children) Waiting Children -------------------------------------------------------------------------------------------- A common use case is to add children at runtime and then wait for the children to complete. This can be handled using the `moveToWaitingChildren` method. However, it is important to note that when a job is being processed by a worker, the worker keeps a lock on this job with a certain token value. For the `moveToWaitingChildren` method to work, we need to pass said token so that it can unlock without error. Finally, we need to exit from the processor by throwing a special error (`WaitingChildrenError`) that will signal to the worker that the job has been moved to _waiting-children_, so that it does not try to complete (or fail) the job instead. TypeScript Python Copy import { WaitingChildrenError, Worker } from 'bullmq'; enum Step { Initial, Second, Third, Finish, } const worker = new Worker( 'parentQueueName', async (job: Job, token?: string) => { let step = job.data.step; while (step !== Step.Finish) { switch (step) { case Step.Initial: { await doInitialStepStuff(); await childrenQueue.add( 'child-1', { foo: 'bar' }, { parent: { id: job.id, queue: job.queueQualifiedName, }, }, ); await job.updateData({ step: Step.Second, }); step = Step.Second; break; } case Step.Second: { await doSecondStepStuff(); await childrenQueue.add( 'child-2', { foo: 'bar' }, { parent: { id: job.id, queue: job.queueQualifiedName, }, }, ); await job.updateData({ step: Step.Third, }); step = Step.Third; break; } case Step.Third: { const shouldWait = await job.moveToWaitingChildren(token); if (!shouldWait) { await job.updateData({ step: Step.Finish, }); step = Step.Finish; return Step.Finish; } else { throw new WaitingChildrenError(); } } default: { throw new Error('invalid step'); } } } }, { connection }, ); Copy from bullmq import Worker, WaitingChildrenError from enum import Enum class Step(int, Enum): Initial = 1 Second = 2 Third = 3 Finish = 4 async def process(job: Job, token: str): step = job.data.get("step") while step != Step.Finish: if step == Step.Initial: await doInitialStepStuff() await children_queue.add('child-1', {"foo": "bar" },{ "parent": { "id": job.id, "queue": job.queueQualifiedName } }) await job.updateData({ "step": Step.Second }) step = Step.Second elif step == Step.Second: await doSecondStepStuff() await children_queue.add('child-2', {"foo": "bar" },{ "parent": { "id": job.id, "queue": job.queueQualifiedName } }) await job.updateData({ "step": Step.Third }) step = Step.Third elif step == Step.Third: should_wait = await job.moveToWaitingChildren(token, {}) if not should_wait: await job.updateData({ "step": Step.Finish }) step = Step.Finish return Step.Finish else: raise WaitingChildrenError else: raise Exception("invalid step") worker = Worker("parentQueueName", process, {"connection": connection}) Bullmq-Pro: this pattern could be handled by using observables; in that case, we do not need to save next step. [](https://docs.bullmq.io/patterns/process-step-jobs#chaining-flows) Chaining Flows ---------------------------------------------------------------------------------------- Another use case is to add flows at runtime and then wait for the children to complete. For example, we can add children dynamically in the worker's processor function: Copy import { FlowProducer, WaitingChildrenError, Worker } from 'bullmq'; enum Step { Initial, Second, Third, Finish, } const flow = new FlowProducer({ connection }); const worker = new Worker( 'parentQueueName', async (job, token) => { let step = job.data.step; while (step !== Step.Finish) { switch (step) { case Step.Initial: { await doInitialStepStuff(); await flow.add({ name: 'child-job', queueName: 'childrenQueueName', data: {}, children: [\ {\ name,\ data: { idx: 0, foo: 'bar' },\ queueName: 'grandchildrenQueueName',\ },\ {\ name,\ data: { idx: 1, foo: 'baz' },\ queueName: 'grandchildrenQueueName',\ },\ ], opts: { parent: { id: job.id, queue: job.queueQualifiedName, }, }, }); await job.updateData({ step: Step.Second, }); step = Step.Second; break; } case Step.Second: { await doSecondStepStuff(); await job.updateData({ step: Step.Third, }); step = Step.Third; break; } case Step.Third: { const shouldWait = await job.moveToWaitingChildren(token); if (!shouldWait) { await job.updateData({ step: Step.Finish, }); step = Step.Finish; return Step.Finish; } else { throw new WaitingChildrenError(); } } default: { throw new Error('invalid step'); } } } }, { connection }, ); Manually moving jobs using special errors does **not** increment the attemptsMade property. This property is incremented on regular job completion or failure (this includes retries using a backoff strategy). To control how many times a job is allowed to skip an attempt made using one of our special errors like: **DelayedError**, **RateLimitError**, **WaitingChildrenError** or **WaitingError**. To control how many times a job is allowed to start processing, use the **maxStartedAttempts** option. [](https://docs.bullmq.io/patterns/process-step-jobs#read-more) Read more: ------------------------------------------------------------------------------- * 💡 [Move To Delayed API Reference](https://api.docs.bullmq.io/classes/v5.Job.html#movetodelayed) * 💡 [Move To Waiting Children API Reference](https://api.docs.bullmq.io/classes/v5.Job.html#movetowaitingchildren) [PreviousManual retrying](https://docs.bullmq.io/patterns/manual-retrying) [NextFailing fast when Redis is down](https://docs.bullmq.io/patterns/failing-fast-when-redis-is-down) Last updated 1 month ago Was this helpful? --- # Support | BullMQ With your BullMQ Pro subscription, you receive commercial support. For the most recent and detailed terms, please refer to the "[BullMQ Pro](https://taskforce.sh/account/bullmqpro) " tab in your account. Here's a broad overview of what's included: * Email-based support: Customers may submit issues or queries via email to our dedicated support email address ([\[email protected\]](https://docs.bullmq.io/cdn-cgi/l/email-protection) ). * Response time: We aim to respond to support emails within one business day (business days are defined in section 4). However, response times may vary, and quicker responses may occur on a best-effort basis. * Resolution time: The time it takes to resolve an issue depends on the nature of the issue. Simple queries or requests for advice may be resolved quickly. More complex issues, such as bugs, may take several days or more to resolve. In some cases, we may not be able to fully resolve an issue but will provide a workaround. [PreviousNew Releases](https://docs.bullmq.io/bullmq-pro/new-releases) [NextIntroduction](https://docs.bullmq.io/bull/introduction) Last updated 11 months ago Was this helpful? --- # Debugging | BullMQ To see debug statements set or add `bull` to the `NODE_DEBUG` environment variable: Copy export NODE_DEBUG=bull or: Copy NODE_DEBUG=bull node ./your-script.js [PreviousCustom backoff strategy](https://docs.bullmq.io/bull/patterns/custom-backoff-strategy) [NextManually fetching jobs](https://docs.bullmq.io/bull/patterns/manually-fetching-jobs) Last updated 4 years ago Was this helpful? --- # Reusing Redis Connections | BullMQ A standard queue requires **3 connections** to the Redis server. In some situations you might want to re-use connections—for example on Heroku where the connection count is restricted. You can do this with the `createClient` option in the `Queue` constructor. #### [](https://docs.bullmq.io/bull/patterns/reusing-redis-connections#notes) Notes: * bclient connections [cannot be re-used](https://github.com/OptimalBits/bull/issues/880) , so you should return a new connection each time this is called. * client and subscriber connections can be shared and will not be closed when the queue is closed. When you are shutting down the process, first close the queues, then the shared connections (if they are shared). * if you are not sharing connections but still using `createClient` to do some custom connection logic, you may still need to keep a list of all the connections you created so you can manually close them later when the queue shuts down, if you need a graceful shutdown for your process * do not set a `keyPrefix` on the connection you create, use bull's built-in prefix feature if you need a key prefix Copy const { REDIS_URL } = process.env; const Redis = require("ioredis"); const client = new Redis(REDIS_URL); const subscriber = new Redis(REDIS_URL); const opts = { // redisOpts here will contain at least a property of // connectionName which will identify the queue based on its name createClient: function (type, redisOpts) { switch (type) { case "client": return client; case "subscriber": return subscriber; case "bclient": return new Redis(REDIS_URL, redisOpts); default: throw new Error("Unexpected connection type: ", type); } }, }; const queueFoo = new Queue("foobar", opts); const queueQux = new Queue("quxbaz", opts); [PreviousReturning Job Completions](https://docs.bullmq.io/bull/patterns/returning-job-completions) [NextRedis cluster](https://docs.bullmq.io/bull/patterns/redis-cluster) Last updated 2 years ago Was this helpful? --- # Message queue | BullMQ Bull can also be used for persistent message queues. This is a quite useful feature in some use cases. For example, you can have two servers that need to communicate with each other. By using a queue, the servers do not need to be online at the same time, so this creates a very robust communication channel. You can treat `add` as _send_ and `process` as _receive_: Server A: Copy const Queue = require('bull'); const sendQueue = new Queue('Server B'); const receiveQueue = new Queue('Server A'); receiveQueue.process(function (job, done) { console.log('Received message', job.data.msg); done(); }); sendQueue.add({ msg: 'Hello' }); Server B: Copy const Queue = require('bull'); const sendQueue = new Queue('Server A'); const receiveQueue = new Queue('Server B'); receiveQueue.process(function (job, done) { console.log('Received message', job.data.msg); done(); }); sendQueue.add({ msg: 'World' }); [PreviousPersistent connections](https://docs.bullmq.io/bull/patterns/persistent-connections) [NextReturning Job Completions](https://docs.bullmq.io/bull/patterns/returning-job-completions) Last updated 2 years ago Was this helpful? --- # Changelog | BullMQ [](https://docs.bullmq.io/bullmq-pro/nestjs/changelog#id-3.1.0-2025-03-06) [3.1.0](https://github.com/taskforcesh/nestjs-bullmq-pro/compare/v3.0.0...v3.1.0) (2025-03-06) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/bullmq-pro/nestjs/changelog#features) Features * support nestjs v11 ([#17](https://github.com/taskforcesh/nestjs-bullmq-pro/issues/17) ) ([4b07bd5](https://github.com/taskforcesh/nestjs-bullmq-pro/commit/4b07bd56075824ca8e8372124cc4c67c843625bb) ) [](https://docs.bullmq.io/bullmq-pro/nestjs/changelog#id-3.0.0-2024-04-05) [3.0.0](https://github.com/taskforcesh/nestjs-bullmq-pro/compare/v2.1.1...v3.0.0) (2024-04-05) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/bullmq-pro/nestjs/changelog#features-1) Features * support bullmq-pro v7 ([#15](https://github.com/taskforcesh/nestjs-bullmq-pro/issues/15) ) ([9ebb21b](https://github.com/taskforcesh/nestjs-bullmq-pro/commit/9ebb21bbd401f2957baca99fffb8e7a8da195e01) ) #### [](https://docs.bullmq.io/bullmq-pro/nestjs/changelog#breaking-changes) BREAKING CHANGES * references: * [Better Queue Markers](https://bullmq.io/news/231204/better-queue-markers/) * [BullMQ v5 Migration Notes](https://bullmq.io/news/231221/bullmqv5-release/) * [BullMQ Pro v7 Release](https://bullmq.io/news/240312/bullmq-prov7-release/) ### [](https://docs.bullmq.io/bullmq-pro/nestjs/changelog#id-2.1.1-2024-04-05) [2.1.1](https://github.com/taskforcesh/nestjs-bullmq-pro/compare/v2.1.0...v2.1.1) (2024-04-05) #### [](https://docs.bullmq.io/bullmq-pro/nestjs/changelog#bug-fixes) Bug Fixes * revert bullmq v7.4.0 to v6 ([#14](https://github.com/taskforcesh/nestjs-bullmq-pro/issues/14) ) ([3f170d9](https://github.com/taskforcesh/nestjs-bullmq-pro/commit/3f170d9fbaa537f06c26fa2a940962d17c2e386f) ) [](https://docs.bullmq.io/bullmq-pro/nestjs/changelog#id-2.1.0-2024-04-05) [2.1.0](https://github.com/taskforcesh/nestjs-bullmq-pro/compare/v2.0.0...v2.1.0) (2024-04-05) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/bullmq-pro/nestjs/changelog#features-2) Features * upgrade bullmq-pro to v7.4.0 ([#10](https://github.com/taskforcesh/nestjs-bullmq-pro/issues/10) ) ([7af2548](https://github.com/taskforcesh/nestjs-bullmq-pro/commit/7af2548397cfde027d80786c7d6a0bcece72988a) ) [](https://docs.bullmq.io/bullmq-pro/nestjs/changelog#id-2.0.0-2023-06-28) [2.0.0](https://github.com/taskforcesh/nestjs-bullmq-pro/compare/v1.1.0...v2.0.0) (2023-06-28) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/bullmq-pro/nestjs/changelog#features-3) Features * **nestjs:** support version 10 and bullmq-pro version 6 ([#9](https://github.com/taskforcesh/nestjs-bullmq-pro/issues/9) ) ([e55489e](https://github.com/taskforcesh/nestjs-bullmq-pro/commit/e55489e634e3feddc05347f4d74f74b67de38293) ) #### [](https://docs.bullmq.io/bullmq-pro/nestjs/changelog#breaking-changes-1) BREAKING CHANGES * **nestjs:** abandon nestjs version 7 [](https://docs.bullmq.io/bullmq-pro/nestjs/changelog#id-1.1.0-2023-02-04) [1.1.0](https://github.com/taskforcesh/nestjs-bullmq-pro/compare/v1.0.3...v1.1.0) (2023-02-04) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/bullmq-pro/nestjs/changelog#features-4) Features * support flow producer pro ([#5](https://github.com/taskforcesh/nestjs-bullmq-pro/issues/5) ) ([99516d2](https://github.com/taskforcesh/nestjs-bullmq-pro/commit/99516d27223dce85019e5d87366cd4a954768e60) ) ### [](https://docs.bullmq.io/bullmq-pro/nestjs/changelog#id-1.0.3-2023-01-18) [1.0.3](https://github.com/taskforcesh/nestjs-bullmq-pro/compare/v1.0.2...v1.0.3) (2023-01-18) #### [](https://docs.bullmq.io/bullmq-pro/nestjs/changelog#bug-fixes-1) Bug Fixes * **package:** point to dist files ([#2](https://github.com/taskforcesh/nestjs-bullmq-pro/issues/2) ) ([c55826c](https://github.com/taskforcesh/nestjs-bullmq-pro/commit/c55826ca8aa7b57a684788f28a0b2d23c009615b) ) ### [](https://docs.bullmq.io/bullmq-pro/nestjs/changelog#id-1.0.2-2023-01-18) [1.0.2](https://github.com/taskforcesh/nestjs-bullmq-pro/compare/v1.0.1...v1.0.2) (2023-01-18) #### [](https://docs.bullmq.io/bullmq-pro/nestjs/changelog#bug-fixes-2) Bug Fixes * scope the package for @taskforcesh org ([177b591](https://github.com/taskforcesh/nestjs-bullmq-pro/commit/177b591d6c24202b7a256177faf4bb19e5f0bcea) ) ### [](https://docs.bullmq.io/bullmq-pro/nestjs/changelog#id-1.0.1-2023-01-18) [1.0.1](https://github.com/taskforcesh/nestjs-bullmq-pro/compare/v1.0.0...v1.0.1) (2023-01-18) #### [](https://docs.bullmq.io/bullmq-pro/nestjs/changelog#bug-fixes-3) Bug Fixes * remove private: true to allow publish ([394e27f](https://github.com/taskforcesh/nestjs-bullmq-pro/commit/394e27f83faa7323ad2ae4b2eca7fdc89624561e) ) [](https://docs.bullmq.io/bullmq-pro/nestjs/changelog#id-1.0.0-2023-01-18) 1.0.0 (2023-01-18) -------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/bullmq-pro/nestjs/changelog#bug-fixes-4) Bug Fixes * use WorkerPro references ([59009fb](https://github.com/taskforcesh/nestjs-bullmq-pro/commit/59009fb981e4f698d114855cc94c09108ed09692) ) #### [](https://docs.bullmq.io/bullmq-pro/nestjs/changelog#features-5) Features * initial setup ([12119d7](https://github.com/taskforcesh/nestjs-bullmq-pro/commit/12119d76cde0679c8f5b6892d029a8d62ce303d2) ) [PreviousQueue Events Listeners](https://docs.bullmq.io/bullmq-pro/nestjs/queue-events-listeners) [NextChangelogs](https://docs.bullmq.io/bullmq-pro/changelog) Last updated 8 months ago Was this helpful? --- # Returning Job Completions | BullMQ A common pattern is where you have a cluster of queue processors that just process jobs as fast as they can, and some other services that need to take the result of these processors and do something with it, maybe storing results in a database. The most robust and scalable way to accomplish this is by combining the standard job queue with the message queue pattern: a service sends jobs to the cluster just by opening a job queue and adding jobs to it, and the cluster will start processing as fast as it can. Everytime a job gets completed in the cluster a message is sent to a results message queue with the result data, and this queue is listened by some other service that stores the results in a database. [PreviousMessage queue](https://docs.bullmq.io/bull/patterns/message-queue) [NextReusing Redis Connections](https://docs.bullmq.io/bull/patterns/reusing-redis-connections) Last updated 2 years ago Was this helpful? --- # Redis cluster | BullMQ Bull internals require atomic operations that span different keys. This behavior breaks Redis's rules for cluster configurations. However, it is still possible to use a cluster environment by using the proper bull prefix option as a cluster "hash tag". Hash tags are used to guarantee that certain keys are placed in the same hash slot, read more about hash tags in the [redis cluster tutorial](https://redis.io/topics/cluster-tutorial) . A hash tag is defined with brackets. I.e. a key that has a substring inside brackets will use that substring to determine in which hash slot the key will be placed. In summary, to make bull compatible with Redis cluster, use a queue prefix inside brackets. For example: You can use two approaches in order to make the Queues compatible with Cluster. Either define a queue prefix: Copy const queue = new Queue('cluster', { prefix: '{myprefix}' }); or wrap the queue name itself: Copy const queue = new Queue('{cluster}'); Note that If you use several queues in the same cluster, you should use different prefixes so that the queues are evenly placed in the cluster nodes, potentially increasing performance and memory usage. [PreviousReusing Redis Connections](https://docs.bullmq.io/bull/patterns/reusing-redis-connections) [NextCustom backoff strategy](https://docs.bullmq.io/bull/patterns/custom-backoff-strategy) Last updated 1 year ago Was this helpful? --- # Pausing groups | BullMQ BullMQ Pro supports pausing groups globally. A group is paused when no workers will pick up any jobs that belongs to the paused group. When you pause a group, the workers that are currently busy processing a job from that group, will continue working on that job until it completes (or failed), and then will just keep idling until the group has been resumed. Pausing a group is performed by calling the `pauseGroup` method on a [`Queue`](https://api.bullmq.pro/classes/v7.QueuePro.html#pausegroup) instance: Copy await myQueue.pauseGroup('groupId'); Even if the `groupId` does not exist at that time, the `groupId` will be added in our paused list as a group could be ephemeral `pauseGroup` will return `false` if the group is already paused. Resuming a group is performed by calling the `resumeGroup` method on a [`Queue`](https://api.bullmq.pro/classes/v7.QueuePro.html#resumegroup) instance: Copy await myQueue.resumeGroup('groupId'); `resumeGroup` will return `false` if the group does not exist or when the group is already resumed. [](https://docs.bullmq.io/bullmq-pro/groups/pausing-groups#read-more) Read more: ------------------------------------------------------------------------------------- * 💡 [Pause Group API Reference](https://api.bullmq.pro/classes/v7.QueuePro.html#pausegroup) * 💡 [Resume Group API Reference](https://api.bullmq.pro/classes/v7.QueuePro.html#resumegroup) [PreviousMax group size](https://docs.bullmq.io/bullmq-pro/groups/max-group-size) [NextPrioritized intra-groups](https://docs.bullmq.io/bullmq-pro/groups/prioritized) Last updated 3 months ago Was this helpful? --- # Producers | BullMQ [](https://docs.bullmq.io/bullmq-pro/nestjs/producers#producers) Producers ------------------------------------------------------------------------------- Job producers add jobs to queues. Producers are typically application services (Nest providers). To add jobs to a queue, first inject the queue into the service as follows: Copy import { Injectable } from '@nestjs/common'; import { QueuePro } from 'taskforcesh/bullmq-pro'; import { InjectQueue } from '@taskforcesh/nestjs-bullmq-pro'; @Injectable() export class AudioService { constructor(@InjectQueue('audio') private audioQueue: QueuePro) {} } The `@InjectQueue()` decorator identifies the queue by its name, as provided in the `registerQueue()`. Now, add a job by calling the queue's `add()` method. Copy const job = await this.audioQueue.add({ foo: 'bar', }); [](https://docs.bullmq.io/bullmq-pro/nestjs/producers#flow-producers) Flow Producers ----------------------------------------------------------------------------------------- To add flows, first inject the flow producer into the service as follows: Copy import { Injectable } from '@nestjs/common'; import { FlowProducerPro } from 'taskforcesh/bullmq-pro'; import { InjectFlowProducer } from '@taskforcesh/nestjs-bullmq-pro'; @Injectable() export class FlowService { constructor( @InjectFlowProducer('flow') private fooFlowProducer: FlowProducerPro, ) {} } The `@InjectFlowProducer()` decorator identifies the flow producer by its name, as provided in the `registerFlowProducer()`. Now, add a flow by calling the flow producer's `add()` method. Copy const job = await this.fooFlowProducer.add({ name: 'root-job', queueName: 'topQueueName', data: {}, children: [\ {\ name,\ data: { idx: 0, foo: 'bar' },\ queueName: 'childrenQueueName',\ },\ ], }); ### [](https://docs.bullmq.io/bullmq-pro/nestjs/producers#read-more) Read more: * 💡 [Queues Technique](https://docs.nestjs.com/techniques/queues) * 💡 [Inject Queue API Reference](https://nestjs.bullmq.pro/functions/InjectQueue.html) * 💡 [Inject Flow Producer API Reference](https://nestjs.bullmq.pro/functions/InjectFlowProducer.html) * 💡 [QueuePro API Reference](https://api.bullmq.pro/classes/v7.QueuePro.html) * 💡 [FlowProducerPro API Reference](https://api.bullmq.pro/classes/v7.FlowProducerPro.html) [PreviousNestJs](https://docs.bullmq.io/bullmq-pro/nestjs) [NextQueue Events Listeners](https://docs.bullmq.io/bullmq-pro/nestjs/queue-events-listeners) Last updated 3 months ago Was this helpful? --- # v0 | BullMQ ### [](https://docs.bullmq.io/python/changelog/changelog-v0#v0.5.6-2023-06-21) v0.5.6 (2023-06-21) #### [](https://docs.bullmq.io/python/changelog/changelog-v0#fix) Fix * **queue:** Pass right params to trimEvents method ([#2004](https://github.com/taskforcesh/bullmq/issues/2004) ) ([`a55fd77`](https://github.com/taskforcesh/bullmq/commit/a55fd777655f7d4bb7af9e4fa2f7b4f48f559189) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v0#v0.5.5-2023-06-16) v0.5.5 (2023-06-16) #### [](https://docs.bullmq.io/python/changelog/changelog-v0#fix-1) Fix * **rate-limit:** Keep priority fifo order (#1991) fixes #1929 (python) ([`56bd7ad`](https://github.com/taskforcesh/bullmq/commit/56bd7ad8c4daffcfb1f9f199abfc5d6495eb291e) ) * **worker:** Set redis version always in initialization (#1989) fixes #1988 ([`a1544a8`](https://github.com/taskforcesh/bullmq/commit/a1544a8c0f29522cd33772b14f559969db852d1d) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v0#v0.5.4-2023-06-14) v0.5.4 (2023-06-14) #### [](https://docs.bullmq.io/python/changelog/changelog-v0#fix-2) Fix * **connection:** Add retry strategy in connection ([#1975](https://github.com/taskforcesh/bullmq/issues/1975) ) ([`7c5ee20`](https://github.com/taskforcesh/bullmq/commit/7c5ee20471b989d297c8c5e87a6ea497a2077ae6) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v0#v0.5.3-2023-06-13) v0.5.3 (2023-06-13) #### [](https://docs.bullmq.io/python/changelog/changelog-v0#fix-3) Fix * **worker:** Use timeout as integer for redis lower than v6.0.0 (python) ([#1981](https://github.com/taskforcesh/bullmq/issues/1981) ) ([`0df6afa`](https://github.com/taskforcesh/bullmq/commit/0df6afad5e71a693b721ba52ffa6be733ee45ccb) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v0#v0.5.2-2023-06-11) v0.5.2 (2023-06-11) #### [](https://docs.bullmq.io/python/changelog/changelog-v0#fix-4) Fix * **retry-job:** Consider priority when moving job to wait (python) ([#1969](https://github.com/taskforcesh/bullmq/issues/1969) ) ([`e753855`](https://github.com/taskforcesh/bullmq/commit/e753855eef248da73a5e9f6b18f4b79319dc2f86) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v0#v0.5.1-2023-06-09) v0.5.1 (2023-06-09) #### [](https://docs.bullmq.io/python/changelog/changelog-v0#fix-5) Fix * **python:** Include lua scripts when releasing ([`bb4f3b2`](https://github.com/taskforcesh/bullmq/commit/bb4f3b2be8e3d5a54a87f0f5d6ba8dfa09900e53) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v0#v0.5.0-2023-06-09) v0.5.0 (2023-06-09) #### [](https://docs.bullmq.io/python/changelog/changelog-v0#feature) Feature * **python:** Add remove job method ([#1965](https://github.com/taskforcesh/bullmq/issues/1965) ) ([`6a172e9`](https://github.com/taskforcesh/bullmq/commit/6a172e97e65684f65ee570c2ae9bcc108720d5df) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v0#v0.4.4-2023-06-08) v0.4.4 (2023-06-08) #### [](https://docs.bullmq.io/python/changelog/changelog-v0#fix-6) Fix * **deps:** Downgrade python-semantic-release to avoid version issue ### [](https://docs.bullmq.io/python/changelog/changelog-v0#v0.4.3-2023-06-07) v0.4.3 (2023-06-07) #### [](https://docs.bullmq.io/python/changelog/changelog-v0#feature-1) Feature * Add changePriority method ([#1943](https://github.com/taskforcesh/bullmq/issues/1943) ) ([`945bcd3`](https://github.com/taskforcesh/bullmq/commit/945bcd39db0f76ef6e9a513304714c120317c7f3) ) #### [](https://docs.bullmq.io/python/changelog/changelog-v0#fix-7) Fix * **rate-limit:** Consider paused queue ([#1931](https://github.com/taskforcesh/bullmq/issues/1931) ) ([`d97864a`](https://github.com/taskforcesh/bullmq/commit/d97864a550992aeb8673557c7d8f186ab4ccb5bf) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v0#v0.4.2-2023-06-01) v0.4.2 (2023-06-01) #### [](https://docs.bullmq.io/python/changelog/changelog-v0#fix-8) Fix * **deps:** Fix 'install\_requires' to include semver ([#1927](https://github.com/taskforcesh/bullmq/issues/1927) ) ([`ce86ece`](https://github.com/taskforcesh/bullmq/commit/ce86eceed40283b5d3276968b65ceae31ce425bb) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v0#v0.4.1-2023-05-29) v0.4.1 (2023-05-29) #### [](https://docs.bullmq.io/python/changelog/changelog-v0#feature-2) Feature * **job:** Add getState method ([#1906](https://github.com/taskforcesh/bullmq/issues/1906) ) ([`f0867a6`](https://github.com/taskforcesh/bullmq/commit/f0867a679c75555fa764078481252110c1e7377f) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v0#v0.4.0-2023-05-18) [v0.4.0](https://github.com/taskforcesh/bullmq/compare/46d6f94...01b621f) (2023-05-18) #### [](https://docs.bullmq.io/python/changelog/changelog-v0#feature-3) Feature * **connection:** accept redis options as string ([`01f549e`](https://github.com/taskforcesh/bullmq/commit/01f549e62a33619a7816758910a2d2b5ac75b589) ) * **job:** add moveToDelayed job method ([#1849](https://github.com/taskforcesh/bullmq/issues/1849) ) ([`5bebf8d`](https://github.com/taskforcesh/bullmq/commit/5bebf8d6560de78448b0413baaabd26f7227575c) ) * **job:** Add retry method into job ([#1877](https://github.com/taskforcesh/bullmq/issues/1877) ) ([`870da45`](https://github.com/taskforcesh/bullmq/commit/870da459f419076f03885a12a4ce5a2930c500f3) ) * **job:** Add updateData method ([#1871](https://github.com/taskforcesh/bullmq/issues/1871) ) ([`800b8c4`](https://github.com/taskforcesh/bullmq/commit/800b8c46e709a8cbc4674d84bd59d5c62251d271) ) * **job:** Add updateProgress method in job class([#1830](https://github.com/taskforcesh/bullmq/issues/1830) ) ([`e1e1aa2`](https://github.com/taskforcesh/bullmq/commit/e1e1aa2e7a41e5418a5a50af4cea347a38bbc7d1) ) * **job:** Save stacktrace when job fails ([#1859](https://github.com/taskforcesh/bullmq/issues/1859) ) ([`0b538ce`](https://github.com/taskforcesh/bullmq/commit/0b538cedf63c3f006838ee3d016e463ee3492f81) ) * Support retryJob logic ([#1869](https://github.com/taskforcesh/bullmq/issues/1869) ) ([`b044a03`](https://github.com/taskforcesh/bullmq/commit/b044a03159bc3a8d8823c71019f64825f318a6c2) ) #### [](https://docs.bullmq.io/python/changelog/changelog-v0#fix-9) Fix * **retry:** Consider when queue is paused ([#1880](https://github.com/taskforcesh/bullmq/issues/1880) ) ([`01b621f`](https://github.com/taskforcesh/bullmq/commit/01b621fea0cbdae602482ff61361c05646823223) ) * **worker:** Stop processes when force stop ([#1837](https://github.com/taskforcesh/bullmq/issues/1837) ) ([`514699c`](https://github.com/taskforcesh/bullmq/commit/514699cd8be96db2320bf0f85d4b6593809a09f1) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v0#v0.3.0-2023-04-18) [v0.3.0](https://github.com/taskforcesh/bullmq/compare/ca48163...46d6f94) (2023-04-18) #### [](https://docs.bullmq.io/python/changelog/changelog-v0#feature-4) Feature * **queue:** Add getJobCounts method ([#1807](https://github.com/taskforcesh/bullmq/issues/1807) ) ([`46d6f94`](https://github.com/taskforcesh/bullmq/commit/46d6f94575454fe2a32be0c5247f16d18739fe27) ) * Improve worker concurrency ([#1809](https://github.com/taskforcesh/bullmq/issues/1809) ) ([`ec7c49e`](https://github.com/taskforcesh/bullmq/commit/ec7c49e284fd1ecdd52b96197281247f5222ea34) ) #### [](https://docs.bullmq.io/python/changelog/changelog-v0#fix-10) Fix * Correct condition so that the worker keeps processing jobs indefinitely ([#1800](https://github.com/taskforcesh/bullmq/issues/1800) ) ([`ef0c5d6`](https://github.com/taskforcesh/bullmq/commit/ef0c5d6cae1dcbae607fa02da32d5236069f2339) ) * Fix scripts typing on array2obj function ([#1786](https://github.com/taskforcesh/bullmq/issues/1786) ) ([`134f6ab`](https://github.com/taskforcesh/bullmq/commit/134f6ab5f3219ddd7a421e61ace6bac72bb51e6d) ) * Pass maxMetricsSize as empty string when it is not provided fixes ([#1754](https://github.com/taskforcesh/bullmq/issues/1754) ) ([`6bda2b2`](https://github.com/taskforcesh/bullmq/commit/6bda2b24be38a78e5fcfc71ed2913f0150a41dfc) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v0#v0.2.0-2023-03-29) [v0.2.0](https://github.com/taskforcesh/bullmq/compare/a97b22f...ca48163) (2023-03-29) #### [](https://docs.bullmq.io/python/changelog/changelog-v0#feature-5) Feature * Add trimEvents method ([#1695](https://github.com/taskforcesh/bullmq/issues/1695) ) ([`ca48163`](https://github.com/taskforcesh/bullmq/commit/ca48163263b12a85533563485176c684e548df0b) ) * **queue:** Add retryJobs method ([#1688](https://github.com/taskforcesh/bullmq/issues/1688) ) ([`2745327`](https://github.com/taskforcesh/bullmq/commit/2745327c7a7080f72e8c265bae77429e597cb6d3) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v0#v0.1.0-2023-02-15) v0.1.0 (2023-02-15) #### [](https://docs.bullmq.io/python/changelog/changelog-v0#feature-6) Feature * Initial python package ([`a97b22f`](https://github.com/taskforcesh/bullmq/commit/a97b22f518a9f6c5d9c30a77bfd03cafdcbc57ff) ) [Previousv1](https://docs.bullmq.io/python/changelog/changelog-v1) Last updated 2 months ago Was this helpful? --- # Custom backoff strategy | BullMQ When the builtin backoff strategies on retries are not sufficient, a custom strategy can be defined. Custom backoff strategies are defined by a function on the queue. The number of attempts already made to process the job is passed to this function as the first parameter, and the error that the job failed with as the second parameter. The function returns either the time to delay the retry with, 0 to retry immediately or -1 to fail the job immediately. Copy const Queue = require('bull'); const myQueue = new Queue('Server B', { settings: { backoffStrategies: { jitter: function (attemptsMade, err) { return 5000 + Math.random() * 500; } } } }); The new backoff strategy can then be specified on the job, using the name defined above: Copy myQueue.add({foo: 'bar'}, { attempts: 3, backoff: { type: 'jitter' } }); You may specify options for your strategy: Copy const Queue = require("bull"); const myQueue = new Queue("Server B", { settings: { backoffStrategies: { // truncated binary exponential backoff binaryExponential: function (attemptsMade, err, options) { // Options can be undefined, you need to handle it by yourself if (!options) { options = {}; } const delay = options.delay || 1000; const truncate = options.truncate || 1000; console.error({ attemptsMade, err, options }); return Math.round( Math.random() * (Math.pow(2, Math.min(attemptsMade, truncate)) - 1) * delay ); }, }, }, }); myQueue.add( { foo: "bar" }, { attempts: 10, backoff: { type: "binaryExponential", options: { delay: 500, truncate: 5, }, }, } ); You may base your backoff strategy on the error that the job throws: Copy const Queue = require('bull'); function MySpecificError() {}; const myQueue = new Queue('Server C', { settings: { backoffStrategies: { foo: function (attemptsMade, err) { if (err instanceof MySpecificError) { return 10000; } return 1000; } } } }); myQueue.process(function (job, done) { if (job.data.msg === 'Specific Error') { throw new MySpecificError(); } else { throw new Error(); } }); myQueue.add({ msg: 'Hello' }, { attempts: 3, backoff: { type: 'foo' } }); myQueue.add({ msg: 'Specific Error' }, { attempts: 3, backoff: { type: 'foo' } }); [PreviousRedis cluster](https://docs.bullmq.io/bull/patterns/redis-cluster) [NextDebugging](https://docs.bullmq.io/bull/patterns/debugging) Last updated 1 year ago Was this helpful? --- # v4 | BullMQ ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v4#id-4.0.3-2022-11-19) [4.0.3](https://github.com/taskforcesh/bullmq-pro/compare/v4.0.2...v4.0.3) (2022-11-19) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v4#bug-fixes) Bug Fixes * **stalled:** use type result as table ([#113](https://github.com/taskforcesh/bullmq-pro/issues/113) ) ([0507801](https://github.com/taskforcesh/bullmq-pro/commit/05078015f0ed687d8151780bb102a43d7da642ca) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v4#id-4.0.2-2022-11-08) [4.0.2](https://github.com/taskforcesh/bullmq-pro/compare/v4.0.1...v4.0.2) (2022-11-08) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v4#bug-fixes-1) Bug Fixes * **promote:** consider groups ([#109](https://github.com/taskforcesh/bullmq-pro/issues/109) ) ([c46c67b](https://github.com/taskforcesh/bullmq-pro/commit/c46c67b785fe521e5742582460c960bc16fd5c60) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v4#id-4.0.1-2022-11-07) [4.0.1](https://github.com/taskforcesh/bullmq-pro/compare/v4.0.0...v4.0.1) (2022-11-07) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v4#features) Features * **flows:** allow parent on root jobs in addBulk method ([#1488](https://github.com/taskforcesh/bullmq/issues/1488) ) ref [#1480](https://github.com/taskforcesh/bullmq/issues/1480) ([92308e5](https://github.com/taskforcesh/bullmq/commit/92308e53acf14e0ce108d94ecd616633ac93e35d) ) [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v4#id-4.0.0-2022-10-27) [4.0.0](https://github.com/taskforcesh/bullmq-pro/compare/v3.0.0...v4.0.0) (2022-10-27) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v4#bug-fixes-2) Bug Fixes * **job:** send failed event when failParentOnFailure ([#1481](https://github.com/taskforcesh/bullmq/issues/1481) ) fixes [#1469](https://github.com/taskforcesh/bullmq/issues/1469) ([b20eb6f](https://github.com/taskforcesh/bullmq/commit/b20eb6f65c7e2c4593d5f9f4d4b940f780bf26d2) ) * **redis:** replace throw exception by console.error ([fafa2f8](https://github.com/taskforcesh/bullmq/commit/fafa2f89e796796f950e6c4abbdda4d3d71ad1b0) ) * **connection:** validate array of strings in Cluster ([#1468](https://github.com/taskforcesh/bullmq/issues/1468) ) fixes [#1467](https://github.com/taskforcesh/bullmq/issues/1467) ([8355182](https://github.com/taskforcesh/bullmq/commit/8355182a372b68ec62e9c3953bacbd69e0abfc74) ) * **worker:** clear stalled jobs timer when closing worker ([1567a0d](https://github.com/taskforcesh/bullmq/commit/1567a0df0ca3c8d43a18990fe488888f4ff68040) ) * **getters:** fix return type of getJobLogs ([d452927](https://github.com/taskforcesh/bullmq/commit/d4529278c59b2c94eee604c7d4455acc490679e9) ) * **sandbox:** get open port using built-in module instead of get-port ([#1446](https://github.com/taskforcesh/bullmq/issues/1446) ) ([6db6288](https://github.com/taskforcesh/bullmq/commit/6db628868a9d64c5a3e47d1c9201017e6d05c1ae) ) * **job:** update delay value when moving to wait ([#1436](https://github.com/taskforcesh/bullmq/issues/1436) ) ([9560915](https://github.com/taskforcesh/bullmq/commit/95609158c1800cf661f22ad7995541fb9474826a) ) * **connection:** throw error when no noeviction policy ([3468390](https://github.com/taskforcesh/bullmq/commit/3468390dd6331291f4cf71a54c32028a06d1d99e) ) * **compat:** remove Queue3 class ([#1421](https://github.com/taskforcesh/bullmq/issues/1421) ) ([fc797f7](https://github.com/taskforcesh/bullmq/commit/fc797f7cd334c19a95cb1290ddb6611cd3417179) ) * **delayed:** promote delayed jobs instead of picking one by one ([1b938af](https://github.com/taskforcesh/bullmq/commit/1b938af75069d69772ddf2b03f95db7f53eada68) ) * **delayed:** remove marker when promoting delayed job ([1aea0dc](https://github.com/taskforcesh/bullmq/commit/1aea0dcc5fb29086cef3d0c432c387d6f8261963) ) * **getters:** compensate for "mark" job id ([231b9aa](https://github.com/taskforcesh/bullmq/commit/231b9aa0f4781e4493d3ea272c33b27c0b7dc0ab) ) * **sandbox:** remove progress method ([b43267b](https://github.com/taskforcesh/bullmq/commit/b43267be50f9eade8233500d189d46940a01cc29) ) * **stalled-jobs:** handle job id 0 ([829e6e0](https://github.com/taskforcesh/bullmq/commit/829e6e0252e78bf2cbc55ab1d3bd153faa0cee4c) ) * **worker:** do not allow stalledInterval to be less than zero ([831ffc5](https://github.com/taskforcesh/bullmq/commit/831ffc520ccd3c6ea63af6b04ddddc9f7829c667) ) * **worker:** use connection closing to determine closing status ([fe1d173](https://github.com/taskforcesh/bullmq/commit/fe1d17321f1eb49bd872c52965392add22729941) ) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v4#features-1) Features * **redis-connection:** allow providing scripts for extension ([#1472](https://github.com/taskforcesh/bullmq/issues/1472) ) ([f193cfb](https://github.com/taskforcesh/bullmq/commit/f193cfb1830e127f9fd47a969baad30011a0e3a4) ) * **flow-producer:** allow parent opts in root job when adding a flow ([#1110](https://github.com/taskforcesh/bullmq/issues/1110) ) ref [#1097](https://github.com/taskforcesh/bullmq/issues/1097) ([3c3ac71](https://github.com/taskforcesh/bullmq/commit/3c3ac718ad84f6bd0cc1575013c948e767b46f38) ) * **job-options:** add failParentOnFailure option ([#1339](https://github.com/taskforcesh/bullmq/issues/1339) ) ([65e5c36](https://github.com/taskforcesh/bullmq/commit/65e5c3678771f26555c9128bdb908dd62e3584f9) ) * improve delayed jobs and remove QueueSchedulerPro ([1f66e5a](https://github.com/taskforcesh/bullmq/commit/1f66e5a6c891d52e0671e58a685dbca511e45e7e) ) * move stalled jobs check and handling to WorkerPro class from QueueSchedulerPro ([13769cb](https://github.com/taskforcesh/bullmq/commit/13769cbe38ba22793cbc66e9706a6be28a7f1512) ) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v4#performance-improvements) Performance Improvements * **scripts:** pre-build scripts ([#1441](https://github.com/taskforcesh/bullmq/issues/1441) ) ([7f72603](https://github.com/taskforcesh/bullmq/commit/7f72603d463f705d0617898cb221f832c49a4aa3) ) * **events:** remove data and opts from added event ([e13d4b8](https://github.com/taskforcesh/bullmq/commit/e13d4b8e0c4f99203f4249ccc86e369d124ff483) ) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v4#breaking-changes) BREAKING CHANGES * Remove QueueSchedulerPro class. WorkerPro class should handle QueueSchedulerPro functionalities. * **compat:** The compatibility class for Bullv3 is no longer available. * Failed and stalled events are now produced by the WorkerPro class instead of by the QueueSchedulerPro. * The minimum Redis recommended version is 6.2.0. [Previousv5](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5) [Nextv3](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v3) Last updated 3 months ago Was this helpful? --- # Local group concurrency | BullMQ It is also possible to set a specific concurrency value to a given group. This is useful if you require that different groups should run with different concurrency factors. Please keep in mind that when specifying a group's concurrency factor, you are storing this value in Redis, so it is your responsibility to remove it if you are not using it anymore. You can use the `setGroupConcurrency` method like this: Copy import { QueuePro } from '@taskforcesh/bullmq-pro'; const queue = new QueuePro('myQueue', { connection }); const groupId = 'my group'; await queue.setGroupConcurrency(groupId, 4); Make sure to also set the [Group Concurrency](https://docs.bullmq.io/bullmq-pro/groups/concurrency) at the worker instance level along side with the desired local concurrency, as it is required for this feature to function properly and also will act as a default concurrency value for the groups that have no local concurrency defined. And you can use the `getGroupConcurrency` method like this: Copy const concurrency = await queue.getGroupConcurrency(groupId); [](https://docs.bullmq.io/bullmq-pro/groups/local-group-concurrency#read-more) Read more: ---------------------------------------------------------------------------------------------- * 💡 [Set Group Concurrency API Reference](https://api.bullmq.pro/classes/v7.QueuePro.html#setgroupconcurrency) * 💡 [Get Group Concurrency API Reference](https://api.bullmq.pro/classes/v7.QueuePro.html#getgroupconcurrency) [PreviousConcurrency](https://docs.bullmq.io/bullmq-pro/groups/concurrency) [NextMax group size](https://docs.bullmq.io/bullmq-pro/groups/max-group-size) Last updated 3 months ago Was this helpful? --- # Introduction | BullMQ The Python package is still in early development and is not recommended for production deployment just yet. In the following months, we will be hardening the code and adding more testing and we expect to have a production-ready package although not all the features are available as in the NodeJS version. ### [](https://docs.bullmq.io/python/introduction#installation) Installation BullMQ is delivered as a pip package and can thus be installed using pip: Copy $ pip install bullmq ### [](https://docs.bullmq.io/python/introduction#get-started) Get started BullMQ uses [asyncio](https://docs.python.org/3/library/asyncio.html) in order to implement concurrency and provide efficient processing of jobs. You can add jobs to a queue like this, assuming you have a Redis host running locally: Copy from bullmq import Queue queue = Queue("myQueue") # Add a job with data { "foo": "bar" } to the queue await queue.add("myJob", { "foo": "bar" }) ... # Close when done adding jobs await queue.close() In order to consume the jobs from the queue you need to use the `Worker` class, providing a "processor" function that will consume the jobs. As soon as the worker is instantiated it will start consuming jobs: Copy from bullmq import Worker import asyncio import signal async def process(job, job_token): # job.data will include the data added to the queue return doSomethingAsync(job) async def main(): # Create an event that will be triggered for shutdown shutdown_event = asyncio.Event() def signal_handler(signal, frame): print("Signal received, shutting down.") shutdown_event.set() # Assign signal handlers to SIGTERM and SIGINT signal.signal(signal.SIGTERM, signal_handler) signal.signal(signal.SIGINT, signal_handler) # Feel free to remove the connection parameter, if your redis runs on localhost worker = Worker("myQueue", process, {"connection": "rediss://:@:"}) # Wait until the shutdown event is set await shutdown_event.wait() # close the worker print("Cleaning up worker...") await worker.close() print("Worker shut down successfully.") if __name__ == "__main__": asyncio.run(main()) If Redis responses are in binary format, you should pass (decode\_responses)\[https://redis-py.readthedocs.io/en/latest/examples/connection\_examples.html#By-default-Redis-return-binary-responses,-to-decode-them-use-decode\_responses=True\] option as _True_. [PreviousManually fetching jobs](https://docs.bullmq.io/bull/patterns/manually-fetching-jobs) [NextChangelogs](https://docs.bullmq.io/python/changelog) Last updated 1 year ago Was this helpful? --- # Manually fetching jobs | BullMQ If you want to manually fetch the jobs from the queue instead of letting the automatic processor taking care of it, this pattern is for you. Manually transitioning states for jobs can be done with a few simple methods. 1. Adding a job to the 'waiting' queue. Grab the queue and call `add`. Copy import Queue from 'bull'; const queue = new Queue({ limiter: { max: 5, duration: 5000, bounceBack: true // important }, ...queueOptions }); queue.add({ random_attr: 'random_value' }); 1. Pulling a job from 'waiting' and moving it to 'active'. Copy const job: Job = await queue.getNextJob(); 1. Move the job to the 'failed' queue if something goes wrong. Copy const (nextJobData, nextJobId) = await job.moveToFailed( { message: 'Call to external service failed!', }, true, ); 1. Move the job to the 'completed' queue. Copy const (nextJobData, nextJobId) = await job.moveToCompleted('succeeded', true); 1. Return the next job if one is returned. Copy if (nextJobdata) { return Job.fromJSON(queue, nextJobData, nextJobId); } **Note** By default the lock duration for a job that has been returned by `getNextJob` or `moveToCompleted` is 30 seconds. If it takes more time than that the job will be automatically marked as stalled and depending on the max stalled options be moved back to the wait state or marked as failed. In order to avoid this you must use `job.extendLock(duration)` in order to give you some more time before the lock expires. It is recommended to extend the lock when half the lock time has passsed. [PreviousDebugging](https://docs.bullmq.io/bull/patterns/debugging) [NextIntroduction](https://docs.bullmq.io/python/introduction) Last updated 2 years ago Was this helpful? --- # v1 | BullMQ [](https://docs.bullmq.io/python/changelog/changelog-v1#id-1.24.0-2023-12-21) [1.24.0](https://github.com/taskforcesh/bullmq/compare/vpy1.23.0...vpy1.24.0) (2023-12-21) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/python/changelog/changelog-v1#features) Features * **job:** add isWaitingChildren method \[python\] ([#2345](https://github.com/taskforcesh/bullmq/issues/2345) ) ([e9c1fa1](https://github.com/taskforcesh/bullmq/commit/e9c1fa10b258ebe171a0396c29b6ccb05aef2608) ) [](https://docs.bullmq.io/python/changelog/changelog-v1#id-1.23.0-2023-12-18) [1.23.0](https://github.com/taskforcesh/bullmq/compare/vpy1.22.0...vpy1.23.0) (2023-12-18) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/python/changelog/changelog-v1#features-1) Features * **queue:** add getRateLimitTtl method \[python\] ([#2340](https://github.com/taskforcesh/bullmq/issues/2340) ) ([f0a1f70](https://github.com/taskforcesh/bullmq/commit/f0a1f7084478f7899233021fbb4d4307c94dfead) ) [](https://docs.bullmq.io/python/changelog/changelog-v1#id-1.22.0-2023-12-14) [1.22.0](https://github.com/taskforcesh/bullmq/compare/vpy1.21.0...vpy1.22.0) (2023-12-14) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/python/changelog/changelog-v1#features-2) Features * **job:** add isFailed method \[python\] ([#2333](https://github.com/taskforcesh/bullmq/issues/2333) ) ([19bfccc](https://github.com/taskforcesh/bullmq/commit/19bfccc2d7734b150a5fbb6ea720fcd9887c9dd3) ) [](https://docs.bullmq.io/python/changelog/changelog-v1#id-1.21.0-2023-12-14) [1.21.0](https://github.com/taskforcesh/bullmq/compare/vpy1.20.0...vpy1.21.0) (2023-12-14) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/python/changelog/changelog-v1#features-3) Features * **job:** add isCompleted method \[python\] ([#2331](https://github.com/taskforcesh/bullmq/issues/2331) ) ([364f0c1](https://github.com/taskforcesh/bullmq/commit/364f0c1f2d4247d2b24041ab9ece0e429110d454) ) [](https://docs.bullmq.io/python/changelog/changelog-v1#id-1.20.0-2023-12-13) [1.20.0](https://github.com/taskforcesh/bullmq/compare/vpy1.19.0...vpy1.20.0) (2023-12-13) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/python/changelog/changelog-v1#features-4) Features * **job:** add isWaiting method \[python\] ([#2328](https://github.com/taskforcesh/bullmq/issues/2328) ) ([5db9f95](https://github.com/taskforcesh/bullmq/commit/5db9f957939cd873eea0224d34569189e5520e84) ) [](https://docs.bullmq.io/python/changelog/changelog-v1#id-1.19.0-2023-12-12) [1.19.0](https://github.com/taskforcesh/bullmq/compare/vpy1.18.0...vpy1.19.0) (2023-12-12) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/python/changelog/changelog-v1#features-5) Features * **job:** add promote method \[python\] ([#2323](https://github.com/taskforcesh/bullmq/issues/2323) ) ([61f4ba3](https://github.com/taskforcesh/bullmq/commit/61f4ba3e99486aa36e5cc3d9b448b8080c567eb1) ) [](https://docs.bullmq.io/python/changelog/changelog-v1#id-1.18.0-2023-12-10) [1.18.0](https://github.com/taskforcesh/bullmq/compare/vpy1.17.0...vpy1.18.0) (2023-12-10) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/python/changelog/changelog-v1#bug-fixes) Bug Fixes * **retry:** pass right redis command name into retryJob script ([#2321](https://github.com/taskforcesh/bullmq/issues/2321) ) \[python\] ([6bb21a0](https://github.com/taskforcesh/bullmq/commit/6bb21a07c9754659fa5aa1734df1046a6da5d16a) ) * **flows:** add meta key to queues created with flows ([272ec69](https://github.com/taskforcesh/bullmq/commit/272ec69557f601a138e1aaba739f7e7878d5344b) ) * **update-progress:** remove old updateProgress script to prevent conflict ([#2298](https://github.com/taskforcesh/bullmq/issues/2298) ) (python) ([e65b819](https://github.com/taskforcesh/bullmq/commit/e65b819101f8e0e8fdef8c51cfdf9a52f5e73f13) ) * **worker:** should cap update progress events ([2cab9e9](https://github.com/taskforcesh/bullmq/commit/2cab9e94f65c7bdd053e3fb5944bcda6e3ebaa39) ) [](https://docs.bullmq.io/python/changelog/changelog-v1#id-1.17.0-2023-11-24) [1.17.0](https://github.com/taskforcesh/bullmq/compare/vpy1.16.1...vpy1.17.0) (2023-11-24) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/python/changelog/changelog-v1#features-6) Features * **worker:** better handling of concurrency when fetching jobs ([#2242](https://github.com/taskforcesh/bullmq/issues/2242) ) ([d2e2035](https://github.com/taskforcesh/bullmq/commit/d2e203588878ee64cb21e67141f73b32867dfb40) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v1#id-1.16.1-2023-11-09) [1.16.1](https://github.com/taskforcesh/bullmq/compare/vpy1.16.0...vpy1.16.1) (2023-11-09) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#bug-fixes-1) Bug Fixes * **job:** set delay value on current job instance when it is retried ([#2266](https://github.com/taskforcesh/bullmq/issues/2266) ) (python) ([76e075f](https://github.com/taskforcesh/bullmq/commit/76e075f54d5745b6cec3cb11305bf3110d963eae) ) [](https://docs.bullmq.io/python/changelog/changelog-v1#id-1.16.0-2023-11-08) [1.16.0](https://github.com/taskforcesh/bullmq/compare/vpy1.15.4...vpy1.16.0) (2023-11-08) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/python/changelog/changelog-v1#bug-fixes-2) Bug Fixes * **backoff:** fix builtin backoff type ([#2265](https://github.com/taskforcesh/bullmq/issues/2265) ) \[python\] ([76959eb](https://github.com/taskforcesh/bullmq/commit/76959eb9d9495eb1b6d2d31fab93c8951b5d3b93) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v1#id-1.15.4-2023-11-05) [1.15.4](https://github.com/taskforcesh/bullmq/compare/vpy1.15.3...vpy1.15.4) (2023-11-05) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#bug-fixes-3) Bug Fixes * update delay job property when moving to delayed set ([#2261](https://github.com/taskforcesh/bullmq/issues/2261) ) ([69ece08](https://github.com/taskforcesh/bullmq/commit/69ece08babd7716c14c38c3dd50630b44c7c1897) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v1#id-1.15.3-2023-11-05) [1.15.3](https://github.com/taskforcesh/bullmq/compare/vpy1.15.2...vpy1.15.3) (2023-11-05) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#bug-fixes-4) Bug Fixes * **add-job:** trim events when waiting-children event is published ([#2262](https://github.com/taskforcesh/bullmq/issues/2262) ) (python) ([198bf05](https://github.com/taskforcesh/bullmq/commit/198bf05fa5a4e1ce50081296033a2e0f26ece498) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v1#v1.15.2-2023-10-18) v1.15.2 (2023-10-18) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#fix) Fix * **events:** Do not publish removed event on non-existent jobs ([#2227](https://github.com/taskforcesh/bullmq/issues/2227) ) ([`c134606`](https://github.com/taskforcesh/bullmq/commit/c1346064c6cd9f93c59b184f150eac11d51c91b4) ) * **events:** Trim events when retrying a job ([#2224](https://github.com/taskforcesh/bullmq/issues/2224) ) ([`1986b05`](https://github.com/taskforcesh/bullmq/commit/1986b05ac03fe4ee48861aa60caadcc9df8170a6) ) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#performance) Performance * **events:** Trim events when removing jobs (#2235) (python) ([`889815c`](https://github.com/taskforcesh/bullmq/commit/889815c412666e5fad8f32d2e3a2d41cf650f001) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v1#v1.15.1-2023-10-04) v1.15.1 (2023-10-04) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#fix-1) Fix * **delayed:** Trim events when moving jobs to delayed (python) ([#2211](https://github.com/taskforcesh/bullmq/issues/2211) ) ([`eca8c2d`](https://github.com/taskforcesh/bullmq/commit/eca8c2d4dfeafbd8ac36a49764dbd4897303628c) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v1#v1.15.0-2023-09-30) v1.15.0 (2023-09-30) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#feature) Feature * Nothing changed ### [](https://docs.bullmq.io/python/changelog/changelog-v1#v1.14.0-2023-09-26) v1.14.0 (2023-09-26) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#feature-1) Feature * **queue:** Add clean method \[python\] ([#2194](https://github.com/taskforcesh/bullmq/issues/2194) ) ([`3b67193`](https://github.com/taskforcesh/bullmq/commit/3b6719379cbec5beb1b7dfb5f06d46cbbf74010f) ) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#fix-2) Fix * **move-to-finished:** Stringify any return value \[python\] (#2198) fixes #2196 ([`07f1335`](https://github.com/taskforcesh/bullmq/commit/07f13356eb1c0136f03dfdf946d163f0ef3c4d62) ) * **queue:** Batched unpack now uses range ([#2188](https://github.com/taskforcesh/bullmq/issues/2188) ) ([`b5e97f4`](https://github.com/taskforcesh/bullmq/commit/b5e97f420bc0c4bc82772f3e87883ee522be43d9) ) * **queue:** Differentiate score purpose per state in clean method (#2133) fixes #2124 ([`862f10b`](https://github.com/taskforcesh/bullmq/commit/862f10b586276314d9bffff2a5e6caf939399f7e) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v1#v1.13.2-2023-09-12) v1.13.2 (2023-09-12) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#fix-3) Fix * **remove:** Change error message when job is locked (python) ([#2175](https://github.com/taskforcesh/bullmq/issues/2175) ) ([`2f5628f`](https://github.com/taskforcesh/bullmq/commit/2f5628feffab66cdcc78abf4d7bb608bdcaa65bb) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v1#v1.13.1-2023-09-11) v1.13.1 (2023-09-11) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#fix-4) Fix * **move-to-finished:** Consider addition of prioritized jobs when processing last active job (#2176) (python) ([`4b01f35`](https://github.com/taskforcesh/bullmq/commit/4b01f359c290cfc62ea74ff3ab0b43ccc6956a02) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v1#v1.13.0-2023-09-07) v1.13.0 (2023-09-07) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#feature-2) Feature * **flow-producer:** Add addBulk method (python) ([#2174](https://github.com/taskforcesh/bullmq/issues/2174) ) ([`c67dfb4`](https://github.com/taskforcesh/bullmq/commit/c67dfb49931ee4cb96573af660e9f2316942687c) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v1#v1.12.0-2023-08-31) v1.12.0 (2023-08-31) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#feature-3) Feature * **queue:** Add addBulk method ([#2161](https://github.com/taskforcesh/bullmq/issues/2161) ) ([`555dd44`](https://github.com/taskforcesh/bullmq/commit/555dd44a0190f4957e43f083e2f59d7f58b90ac9) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v1#v1.11.0-2023-08-26) v1.11.0 (2023-08-26) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#feature-4) Feature * Add flow producer class ([#2115](https://github.com/taskforcesh/bullmq/issues/2115) ) ([`14a769b`](https://github.com/taskforcesh/bullmq/commit/14a769b193d97576ff9b3f2a65de47463ba04ffd) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v1#v1.10.1-2023-08-19) v1.10.1 (2023-08-19) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#fix-5) Fix * **job:** Job getReturnValue not returning returnvalue ([#2143](https://github.com/taskforcesh/bullmq/issues/2143) ) ([`dcb8e6a`](https://github.com/taskforcesh/bullmq/commit/dcb8e6a8e62346fac8574bd9aac56c5a25589a2c) ) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#performance-1) Performance * **rate-limit:** Get pttl only if needed ([#2129](https://github.com/taskforcesh/bullmq/issues/2129) ) ([`12ce2f3`](https://github.com/taskforcesh/bullmq/commit/12ce2f3746626a81ea961961bb1a629077eed68a) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v1#v1.10.0-2023-08-03) v1.10.0 (2023-08-03) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#feature-5) Feature * **redis-connection:** Add username option into redisOpts ([#2108](https://github.com/taskforcesh/bullmq/issues/2108) ) ([`d27f33e`](https://github.com/taskforcesh/bullmq/commit/d27f33e997d30e6c0c7d4484bea338347c3fe67e) ) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#performance-2) Performance * **retry:** Compare prev state instead of regex expression ([#2099](https://github.com/taskforcesh/bullmq/issues/2099) ) ([`c141283`](https://github.com/taskforcesh/bullmq/commit/c1412831903d1fae0955af097e0be049024839fe) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v1#v1.9.0-2023-07-18) v1.9.0 (2023-07-18) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#feature-6) Feature * **job:** Add option for removing children in remove method (python) ([#2064](https://github.com/taskforcesh/bullmq/issues/2064) ) ([`841dc87`](https://github.com/taskforcesh/bullmq/commit/841dc87a689897df81438ad1f43e45a4da77c388) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v1#v1.8.0-2023-07-17) v1.8.0 (2023-07-17) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#fix-6) Fix * **worker:** Respect concurrency (#2062) fixes #2063 ([`1b95185`](https://github.com/taskforcesh/bullmq/commit/1b95185e8f4a4349037b59e61455bdec79792644) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v1#v1.7.0-2023-07-14) v1.7.0 (2023-07-14) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#feature-7) Feature * **queue:** Add remove method ([#2066](https://github.com/taskforcesh/bullmq/issues/2066) ) ([`808ee72`](https://github.com/taskforcesh/bullmq/commit/808ee7231c75d4d826881f25e346f01b2fd2dc23) ) * **worker:** Add id as part of token ([#2061](https://github.com/taskforcesh/bullmq/issues/2061) ) ([`e255356`](https://github.com/taskforcesh/bullmq/commit/e2553562271e1e4143a8fef616349bb30de4899d) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v1#v1.6.1-2023-07-10) v1.6.1 (2023-07-10) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#fix-7) Fix * **pyproject:** Add requires-python config (#2056) fixes #1979 ([`a557970`](https://github.com/taskforcesh/bullmq/commit/a557970c755d370ed23850e2f32af35774002bc9) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v1#v1.6.0-2023-07-06) v1.6.0 (2023-07-06) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#feature-8) Feature * **job:** Add moveToWaitingChildren method ([#2049](https://github.com/taskforcesh/bullmq/issues/2049) ) ([`6d0e224`](https://github.com/taskforcesh/bullmq/commit/6d0e224cd985069055786f447b0ba7c394a76b8a) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v1#v1.5.0-2023-07-04) v1.5.0 (2023-07-04) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#fix-8) Fix * **queue:** Fix isPaused method when custom prefix is present ([#2047](https://github.com/taskforcesh/bullmq/issues/2047) ) ([`7ec1c5b`](https://github.com/taskforcesh/bullmq/commit/7ec1c5b2ccbd575ecd50d339f5377e204ca7aa16) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v1#v1.4.0-2023-06-30) v1.4.0 (2023-06-30) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#feature-9) Feature * **queue:** Add getJobState method ([#2040](https://github.com/taskforcesh/bullmq/issues/2040) ) ([`8ec9ed6`](https://github.com/taskforcesh/bullmq/commit/8ec9ed67d2803224a3b866c51f67239a5c4b7042) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v1#v1.3.1-2023-06-29) v1.3.1 (2023-06-29) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#fix-9) Fix * **pyproject:** Build egg-info at the root location ([`3c2d06e`](https://github.com/taskforcesh/bullmq/commit/3c2d06e7e6e0944135fe6bd8045d08dd43fe7d9c) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v1#v1.3.0-2023-06-29) v1.3.0 (2023-06-29) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#feature-10) Feature * **queue:** Add getFailedCount method ([#2036](https://github.com/taskforcesh/bullmq/issues/2036) ) ([`92d7227`](https://github.com/taskforcesh/bullmq/commit/92d7227bf5ec63a75b7af3fc7c312d9b4a81d69f) ) * **queue:** Add getCompletedCount method ([#2033](https://github.com/taskforcesh/bullmq/issues/2033) ) ([`3e9db5e`](https://github.com/taskforcesh/bullmq/commit/3e9db5ef4d868f8b420e368a711c20c2568a5910) ) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#fix-10) Fix * **release:** Add recommended pyproject.toml configuration ([#2029](https://github.com/taskforcesh/bullmq/issues/2029) ) ([`d03ffc9`](https://github.com/taskforcesh/bullmq/commit/d03ffc9c98425a96d6e9dd47a6625382556a4cbf) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v1#v1.2.0-2023-06-24) v1.2.0 (2023-06-24) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#feature-11) Feature * **queue:** Add get job methods by state ([#2012](https://github.com/taskforcesh/bullmq/issues/2012) ) ([`57b2b72`](https://github.com/taskforcesh/bullmq/commit/57b2b72f79afb683067d49170df5d2eed46e3712) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v1#v1.1.0-2023-06-23) v1.1.0 (2023-06-23) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#feature-12) Feature * **queue:** Add getJobs method ([#2011](https://github.com/taskforcesh/bullmq/issues/2011) ) ([`8d5d6c1`](https://github.com/taskforcesh/bullmq/commit/8d5d6c14442b7b967c42cb6ec3907a4d1a5bd575) ) ### [](https://docs.bullmq.io/python/changelog/changelog-v1#v1.0.0-2023-06-21) v1.0.0 (2023-06-21) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#breaking) Breaking * priority is separeted in its own zset, no duplication needed ([`42a890a`](https://github.com/taskforcesh/bullmq/commit/42a890a2bfe45b29348030f886766400f5d41aa3) ) #### [](https://docs.bullmq.io/python/changelog/changelog-v1#performance-3) Performance * **priority:** Add prioritized as a new state (#1984) (python) ([`42a890a`](https://github.com/taskforcesh/bullmq/commit/42a890a2bfe45b29348030f886766400f5d41aa3) ) [PreviousChangelogs](https://docs.bullmq.io/python/changelog) [Nextv0](https://docs.bullmq.io/python/changelog/changelog-v0) Last updated 1 month ago Was this helpful? --- # Quick Guide | BullMQ ### [](https://docs.bullmq.io/bull/quick-guide#basic-usage) **Basic Usage** Copy const Queue = require("bull"); const videoQueue = new Queue("video transcoding", "redis://127.0.0.1:6379"); const audioQueue = new Queue("audio transcoding", { redis: { port: 6379, host: "127.0.0.1", password: "foobared" }, }); // Specify Redis connection using object const imageQueue = new Queue("image transcoding"); const pdfQueue = new Queue("pdf transcoding"); videoQueue.process(function (job, done) { // job.data contains the custom data passed when the job was created // job.id contains id of this job. // transcode video asynchronously and report progress job.progress(42); // call done when finished done(); // or give a error if error done(new Error("error transcoding")); // or pass it a result done(null, { framerate: 29.5 /* etc... */ }); // If the job throws an unhandled exception it is also handled correctly throw new Error("some unexpected error"); }); audioQueue.process(function (job, done) { // transcode audio asynchronously and report progress job.progress(42); // call done when finished done(); // or give a error if error done(new Error("error transcoding")); // or pass it a result done(null, { samplerate: 48000 /* etc... */ }); // If the job throws an unhandled exception it is also handled correctly throw new Error("some unexpected error"); }); imageQueue.process(function (job, done) { // transcode image asynchronously and report progress job.progress(42); // call done when finished done(); // or give a error if error done(new Error("error transcoding")); // or pass it a result done(null, { width: 1280, height: 720 /* etc... */ }); // If the job throws an unhandled exception it is also handled correctly throw new Error("some unexpected error"); }); pdfQueue.process(function (job) { // Processors can also return promises instead of using the done callback return pdfAsyncProcessor(); }); videoQueue.add({ video: "http://example.com/video1.mov" }); audioQueue.add({ audio: "http://example.com/audio1.mp3" }); imageQueue.add({ image: "http://example.com/image1.tiff" }); ### [](https://docs.bullmq.io/bull/quick-guide#using-promises) **Using promises** Alternatively, you can use return promises instead of using the `done` callback: Copy videoQueue.process(function (job) { // don't forget to remove the done callback! // Simply return a promise return fetchVideo(job.data.url).then(transcodeVideo); // Handles promise rejection return Promise.reject(new Error("error transcoding")); // Passes the value the promise is resolved with to the "completed" event return Promise.resolve({ framerate: 29.5 /* etc... */ }); // If the job throws an unhandled exception it is also handled correctly throw new Error("some unexpected error"); // same as return Promise.reject(new Error("some unexpected error")); }); ### [](https://docs.bullmq.io/bull/quick-guide#sandboxed-processes) **Sandboxed processes** The process function can also be run in a separate process. This has several advantages: * The process is sandboxed so if it crashes it does not affect the worker. * You can run blocking code without affecting the queue (jobs will not stall). * Better utilization of multi-core CPUs. * Less connections to redis. In order to use this feature just create a separate file with the processor: Copy // processor.js module.exports = function (job) { // Do some heavy work return Promise.resolve(result); } And define the processor like this: Copy // Single process: queue.process('/path/to/my/processor.js'); // You can use concurrency as well: queue.process(5, '/path/to/my/processor.js'); // and named processors: queue.process('my processor', 5, '/path/to/my/processor.js'); ### [](https://docs.bullmq.io/bull/quick-guide#repeated-jobs) **Repeated jobs** A job can be added to a queue and processed repeatedly according to a cron specification: Copy paymentsQueue.process(function (job) { // Check payments }); // Repeat payment job once every day at 3:15 (am) paymentsQueue.add(paymentsData, { repeat: { cron: "15 3 * * *" } }); As a tip, check your expressions here to verify they are correct: [cron expression generator](https://crontab.cronhub.io/) ### [](https://docs.bullmq.io/bull/quick-guide#pause-resume) **Pause / Resume** A queue can be paused and resumed globally (pass `true` to pause processing for just this worker): Copy queue.pause().then(function () { // queue is paused now }); queue.resume().then(function () { // queue is resumed now }); ### [](https://docs.bullmq.io/bull/quick-guide#events) **Events** A queue emits some useful events, for example... Copy myqueue.on('completed', function (job, result) { // Job completed with output result! }) For more information on events, including the full list of events that are fired, check out the Events reference ### [](https://docs.bullmq.io/bull/quick-guide#queues-performance) **Queues performance** Queues are relative cheap, so if you need many of them just create new ones with different names. However having too many queues can become unmanageable. Up to a dozen is normally ok. Copy const userJohn = new Queue('john'); const userLisa = new Queue('lisa'); . . . Also keep in mind that every queue instance will require new redis connections, check how to [reuse connections](https://github.com/OptimalBits/bull/blob/master/PATTERNS.md#reusing-redis-connections) or you can also use [named processors](https://github.com/OptimalBits/bull/blob/master/REFERENCE.md#queueprocess) to achieve a similar result. ### [](https://docs.bullmq.io/bull/quick-guide#cluster-support) **Cluster support** From version 3.2.0 and above it is recommended to use threaded processors instead. Queues are robust and can be run in parallel in several threads or processes without any risk of hazards or queue corruption. Check this simple example using cluster to parallelize jobs across processes: Copy const Queue = require("bull"); const cluster = require("cluster"); const numWorkers = 8; const queue = new Queue("test concurrent queue"); if (cluster.isMaster) { for (let i = 0; i < numWorkers; i++) { cluster.fork(); } cluster.on("online", function (worker) { // Let's create a few jobs for the queue workers for (let i = 0; i < 500; i++) { queue.add({ foo: "bar" }); } }); cluster.on("exit", function (worker, code, signal) { console.log("worker " + worker.process.pid + " died"); }); } else { queue.process(function (job, jobDone) { console.log("Job done by worker", cluster.worker.id, job.id); jobDone(); }); } * * * [PreviousInstall](https://docs.bullmq.io/bull/install) [NextImportant Notes](https://docs.bullmq.io/bull/important-notes) Last updated 3 years ago Was this helpful? --- # v1 | BullMQ ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#id-1.4.1-2022-04-19) [1.4.1](https://github.com/taskforcesh/bullmq-pro/compare/v1.4.0...v1.4.1) (2022-04-19) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#bug-fixes) Bug Fixes * **deps:** upgrade bullmq to 1.80.4 ([#46](https://github.com/taskforcesh/bullmq-pro/issues/46) ) ([fc2818f](https://github.com/taskforcesh/bullmq-pro/commit/fc2818f09ddd273b7cc16c54fcd9650e1d456d04) ) [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#id-1.4.0-2022-04-12) [1.4.0](https://github.com/taskforcesh/bullmq-pro/compare/v1.3.5...v1.4.0) (2022-04-12) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#features) Features * **worker:** allow ttl per job name ([#43](https://github.com/taskforcesh/bullmq-pro/issues/43) ) ([93a61ad](https://github.com/taskforcesh/bullmq-pro/commit/93a61ad1b0c788af39ea944fb908ec78398a47a9) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#id-1.3.5-2022-04-06) [1.3.5](https://github.com/taskforcesh/bullmq-pro/compare/v1.3.4...v1.3.5) (2022-04-06) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#bug-fixes-1) Bug Fixes * **deps:** upgrade bullmq to 1.78.2 ([#42](https://github.com/taskforcesh/bullmq-pro/issues/42) ) ([10771c2](https://github.com/taskforcesh/bullmq-pro/commit/10771c29212686584fbd278d689655b8e97de62a) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#id-1.3.4-2022-03-26) [1.3.4](https://github.com/taskforcesh/bullmq-pro/compare/v1.3.3...v1.3.4) (2022-03-26) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#bug-fixes-2) Bug Fixes * **groups:** consider delay option ([#40](https://github.com/taskforcesh/bullmq-pro/issues/40) ) fixes [#39](https://github.com/taskforcesh/bullmq-pro/issues/39) ([c2a2b93](https://github.com/taskforcesh/bullmq-pro/commit/c2a2b9393a0ca3febcc486ea2f1afd515c294473) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#id-1.3.3-2022-03-23) [1.3.3](https://github.com/taskforcesh/bullmq-pro/compare/v1.3.2...v1.3.3) (2022-03-23) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#bug-fixes-3) Bug Fixes * upgrade bullmq to 1.77.3 ([#38](https://github.com/taskforcesh/bullmq-pro/issues/38) ) ([b7af3d2](https://github.com/taskforcesh/bullmq-pro/commit/b7af3d2c01cfc2fbfadd4643fd8915ed1a2c3098) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#id-1.3.2-2022-03-16) [1.3.2](https://github.com/taskforcesh/bullmq-pro/compare/v1.3.1...v1.3.2) (2022-03-16) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#bug-fixes-4) Bug Fixes * correctly handle stalled jobs when using groups ([#37](https://github.com/taskforcesh/bullmq-pro/issues/37) ) ([97ed889](https://github.com/taskforcesh/bullmq-pro/commit/97ed8890061aea2e5833808df4400bf1e86caada) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#id-1.3.1-2022-03-16) [1.3.1](https://github.com/taskforcesh/bullmq-pro/compare/v1.3.0...v1.3.1) (2022-03-16) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#bug-fixes-5) Bug Fixes * upgrade bullmq to 1.76.6 ([#36](https://github.com/taskforcesh/bullmq-pro/issues/36) ) ([6c7f008](https://github.com/taskforcesh/bullmq-pro/commit/6c7f00885cc8833e5b23322116687d1fc4f7d03c) ) [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#id-1.3.0-2022-03-10) [1.3.0](https://github.com/taskforcesh/bullmq-pro/compare/v1.2.5...v1.3.0) (2022-03-10) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#features-1) Features * add support for max concurrency per group ([d4afb21](https://github.com/taskforcesh/bullmq-pro/commit/d4afb21f7162eda0b080c844c3f6dd90c87003e1) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#id-1.2.5-2022-03-08) [1.2.5](https://github.com/taskforcesh/bullmq-pro/compare/v1.2.4...v1.2.5) (2022-03-08) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#bug-fixes-6) Bug Fixes * upgrade bullmq to 1.76.1 ([#34](https://github.com/taskforcesh/bullmq-pro/issues/34) ) ([8eb36d2](https://github.com/taskforcesh/bullmq-pro/commit/8eb36d280bcbc2bdd7c640d9de12022dbbe79581) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#id-1.2.4-2022-02-17) [1.2.4](https://github.com/taskforcesh/bullmq-pro/compare/v1.2.3...v1.2.4) (2022-02-17) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#bug-fixes-7) Bug Fixes * upgrade bullmq to 1.73.0 ([#33](https://github.com/taskforcesh/bullmq-pro/issues/33) ) ([6afa980](https://github.com/taskforcesh/bullmq-pro/commit/6afa980f3241f3ed364bd8195375fa800b7369d8) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#id-1.2.3-2022-02-03) [1.2.3](https://github.com/taskforcesh/bullmq-pro/compare/v1.2.2...v1.2.3) (2022-02-03) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#bug-fixes-8) Bug Fixes * upgrade bullmq to 1.67.1 ([#31](https://github.com/taskforcesh/bullmq-pro/issues/31) ) ([905d805](https://github.com/taskforcesh/bullmq-pro/commit/905d80588f240464ed5b08359cc315edd4016c7c) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#id-1.2.2-2022-01-29) [1.2.2](https://github.com/taskforcesh/bullmq-pro/compare/v1.2.1...v1.2.2) (2022-01-29) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#bug-fixes-9) Bug Fixes * upgrade bullmq to 1.66.1 ([#32](https://github.com/taskforcesh/bullmq-pro/issues/32) ) ([a830bc6](https://github.com/taskforcesh/bullmq-pro/commit/a830bc64a3588f9b4340e1a9d7a609e8cbc73836) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#id-1.2.1-2022-01-13) [1.2.1](https://github.com/taskforcesh/bullmq-pro/compare/v1.2.0...v1.2.1) (2022-01-13) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#bug-fixes-10) Bug Fixes * upgrade bullmq to 1.64.0 ([#28](https://github.com/taskforcesh/bullmq-pro/issues/28) ) ([893cbeb](https://github.com/taskforcesh/bullmq-pro/commit/893cbeb279196e1aa4c0978c57fc3183a4a4615c) ) [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#id-1.2.0-2021-12-17) [1.2.0](https://github.com/taskforcesh/bullmq-pro/compare/v1.1.6...v1.2.0) (2021-12-17) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#features-2) Features * groups rate limit ([#22](https://github.com/taskforcesh/bullmq-pro/issues/22) ) ([a9268b0](https://github.com/taskforcesh/bullmq-pro/commit/a9268b01093e64f45afbde5e36a60451fcaca880) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#id-1.1.6-2021-12-08) [1.1.6](https://github.com/taskforcesh/bullmq-pro/compare/v1.1.5...v1.1.6) (2021-12-08) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#bug-fixes-11) Bug Fixes * **bullmq:** use fixed version for 1.55.1 ([#23](https://github.com/taskforcesh/bullmq-pro/issues/23) ) ([81368de](https://github.com/taskforcesh/bullmq-pro/commit/81368de9045c192c74d4c171333c3a1dd359565b) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#id-1.1.5-2021-12-04) [1.1.5](https://github.com/taskforcesh/bullmq-pro/compare/v1.1.4...v1.1.5) (2021-12-04) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#bug-fixes-12) Bug Fixes * **bullmq:** upgrade to 1.55.1 ([#21](https://github.com/taskforcesh/bullmq-pro/issues/21) ) ([5bb682a](https://github.com/taskforcesh/bullmq-pro/commit/5bb682abc1ced6d46fecbfbb8f955c62e0956e01) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#id-1.1.4-2021-12-02) [1.1.4](https://github.com/taskforcesh/bullmq-pro/compare/v1.1.3...v1.1.4) (2021-12-02) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#bug-fixes-13) Bug Fixes * expose missing classes ([#20](https://github.com/taskforcesh/bullmq-pro/issues/20) ) ([1a5d80d](https://github.com/taskforcesh/bullmq-pro/commit/1a5d80d95177d3ad4ebc8923df88065d1e38e08b) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#id-1.1.3-2021-12-01) [1.1.3](https://github.com/taskforcesh/bullmq-pro/compare/v1.1.2...v1.1.3) (2021-12-01) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#bug-fixes-14) Bug Fixes * point correct main files ([#19](https://github.com/taskforcesh/bullmq-pro/issues/19) ) ([86075af](https://github.com/taskforcesh/bullmq-pro/commit/86075af5acb85bb620071623a3ce2d4c2c88ec71) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#id-1.1.2-2021-11-26) [1.1.2](https://github.com/taskforcesh/bullmq-pro/compare/v1.1.1...v1.1.2) (2021-11-26) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#bug-fixes-15) Bug Fixes * **bullmq:** upgrade to 1.54.5 version ([#17](https://github.com/taskforcesh/bullmq-pro/issues/17) ) ([f2c29a4](https://github.com/taskforcesh/bullmq-pro/commit/f2c29a424bb3350f097eb446cb8d81f40cc0adb6) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#id-1.1.1-2021-11-25) [1.1.1](https://github.com/taskforcesh/bullmq-pro/compare/v1.1.0...v1.1.1) (2021-11-25) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#bug-fixes-16) Bug Fixes * use es6 interpolation ([f7a533a](https://github.com/taskforcesh/bullmq-pro/commit/f7a533a4b06ccfee1fa386e439af266fd502701e) ) [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#id-1.1.0-2021-11-24) [1.1.0](https://github.com/taskforcesh/bullmq-pro/compare/v1.0.4...v1.1.0) (2021-11-24) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#features-3) Features * **rxjs:** support observables ([#11](https://github.com/taskforcesh/bullmq-pro/issues/11) ) ([df4cf07](https://github.com/taskforcesh/bullmq-pro/commit/df4cf07311a41992547b4a5d326408952653f6fd) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#id-1.0.4-2021-11-22) [1.0.4](https://github.com/taskforcesh/bullmq-pro/compare/v1.0.3...v1.0.4) (2021-11-22) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#bug-fixes-17) Bug Fixes * **add:** send missing events in lua script ([#13](https://github.com/taskforcesh/bullmq-pro/issues/13) ) ([a6658dd](https://github.com/taskforcesh/bullmq-pro/commit/a6658dda455e864be5cf34784bd8332575c5d3d2) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#id-1.0.3-2021-11-18) [1.0.3](https://github.com/taskforcesh/bullmq-pro/compare/v1.0.2...v1.0.3) (2021-11-18) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#bug-fixes-18) Bug Fixes * **bullmq:** upgrade to 1.54.0 version ([#10](https://github.com/taskforcesh/bullmq-pro/issues/10) ) ([1fc73ad](https://github.com/taskforcesh/bullmq-pro/commit/1fc73ad86f4130f3e3849890162a37bb4ceed7a6) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#id-1.0.2-2021-10-18) [1.0.2](https://github.com/taskforcesh/bullmq-pro/compare/v1.0.1...v1.0.2) (2021-10-18) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#bug-fixes-19) Bug Fixes * **npm:** replace npm registry ([01518eb](https://github.com/taskforcesh/bullmq-pro/commit/01518eba488d250d1afa675c7fba237a6d442fe3) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#id-1.0.1-2021-10-18) [1.0.1](https://github.com/taskforcesh/bullmq-pro/compare/v1.0.0...v1.0.1) (2021-10-18) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#bug-fixes-20) Bug Fixes * **bullmq:** upgrade bullmq ([6b93b5f](https://github.com/taskforcesh/bullmq-pro/commit/6b93b5f4c8712fc55ea76ed89a0eba38b7b4df35) ) [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#id-1.0.0-2021-10-18) 1.0.0 (2021-10-18) -------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#bug-fixes-21) Bug Fixes * remove nodejs 10 support ([49eaf78](https://github.com/taskforcesh/bullmq-pro/commit/49eaf7893e287983febe05c034733ec958114882) ) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1#features-4) Features * **groups:** initial implementation ([b1da106](https://github.com/taskforcesh/bullmq-pro/commit/b1da106d7870e8e2783a6028b5088e6c4fa82086) ) * add queue-pro, worker-pro and redis-connection-pro ([b9de319](https://github.com/taskforcesh/bullmq-pro/commit/b9de3193b7820dc0dcf5ae1a2f673f7ed5a82aed) ) * initial commit ([7924260](https://github.com/taskforcesh/bullmq-pro/commit/7924260d621dd98b8acc6aefb53c21f6d3e06186) ) [Previousv2](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2) [NextNew Releases](https://docs.bullmq.io/bullmq-pro/new-releases) Last updated 3 months ago Was this helpful? --- # v5 | BullMQ ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.3.5-2023-06-16) [5.3.5](https://github.com/taskforcesh/bullmq-pro/compare/v5.3.4...v5.3.5) (2023-06-16) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#bug-fixes) Bug Fixes * **rate-limit:** keep priority fifo order ([#1991](https://github.com/taskforcesh/bullmq/issues/1991) ) fixes [#1929](https://github.com/taskforcesh/bullmq/issues/1929) (python) ([56bd7ad](https://github.com/taskforcesh/bullmq/commit/56bd7ad8c4daffcfb1f9f199abfc5d6495eb291e) ) * **worker:** set redis version always in initialization ([#1989](https://github.com/taskforcesh/bullmq/issues/1989) ) fixes [#1988](https://github.com/taskforcesh/bullmq/issues/1988) ([a1544a8](https://github.com/taskforcesh/bullmq/commit/a1544a8c0f29522cd33772b14f559969db852d1d) ) * **worker:** use timeout as integer for redis lower than v6.0.0 (python) ([#1981](https://github.com/taskforcesh/bullmq/issues/1981) ) ([0df6afa](https://github.com/taskforcesh/bullmq/commit/0df6afad5e71a693b721ba52ffa6be733ee45ccb) ) * **retry-job:** consider priority when moving job to wait (python) ([#1969](https://github.com/taskforcesh/bullmq/issues/1969) ) ([e753855](https://github.com/taskforcesh/bullmq/commit/e753855eef248da73a5e9f6b18f4b79319dc2f86) ) * **job:** import right reference of QueueEvents ([#1964](https://github.com/taskforcesh/bullmq/issues/1964) ) ([689c845](https://github.com/taskforcesh/bullmq/commit/689c84567f3a9fea51f349ca93b3008d5c187f62) ) * **job:** use QueueEvents type for waitUntilFinished ([#1958](https://github.com/taskforcesh/bullmq/issues/1958) ) ([881848c](https://github.com/taskforcesh/bullmq/commit/881848c1ee3835dac24daf6807b1f35da967f68b) ) * **worker:** better worker client naming ([c5f63af](https://github.com/taskforcesh/bullmq/commit/c5f63affe72f7b6616f4c5f3aafde858dcc0b200) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.3.4-2023-06-06) [5.3.4](https://github.com/taskforcesh/bullmq-pro/compare/v5.3.3...v5.3.4) (2023-06-06) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#features) Features * **job:** add changePriority method ([#1901](https://github.com/taskforcesh/bullmq/issues/1901) ) ref [#1899](https://github.com/taskforcesh/bullmq/issues/1899) ([9485ad5](https://github.com/taskforcesh/bullmq/commit/9485ad567e2d8c78d601cc9eb2b7dd37f96d00c9) ) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#bug-fixes-1) Bug Fixes * **rate-limit:** consider paused queue ([#1931](https://github.com/taskforcesh/bullmq/issues/1931) ) ([d97864a](https://github.com/taskforcesh/bullmq/commit/d97864a550992aeb8673557c7d8f186ab4ccb5bf) ) * **rate-limit:** take in count priority ([#1919](https://github.com/taskforcesh/bullmq/issues/1919) ) fixes [#1915](https://github.com/taskforcesh/bullmq/issues/1915) ([b8157a3](https://github.com/taskforcesh/bullmq/commit/b8157a3424ceb60e662e80a3b0db918241b87ecc) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.3.3-2023-05-30) [5.3.3](https://github.com/taskforcesh/bullmq-pro/compare/v5.3.2...v5.3.3) (2023-05-30) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#bug-fixes-2) Bug Fixes * **rate-limit:** take groups in count in global rate limit counter ([#151](https://github.com/taskforcesh/bullmq-pro/issues/151) ) ([3d8b28d](https://github.com/taskforcesh/bullmq-pro/commit/3d8b28de087b6f97570dee74a356e11d92daf7fa) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.3.2-2023-05-24) [5.3.2](https://github.com/taskforcesh/bullmq-pro/compare/v5.3.1...v5.3.2) (2023-05-24) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#bug-fixes-3) Bug Fixes * **job-pro:** use saveStacktrace script ([#150](https://github.com/taskforcesh/bullmq-pro/issues/150) ) ([146d9a9](https://github.com/taskforcesh/bullmq-pro/commit/146d9a9596387026e842ab63f13212f7ed66c2c9) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.3.1-2023-05-23) [5.3.1](https://github.com/taskforcesh/bullmq-pro/compare/v5.3.0...v5.3.1) (2023-05-23) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#bug-fixes-4) Bug Fixes * **remove-job:** consider decreasing group concurrency ([#149](https://github.com/taskforcesh/bullmq-pro/issues/149) ) ([25068e2](https://github.com/taskforcesh/bullmq-pro/commit/25068e243f993e6a22531bb8d2c6c60ffba36b9b) ) [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.3.0-2023-05-23) [5.3.0](https://github.com/taskforcesh/bullmq-pro/compare/v5.2.5...v5.3.0) (2023-05-23) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#features-1) Features * add support for job batches ([1db0c94](https://github.com/taskforcesh/bullmq-pro/commit/1db0c9436461262b3393628d3eff2191cc3247a2) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.2.5-2023-05-20) [5.2.5](https://github.com/taskforcesh/bullmq-pro/compare/v5.2.4...v5.2.5) (2023-05-20) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#bug-fixes-5) Bug Fixes * **retry-job:** consider promoting delayed jobs ([#147](https://github.com/taskforcesh/bullmq-pro/issues/147) ) ([3efd39e](https://github.com/taskforcesh/bullmq-pro/commit/3efd39eb2552d852b916f974fc16fab6768434c5) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.2.4-2023-05-16) [5.2.4](https://github.com/taskforcesh/bullmq-pro/compare/v5.2.3...v5.2.4) (2023-05-16) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#bug-fixes-6) Bug Fixes * **rate-limit:** consider groups when global dynamic rate limit ([#145](https://github.com/taskforcesh/bullmq-pro/issues/145) ) ([6f5d1e3](https://github.com/taskforcesh/bullmq-pro/commit/6f5d1e3014824149d1c857b79587bd5fd01f9bba) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.2.3-2023-05-09) [5.2.3](https://github.com/taskforcesh/bullmq-pro/compare/v5.2.2...v5.2.3) (2023-05-09) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#features-2) Features * **worker:** add worker threads support ([0820985](https://github.com/taskforcesh/bullmq/commit/0820985e073582fdf841affad38ecc7ab64691ec) ) * upgrade ioredis to 5.3.2 ([375b1be](https://github.com/taskforcesh/bullmq/commit/375b1be52035e93c5fef6024e0d06aa723f602a9) ) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#bug-fixes-7) Bug Fixes * **rate-limit:** consider paused queue when dynamic rate limit ([#1884](https://github.com/taskforcesh/bullmq/issues/1884) ) ([a23f37e](https://github.com/taskforcesh/bullmq/commit/a23f37e4079d34c8589efc85e4d726a62244f0d2) ) * **retry:** consider when queue is paused ([#1880](https://github.com/taskforcesh/bullmq/issues/1880) ) ([01b621f](https://github.com/taskforcesh/bullmq/commit/01b621fea0cbdae602482ff61361c05646823223) ) * **worker:** close open handles after closing ([#1861](https://github.com/taskforcesh/bullmq/issues/1861) ) fixes [#1312](https://github.com/taskforcesh/bullmq/issues/1312) ([39286e8](https://github.com/taskforcesh/bullmq/commit/39286e87e8ffabf641f229cf2da3db4c280f4637) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.2.2-2023-04-18) [5.2.2](https://github.com/taskforcesh/bullmq-pro/compare/v5.2.1...v5.2.2) (2023-04-18) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#features-3) Features * **upstash:** don't throw an error when detecting an upstash host ([2e06bca](https://github.com/taskforcesh/bullmq/commit/2e06bca3615aafecd725d093045a510a67053fed) ) ref ([#143](https://github.com/taskforcesh/bullmq-pro/issues/143) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.2.1-2023-04-15) [5.2.1](https://github.com/taskforcesh/bullmq-pro/compare/v5.2.0...v5.2.1) (2023-04-15) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#bug-fixes-8) Bug Fixes * **flow-producer-pro:** fix opts assignment ([#140](https://github.com/taskforcesh/bullmq-pro/issues/140) ) ([9f8896c](https://github.com/taskforcesh/bullmq-pro/commit/9f8896c5f082d807bb6945780b30c2768015b95f) ) [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.2.0-2023-03-23) [5.2.0](https://github.com/taskforcesh/bullmq-pro/compare/v5.1.15...v5.2.0) (2023-03-23) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#features-4) Features * **groups:** add repair maxed group function ([a1fa1d8](https://github.com/taskforcesh/bullmq-pro/commit/a1fa1d80cf8ad79c7b9844df163765f61231350a) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.1.15-2023-03-23) [5.1.15](https://github.com/taskforcesh/bullmq-pro/compare/v5.1.14...v5.1.15) (2023-03-23) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#bug-fixes-9) Bug Fixes * **job:** avoid error when job is moved when processing ([#1354](https://github.com/taskforcesh/bullmq/issues/1354) ) fixes [#1343](https://github.com/taskforcesh/bullmq/issues/1343) [#1602](https://github.com/taskforcesh/bullmq/issues/1602) ([78085e4](https://github.com/taskforcesh/bullmq/commit/78085e4304357dd3695df61057f91e706c3a52bf) ) * **worker:** throw error with invalid concurrency fixes [#1723](https://github.com/taskforcesh/bullmq/issues/1723) ([2a1cdbe](https://github.com/taskforcesh/bullmq/commit/2a1cdbe3e871309f460aadc14b4d632238c32aa9) ) * **worker:** close lock extended timer ([7995f18](https://github.com/taskforcesh/bullmq/commit/7995f18bb7712bd50d0fa3d17c4ab565b16ab379) ) * **worker:** correct lock extender logic ([6aa3569](https://github.com/taskforcesh/bullmq/commit/6aa3569db0fe0137790e61a4b5982d2b35ee5646) ) * **worker:** start stalled check timer ([4763be0](https://github.com/taskforcesh/bullmq/commit/4763be028b0c7b0460fd0804d4569c446a06ef4a) ) * **worker:** run stalled check directly first time ([f71ec03](https://github.com/taskforcesh/bullmq/commit/f71ec03111a22897cbf2fad39073185e4aeac6d6) ) * **worker:** restore failed event job parameter typing ([#1707](https://github.com/taskforcesh/bullmq/issues/1707) ) ([44c2203](https://github.com/taskforcesh/bullmq/commit/44c2203ab65d406be9a913254600fe07c83e62d5) ) * **worker:** failed event receives an optional job parameter ([#1702](https://github.com/taskforcesh/bullmq/issues/1702) ) fixes [#1690](https://github.com/taskforcesh/bullmq/issues/1690) ([6009906](https://github.com/taskforcesh/bullmq/commit/6009906355765bf00cba5c1505e9e0c6bf8f14db) ) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#features-5) Features * **worker:** replace Promise.race with efficient an async fifo ([0d94e35](https://github.com/taskforcesh/bullmq/commit/0d94e35e805b09c3b4c7404b8a2eeb71a1aff5c4) ) ref ([#138](https://github.com/taskforcesh/bullmq-pro/issues/138) ) * **worker:** simplify lock extension to one call independent of concurrency ([ebf1aeb](https://github.com/taskforcesh/bullmq/commit/ebf1aeb2400383d0ae90ab68aeb4822aea03ba44) ) * **worker:** add remove on complete and fail options ([#1703](https://github.com/taskforcesh/bullmq/issues/1703) ) ([cf13494](https://github.com/taskforcesh/bullmq/commit/cf1349471dcbf0e43feea9972eaa71d2299d619f) ) * **worker:** add a public method to run the stalled checker ([3159266](https://github.com/taskforcesh/bullmq/commit/3159266ccb002d4fc71b7ee7ac63c465c536dbd1) ) * **worker:** add support to disable stalled checks ([49e860c](https://github.com/taskforcesh/bullmq/commit/49e860c6675853971e992c2945b445660504e3b2) ) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#performance-improvements) Performance Improvements * **scripts:** reuse keys array to avoid allocations ([feac7b4](https://github.com/taskforcesh/bullmq/commit/feac7b4070a6a3720597af36c43d095e9ea37173) ) * **worker:** improve worker memory consumption ([4846cf1](https://github.com/taskforcesh/bullmq/commit/4846cf1fe3f9ea35f58a679c11706e1a7101c898) ) * **move-to-active:** remove deprecated limiter reference ([#1673](https://github.com/taskforcesh/bullmq/issues/1673) ) ([a97b22f](https://github.com/taskforcesh/bullmq/commit/a97b22f518a9f6c5d9c30a77bfd03cafdcbc57ff) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.1.14-2023-02-15) [5.1.14](https://github.com/taskforcesh/bullmq-pro/compare/v5.1.13...v5.1.14) (2023-02-15) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#bug-fixes-10) Bug Fixes * **job:** check jobKey when saving stacktrace ([#1681](https://github.com/taskforcesh/bullmq/issues/1681) ) fixes [#1676](https://github.com/taskforcesh/bullmq/issues/1676) ([1856c76](https://github.com/taskforcesh/bullmq/commit/1856c7684c377ca4fd36294cca8e128404be27b8) ) * infinite worker process spawned for invalid JS file ([a445ba8](https://github.com/taskforcesh/bullmq/commit/a445ba8b7a261b370dec7d88091ae5f5af8b2728) ) * **worker:** add a maximum block time ([1a2618b](https://github.com/taskforcesh/bullmq/commit/1a2618bc5473288a62dddb85e3cb78d6cdb4f39f) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.1.13-2023-02-07) [5.1.13](https://github.com/taskforcesh/bullmq-pro/compare/v5.1.12...v5.1.13) (2023-02-07) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#bug-fixes-11) Bug Fixes * **master:** copy type declaration ([23ade6e](https://github.com/taskforcesh/bullmq/commit/23ade6e3e45df14bd3fbc2c3e7be47307b642872) ) * **redis:** increase minimum default retry time ([d521531](https://github.com/taskforcesh/bullmq/commit/d521531e22ba9eda8ad8d6e8eddf450fdc3f50f4) ) * **connection:** apply console.warn in noeviction message ([95f171c](https://github.com/taskforcesh/bullmq/commit/95f171cbc8cdd7d04865618b715dd21229f36a4a) ) * **error:** remove global prototype toJSON ([#1642](https://github.com/taskforcesh/bullmq/issues/1642) ) fixes [#1414](https://github.com/taskforcesh/bullmq/issues/1414) ([d4e7108](https://github.com/taskforcesh/bullmq/commit/d4e7108a37aeabdd3085a26c9daf09cea5976f3e) ) * **rate-limit:** update group concurrency after manual rate-limit ([de66ec4](https://github.com/taskforcesh/bullmq-pro/commit/de66ec494b8400e3cbb916f5937dc3834a213389) ) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#features-6) Features * **job:** allow clearing job's log ([#1600](https://github.com/taskforcesh/bullmq/issues/1600) ) ([0ded2d7](https://github.com/taskforcesh/bullmq/commit/0ded2d7709322bf105e0decac44d801ece5615f2) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.1.12-2023-01-26) [5.1.12](https://github.com/taskforcesh/bullmq-pro/compare/v5.1.11...v5.1.12) (2023-01-26) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#bug-fixes-12) Bug Fixes * **move-to-finished:** return correct delayUntil ([#1643](https://github.com/taskforcesh/bullmq/issues/1643) ) ([c4bf9fa](https://github.com/taskforcesh/bullmq/commit/c4bf9fa6563eda1630d8eb2189b16e9324b01c7f) ) * **worker:** fix delayed jobs with concurrency fixes [#1627](https://github.com/taskforcesh/bullmq/issues/1627) ([99a8e6d](https://github.com/taskforcesh/bullmq/commit/99a8e6d3a339be51fb46f69c8afac4ecdebff6d3) ) * **move-to-active:** delete marker when it is moved to active ([#1634](https://github.com/taskforcesh/bullmq/issues/1634) ) ([ad1fcea](https://github.com/taskforcesh/bullmq/commit/ad1fcea4500d4ceed51d5d5b0a03dbb5e1735a42) ) * **move-to-active:** validate next marker and return delayUntil ([#1630](https://github.com/taskforcesh/bullmq/issues/1630) ) ([3cd3305](https://github.com/taskforcesh/bullmq/commit/3cd33052fc711a9ba560c9a431630be5cdd02193) ) * **worker:** add max concurrency from the beginning ([#1597](https://github.com/taskforcesh/bullmq/issues/1597) ) fixes [#1589](https://github.com/taskforcesh/bullmq/issues/1589) ([6f49db3](https://github.com/taskforcesh/bullmq/commit/6f49db3fb15119d13f99cd83d49f2a7bdcb614cd) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.1.11-2023-01-10) [5.1.11](https://github.com/taskforcesh/bullmq-pro/compare/v5.1.10...v5.1.11) (2023-01-10) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#bug-fixes-13) Bug Fixes * circular references ([#1622](https://github.com/taskforcesh/bullmq/issues/1622) ) ([f607ec7](https://github.com/taskforcesh/bullmq/commit/f607ec7530fb4430e8cab7ed325583bd9d171ccf) ) * [#1603](https://github.com/taskforcesh/bullmq/issues/1603) performance issues in `remove()` ([#1607](https://github.com/taskforcesh/bullmq/issues/1607) ) ([2541215](https://github.com/taskforcesh/bullmq/commit/2541215bcf81dcd52eaefa02530c3812a5135fbf) ) * **delayed:** remove marker after being consumed ([#1620](https://github.com/taskforcesh/bullmq/issues/1620) ) fixes [#1615](https://github.com/taskforcesh/bullmq/issues/1615) ([9fce0f0](https://github.com/taskforcesh/bullmq/commit/9fce0f05e5acc1918a276b03e8cb9c16083cb509) ) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#performance-improvements-1) Performance Improvements * **get-dependencies:** replace slow object destructuring with single object ([#1612](https://github.com/taskforcesh/bullmq/issues/1612) ) ([621748e](https://github.com/taskforcesh/bullmq/commit/621748ec7727b46ce57eb9d2b46ef981874cdf4c) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.1.10-2022-12-29) [5.1.10](https://github.com/taskforcesh/bullmq-pro/compare/v5.1.9...v5.1.10) (2022-12-29) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#bug-fixes-14) Bug Fixes * **stalled:** add activeKey local reference ([#131](https://github.com/taskforcesh/bullmq-pro/issues/131) ) ([6554ea4](https://github.com/taskforcesh/bullmq-pro/commit/6554ea4d155e905312dd3398189b611bd54942e0) ), closes [taskforcesh/bullmq-pro-support#34](https://github.com/taskforcesh/bullmq-pro-support/issues/34) * **connection:** throw exception if using keyPrefix in ioredis ([eb6a130](https://github.com/taskforcesh/bullmq/commit/eb6a1305541547725e1717eefe2b678bc445f4d0) ) * **connection:** use includes to check for upstash more reliably ([12efb5c](https://github.com/taskforcesh/bullmq/commit/12efb5c539cb6f031ea6f3a80e4128d2e556e627) ) * **job:** fetch parent before job moves to complete ([#1580](https://github.com/taskforcesh/bullmq/issues/1580) ) ([6a6c0dc](https://github.com/taskforcesh/bullmq/commit/6a6c0dca30bb0a2417e0c62d4c80202c750322dd) ) * **sandbox:** throw error when no exported function ([#1588](https://github.com/taskforcesh/bullmq/issues/1588) ) fixes [#1587](https://github.com/taskforcesh/bullmq/issues/1587) ([c031891](https://github.com/taskforcesh/bullmq/commit/c03189184c8eeeb324f005b86e93d114abbe2154) ) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#features-7) Features * **queue:** add getJobState method ([#1593](https://github.com/taskforcesh/bullmq/issues/1593) ) ref [#1532](https://github.com/taskforcesh/bullmq/issues/1532) ([b741e84](https://github.com/taskforcesh/bullmq/commit/b741e8456f262b51aa7c68f571c76a3c54d02d37) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.1.9-2022-12-23) [5.1.9](https://github.com/taskforcesh/bullmq-pro/compare/v5.1.8...v5.1.9) (2022-12-23) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#bug-fixes-15) Bug Fixes * **job-pro:** fix opts type ([#129](https://github.com/taskforcesh/bullmq-pro/issues/129) ) ([262de56](https://github.com/taskforcesh/bullmq-pro/commit/262de56bcb33f107d88fc765215bb809adc502a1) ), closes [taskforcesh/issues#114](https://github.com/taskforcesh/issues/issues/114) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.1.8-2022-12-22) [5.1.8](https://github.com/taskforcesh/bullmq-pro/compare/v5.1.7...v5.1.8) (2022-12-22) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#bug-fixes-16) Bug Fixes * **worker:** avoid calling run on base class ([aba70f3](https://github.com/taskforcesh/bullmq-pro/commit/aba70f3df50f97221b1b998a416eb8e74ee66465) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.1.7-2022-12-16) [5.1.7](https://github.com/taskforcesh/bullmq-pro/compare/v5.1.6...v5.1.7) (2022-12-16) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#performance-improvements-2) Performance Improvements * **counts:** delete delayed marker when needed ([#1583](https://github.com/taskforcesh/bullmq/issues/1583) ) ([cc26f1c](https://github.com/taskforcesh/bullmq/commit/cc26f1cd550de76c7588d3a98187b80ee78c40c4) ) * **get-children-values:** replace slow object destructuring with single object ([#1586](https://github.com/taskforcesh/bullmq/issues/1586) ) ([857d403](https://github.com/taskforcesh/bullmq/commit/857d40377a6eb2c0101e6d16d9085ecd4b52b016) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.1.6-2022-12-15) [5.1.6](https://github.com/taskforcesh/bullmq-pro/compare/v5.1.5...v5.1.6) (2022-12-15) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#bug-fixes-17) Bug Fixes * **remove-job:** check groupId is different than false on removed children ([#126](https://github.com/taskforcesh/bullmq-pro/issues/126) ) ([efb54cb](https://github.com/taskforcesh/bullmq-pro/commit/efb54cbbd9486a608beace7f975247f5c6995470) ), closes [taskforcesh/bullmq-pro-support#32](https://github.com/taskforcesh/bullmq-pro-support/issues/32) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.1.5-2022-12-13) [5.1.5](https://github.com/taskforcesh/bullmq-pro/compare/v5.1.4...v5.1.5) (2022-12-13) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#bug-fixes-18) Bug Fixes * **exponential:** respect exponential backoff delay ([#1581](https://github.com/taskforcesh/bullmq/issues/1581) ) ([145dd32](https://github.com/taskforcesh/bullmq/commit/145dd329bb9f8254b404f4c5fbf7a50359202d37) ) * **get-jobs:** filter marker ([#1551](https://github.com/taskforcesh/bullmq/issues/1551) ) ([4add0ef](https://github.com/taskforcesh/bullmq/commit/4add0efa7857cc2f7b6d3c0c78a7f82cb7a46933) ) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#features-8) Features * **worker:** add ready event for blockingConnection ([#1577](https://github.com/taskforcesh/bullmq/issues/1577) ) ([992cc9e](https://github.com/taskforcesh/bullmq/commit/992cc9e9b3046185d3b67f2cc956f30337f458e1) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.1.4-2022-12-08) [5.1.4](https://github.com/taskforcesh/bullmq-pro/compare/v5.1.3...v5.1.4) (2022-12-08) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#bug-fixes-19) Bug Fixes * **rate-limit-group:** several small fixes related to manual group rate limit. ([5b338d6](https://github.com/taskforcesh/bullmq-pro/commit/5b338d6b68af6762ae1c12367cff010596d8a15e) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.1.3-2022-12-08) [5.1.3](https://github.com/taskforcesh/bullmq-pro/compare/v5.1.2...v5.1.3) (2022-12-08) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#bug-fixes-20) Bug Fixes * **worker:** try catch setname call ([#1576](https://github.com/taskforcesh/bullmq/issues/1576) ) fixes [#1574](https://github.com/taskforcesh/bullmq/issues/1574) ([0c42fd8](https://github.com/taskforcesh/bullmq/commit/0c42fd8c07dbac7ace81e97e45440af93fc622a5) ) * do not allow move from active to wait if not owner of the job ([dc1a307](https://github.com/taskforcesh/bullmq/commit/dc1a3077d1521c5dc99824a7fc05d17da03906bc) ) * floor pexpire to integer ([1d5de42](https://github.com/taskforcesh/bullmq/commit/1d5de425a19ebf879a8f9a7e0543d87a4d358be1) ) * **get-workers:** set name when ready event in connection ([#1564](https://github.com/taskforcesh/bullmq/issues/1564) ) ([de93c17](https://github.com/taskforcesh/bullmq/commit/de93c172901650e1666c48423a39076f2c7b9c7b) ) * **job:** console warn custom job ids when they represent integers ([#1569](https://github.com/taskforcesh/bullmq/issues/1569) ) ([6e677d2](https://github.com/taskforcesh/bullmq/commit/6e677d2800957b368bef4247b8e4328c5758f262) ) * **add-job:** throw error when jobId represents an integer ([#1556](https://github.com/taskforcesh/bullmq/issues/1556) ) ([db617d7](https://github.com/taskforcesh/bullmq/commit/db617d79e8f55b5c9e0df4b6bfd4247612016da1) ) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#features-9) Features * **queue-events:** support duplicated event ([#1549](https://github.com/taskforcesh/bullmq/issues/1549) ) ([18bc4eb](https://github.com/taskforcesh/bullmq/commit/18bc4eb50432f8aa27f2395750a7617317b66ca1) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.1.2-2022-12-07) [5.1.2](https://github.com/taskforcesh/bullmq-pro/compare/v5.1.1...v5.1.2) (2022-12-07) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#bug-fixes-21) Bug Fixes * **add-job:** do not update job that already exist ([#1550](https://github.com/taskforcesh/bullmq/issues/1550) ) ([26f6311](https://github.com/taskforcesh/bullmq/commit/26f6311cd0d2b936e404d0abebca9637f314a209) ) * **rate-limit:** delete rateLimiterKey when 0 ([#1553](https://github.com/taskforcesh/bullmq/issues/1553) ) ([0b88e5b](https://github.com/taskforcesh/bullmq/commit/0b88e5b94b4a0dc0d4000f7fd4b327f402248ad2) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.1.1-2022-12-05) [5.1.1](https://github.com/taskforcesh/bullmq-pro/compare/v5.1.0...v5.1.1) (2022-12-05) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#bug-fixes-22) Bug Fixes * **remove-job:** check for nil as groupId instead of empty string ([#119](https://github.com/taskforcesh/bullmq-pro/issues/119) ) ([dd63c23](https://github.com/taskforcesh/bullmq-pro/commit/dd63c238fdda33313cc06cfe3c69f1c0243fd9f3) ) [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.1.0-2022-11-29) [5.1.0](https://github.com/taskforcesh/bullmq-pro/compare/v5.0.3...v5.1.0) (2022-11-29) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#features-10) Features * add support for manually rate-limit groups ([64006ee](https://github.com/taskforcesh/bullmq-pro/commit/64006ee49f5f3a83816bfcb1e16488ffa9460273) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.0.3-2022-11-26) [5.0.3](https://github.com/taskforcesh/bullmq-pro/compare/v5.0.2...v5.0.3) (2022-11-26) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#bug-fixes-23) Bug Fixes * **global-rate-limit:** move job into group list ([#116](https://github.com/taskforcesh/bullmq-pro/issues/116) ) ([75384c4](https://github.com/taskforcesh/bullmq-pro/commit/75384c494e78eacb2f183d1a6504c0918561272b) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.0.2-2022-11-25) [5.0.2](https://github.com/taskforcesh/bullmq-pro/compare/v5.0.1...v5.0.2) (2022-11-25) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#performance-improvements-3) Performance Improvements * **groups:** check rate-limit when moving job to active ([#117](https://github.com/taskforcesh/bullmq-pro/issues/117) ) ([d247983](https://github.com/taskforcesh/bullmq-pro/commit/d247983f1ceba109511669944e5cf3be756815d2) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.0.1-2022-11-23) [5.0.1](https://github.com/taskforcesh/bullmq-pro/compare/v5.0.0...v5.0.1) (2022-11-23) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#bug-fixes-24) Bug Fixes * **ttl:** throw error when it's not provided as positive number ([#115](https://github.com/taskforcesh/bullmq-pro/issues/115) ) ([2d8ef2a](https://github.com/taskforcesh/bullmq-pro/commit/2d8ef2a9e8116e272cdf423de08a19775667d75a) ) [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#id-5.0.0-2022-11-22) [5.0.0](https://github.com/taskforcesh/bullmq-pro/compare/v4.0.3...v5.0.0) (2022-11-22) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#bug-fixes-25) Bug Fixes * upgrade bullmq to 3.2.2 ([#111](https://github.com/taskforcesh/bullmq-pro/issues/111) ) ([cac9167](https://github.com/taskforcesh/bullmq-pro/commit/cac91672be7962d2fa3234870d811cd3e690b7b5) ) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5#breaking-changes) BREAKING CHANGES * Change global rate limit Move jobs to wait or groups when global rate limit [Previousv6](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6) [Nextv4](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v4) Last updated 3 months ago Was this helpful? --- # v6 | BullMQ [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.11.0-2024-02-26) [6.11.0](https://github.com/taskforcesh/bullmq-pro/compare/v6.10.0...v6.11.0) (2024-02-26) ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#features) Features * expose sandboxed-job-pro interface ([6652e0a](https://github.com/taskforcesh/bullmq-pro/commit/6652e0afbdd45664c1e9436f9da0161adfea8c0d) ) [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.10.0-2024-02-21) [6.10.0](https://github.com/taskforcesh/bullmq-pro/compare/v6.9.6...v6.10.0) (2024-02-21) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#features-1) Features * **groups:** add sandbox support for groups ([53be7a0](https://github.com/taskforcesh/bullmq-pro/commit/53be7a095fca70df7bf8e52dfec45dde8dac064a) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.9.6-2024-01-31) [6.9.6](https://github.com/taskforcesh/bullmq-pro/compare/v6.9.5...v6.9.6) (2024-01-31) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#bug-fixes) Bug Fixes * **groups:** remove group when removing last job ([#199](https://github.com/taskforcesh/bullmq-pro/issues/199) ) ([3066686](https://github.com/taskforcesh/bullmq-pro/commit/3066686df4851334efadd7024cc8566407eabd7f) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.9.5-2024-01-27) [6.9.5](https://github.com/taskforcesh/bullmq-pro/compare/v6.9.4...v6.9.5) (2024-01-27) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#bug-fixes-1) Bug Fixes * **batches:** differentiate movetoBatchFinished responses ([#198](https://github.com/taskforcesh/bullmq-pro/issues/198) ) ([bb74c50](https://github.com/taskforcesh/bullmq-pro/commit/bb74c501f19fabbb61c4cb637598591f508bd59d) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.9.4-2024-01-20) [6.9.4](https://github.com/taskforcesh/bullmq-pro/compare/v6.9.3...v6.9.4) (2024-01-20) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#bug-fixes-2) Bug Fixes * **backoff:** set marker after adding delayed job ([#197](https://github.com/taskforcesh/bullmq-pro/issues/197) ) ([50a012e](https://github.com/taskforcesh/bullmq-pro/commit/50a012e352b9608a2a7f36db0cd7643078e183ee) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.9.3-2024-01-18) [6.9.3](https://github.com/taskforcesh/bullmq-pro/compare/v6.9.2...v6.9.3) (2024-01-18) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#performance-improvements) Performance Improvements * **prioritized:** get target list once in addPrioritizedJob ([#195](https://github.com/taskforcesh/bullmq-pro/issues/195) ) ([51cf4a3](https://github.com/taskforcesh/bullmq-pro/commit/51cf4a34d645013a49c01b740cf280666ebc4c97) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.9.2-2024-01-17) [6.9.2](https://github.com/taskforcesh/bullmq-pro/compare/v6.9.1...v6.9.2) (2024-01-17) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#bug-fixes-3) Bug Fixes * **groups:** consider prioritized groups when maxSize is provided ([#194](https://github.com/taskforcesh/bullmq-pro/issues/194) ) ([1c345c5](https://github.com/taskforcesh/bullmq-pro/commit/1c345c550685f5fdb2d2ff95056261af324d2ca5) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.9.1-2024-01-15) [6.9.1](https://github.com/taskforcesh/bullmq-pro/compare/v6.9.0...v6.9.1) (2024-01-15) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#bug-fixes-4) Bug Fixes * **events:** emit drained event only when finishing jobs ([#192](https://github.com/taskforcesh/bullmq-pro/issues/192) ) ([22a503d](https://github.com/taskforcesh/bullmq-pro/commit/22a503d4f462cea68f03015af93e0701ed27e2d6) ) [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.9.0-2023-12-28) [6.9.0](https://github.com/taskforcesh/bullmq-pro/compare/v6.8.0...v6.9.0) (2023-12-28) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#bug-fixes-5) Bug Fixes * **flows:** update constructor and methods to match queue base ([#2324](https://github.com/taskforcesh/bullmq/issues/2324) ) ([d6c2064](https://github.com/taskforcesh/bullmq/commit/d6c2064b1fdd88bd4cc61e049ce055ff620b0062) ) * **sandboxed:** better compatibility with esbuild ([8eaf955](https://github.com/taskforcesh/bullmq/commit/8eaf9550fe8b322df624893c507c55d2cce34b11) ) * **child-processor:** preserve dynamic imports in commonjs ([d97a5e0](https://github.com/taskforcesh/bullmq/commit/d97a5e06816cff04d86facdb8d32b512f29c6fb9) ) * **flows:** add meta key to queues created with flows ([272ec69](https://github.com/taskforcesh/bullmq/commit/272ec69557f601a138e1aaba739f7e7878d5344b) ) * **repeat-strategy:** add missing Promise return type ([#2301](https://github.com/taskforcesh/bullmq/issues/2301) ) ([6f8f534](https://github.com/taskforcesh/bullmq/commit/6f8f5342cc8aa03f596d9ed5b8831f96a1d4c736) ) * **update-progress:** remove old updateProgress script to prevent conflict ([#2298](https://github.com/taskforcesh/bullmq/issues/2298) ) (python) ([e65b819](https://github.com/taskforcesh/bullmq/commit/e65b819101f8e0e8fdef8c51cfdf9a52f5e73f13) ) * **worker:** get dirname by using module.filename ([#2296](https://github.com/taskforcesh/bullmq/issues/2296) ) fixes [#2288](https://github.com/taskforcesh/bullmq/issues/2288) ([6e4db5a](https://github.com/taskforcesh/bullmq/commit/6e4db5a3f3648c6a7e10991f2e18f3dab96fb1d7) ) * **worker:** should cap update progress events ([2cab9e9](https://github.com/taskforcesh/bullmq/commit/2cab9e94f65c7bdd053e3fb5944bcda6e3ebaa39) ) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#features-2) Features * **sandbox:** support URL (local files) as processor file ([7eea670](https://github.com/taskforcesh/bullmq/commit/7eea6700b33bfd7f36b030b647b819a4c5fd9606) ) * **queue:** add a paginated getDependencies ([#2327](https://github.com/taskforcesh/bullmq/issues/2327) ) ([c5b8ba3](https://github.com/taskforcesh/bullmq/commit/c5b8ba318b12a84a3a6a928345377fa0eaa08ee3) ) * **sandboxes:** use the more compatible dynamic import instead of require ([6d2fe6e](https://github.com/taskforcesh/bullmq/commit/6d2fe6e7c0473b75aeb9a6d3080b0676f9521065) ) [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.8.0-2023-11-30) [6.8.0](https://github.com/taskforcesh/bullmq-pro/compare/v6.7.0...v6.8.0) (2023-11-30) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#bug-fixes-6) Bug Fixes * **worker:** do not wait for slow jobs fixes [#2290](https://github.com/taskforcesh/bullmq/issues/2290) ([568d758](https://github.com/taskforcesh/bullmq/commit/568d7585edb1f2ef15991d4ae4a2425e6834046a) ) * **utils:** use EventEmitter as a type instead of a namespace ([#2283](https://github.com/taskforcesh/bullmq/issues/2283) ) ([41c9d1d](https://github.com/taskforcesh/bullmq/commit/41c9d1d05eedc7351272708e667e8d65eb6773fc) ) * **job:** set delay value on current job instance when it is retried ([#2266](https://github.com/taskforcesh/bullmq/issues/2266) ) (python) ([76e075f](https://github.com/taskforcesh/bullmq/commit/76e075f54d5745b6cec3cb11305bf3110d963eae) ) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#features-3) Features * **worker:** better handling of concurrency when fetching jobs ([#2242](https://github.com/taskforcesh/bullmq/issues/2242) ) ([d2e2035](https://github.com/taskforcesh/bullmq/commit/d2e203588878ee64cb21e67141f73b32867dfb40) ) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#performance-improvements-1) Performance Improvements * **rate-limit:** continue promoting groups when there are in paused state ([#187](https://github.com/taskforcesh/bullmq-pro/issues/187) ) ([17f9e81](https://github.com/taskforcesh/bullmq-pro/commit/17f9e81ad330eb273c3863f448fd3eee546d420f) ) [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.7.0-2023-11-22) [6.7.0](https://github.com/taskforcesh/bullmq-pro/compare/v6.6.2...v6.7.0) (2023-11-22) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#features-4) Features * **queue:** improve clean to work iteratively ([#2260](https://github.com/taskforcesh/bullmq/issues/2260) ) ([0cfa66f](https://github.com/taskforcesh/bullmq/commit/0cfa66fd0fa0dba9b3941f183cf6f06d8a4f281d) ) ref ([#186](https://github.com/taskforcesh/bullmq-pro/issues/186) ) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#bug-fixes-7) Bug Fixes * **utils:** use EventEmitter as a type instead of a namespace ([#2283](https://github.com/taskforcesh/bullmq/issues/2283) ) ([41c9d1d](https://github.com/taskforcesh/bullmq/commit/41c9d1d05eedc7351272708e667e8d65eb6773fc) ) * **job:** set delay value on current job instance when it is retried ([#2266](https://github.com/taskforcesh/bullmq/issues/2266) ) (python) ([76e075f](https://github.com/taskforcesh/bullmq/commit/76e075f54d5745b6cec3cb11305bf3110d963eae) ) * **connection:** better handling of attached listeners ([02474ad](https://github.com/taskforcesh/bullmq/commit/02474ad59a7b340d7bb2a7415ae7a88e14200398) ) * **connection:** move redis instance check to queue base ([13a339a](https://github.com/taskforcesh/bullmq/commit/13a339a730f46ff22acdd4a046e0d9c4b7d88679) ) * update delay job property when moving to delayed set ([#2261](https://github.com/taskforcesh/bullmq/issues/2261) ) ([69ece08](https://github.com/taskforcesh/bullmq/commit/69ece08babd7716c14c38c3dd50630b44c7c1897) ) * **add-job:** trim events when waiting-children event is published ([#2262](https://github.com/taskforcesh/bullmq/issues/2262) ) (python) ([198bf05](https://github.com/taskforcesh/bullmq/commit/198bf05fa5a4e1ce50081296033a2e0f26ece498) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.6.2-2023-11-03) [6.6.2](https://github.com/taskforcesh/bullmq-pro/compare/v6.6.1...v6.6.2) (2023-11-03) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#bug-fixes-8) Bug Fixes * **worker:** keep extending locks while closing workers ([#2259](https://github.com/taskforcesh/bullmq/issues/2259) ) ([c4d12ea](https://github.com/taskforcesh/bullmq/commit/c4d12ea3a9837ffd7f58e2134796137c4181c3de) ) * **sandbox:** do not return empty object result when it is undefined ([#2247](https://github.com/taskforcesh/bullmq/issues/2247) ) ([308db7f](https://github.com/taskforcesh/bullmq/commit/308db7f58758a72b8abb272da8e92509813a2178) ) * **events:** do not publish removed event on non-existent jobs ([#2227](https://github.com/taskforcesh/bullmq/issues/2227) ) ([c134606](https://github.com/taskforcesh/bullmq/commit/c1346064c6cd9f93c59b184f150eac11d51c91b4) ) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#performance-improvements-2) Performance Improvements * **redis-connection:** check redis version greater or equal than v6 only once ([#2252](https://github.com/taskforcesh/bullmq/issues/2252) ) ([a09b15a](https://github.com/taskforcesh/bullmq/commit/a09b15af0d5dedfa83bce7130ee9094f3fb69e10) ) * **events:** trim events when removing jobs ([#2235](https://github.com/taskforcesh/bullmq/issues/2235) ) (python) ([889815c](https://github.com/taskforcesh/bullmq/commit/889815c412666e5fad8f32d2e3a2d41cf650f001) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.6.1-2023-10-11) [6.6.1](https://github.com/taskforcesh/bullmq-pro/compare/v6.6.0...v6.6.1) (2023-10-11) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#bug-fixes-9) Bug Fixes * **events:** trim events when retrying a job ([#2224](https://github.com/taskforcesh/bullmq/issues/2224) ) ([1986b05](https://github.com/taskforcesh/bullmq/commit/1986b05ac03fe4ee48861aa60caadcc9df8170a6) ) * **sandbox:** update progress value on job instance ([#2214](https://github.com/taskforcesh/bullmq/issues/2214) ) fixes [#2213](https://github.com/taskforcesh/bullmq/issues/2213) ([3d0f36a](https://github.com/taskforcesh/bullmq/commit/3d0f36a134b7f5c6b6de26967c9d71bcfb346e72) ) [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.6.0-2023-10-06) [6.6.0](https://github.com/taskforcesh/bullmq-pro/compare/v6.5.0...v6.6.0) (2023-10-06) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#bug-fixes-10) Bug Fixes * **delayed:** trim events when moving jobs to delayed (python) ([#2211](https://github.com/taskforcesh/bullmq/issues/2211) ) ([eca8c2d](https://github.com/taskforcesh/bullmq/commit/eca8c2d4dfeafbd8ac36a49764dbd4897303628c) ) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#features-5) Features * **queue:** expose addJobLog and updateJobProgress ([#2202](https://github.com/taskforcesh/bullmq/issues/2202) ) ([2056939](https://github.com/taskforcesh/bullmq/commit/205693907a4d6c2da9bd0690fb552b1d1e369c08) ) [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.5.0-2023-09-28) [6.5.0](https://github.com/taskforcesh/bullmq-pro/compare/v6.4.0...v6.5.0) (2023-09-28) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#features-6) Features * **sandbox:** convert wrapJob method as protected for extension ([#2182](https://github.com/taskforcesh/bullmq/issues/2182) ) ([1494b55](https://github.com/taskforcesh/bullmq/commit/1494b5566573356e0248b4a5cab48ae21d82f1da) ) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#bug-fixes-11) Bug Fixes * **queue:** batched unpack now uses range ([#2188](https://github.com/taskforcesh/bullmq/issues/2188) ) ([b5e97f4](https://github.com/taskforcesh/bullmq/commit/b5e97f420bc0c4bc82772f3e87883ee522be43d9) ) * **worker:** forward skipVersionCheck to blockingConnection ([#2189](https://github.com/taskforcesh/bullmq/issues/2189) ) ref [#2149](https://github.com/taskforcesh/bullmq/issues/2149) ([c8aa9a3](https://github.com/taskforcesh/bullmq/commit/c8aa9a36224cba8ecb19af1bf652f4f1c4c20d40) ) * **worker:** throw exception with NaN as concurrency ([#2184](https://github.com/taskforcesh/bullmq/issues/2184) ) ([f36ac8b](https://github.com/taskforcesh/bullmq/commit/f36ac8b61dcd4bb3d9e283278310cd50cfc83fae) ) * **queue:** differentiate score purpose per state in clean method ([#2133](https://github.com/taskforcesh/bullmq/issues/2133) ) fixes [#2124](https://github.com/taskforcesh/bullmq/issues/2124) ([862f10b](https://github.com/taskforcesh/bullmq/commit/862f10b586276314d9bffff2a5e6caf939399f7e) ) [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.4.0-2023-09-16) [6.4.0](https://github.com/taskforcesh/bullmq-pro/compare/v6.3.4...v6.4.0) (2023-09-16) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#features-7) Features * **connection:** provide skipVersionCheck option for shared connections ([#2149](https://github.com/taskforcesh/bullmq/issues/2149) ) ref [#2148](https://github.com/taskforcesh/bullmq/issues/2148) ([914820f](https://github.com/taskforcesh/bullmq/commit/914820f720cbc48b49f4bd1c46d148eb2bb5b79c) ) * **sandbox:** emulate moveToDelayed method ([#180](https://github.com/taskforcesh/bullmq-pro/issues/180) ) ([d61de09](https://github.com/taskforcesh/bullmq-pro/commit/d61de095115481b688101bfaf0b126a02545cc6f) ) ref [#2118](https://github.com/taskforcesh/bullmq/issues/2118) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#bug-fixes-12) Bug Fixes * **remove:** change error message when job is locked (python) ([#2175](https://github.com/taskforcesh/bullmq/issues/2175) ) ([2f5628f](https://github.com/taskforcesh/bullmq/commit/2f5628feffab66cdcc78abf4d7bb608bdcaa65bb) ) * **sandbox:** ignore extra params on processor ([#2142](https://github.com/taskforcesh/bullmq/issues/2142) ) ([3602c20](https://github.com/taskforcesh/bullmq/commit/3602c20ab80cbe0a0d3de66210a01ad119e1090b) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.3.4-2023-08-18) [6.3.4](https://github.com/taskforcesh/bullmq-pro/compare/v6.3.3...v6.3.4) (2023-08-18) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#bug-fixes-13) Bug Fixes * **worker:** abort rate-limit delay when closing worker ([#179](https://github.com/taskforcesh/bullmq-pro/issues/179) ) ([4ad650b](https://github.com/taskforcesh/bullmq-pro/commit/4ad650b7bc0c6a950536df252e510cb96e2e0054) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.3.3-2023-08-15) [6.3.3](https://github.com/taskforcesh/bullmq-pro/compare/v6.3.2...v6.3.3) (2023-08-15) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#bug-fixes-14) Bug Fixes * **queue:** throw error when name is not provided ([#178](https://github.com/taskforcesh/bullmq-pro/issues/178) ) ([9715bf1](https://github.com/taskforcesh/bullmq-pro/commit/9715bf15edef1f54a9ebc618eb7d47c7b45a35ca) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.3.2-2023-08-11) [6.3.2](https://github.com/taskforcesh/bullmq-pro/compare/v6.3.1...v6.3.2) (2023-08-11) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#bug-fixes-15) Bug Fixes * correct group rate limit in some edge cases ([#177](https://github.com/taskforcesh/bullmq-pro/issues/177) ) ([c3c87a7](https://github.com/taskforcesh/bullmq-pro/commit/c3c87a7f0a6de5c35ac389efbac594d6d987cf49) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.3.1-2023-08-10) [6.3.1](https://github.com/taskforcesh/bullmq-pro/compare/v6.3.0...v6.3.1) (2023-08-10) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#performance-improvements-3) Performance Improvements * **rate-limit:** get pttl only if needed ([#175](https://github.com/taskforcesh/bullmq-pro/issues/175) ) ([0439823](https://github.com/taskforcesh/bullmq-pro/commit/0439823c32a82e48abcae43c29e50ef912c31d15) ) [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.3.0-2023-08-03) [6.3.0](https://github.com/taskforcesh/bullmq-pro/compare/v6.2.4...v6.3.0) (2023-08-03) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#features-8) Features * **queue:** add getRateLimitTtl method ([#173](https://github.com/taskforcesh/bullmq-pro/issues/173) ) ([3327350](https://github.com/taskforcesh/bullmq-pro/commit/3327350d06526651353974aa7822fdeeec881fb0) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.2.4-2023-07-29) [6.2.4](https://github.com/taskforcesh/bullmq-pro/compare/v6.2.3...v6.2.4) (2023-07-29) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#bug-fixes-16) Bug Fixes * **group:** add priority option into group option ([#171](https://github.com/taskforcesh/bullmq-pro/issues/171) ) ([2e632f1](https://github.com/taskforcesh/bullmq-pro/commit/2e632f11013a0db520f7a642741df21bbdd38a78) ), closes [taskforcesh/bullmq-pro-support#23](https://github.com/taskforcesh/bullmq-pro-support/issues/23) [taskforcesh/bullmq-pro-support#13](https://github.com/taskforcesh/bullmq-pro-support/issues/13) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.2.3-2023-07-27) [6.2.3](https://github.com/taskforcesh/bullmq-pro/compare/v6.2.2...v6.2.3) (2023-07-27) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#performance-improvements-4) Performance Improvements * **groups:** do not move job to paused when promoting rate-limited group ([#169](https://github.com/taskforcesh/bullmq-pro/issues/169) ) ([fa2bb3c](https://github.com/taskforcesh/bullmq-pro/commit/fa2bb3c98761615822035312dfdb782934d9a774) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.2.2-2023-07-26) [6.2.2](https://github.com/taskforcesh/bullmq-pro/compare/v6.2.1...v6.2.2) (2023-07-26) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#features-9) Features * **queue:** add promoteJobs to promote all delayed jobs ([6074592](https://github.com/taskforcesh/bullmq/commit/6074592574256ec4b1c340126288e803e56b1a64) ) * **job:** add option for removing children in remove method (python) ([#2064](https://github.com/taskforcesh/bullmq/issues/2064) ) ([841dc87](https://github.com/taskforcesh/bullmq/commit/841dc87a689897df81438ad1f43e45a4da77c388) ) * **job:** add removeDependencyOnFailure option ([#1953](https://github.com/taskforcesh/bullmq/issues/1953) ) ([ffd49e2](https://github.com/taskforcesh/bullmq/commit/ffd49e289c57252487200d47b92193228ae7451f) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.2.1-2023-07-25) [6.2.1](https://github.com/taskforcesh/bullmq-pro/compare/v6.2.0...v6.2.1) (2023-07-25) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#bug-fixes-17) Bug Fixes * **flow:** emit delayed event when parent is moved to delayed ([#166](https://github.com/taskforcesh/bullmq-pro/issues/166) ) ([38afe1c](https://github.com/taskforcesh/bullmq-pro/commit/38afe1cf3187ddee33d33a39438587d921b8743a) ) [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.2.0-2023-07-25) [6.2.0](https://github.com/taskforcesh/bullmq-pro/compare/v6.1.1...v6.2.0) (2023-07-25) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#features-10) Features * **groups:** support local priorities ([#156](https://github.com/taskforcesh/bullmq-pro/issues/156) ) ([260bd24](https://github.com/taskforcesh/bullmq-pro/commit/260bd24a76d703ec87385c05cdd3b4589f142aa8) ), closes [taskforcesh/bullmq-pro-support#23](https://github.com/taskforcesh/bullmq-pro-support/issues/23) [taskforcesh/bullmq-pro-support#13](https://github.com/taskforcesh/bullmq-pro-support/issues/13) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.1.1-2023-07-18) [6.1.1](https://github.com/taskforcesh/bullmq-pro/compare/v6.1.0...v6.1.1) (2023-07-18) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#bug-fixes-18) Bug Fixes * add missing error export ([2b8c51f](https://github.com/taskforcesh/bullmq-pro/commit/2b8c51fd43c538fd3eed122ab96d55325da97b90) ) [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.1.0-2023-07-18) [6.1.0](https://github.com/taskforcesh/bullmq-pro/compare/v6.0.5...v6.1.0) (2023-07-18) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#bug-fixes-19) Bug Fixes * fix the GroupMaxSizeExceededError prototype ([a1b6a96](https://github.com/taskforcesh/bullmq-pro/commit/a1b6a96f1a0b632940177a1057a4995d59957964) ) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#features-11) Features * add getGroupStatus ([3bac19d](https://github.com/taskforcesh/bullmq-pro/commit/3bac19d32e9a620a393736152844973951e45d47) ) * add support max sized groups ([7bc654c](https://github.com/taskforcesh/bullmq-pro/commit/7bc654c2191c9a2e8c80f4e0843c4beda7d61565) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.0.5-2023-07-11) [6.0.5](https://github.com/taskforcesh/bullmq-pro/compare/v6.0.4...v6.0.5) (2023-07-11) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#bug-fixes-20) Bug Fixes * **pause-group:** do not move job to wait when queue is paused ([#162](https://github.com/taskforcesh/bullmq-pro/issues/162) ) ([458b381](https://github.com/taskforcesh/bullmq-pro/commit/458b3813eef982dc661a019349776d44d6ddb194) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.0.4-2023-07-07) [6.0.4](https://github.com/taskforcesh/bullmq-pro/compare/v6.0.3...v6.0.4) (2023-07-07) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#bug-fixes-21) Bug Fixes * **group:** move job into group list when paused and dynamic rate limit ([#161](https://github.com/taskforcesh/bullmq-pro/issues/161) ) ([1625f36](https://github.com/taskforcesh/bullmq-pro/commit/1625f36b3014ac191828d8ce070f237c19494c67) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.0.3-2023-07-05) [6.0.3](https://github.com/taskforcesh/bullmq-pro/compare/v6.0.2...v6.0.3) (2023-07-05) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#bug-fixes-22) Bug Fixes * **rate-limit:** emit waiting event in rateLimitGroup ([#160](https://github.com/taskforcesh/bullmq-pro/issues/160) ) ([eaf3cd7](https://github.com/taskforcesh/bullmq-pro/commit/eaf3cd74e3bcd40e6ba46bb2f540cae9cb945962) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.0.2-2023-07-04) [6.0.2](https://github.com/taskforcesh/bullmq-pro/compare/v6.0.1...v6.0.2) (2023-07-04) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#performance-improvements-5) Performance Improvements * **remove-job:** do not remove last group id ([#159](https://github.com/taskforcesh/bullmq-pro/issues/159) ) ([f5a3cd5](https://github.com/taskforcesh/bullmq-pro/commit/f5a3cd50d78bcadfd09ca9c5de1054f9620c191b) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.0.1-2023-06-29) [6.0.1](https://github.com/taskforcesh/bullmq-pro/compare/v6.0.0...v6.0.1) (2023-06-29) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#bug-fixes-23) Bug Fixes * **job:** save groupId even when the job is a parent ([#157](https://github.com/taskforcesh/bullmq-pro/issues/157) ) ([1debbf4](https://github.com/taskforcesh/bullmq-pro/commit/1debbf40ca4aa8f8b5ab45c36e8732f7ffe5442b) ) [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#id-6.0.0-2023-06-26) [6.0.0](https://github.com/taskforcesh/bullmq-pro/compare/v5.3.5...v6.0.0) (2023-06-26) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#performance-improvements-6) Performance Improvements * **priority:** add prioritized as a new state ([#155](https://github.com/taskforcesh/bullmq-pro/issues/155) ) ([b2391ca](https://github.com/taskforcesh/bullmq-pro/commit/b2391cab4d63e97f807eaed3a6e814be01de0f32) ) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v6#breaking-changes) BREAKING CHANGES * **priority:** priority is separated in its own zset, no duplication needed * change job method name update to updateData ref [faster priority jobs](https://bullmq.io/news/062123/faster-priority-jobs/) [PreviousChangelogs](https://docs.bullmq.io/bullmq-pro/changelog) [Nextv5](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v5) Last updated 3 months ago Was this helpful? --- # v2 | BullMQ ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.7.1-2022-10-13) [2.7.1](https://github.com/taskforcesh/bullmq-pro/compare/v2.7.0...v2.7.1) (2022-10-13) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes) Bug Fixes * **delete-groups:** consider rate-limit, max concurrency and paused ([#104](https://github.com/taskforcesh/bullmq-pro/issues/104) ) ([29873f8](https://github.com/taskforcesh/bullmq-pro/commit/29873f8c900025f70cd88f8328fa8c6b3841bc7b) ) [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.7.0-2022-10-11) [2.7.0](https://github.com/taskforcesh/bullmq-pro/compare/v2.6.0...v2.7.0) (2022-10-11) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#features) Features * add getGroupStatus ([a7cd882](https://github.com/taskforcesh/bullmq-pro/commit/a7cd882f80b182612a19924823000cec15d2cf90) ) [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.6.0-2022-10-11) [2.6.0](https://github.com/taskforcesh/bullmq-pro/compare/v2.5.0...v2.6.0) (2022-10-11) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#features-1) Features * add version support ([b7e1831](https://github.com/taskforcesh/bullmq-pro/commit/b7e183116137d8774a12d09a4d97d29d1cdb2999) ) [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.5.0-2022-10-11) [2.5.0](https://github.com/taskforcesh/bullmq-pro/compare/v2.4.14...v2.5.0) (2022-10-11) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#features-2) Features * add getGroupsByStatus method to getters ([949e93b](https://github.com/taskforcesh/bullmq-pro/commit/949e93bc3478607f95ee59eab41a1ac7e271e74d) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.4.14-2022-10-07) [2.4.14](https://github.com/taskforcesh/bullmq-pro/compare/v2.4.13...v2.4.14) (2022-10-07) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-1) Bug Fixes * **delete-group:** consider max-concurrency state ([#98](https://github.com/taskforcesh/bullmq-pro/issues/98) ) ([d897dd9](https://github.com/taskforcesh/bullmq-pro/commit/d897dd9bef0f6844d9752bfb3c22f0be6368889b) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.4.13-2022-10-05) [2.4.13](https://github.com/taskforcesh/bullmq-pro/compare/v2.4.12...v2.4.13) (2022-10-05) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-2) Bug Fixes * **delete-group:** consider rate-limit state ([#97](https://github.com/taskforcesh/bullmq-pro/issues/97) ) ([85f7f32](https://github.com/taskforcesh/bullmq-pro/commit/85f7f32a0c2e893f7921c8eee9bc0655fdff7a39) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.4.12-2022-09-30) [2.4.12](https://github.com/taskforcesh/bullmq-pro/compare/v2.4.11...v2.4.12) (2022-09-30) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-3) Bug Fixes * **global-rate-limit:** consider groups ([#95](https://github.com/taskforcesh/bullmq-pro/issues/95) ) ([de95fde](https://github.com/taskforcesh/bullmq-pro/commit/de95fde1f07096f6d2dfff278b1d969a5b2a0c0f) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.4.11-2022-09-29) [2.4.11](https://github.com/taskforcesh/bullmq-pro/compare/v2.4.10...v2.4.11) (2022-09-29) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-4) Bug Fixes * **drain:** consider empty active list ([#1412](https://github.com/taskforcesh/bullmq/issues/1412) ) ([f919a50](https://github.com/taskforcesh/bullmq/commit/f919a50b2f4972dcb9ecd5848b0f7fd9a0e137ea) ) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#features-3) Features * **sandbox:** support update method ([#1416](https://github.com/taskforcesh/bullmq/issues/1416) ) ([606b75d](https://github.com/taskforcesh/bullmq/commit/606b75d53e12dfc109f01eda38736c07e829e9b7) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.4.10-2022-09-14) [2.4.10](https://github.com/taskforcesh/bullmq-pro/compare/v2.4.9...v2.4.10) (2022-09-14) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-5) Bug Fixes * **timeout:** delete unused option ([#94](https://github.com/taskforcesh/bullmq-pro/issues/94) ) ([4f8dc50](https://github.com/taskforcesh/bullmq-pro/commit/4f8dc5021c311fe10d20568c4dae4055d01ef98f) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.4.9-2022-09-13) [2.4.9](https://github.com/taskforcesh/bullmq-pro/compare/v2.4.8...v2.4.9) (2022-09-13) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#performance-improvements) Performance Improvements * **script-loader:** use cache to read script once ([#93](https://github.com/taskforcesh/bullmq-pro/issues/93) ) ([04bbeec](https://github.com/taskforcesh/bullmq-pro/commit/04bbeece1dfc8e06d8590eb486879593d4dae437) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.4.8-2022-09-09) [2.4.8](https://github.com/taskforcesh/bullmq-pro/compare/v2.4.7...v2.4.8) (2022-09-09) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-6) Bug Fixes * **concurrency:** consider base rate limit ([#90](https://github.com/taskforcesh/bullmq-pro/issues/90) ) ([74a4a0b](https://github.com/taskforcesh/bullmq-pro/commit/74a4a0ba01f3a447f9dc24f5bbb898bc6afaeaa6) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.4.7-2022-09-06) [2.4.7](https://github.com/taskforcesh/bullmq-pro/compare/v2.4.6...v2.4.7) (2022-09-06) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-7) Bug Fixes * **flow-producer-pro:** use interim class ([#92](https://github.com/taskforcesh/bullmq-pro/issues/92) ) ([2406cc3](https://github.com/taskforcesh/bullmq-pro/commit/2406cc3f1b4c78feed8a4fbd91422e3ca1970b19) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.4.6-2022-09-06) [2.4.6](https://github.com/taskforcesh/bullmq-pro/compare/v2.4.5...v2.4.6) (2022-09-06) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#performance-improvements-1) Performance Improvements * **add-job:** handle parent split on js ([#1397](https://github.com/taskforcesh/bullmq/issues/1397) ) ([566f074](https://github.com/taskforcesh/bullmq/commit/566f0747110679e5b07e7642fef793744565fffe) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.4.5-2022-08-30) [2.4.5](https://github.com/taskforcesh/bullmq-pro/compare/v2.4.4...v2.4.5) (2022-08-30) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-8) Bug Fixes * **delete-group:** consider children ([#88](https://github.com/taskforcesh/bullmq-pro/issues/88) ) ([83de2a9](https://github.com/taskforcesh/bullmq-pro/commit/83de2a9c9b42775996a8c8893caf66d1af6bea15) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.4.4-2022-08-30) [2.4.4](https://github.com/taskforcesh/bullmq-pro/compare/v2.4.3...v2.4.4) (2022-08-30) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-9) Bug Fixes * **deps:** upgrade bullmq to 1.90.0 ([#84](https://github.com/taskforcesh/bullmq-pro/issues/84) ) ([69a01c5](https://github.com/taskforcesh/bullmq-pro/commit/69a01c5d91c3e6ad2b1fb7a32ced8a04021d91ec) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.4.3-2022-08-26) [2.4.3](https://github.com/taskforcesh/bullmq-pro/compare/v2.4.2...v2.4.3) (2022-08-26) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-10) Bug Fixes * **waiting-children:** consider decreasing group concurrency ([#86](https://github.com/taskforcesh/bullmq-pro/issues/86) ) ([be430a7](https://github.com/taskforcesh/bullmq-pro/commit/be430a72f7bda55e22a0ae5e5623e8a2b835e98e) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.4.2-2022-08-25) [2.4.2](https://github.com/taskforcesh/bullmq-pro/compare/v2.4.1...v2.4.2) (2022-08-25) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-11) Bug Fixes * **deps:** upgrade bullmq to 1.89.1 ([#87](https://github.com/taskforcesh/bullmq-pro/issues/87) ) ([228aca3](https://github.com/taskforcesh/bullmq-pro/commit/228aca3e72ef9401fe3c67e5ca72be6b1068b6c6) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.4.1-2022-08-18) [2.4.1](https://github.com/taskforcesh/bullmq-pro/compare/v2.4.0...v2.4.1) (2022-08-18) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-12) Bug Fixes * **job:** remove from group ([#57](https://github.com/taskforcesh/bullmq-pro/issues/57) ) ([7c38aa1](https://github.com/taskforcesh/bullmq-pro/commit/7c38aa19ea9aba53689e14208892ab7f6547b699) ) [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.4.0-2022-08-16) [2.4.0](https://github.com/taskforcesh/bullmq-pro/compare/v2.3.13...v2.4.0) (2022-08-16) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#features-4) Features * **groups:** support flows ([#81](https://github.com/taskforcesh/bullmq-pro/issues/81) ) ([3db9478](https://github.com/taskforcesh/bullmq-pro/commit/3db947863093c7c7db83773876dd7593b5a33210) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.3.13-2022-08-13) [2.3.13](https://github.com/taskforcesh/bullmq-pro/compare/v2.3.12...v2.3.13) (2022-08-13) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-13) Bug Fixes * **deps:** upgrade bullmq to 1.87.2 ([#83](https://github.com/taskforcesh/bullmq-pro/issues/83) ) ([5b3c866](https://github.com/taskforcesh/bullmq-pro/commit/5b3c866016837bdafa93bc315d31d9eee2465ed5) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.3.12-2022-08-11) [2.3.12](https://github.com/taskforcesh/bullmq-pro/compare/v2.3.11...v2.3.12) (2022-08-11) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-14) Bug Fixes * **observables:** guarantee store result order ([f963557](https://github.com/taskforcesh/bullmq-pro/commit/f9635571ae359cdf6de9cd18463ef879c166a4f4) ) * **observables:** store last value as returnvalue ([7306ae2](https://github.com/taskforcesh/bullmq-pro/commit/7306ae233b5a2ecb96d402a30d7db61bb8c74567) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.3.11-2022-08-09) [2.3.11](https://github.com/taskforcesh/bullmq-pro/compare/v2.3.10...v2.3.11) (2022-08-09) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-15) Bug Fixes * **deps:** upgrade bullmq to 1.87.1 ([#79](https://github.com/taskforcesh/bullmq-pro/issues/79) ) ([3affc37](https://github.com/taskforcesh/bullmq-pro/commit/3affc37ab682f1d58c0dfa29d3db714c8e7f8c91) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.3.10-2022-08-03) [2.3.10](https://github.com/taskforcesh/bullmq-pro/compare/v2.3.9...v2.3.10) (2022-08-03) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#performance-improvements-2) Performance Improvements * **move-to-finished:** pass keepJobs into opts arg ([#78](https://github.com/taskforcesh/bullmq-pro/issues/78) ) ([08eb23f](https://github.com/taskforcesh/bullmq-pro/commit/08eb23fa54bfe1e46c1e79bfee9d72fb0dbba52b) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.3.9-2022-08-01) [2.3.9](https://github.com/taskforcesh/bullmq-pro/compare/v2.3.8...v2.3.9) (2022-08-01) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-16) Bug Fixes * **deps:** upgrade bullmq to 1.86.10 ([#76](https://github.com/taskforcesh/bullmq-pro/issues/76) ) ([d3df585](https://github.com/taskforcesh/bullmq-pro/commit/d3df5850fd92b6d98e77c6d7e7355f205f7df4c4) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.3.8-2022-08-01) [2.3.8](https://github.com/taskforcesh/bullmq-pro/compare/v2.3.7...v2.3.8) (2022-08-01) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-17) Bug Fixes * **move-to-active:** use local jobId instead of global reference ([#77](https://github.com/taskforcesh/bullmq-pro/issues/77) ) ([1f0b8dd](https://github.com/taskforcesh/bullmq-pro/commit/1f0b8dd747ce9ad9fdacdb7774cb1f34e989ceb5) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.3.7-2022-07-28) [2.3.7](https://github.com/taskforcesh/bullmq-pro/compare/v2.3.6...v2.3.7) (2022-07-28) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-18) Bug Fixes * **deps:** upgrade bullmq to 1.86.9 ([#73](https://github.com/taskforcesh/bullmq-pro/issues/73) ) ([bbc0784](https://github.com/taskforcesh/bullmq-pro/commit/bbc07845f6cce0cc003681255b892330c729b30e) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.3.6-2022-07-26) [2.3.6](https://github.com/taskforcesh/bullmq-pro/compare/v2.3.5...v2.3.6) (2022-07-26) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#performance-improvements-3) Performance Improvements * **retry-jobs:** add jobs in batches when groupId is present ([#72](https://github.com/taskforcesh/bullmq-pro/issues/72) ) ([3961da0](https://github.com/taskforcesh/bullmq-pro/commit/3961da022843048597033e8f13034f245198bca3) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.3.5-2022-07-20) [2.3.5](https://github.com/taskforcesh/bullmq-pro/compare/v2.3.4...v2.3.5) (2022-07-20) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-19) Bug Fixes * **retry-jobs:** consider groups ([#70](https://github.com/taskforcesh/bullmq-pro/issues/70) ) ([7b03017](https://github.com/taskforcesh/bullmq-pro/commit/7b030179d1a2de23aba2f9c5e71b5d13d6de67d3) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.3.4-2022-07-16) [2.3.4](https://github.com/taskforcesh/bullmq-pro/compare/v2.3.3...v2.3.4) (2022-07-16) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-20) Bug Fixes * **scripts:** use tonumber on timestamp args ([#71](https://github.com/taskforcesh/bullmq-pro/issues/71) ) ([5c6a62d](https://github.com/taskforcesh/bullmq-pro/commit/5c6a62de4d7df43343cca58f53ef39201c2aa6d1) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.3.3-2022-07-12) [2.3.3](https://github.com/taskforcesh/bullmq-pro/compare/v2.3.2...v2.3.3) (2022-07-12) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-21) Bug Fixes * **deps:** upgrade bullmq to 1.86.5 ([#69](https://github.com/taskforcesh/bullmq-pro/issues/69) ) ([2ed4bf3](https://github.com/taskforcesh/bullmq-pro/commit/2ed4bf36a1a0245e0303a8bc5fe120dbf84d8e1d) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.3.2-2022-07-09) [2.3.2](https://github.com/taskforcesh/bullmq-pro/compare/v2.3.1...v2.3.2) (2022-07-09) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-22) Bug Fixes * **concurrency:** consider retry backoff strategy ([#68](https://github.com/taskforcesh/bullmq-pro/issues/68) ) ([99f17bd](https://github.com/taskforcesh/bullmq-pro/commit/99f17bdd085ef1376bb1f35e2c679ab04e3a2d03) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.3.1-2022-07-01) [2.3.1](https://github.com/taskforcesh/bullmq-pro/compare/v2.3.0...v2.3.1) (2022-07-01) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-23) Bug Fixes * **job-pro:** fix gid parse ([#67](https://github.com/taskforcesh/bullmq-pro/issues/67) ) ([5532eaf](https://github.com/taskforcesh/bullmq-pro/commit/5532eaf5d61790a9bf63604838c2c3cd5546697e) ) [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.3.0-2022-07-01) [2.3.0](https://github.com/taskforcesh/bullmq-pro/compare/v2.2.3...v2.3.0) (2022-07-01) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#features-5) Features * **job-pro:** expose gid value ([#65](https://github.com/taskforcesh/bullmq-pro/issues/65) ) ([ea7ab29](https://github.com/taskforcesh/bullmq-pro/commit/ea7ab29d7d15c42fba6823de53c243c0eb20d2fa) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.2.3-2022-06-30) [2.2.3](https://github.com/taskforcesh/bullmq-pro/compare/v2.2.2...v2.2.3) (2022-06-30) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-24) Bug Fixes * **queue-pro:** fix addBulk opts typing ([#66](https://github.com/taskforcesh/bullmq-pro/issues/66) ) ([8b73ed9](https://github.com/taskforcesh/bullmq-pro/commit/8b73ed9b807375f1a18a62feef26c48c9b324fe8) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.2.2-2022-06-28) [2.2.2](https://github.com/taskforcesh/bullmq-pro/compare/v2.2.1...v2.2.2) (2022-06-28) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-25) Bug Fixes * **pause-group:** return boolean for execution success ([#64](https://github.com/taskforcesh/bullmq-pro/issues/64) ) ([b665b82](https://github.com/taskforcesh/bullmq-pro/commit/b665b828ba950411567f3424f0e8a1f80467021b) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.2.1-2022-06-25) [2.2.1](https://github.com/taskforcesh/bullmq-pro/compare/v2.2.0...v2.2.1) (2022-06-25) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-26) Bug Fixes * **groups:** rename paused and resumed events in QueueEventsPro ([#63](https://github.com/taskforcesh/bullmq-pro/issues/63) ) ([e2d6abf](https://github.com/taskforcesh/bullmq-pro/commit/e2d6abff3d59a8417896f7405ffcab35f2a780f3) ) [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.2.0-2022-06-24) [2.2.0](https://github.com/taskforcesh/bullmq-pro/compare/v2.1.6...v2.2.0) (2022-06-24) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#features-6) Features * **pause-group:** allow pausing specific group ([#61](https://github.com/taskforcesh/bullmq-pro/issues/61) ) ref [#25](https://github.com/taskforcesh/bullmq-pro/issues/25) ([a5ec201](https://github.com/taskforcesh/bullmq-pro/commit/a5ec2018935241b01be1c38323e6d1e31fffe89f) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.1.6-2022-06-10) [2.1.6](https://github.com/taskforcesh/bullmq-pro/compare/v2.1.5...v2.1.6) (2022-06-10) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-27) Bug Fixes * **deps:** upgrade bullmq to 1.86.0 ([#60](https://github.com/taskforcesh/bullmq-pro/issues/60) ) ([ea07b00](https://github.com/taskforcesh/bullmq-pro/commit/ea07b0090e21efabfe25f65d277856eaab0d8fc5) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.1.5-2022-06-09) [2.1.5](https://github.com/taskforcesh/bullmq-pro/compare/v2.1.4...v2.1.5) (2022-06-09) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-28) Bug Fixes * **deps:** upgrade bullmq to 1.85.4 ([#59](https://github.com/taskforcesh/bullmq-pro/issues/59) ) ([b45b363](https://github.com/taskforcesh/bullmq-pro/commit/b45b36369909a7db9fa01968065af0ff9ad2cafd) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.1.4-2022-06-08) [2.1.4](https://github.com/taskforcesh/bullmq-pro/compare/v2.1.3...v2.1.4) (2022-06-08) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-29) Bug Fixes * **worker:** use isObservable ([#58](https://github.com/taskforcesh/bullmq-pro/issues/58) ) ([8bed7ce](https://github.com/taskforcesh/bullmq-pro/commit/8bed7ce5a933c0126abd441488180fb5036eb3f1) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.1.3-2022-05-25) [2.1.3](https://github.com/taskforcesh/bullmq-pro/compare/v2.1.2...v2.1.3) (2022-05-25) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-30) Bug Fixes * **deps:** upgrade bullmq to 1.83.2 ([#56](https://github.com/taskforcesh/bullmq-pro/issues/56) ) ([a98c917](https://github.com/taskforcesh/bullmq-pro/commit/a98c9177bbb526692a22b9407d0f0374db7ee8d2) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.1.2-2022-05-20) [2.1.2](https://github.com/taskforcesh/bullmq-pro/compare/v2.1.1...v2.1.2) (2022-05-20) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-31) Bug Fixes * **deps:** upgrade bullmq to 1.83.0 ([#55](https://github.com/taskforcesh/bullmq-pro/issues/55) ) ([dc3b02d](https://github.com/taskforcesh/bullmq-pro/commit/dc3b02d28b583862ea2fab2e6557d5d35ff811e6) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.1.1-2022-05-18) [2.1.1](https://github.com/taskforcesh/bullmq-pro/compare/v2.1.0...v2.1.1) (2022-05-18) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-32) Bug Fixes * **flow-producer:** use JobPro instances ([#54](https://github.com/taskforcesh/bullmq-pro/issues/54) ) ([578d3db](https://github.com/taskforcesh/bullmq-pro/commit/578d3db5941752b72d1925a1026e013a590d55d5) ) [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.1.0-2022-05-17) [2.1.0](https://github.com/taskforcesh/bullmq-pro/compare/v2.0.3...v2.1.0) (2022-05-17) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#features-7) Features * **get-state:** consider checking groups ([#53](https://github.com/taskforcesh/bullmq-pro/issues/53) ) ([1dad072](https://github.com/taskforcesh/bullmq-pro/commit/1dad072cad84b3b18219bd8c0caf883c2b5179fc) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.0.3-2022-05-07) [2.0.3](https://github.com/taskforcesh/bullmq-pro/compare/v2.0.2...v2.0.3) (2022-05-07) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-33) Bug Fixes * **deps:** upgrade bullmq to 1.81.4 ([#52](https://github.com/taskforcesh/bullmq-pro/issues/52) ) ([8d92b21](https://github.com/taskforcesh/bullmq-pro/commit/8d92b21571a1263a3be097bf7e1c7d7f60c06816) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.0.2-2022-04-27) [2.0.2](https://github.com/taskforcesh/bullmq-pro/compare/v2.0.1...v2.0.2) (2022-04-27) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-34) Bug Fixes * **stalled:** allow easy transition for stalled changes ([#50](https://github.com/taskforcesh/bullmq-pro/issues/50) ) ([ce40ead](https://github.com/taskforcesh/bullmq-pro/commit/ce40ead2c26bffbc80d3953ed80a63bceedbb73b) ) ### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.0.1-2022-04-22) [2.0.1](https://github.com/taskforcesh/bullmq-pro/compare/v2.0.0...v2.0.1) (2022-04-22) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#bug-fixes-35) Bug Fixes * **deps:** upgrade bullmq to 1.80.6 ([#48](https://github.com/taskforcesh/bullmq-pro/issues/48) ) ([4aed9b0](https://github.com/taskforcesh/bullmq-pro/commit/4aed9b0c11d77b96f0859ff6b1b32e4b7c95249d) ) [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#id-2.0.0-2022-04-20) [2.0.0](https://github.com/taskforcesh/bullmq-pro/compare/v1.4.1...v2.0.0) (2022-04-20) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#features-8) Features * **groups:** improve addGroups to return all groups statuses ([3f01d66](https://github.com/taskforcesh/bullmq-pro/commit/3f01d66fee33965a68de634e6771ab9da158a0e1) ) #### [](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v2#breaking-changes) BREAKING CHANGES * **groups:** In order to make the group getter consistent for all statuses we are changing the SET type for groups:active to ZSET. Also we rename the ZSET to groups:max as it represent groups that have maxed the concurrency. [Previousv3](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v3) [Nextv1](https://docs.bullmq.io/bullmq-pro/changelog/changelog-v1) Last updated 3 months ago Was this helpful? --- # Changelogs | BullMQ ### [](https://docs.bullmq.io/python/changelog#id-2.16.1-2025-09-08) [2.16.1](https://github.com/taskforcesh/bullmq/compare/vpy2.16.0...vpy2.16.1) (2025-09-08) #### [](https://docs.bullmq.io/python/changelog#bug-fixes) Bug Fixes * **deps:** upgrade semver to v3 \[python\] ([#3414](https://github.com/taskforcesh/bullmq/issues/3414) ) ([7dfd251](https://github.com/taskforcesh/bullmq/commit/7dfd2510a7bb03301365a2c24f045a0cd20580ef) ) [](https://docs.bullmq.io/python/changelog#id-2.16.0-2025-09-07) [2.16.0](https://github.com/taskforcesh/bullmq/compare/vpy2.15.1...vpy2.16.0) (2025-09-07) ----------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/python/changelog#features) Features * **queue:** support getWaitingCount method \[python\] ([#3434](https://github.com/taskforcesh/bullmq/issues/3434) ) ([1c75abb](https://github.com/taskforcesh/bullmq/commit/1c75abb4806971a2ac14c47b302c87f86095a2c3) ) ### [](https://docs.bullmq.io/python/changelog#id-2.15.1-2025-09-06) [2.15.1](https://github.com/taskforcesh/bullmq/compare/vpy2.15.0...vpy2.15.1) (2025-09-06) #### [](https://docs.bullmq.io/python/changelog#bug-fixes-1) Bug Fixes * **job:** consider parent update when retrying ([#3402](https://github.com/taskforcesh/bullmq/issues/3402) ) (python) fixes [#3320](https://github.com/taskforcesh/bullmq/issues/3320) ([316d1ed](https://github.com/taskforcesh/bullmq/commit/316d1ed32680e690b1d2ab92c79a53e0d4c00c2d) ) * **job:** pass stacktrace into scripts on failures \[python\] ([#3294](https://github.com/taskforcesh/bullmq/issues/3294) ) ([97b215d](https://github.com/taskforcesh/bullmq/commit/97b215d5a7aeaf4dceb4543bef1a00e463f12197) ) [](https://docs.bullmq.io/python/changelog#id-2.15.0-2025-05-13) [2.15.0](https://github.com/taskforcesh/bullmq/compare/vpy2.14.0...vpy2.15.0) (2025-05-13) ----------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/python/changelog#bug-fixes-2) Bug Fixes * **worker:** maxStalledCount no less than 0 ([#3249](https://github.com/taskforcesh/bullmq/issues/3249) ) fixes [#3248](https://github.com/taskforcesh/bullmq/issues/3248) ([34dcb8c](https://github.com/taskforcesh/bullmq/commit/34dcb8c3d01a822b07852bc928d882bd6e4049d2) ) * **remove:** pass correct children meta references ([#3245](https://github.com/taskforcesh/bullmq/issues/3245) ) ([01c62ad](https://github.com/taskforcesh/bullmq/commit/01c62ada0cea80c73ba28d79fd14ea5ba78fdc7d) ) #### [](https://docs.bullmq.io/python/changelog#features-1) Features * **job:** add moveToCompleted method \[python\] ([#3251](https://github.com/taskforcesh/bullmq/issues/3251) ) ([6a8e3e2](https://github.com/taskforcesh/bullmq/commit/6a8e3e206384b56063c6f5a46ca030d2b330c712) ) [](https://docs.bullmq.io/python/changelog#id-2.14.0-2025-05-01) [2.14.0](https://github.com/taskforcesh/bullmq/compare/vpy2.13.1...vpy2.14.0) (2025-05-01) ----------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/python/changelog#bug-fixes-3) Bug Fixes * **connection:** add str type in connection option \[python\] ([#3212](https://github.com/taskforcesh/bullmq/issues/3212) ) ([72fac42](https://github.com/taskforcesh/bullmq/commit/72fac4297f5a60e0c2ae0831507cb16ce8baed5f) ) * **flow:** remove job from dependencies when failParentOnFailure or continueParentOnFailure ([#3201](https://github.com/taskforcesh/bullmq/issues/3201) ) ([1fbcbec](https://github.com/taskforcesh/bullmq/commit/1fbcbec56969fc4aa628f77e4b05d2c6844894ae) ) * **flow:** validate pending dependencies before removing lock ([#3182](https://github.com/taskforcesh/bullmq/issues/3182) ) ([8d59e3b](https://github.com/taskforcesh/bullmq/commit/8d59e3b8084c60afad16372b4f7fc22f1b9d3f4e) ) * **flow:** only validate pending dependencies when moving to completed ([#3164](https://github.com/taskforcesh/bullmq/issues/3164) ) ([d3c397f](https://github.com/taskforcesh/bullmq/commit/d3c397fa3f122287026018aaae5ed2c5dfad19aa) ) * **flow:** consider prioritized state when moving a parent to failed ([#3160](https://github.com/taskforcesh/bullmq/issues/3160) ) ([d91d9f4](https://github.com/taskforcesh/bullmq/commit/d91d9f4398584506f5af8b46e4d47b769beaa212) ) #### [](https://docs.bullmq.io/python/changelog#features-2) Features * **flows:** add continueParentOnFailure option ([#3181](https://github.com/taskforcesh/bullmq/issues/3181) ) ([738d375](https://github.com/taskforcesh/bullmq/commit/738d3752934746a347fd04e59e9dcd4726777508) ) #### [](https://docs.bullmq.io/python/changelog#performance-improvements) Performance Improvements * **flow:** change parent failure in a lazy way ([#3228](https://github.com/taskforcesh/bullmq/issues/3228) ) ([6b37a37](https://github.com/taskforcesh/bullmq/commit/6b37a379cc65abe7b4c60ba427065957c9080a08) ) * **flow:** validate parentKey existence before trying to move it to failed ([#3163](https://github.com/taskforcesh/bullmq/issues/3163) ) ([5a88e47](https://github.com/taskforcesh/bullmq/commit/5a88e4745d9449e41c5e2c467b5d02ca21357703) ) ### [](https://docs.bullmq.io/python/changelog#id-2.13.1-2025-03-15) [2.13.1](https://github.com/taskforcesh/bullmq/compare/vpy2.13.0...vpy2.13.1) (2025-03-15) #### [](https://docs.bullmq.io/python/changelog#bug-fixes-4) Bug Fixes * Nothing changed [](https://docs.bullmq.io/python/changelog#id-2.13.0-2025-03-15) [2.13.0](https://github.com/taskforcesh/bullmq/compare/vpy2.12.1...vpy2.13.0) (2025-03-15) ----------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/python/changelog#bug-fixes-5) Bug Fixes * **flow:** consider to fail a parent not in waiting-children when failParentOnFailure is provided ([#3098](https://github.com/taskforcesh/bullmq/issues/3098) ) ([589adb4](https://github.com/taskforcesh/bullmq/commit/589adb4f89bcb7d7721200333c2d605eb6ba7864) ) ### [](https://docs.bullmq.io/python/changelog#id-2.12.1-2025-02-28) [2.12.1](https://github.com/taskforcesh/bullmq/compare/vpy2.12.0...vpy2.12.1) (2025-02-28) #### [](https://docs.bullmq.io/python/changelog#bug-fixes-6) Bug Fixes * **worker:** cast delay\_until to integer \[python\] ([#3116](https://github.com/taskforcesh/bullmq/issues/3116) ) ([db617e4](https://github.com/taskforcesh/bullmq/commit/db617e48ef1dd52446bfd73e15f24957df2ca315) ) * **flow:** consider delayed state when moving a parent to failed ([#3112](https://github.com/taskforcesh/bullmq/issues/3112) ) ([6a28b86](https://github.com/taskforcesh/bullmq/commit/6a28b861346a3efa89574a78b396954d6c4ed113) ) [](https://docs.bullmq.io/python/changelog#id-2.12.0-2025-02-21) [2.12.0](https://github.com/taskforcesh/bullmq/compare/vpy2.11.0...vpy2.12.0) (2025-02-21) ----------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/python/changelog#bug-fixes-7) Bug Fixes * **flow:** allow using removeOnFail and failParentOnFailure in parents ([#2947](https://github.com/taskforcesh/bullmq/issues/2947) ) fixes [#2229](https://github.com/taskforcesh/bullmq/issues/2229) ([85f6f6f](https://github.com/taskforcesh/bullmq/commit/85f6f6f181003fafbf75304a268170f0d271ccc3) ) #### [](https://docs.bullmq.io/python/changelog#performance-improvements-1) Performance Improvements * **delayed:** add marker once when promoting delayed jobs ([#3096](https://github.com/taskforcesh/bullmq/issues/3096) ) (python) ([38912fb](https://github.com/taskforcesh/bullmq/commit/38912fba969d614eb44d05517ba2ec8bc418a16e) ) * **add-job:** add job into wait or prioritized state when delay is provided as 0 ([#3052](https://github.com/taskforcesh/bullmq/issues/3052) ) ([3e990eb](https://github.com/taskforcesh/bullmq/commit/3e990eb742b3a12065110f33135f282711fdd7b9) ) [](https://docs.bullmq.io/python/changelog#id-2.11.0-2024-11-26) [2.11.0](https://github.com/taskforcesh/bullmq/compare/vpy2.10.1...vpy2.11.0) (2024-11-26) ----------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/python/changelog#features-3) Features * **queue:** add getDelayedCount method \[python\] ([#2934](https://github.com/taskforcesh/bullmq/issues/2934) ) ([71ce75c](https://github.com/taskforcesh/bullmq/commit/71ce75c04b096b5593da0986c41a771add1a81ce) ) #### [](https://docs.bullmq.io/python/changelog#performance-improvements-2) Performance Improvements * **marker:** add base markers while consuming jobs to get workers busy ([#2904](https://github.com/taskforcesh/bullmq/issues/2904) ) fixes [#2842](https://github.com/taskforcesh/bullmq/issues/2842) ([1759c8b](https://github.com/taskforcesh/bullmq/commit/1759c8bc111cab9e43d5fccb4d8d2dccc9c39fb4) ) ### [](https://docs.bullmq.io/python/changelog#id-2.10.1-2024-10-26) [2.10.1](https://github.com/taskforcesh/bullmq/compare/vpy2.10.0...vpy2.10.1) (2024-10-26) #### [](https://docs.bullmq.io/python/changelog#bug-fixes-8) Bug Fixes * **commands:** add missing build statement when releasing \[python\] ([#2869](https://github.com/taskforcesh/bullmq/issues/2869) ) fixes [#2868](https://github.com/taskforcesh/bullmq/issues/2868) ([ff2a47b](https://github.com/taskforcesh/bullmq/commit/ff2a47b37c6b36ee1a725f91de2c6e4bcf8b011a) ) [](https://docs.bullmq.io/python/changelog#id-2.10.0-2024-10-24) [2.10.0](https://github.com/taskforcesh/bullmq/compare/vpy2.9.4...vpy2.10.0) (2024-10-24) ---------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/python/changelog#features-4) Features * **job:** add getChildrenValues method \[python\] ([#2853](https://github.com/taskforcesh/bullmq/issues/2853) ) ([0f25213](https://github.com/taskforcesh/bullmq/commit/0f25213b28900a1c35922bd33611701629d83184) ) ### [](https://docs.bullmq.io/python/changelog#id-2.9.4-2024-09-10) [2.9.4](https://github.com/taskforcesh/bullmq/compare/vpy2.9.3...vpy2.9.4) (2024-09-10) #### [](https://docs.bullmq.io/python/changelog#bug-fixes-9) Bug Fixes * **metrics:** differentiate points in different minutes to be more accurate ([#2766](https://github.com/taskforcesh/bullmq/issues/2766) ) (python) ([7cb670e](https://github.com/taskforcesh/bullmq/commit/7cb670e1bf9560a24de3da52427b4f6b6152a59a) ) #### [](https://docs.bullmq.io/python/changelog#performance-improvements-3) Performance Improvements * **metrics:** save zeros as much as max data points ([#2758](https://github.com/taskforcesh/bullmq/issues/2758) ) ([3473054](https://github.com/taskforcesh/bullmq/commit/347305451a9f5d7f2c16733eb139b5de96ea4b9c) ) ### [](https://docs.bullmq.io/python/changelog#id-2.9.3-2024-08-31) [2.9.3](https://github.com/taskforcesh/bullmq/compare/vpy2.9.2...vpy2.9.3) (2024-08-31) #### [](https://docs.bullmq.io/python/changelog#bug-fixes-10) Bug Fixes * **flow:** recursive ignoreDependencyOnFailure option ([#2712](https://github.com/taskforcesh/bullmq/issues/2712) ) ([53bc9eb](https://github.com/taskforcesh/bullmq/commit/53bc9eb68b5bb0a470a8fe64ef78ece5cde44632) ) ### [](https://docs.bullmq.io/python/changelog#id-2.9.2-2024-08-10) [2.9.2](https://github.com/taskforcesh/bullmq/compare/vpy2.9.1...vpy2.9.2) (2024-08-10) #### [](https://docs.bullmq.io/python/changelog#bug-fixes-11) Bug Fixes * **flow:** validate parentData before ignoreDependencyOnFailure when stalled check happens ([#2702](https://github.com/taskforcesh/bullmq/issues/2702) ) (python) ([9416501](https://github.com/taskforcesh/bullmq/commit/9416501551b1ad464e59bdba1045a5a9955e2ea4) ) ### [](https://docs.bullmq.io/python/changelog#id-2.9.1-2024-08-08) [2.9.1](https://github.com/taskforcesh/bullmq/compare/vpy2.9.0...vpy2.9.1) (2024-08-08) #### [](https://docs.bullmq.io/python/changelog#bug-fixes-12) Bug Fixes * Nothing changed [](https://docs.bullmq.io/python/changelog#id-2.9.0-2024-08-02) [2.9.0](https://github.com/taskforcesh/bullmq/compare/vpy2.8.1...vpy2.9.0) (2024-08-02) ------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/python/changelog#bug-fixes-13) Bug Fixes * **job:** make sure json.dumps return JSON compliant JSON \[python\] ([#2683](https://github.com/taskforcesh/bullmq/issues/2683) ) ([4441711](https://github.com/taskforcesh/bullmq/commit/4441711a986a9f6a326100308d639eb0a2ea8c8d) ) ### [](https://docs.bullmq.io/python/changelog#id-2.8.1-2024-07-11) [2.8.1](https://github.com/taskforcesh/bullmq/compare/vpy2.8.0...vpy2.8.1) (2024-07-11) #### [](https://docs.bullmq.io/python/changelog#bug-fixes-14) Bug Fixes * **delayed:** avoid using jobId in order to schedule delayed jobs ([#2587](https://github.com/taskforcesh/bullmq/issues/2587) ) (python) ([228db2c](https://github.com/taskforcesh/bullmq/commit/228db2c780a1ca8323900fc568156495a13355a3) ) #### [](https://docs.bullmq.io/python/changelog#performance) Performance * **delayed:** keep moving delayed jobs to waiting when queue is paused ([#2640](https://github.com/taskforcesh/bullmq/issues/2640) ) (python) ([b89e2e0](https://github.com/taskforcesh/bullmq/commit/b89e2e0913c0886561fc1c2470771232f17f5b3b) ) [](https://docs.bullmq.io/python/changelog#id-2.8.0-2024-07-10) [2.8.0](https://github.com/taskforcesh/bullmq/compare/vpy2.7.8...vpy2.8.0) (2024-07-10) ------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/python/changelog#bug-fixes-15) Bug Fixes * **parent:** consider re-adding child that is in completed state using same jobIds ([#2627](https://github.com/taskforcesh/bullmq/issues/2627) ) (python) fixes [#2554](https://github.com/taskforcesh/bullmq/issues/2554) ([00cd017](https://github.com/taskforcesh/bullmq/commit/00cd0174539fbe1cc4628b9b6e1a7eb87a5ef705) ) * **priority:** consider paused state when calling getCountsPerPriority (python) ([#2609](https://github.com/taskforcesh/bullmq/issues/2609) ) ([6e99250](https://github.com/taskforcesh/bullmq/commit/6e992504b2a7a2fa76f1d04ad53d1512e98add7f) ) * **priority:** use module instead of bit.band to keep order (python) ([#2597](https://github.com/taskforcesh/bullmq/issues/2597) ) ([9ece15b](https://github.com/taskforcesh/bullmq/commit/9ece15b17420fe0bee948a5307e870915e3bce87) ) #### [](https://docs.bullmq.io/python/changelog#features-5) Features * **queue:** add getCountsPerPriority method \[python\] ([#2607](https://github.com/taskforcesh/bullmq/issues/2607) ) ([02b8338](https://github.com/taskforcesh/bullmq/commit/02b83380334879cc2434043141566f2a375db958) ) ### [](https://docs.bullmq.io/python/changelog#id-2.7.8-2024-06-05) [2.7.8](https://github.com/taskforcesh/bullmq/compare/vpy2.7.7...vpy2.7.8) (2024-06-05) #### [](https://docs.bullmq.io/python/changelog#bug-fixes-16) Bug Fixes * remove print calls \[python\] ([#2579](https://github.com/taskforcesh/bullmq/issues/2579) ) ([f957186](https://github.com/taskforcesh/bullmq/commit/f95718689864dbaca8a6b4113a6b37727919d6df) ) ### [](https://docs.bullmq.io/python/changelog#id-2.7.7-2024-06-04) [2.7.7](https://github.com/taskforcesh/bullmq/compare/vpy2.7.6...vpy2.7.7) (2024-06-04) #### [](https://docs.bullmq.io/python/changelog#bug-fixes-17) Bug Fixes * **retry-job:** throw error when job is not in active state ([#2576](https://github.com/taskforcesh/bullmq/issues/2576) ) ([ca207f5](https://github.com/taskforcesh/bullmq/commit/ca207f593d0ed455ecc59d9e0ef389a9a50d9634) ) ### [](https://docs.bullmq.io/python/changelog#id-2.7.6-2024-05-09) [2.7.6](https://github.com/taskforcesh/bullmq/compare/vpy2.7.5...vpy2.7.6) (2024-05-09) #### [](https://docs.bullmq.io/python/changelog#bug-fixes-18) Bug Fixes * **connection:** use async Retry ([#2555](https://github.com/taskforcesh/bullmq/issues/2555) ) \[python\] ([d6dd21d](https://github.com/taskforcesh/bullmq/commit/d6dd21d3ac28660bbfa7825bba0b586328769709) ) ### [](https://docs.bullmq.io/python/changelog#id-2.7.5-2024-04-28) [2.7.5](https://github.com/taskforcesh/bullmq/compare/vpy2.7.4...vpy2.7.5) (2024-04-28) #### [](https://docs.bullmq.io/python/changelog#bug-fixes-19) Bug Fixes * **worker:** wait for jobs to finalize on close ([#2545](https://github.com/taskforcesh/bullmq/issues/2545) ) \[python\] ([d81f210](https://github.com/taskforcesh/bullmq/commit/d81f210a5f5968fc040e820946fb672deb24bd01) ) ### [](https://docs.bullmq.io/python/changelog#id-2.7.4-2024-04-26) [2.7.4](https://github.com/taskforcesh/bullmq/compare/vpy2.7.3...vpy2.7.4) (2024-04-26) #### [](https://docs.bullmq.io/python/changelog#bug-fixes-20) Bug Fixes * **redis-connection:** increase redis retry strategy backoff ([#2546](https://github.com/taskforcesh/bullmq/issues/2545) ) \[python\] ([6cf7712](https://github.com/taskforcesh/bullmq/commit/6cf77122da845e5b0afa1607348cf06602679329) ) ### [](https://docs.bullmq.io/python/changelog#id-2.7.3-2024-04-24) [2.7.3](https://github.com/taskforcesh/bullmq/compare/vpy2.7.2...vpy2.7.3) (2024-04-24) #### [](https://docs.bullmq.io/python/changelog#bug-fixes-21) Bug Fixes * **stalled:** consider ignoreDependencyOnFailure option (python) ([#2540](https://github.com/taskforcesh/bullmq/issues/2540) ) fixes [#2531](https://github.com/taskforcesh/bullmq/issues/2531) ([0140959](https://github.com/taskforcesh/bullmq/commit/0140959cabd2613794631e41ebe4c2ddee6f91da) ) ### [](https://docs.bullmq.io/python/changelog#id-2.7.2-2024-04-20) [2.7.2](https://github.com/taskforcesh/bullmq/compare/vpy2.7.1...vpy2.7.2) (2024-04-20) #### [](https://docs.bullmq.io/python/changelog#bug-fixes-22) Bug Fixes * **worker:** return minimumBlockTimeout depending on redis version (python) ([#2532](https://github.com/taskforcesh/bullmq/issues/2532) ) ([83dfb63](https://github.com/taskforcesh/bullmq/commit/83dfb63e72a1a36a4dfc40f122efb54fbb796339) ) ### [](https://docs.bullmq.io/python/changelog#id-2.7.1-2024-04-18) [2.7.1](https://github.com/taskforcesh/bullmq/compare/vpy2.7.0...vpy2.7.1) (2024-04-18) #### [](https://docs.bullmq.io/python/changelog#bug-fixes-23) Bug Fixes * **stalled:** consider failParentOnFailure when moving child into failed ([#2526](https://github.com/taskforcesh/bullmq/issues/2526) ) fixes [#2464](https://github.com/taskforcesh/bullmq/issues/2464) (python) ([5e31eb0](https://github.com/taskforcesh/bullmq/commit/5e31eb096169ea57350db591bcebfc2264a6b6dc) ) [](https://docs.bullmq.io/python/changelog#id-2.7.0-2024-04-13) [2.7.0](https://github.com/taskforcesh/bullmq/compare/vpy2.6.0...vpy2.7.0) (2024-04-13) ------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/python/changelog#features-6) Features * **queue:** add getJobLogs method \[python\] ([#2523](https://github.com/taskforcesh/bullmq/issues/2523) ) ref [#2472](https://github.com/taskforcesh/bullmq/issues/2472) ([a24a16e](https://github.com/taskforcesh/bullmq/commit/a24a16ea2707541ee06ec3c4d636cd30dcdaade5) ) [](https://docs.bullmq.io/python/changelog#id-2.6.0-2024-04-13) [2.6.0](https://github.com/taskforcesh/bullmq/compare/vpy2.5.0...vpy2.6.0) (2024-04-13) ------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/python/changelog#features-7) Features * **worker:** use 0.002 as minimum timeout for redis version lower than 7.0.8 \[python\] ([#2521](https://github.com/taskforcesh/bullmq/issues/2521) ) ([f3862dd](https://github.com/taskforcesh/bullmq/commit/f3862dd0c85cf2c2122fb0306c5f4b5eb8ad0bcd) ) [](https://docs.bullmq.io/python/changelog#id-2.5.0-2024-04-08) [2.5.0](https://github.com/taskforcesh/bullmq/compare/vpy2.4.0...vpy2.5.0) (2024-04-08) ------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/python/changelog#features-8) Features * **python:** support reusable redis connections ([29ad8c8](https://github.com/taskforcesh/bullmq/commit/29ad8c83596b14a312ad1cd375e0e34d4fdecc52) ) [](https://docs.bullmq.io/python/changelog#id-2.4.0-2024-04-07) [2.4.0](https://github.com/taskforcesh/bullmq/compare/vpy2.3.3...vpy2.4.0) (2024-04-07) ------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/python/changelog#performance-improvements-4) Performance Improvements * **stalled:** remove jobId from stalled after removing lock when moved from active ([#2512](https://github.com/taskforcesh/bullmq/issues/2512) ) (python) ([64feec9](https://github.com/taskforcesh/bullmq/commit/64feec91b0b034fe640a846166bd95b546ff6d71) ) ### [](https://docs.bullmq.io/python/changelog#id-2.3.3-2024-03-24) [2.3.3](https://github.com/taskforcesh/bullmq/compare/vpy2.3.2...vpy2.3.3) (2024-03-24) #### [](https://docs.bullmq.io/python/changelog#bug-fixes-24) Bug Fixes * **connection:** accept all parameters for redis connection \[python\] ([#2486](https://github.com/taskforcesh/bullmq/issues/2486) ) ([ce30192](https://github.com/taskforcesh/bullmq/commit/ce30192ad30f66fb0f39c8c9ed669ddd133346c8) ) ### [](https://docs.bullmq.io/python/changelog#id-2.3.2-2024-03-23) [2.3.2](https://github.com/taskforcesh/bullmq/compare/vpy2.3.1...vpy2.3.2) (2024-03-23) #### [](https://docs.bullmq.io/python/changelog#bug-fixes-25) Bug Fixes * Nothing changed ### [](https://docs.bullmq.io/python/changelog#id-2.3.1-2024-03-19) [2.3.1](https://github.com/taskforcesh/bullmq/compare/vpy2.3.0...vpy2.3.1) (2024-03-19) #### [](https://docs.bullmq.io/python/changelog#bug-fixes-26) Bug Fixes * **worker:** set blockTimeout as 0.001 when reach the time to get delayed jobs \[python\] ([#2478](https://github.com/taskforcesh/bullmq/issues/2478) ) ([b385034](https://github.com/taskforcesh/bullmq/commit/b385034006ac183a26093f593269349eb78f8b54) ) [](https://docs.bullmq.io/python/changelog#id-2.3.0-2024-03-16) [2.3.0](https://github.com/taskforcesh/bullmq/compare/vpy2.2.4...vpy2.3.0) (2024-03-16) ------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/python/changelog#features-9) Features * **job:** add log method \[python\] ([#2476](https://github.com/taskforcesh/bullmq/issues/2476) ) ref [#2472](https://github.com/taskforcesh/bullmq/issues/2472) ([34946c4](https://github.com/taskforcesh/bullmq/commit/34946c4b29cc9e7d5ae81f8fd170a2e539ac6279) ) ### [](https://docs.bullmq.io/python/changelog#id-2.2.4-2024-02-13) [2.2.4](https://github.com/taskforcesh/bullmq/compare/vpy2.2.3...vpy2.2.4) (2024-02-13) #### [](https://docs.bullmq.io/python/changelog#bug-fixes-27) Bug Fixes * **flow:** parent job cannot be replaced (python) ([#2417](https://github.com/taskforcesh/bullmq/issues/2417) ) ([2696ef8](https://github.com/taskforcesh/bullmq/commit/2696ef8200058b7f616938c2166a3b0454663b39) ) ### [](https://docs.bullmq.io/python/changelog#id-2.2.3-2024-02-10) [2.2.3](https://github.com/taskforcesh/bullmq/compare/vpy2.2.2...vpy2.2.3) (2024-02-10) #### [](https://docs.bullmq.io/python/changelog#performance-improvements-5) Performance Improvements * **marker:** differentiate standard and delayed markers (python) ([#2389](https://github.com/taskforcesh/bullmq/issues/2389) ) ([18ebee8](https://github.com/taskforcesh/bullmq/commit/18ebee8c242f66f1b5b733d68e48c574b1f1fdef) ) ### [](https://docs.bullmq.io/python/changelog#id-2.2.2-2024-02-03) [2.2.2](https://github.com/taskforcesh/bullmq/compare/vpy2.2.1...vpy2.2.2) (2024-02-03) #### [](https://docs.bullmq.io/python/changelog#bug-fixes-28) Bug Fixes * **reprocess-job:** add marker if needed ([#2406](https://github.com/taskforcesh/bullmq/issues/2406) ) ([5923ed8](https://github.com/taskforcesh/bullmq/commit/5923ed885f5451eee2f14258767d7d5f8d80ae13) ) * **stalled:** consider adding marker when moving job back to wait ([#2384](https://github.com/taskforcesh/bullmq/issues/2384) ) ([4914df8](https://github.com/taskforcesh/bullmq/commit/4914df87e416711835291e81da93b279bd758254) ) #### [](https://docs.bullmq.io/python/changelog#performance-improvements-6) Performance Improvements * **flow:** add marker when moving parent to wait (python) ([#2408](https://github.com/taskforcesh/bullmq/issues/2408) ) ([6fb6896](https://github.com/taskforcesh/bullmq/commit/6fb6896701ae7595e1cb5e2cdbef44625c48d673) ) * **move-to-active:** check rate limited once ([#2391](https://github.com/taskforcesh/bullmq/issues/2391) ) ([`ca6c17a`](https://github.com/taskforcesh/bullmq/commit/ca6c17a43e38d5339e62471ea9f59c62a169b797) ) ### [](https://docs.bullmq.io/python/changelog#id-2.2.1-2024-01-16) [2.2.1](https://github.com/taskforcesh/bullmq/compare/vpy2.2.0...vpy2.2.1) (2024-01-16) #### [](https://docs.bullmq.io/python/changelog#bug-fixes-29) Bug Fixes * **retry-jobs:** add marker when needed ([#2374](https://github.com/taskforcesh/bullmq/issues/2374) ) ([`1813d5f`](https://github.com/taskforcesh/bullmq/commit/1813d5fa12b7db69ee6c8c09273729cda8e3e3b5) ) [](https://docs.bullmq.io/python/changelog#id-2.2.0-2024-01-14) [2.2.0](https://github.com/taskforcesh/bullmq/compare/vpy2.1.0...vpy2.2.0) (2024-01-14) ------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/python/changelog#features-10) Features * **queue:** add promoteJobs method \[python\] ([#2377](https://github.com/taskforcesh/bullmq/issues/2377) ) ([3b9de96](https://github.com/taskforcesh/bullmq/commit/3b9de967efa34ea22cdab1fbc7ff65d49927d787) ) [](https://docs.bullmq.io/python/changelog#id-2.1.0-2024-01-12) [2.1.0](https://github.com/taskforcesh/bullmq/compare/vpy2.0.0...vpy2.1.0) (2024-01-12) ------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/python/changelog#bug-fixes-30) Bug Fixes * **redis:** upgrade to v5 \[python\] ([#2364](https://github.com/taskforcesh/bullmq/issues/2364) ) ([d5113c8](https://github.com/taskforcesh/bullmq/commit/d5113c88ad108b281b292e2890e0eef3be41c8fb) ) [](https://docs.bullmq.io/python/changelog#id-2.0.0-2023-12-23) [2.0.0](https://github.com/taskforcesh/bullmq/compare/vpy1.24.0...vpy2.0.0) (2023-12-23) -------------------------------------------------------------------------------------------------------------------------------------------------------------- #### [](https://docs.bullmq.io/python/changelog#bug-fixes-31) Bug Fixes * **connection:** unify redis connection args for Queue and Worker ([#2282](https://github.com/taskforcesh/bullmq/issues/2282) ) ([8eee20f](https://github.com/taskforcesh/bullmq/commit/8eee20f1210a49024eeee6647817f0659b8c3893) ) #### [](https://docs.bullmq.io/python/changelog#features-11) Features * **job:** add isActive method \[python\] ([#2352](https://github.com/taskforcesh/bullmq/issues/2352) ) ([afb5e31](https://github.com/taskforcesh/bullmq/commit/afb5e31484ed2e5a1c381c732321225c0a8b78ff) ) * **job:** separate attemptsMade from attemptsStarted when manually moving a job ([#2203](https://github.com/taskforcesh/bullmq/issues/2203) ) ([0e88e4f](https://github.com/taskforcesh/bullmq/commit/0e88e4fe4ed940487dfc79d1345d0686de22d0c6) ) * **scripts:** use new queue markers ([4276eb7](https://github.com/taskforcesh/bullmq/commit/4276eb725ca294ddbfc00c4edc627bb2cb5d403a) ) * **worker:** improved markers handling ([73cf5fc](https://github.com/taskforcesh/bullmq/commit/73cf5fc1e6e13d8329e1e4e700a8db92173e0624) ) ([0bac0fb](https://github.com/taskforcesh/bullmq/commit/0bac0fbb97afa968aa7644f1438b86d7bc18bbc5) ) #### [](https://docs.bullmq.io/python/changelog#breaking-changes) BREAKING CHANGES * **connection:** connection must be provided as part of options ([#2282](https://github.com/taskforcesh/bullmq/issues/2282) ) ([8eee20f](https://github.com/taskforcesh/bullmq/commit/8eee20f1210a49024eeee6647817f0659b8c3893) ) * **worker:** markers use now a dedicated key in redis instead of using a special Job ID. ([`73cf5fc`](https://github.com/taskforcesh/bullmq/commit/73cf5fc1e6e13d8329e1e4e700a8db92173e0624) ) ([`0bac0fb`](https://github.com/taskforcesh/bullmq/commit/0bac0fbb97afa968aa7644f1438b86d7bc18bbc5) ) * references: * [Better Queue Markers](https://bullmq.io/news/231204/better-queue-markers/) [PreviousIntroduction](https://docs.bullmq.io/python/introduction) [Nextv1](https://docs.bullmq.io/python/changelog/changelog-v1) Last updated 2 months ago Was this helpful? --- # Email Protection | Cloudflare Please enable cookies. Email Protection ================ You are unable to access this email address docs.bullmq.io ---------------------------------------------------------- The website from which you got to this page is protected by Cloudflare. Email addresses on that page have been hidden in order to keep them from being accessed by malicious bots. **You must enable Javascript in your browser in order to decode the e-mail address**. If you have a website and are interested in protecting it in a similar way, you can [sign up for Cloudflare](https://www.cloudflare.com/sign-up?utm_source=email_protection) . * [How does Cloudflare protect email addresses on website from spammers?](https://developers.cloudflare.com/waf/tools/scrape-shield/email-address-obfuscation/) * [Can I sign up for Cloudflare?](https://developers.cloudflare.com/fundamentals/setup/account/create-account/) Cloudflare Ray ID: **9a5ca464480686f1** • Your IP: Click to reveal 54.237.218.47 • Performance & security by [Cloudflare](https://www.cloudflare.com/5xx-error-landing) ---