Bull MQ
Last updated
Was this helpful?
Last updated
Was this helpful?
BullMQ is a library that implements a fast and robust queue system built on top of that helps in resolving many modern age micro-services architectures.
Redis is used to store the history of execution history of job, the detail and configuration of job, job will be distinguished by using job id.
Break up monolithic tasks that may otherwise block the Node.js event loop. For example, if a user request requires CPU intensive work like audio transcoding, you can delegate this task to other processes, freeing up user-facing processes to remain responsive.
It is using publisher-subscriber mechanism, The producer is to create a job with unique id and data and transfer through the queue. The consumer is to subscribe to a queue in order to receive the job and execute the logic.
Job producers add jobs to queues.
A consumer is a class defining methods that either process jobs added into the queue,
It is also for listen for events on the queue
When a job is added to a queue it can be in one of three states:
“wait”: a waiting list, where all jobs must enter before they can be processed.
“prioritized”: implies that a job with higher priority will be processed first.
“delayed”: implies that the job is waiting for some timeout or to be promoted for being processed. These jobs are not processed directly, but instead are placed at the beginning of the waiting list, or in a prioritized set, and processed as soon as a worker is idle.
The next state for a job is the “active” state. The active state is represented by a set, and are jobs that are currently being processed
the job will end in either the “completed” or the “failed” status.
Redis acts as a persist data store to store the job status, job detail, etc....
The detail of job, which key is the unique job id
The repeatable job configuration and its related job executed (job ids are generated in random string)
The completed job list, the history of execution of job and the repeatable configuration list
To visualise the job datas with redis data