How to wait for a queue to be idle for a serverless functions?

[knthmn]

I'm using BullMQ with serverless functions. How can I ensure all tasks in a queue to finish (either completed or failed) before shutting down? Is there a waitForAllFinished(queue) or is there a better approach?

Cheers.

[roggervalf]

hi @knthmn, since v5.40.1 it should be safe to close your workers and they will wait for all pending jobs before closing entirely

[knthmn]

I looked at the documentation and it says it will not pick up new jobs. Are existing jobs in queue considered new jobs? I need the worker to finish processing all the jobs inside the queue.

[roggervalf]

Now I see what you meant. Yes, when closing, workers won't pick new jobs. We have this event https://api.docs.bullmq.io/interfaces/v5.WorkerListener.html#drained, but take in count that any job apart of waiting ones are not considered by this event.

[manast]

@knthmn one thing you could do is every time a job completes check if there are any jobs in any relevant state, such as waiting, delayed, prioritized or active. You can use getJobCounts for that. Just remember that if you call this method from the processor function, you will always have at least 1 active job.