Objectives:

By the end of this chapter, you should be able to:

• Define what a background job is
• Explain what redis is and its use in background jobs
• Use kue to run background jobs

Background jobs

Another common issue when building applications is ensuring that long processes or tasks are not blocking or slowing down the entire application. This could happen when many emails are being sent, large files are being uploaded, or when you want to execute a process and you know there will be less traffic. Background job library often involve using another data store (usually a queue) to handle the order and management of jobs being processed. Kue is a very common tool (written by the same people who made Mongoose!) for handling background jobs. You can read more about it here

Getting started with kue

To get started with kue we need to npm install --save kue and require the kue module and create a queue (which is backed by an in memory data store called redis)

var kue = require('kue');
var queue = kue.createQueue();

Once you have created the queue - it's time to queue up some tasks! These tasks can be time consuming web scraping, gathering analytics, making bulk database writes, uploading files or sending emails.

function sendEmail(title,to,subject,message, done){
    var email = queue.create('email', {title, to, subject, message});
    done();
}

router.post('/', function(req, res, next) {
  queue.process('email', function(job, done){
    const {title,to,subject,message} = req.body;
    // use nodemailer or another tool to send an email
    sendEmail(title,to,subject,message);
  });
});

Kue UI

Kue also ships with a nice package called kue-dashboard which provides an interface for you to see jobs running, stalled, failed, completed and much more. You can access it by starting a new server with node_modules/kue/bin/kue-dashboard -p 3001

You can read more about the UI here