Node.js and Redis Tutorial: Caching With Node.js and Redis

gowebing

stack-of-euros

Cold, hard cache

Through this short and introductory article, I would like to mention how caching can be achieved in Node.js applications using Redis and how our applications can benefit from it in terms of performance.

What Is Cache and How Can it Help us?

With a few words, we can say that cache is the process with which data is stored temporarily at a storage component area in order to be used in future mutch more faster. E.g. if we have some data coming from a third-party API, and that data won't be changed in the "near future," we can store them in cache once we retrieve them and avoid unnecessary service calls. In this way, we do not have to wait for the API call to complete, as we already have them, and we can retrieve them from cache. This will make our application much faster and more performant.

You may also like: Where Is My Cache? Architectural Patterns for Caching Microservices

What Is Redis?

Redis is a high-performance, open source NoSQL database primarily used as a caching solution for various types of applications. What is surprising, is that it stores all its data in RAM and promises highly optimized data reads and writes. Before we start describing how we can implement caching in Node.js applications, let's first see what how Redis.io defines their database.

Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache and message broker. It supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes with radius queries and streams. Redis has built-in replication, Lua scripting, LRU eviction, transactions and different levels of on-disk persistence, and provides high availability via Redis Sentinel and automatic partitioning with Redis Cluster.

What We Will Implement 

Today, I am going to implement a very simple cache mechanism on a Node.js application that requests data from a third-party API. More specifically, I will use GitHub's API in order to fetch the number of a user's public repositories. So, let's get started.

Redis Setup

If you already have Redis installed on your local mache or if you are using a Redis cloud solution, you can skip this step.

Install Redis on Windows

If you are a Windows user, you can check this short guide from redislabs.com in order to get started with Redis on Windows. The installation process of Redis is not the purpose of this article, so I will not dwell on that point.

Install Redis on Mac

Redis can be installed on your Mac using Homebrew. If you do not have Homebrew installed on your Mac, you can run the following command on your terminal to install it.

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"


After you have Homebrew installed, you can install Redis by using this command.

brew install redis


Install Redis on Linux

There is also an amazing, short guide on how to install Redis on a Linux machine from digitalocean.com, which is very easy to follow, so check it out.

Starting Redis Server and Redis CLI

Suppose we already have Redis installed locally, or, we have, as mentioned above, access to a Redis cloud solution. We can then start the Redis server with the following command.

redis-server


It is also so simple to start the Redis CLI with the second command.

redis-cli


Project Setup

As we already mentioned, we will create a simple Node.js application that will retrieve the number of public repositories of a GitHub user via a GitHub API.

First of all, we need to create a new folder for our Node.js application. After that, we can initialize the application by typing in the command prompt the following command to create a default package.json file.

npm init -y


Or, we can run npm init and follow the guide to initialize the package.json file as we need.

After that, we will have generated a package.json file which will be like the following one.


{
  "name": "nodejs-redis-cache-example",
  "version": "1.0.0",
  "description": "A demo application showing how to use redis to cache node.js web application",
  "main": "index.js",
  "scripts": {
    "start": "nodemon index.js"
  },
  "repository": {
    "type": "git",
    "url": "git+https://github.com/atheodosiou/nodejs-redis-cache-example.git"
  },
  "keywords": [
    "node.js",
    "redis",
    "cache",
    "tutorial"
  ],
  "author": "Anastasios Theodosiou",
  "license": "MIT",
  "bugs": {
    "url": "https://github.com/atheodosiou/nodejs-redis-cache-example/issues"
  },
  "homepage": "https://github.com/atheodosiou/nodejs-redis-cache-example#readme",
}


Installing Dependencies

We will continue by installing all the necessary dependencies by running  npm i. For this article, I will use three dependencies, and I am going to install them with the next command.

npm i --save express node-fetch redis

Installing Dev-Dependencies

In order not to have to manually refresh our development server, we will install an extra dependency for development. It is called nodemon.

npm i -D nodemon


Creating a Start Script

After we have all of the needed dependencies installed, we can create a start script inside the pacakge.json file to help us start our application with nodemon. So, inside the package.json file, create a new script like below.

  "start": "nodemon index.js"


The file, index.js, is the starting point of our application. The next step is to create this file in the same folder with the package.json file by running touch index.js. After, our package.json file should look like this:

{
  "name": "nodejs-redis-cache-example",
  "version": "1.0.0",
  "description": "A demo application showing how to use redis to cache node.js",
  "main": "index.js",
  "scripts": {
    "start": "nodemon index.js"
  },
  "repository": {
    "type": "git",
    "url": "git+https://github.com/atheodosiou/nodejs-redis-cache-example.git"
  },
  "keywords": [
    "node.js",
    "redis",
    "cache",
    "tutorial"
  ],
  "author": "Anastasios Theodosiou",
  "license": "MIT",
  "bugs": {
    "url": "https://github.com/atheodosiou/nodejs-redis-cache-example/issues"
  },
  "homepage": "https://github.com/atheodosiou/nodejs-redis-cache-example#readme",
  "dependencies": {
    "express": "^4.17.1",
    "node-fetch": "^2.6.0",
    "redis": "^2.8.0"
  },
  "devDependencies": {
    "nodemon": "^1.19.4"
  }
}


Add Application Logic

At this point, we can start implementing our application. We start by declaring three variables for our packages and importing them.

//Import packages
const express = require("express");
const fetch = require("node-fetch");
const redis = require("redis");


The next step is to create a .env file that will contain the port number for our server and Redis client.

//Declare express server port and redis client port
const PORT = process.env.PORT || 3000;
const REDIS_PORT = process.env.REDIS_PORT || 6379;


The code above checks if environment variables exist. Otherwise, it sets a default value. To work with these variables without more server customization, we can use another dev dependency called  dotenv. From there, we can install it with the following command:

npm i -D dotenv


Now, we can set up our server to listen on our desired port like below.

app.listen(PORT, () => {
  console.log(`App listening on port ${PORT}...`);
});


At this point, our index.js file should be like the following code block:

//Import packages
const express = require("express");
const fetch = require("node-fetch");
const redis = require("redis");

//Declare express server port and redis client port
const PORT = process.env.PORT || 3000;
const REDIS_PORT = process.env.REDIS_PORT || 6379;


app.listen(PORT, () => {
  console.log(`App listening on port ${PORT}...`);
});


We can verify that our application is running correctly by executing the npm run start command; we should see the following output in our terminal.

Development server output

Development server output

Creating a Redis Client and Express App

After we have already created a development server, we must create a Redis client and an Express app. We can achieve this with the following code.

//Create Redis client on Redis port
const redisClient = redis.createClient(REDIS_PORT);
//Create an app const by excecuting express like a function
const app = express();


Registring Routes and Adding Handlers

Next, we need to create a route in which our server will be listening and respond with the total number of a GitHub user's public repositories. For this, we will add some new lines of code.

app.get("/repos/:username", cache, getPublicReposNumber);


This means that when we are targeting http://localhost:3000/repos/atheodosiou, the server will trigger the associated handler to send us the response. This handler is a callback function with the name getPublicReposNumber.

Callback Function

This is the point in which we will try to request GitHub's public API for data.

//Make request to GitHub for data
async function getPublicReposNumber(req, res, next) {
  try {
    console.log("Fetching data...");

    const { username } = req.params;
    const response = await fetch(`https://api.github.com/users/${username}`);
    const data = await response.json();

    //set to redis
    redisClient.setex(username, 3600, data);

    res.status(200).send(setResponse(username, data));
  } catch (error) {
    console.error(error);
    req.status(500).json({ error: error });
  }
}


As you can see, we are getting the data to send it back to the client as a response. At this point, we are not using a cache mechanism, but we are using the Redis client in order to store the retrieved data. This is implemented on row 11 in the above code block.

We use the setx method of the Redis driver which allows us to set an expiration time for our stored data. In our case, we set this in 3600/sec, which means one hour.

Implement a Cache Mechanism

In order to maximize our application performance, we will implement a cache middleware that will search for a key on the Redis local server before requesting new data. This is how we create cache in a Node.js application.

I want to mention that this is not the only way you can to implement caching. It may depend on other application logic or your preferences, but, this tutorial is a good way to show you how Redis works on Node.js applications.

Before we add the cache middleware, let's run our application and check the total time taken to serve the request.

Total excecution time without cache

Total excecution time without cache

It took one second to respond without using cache, which is a normal time. Let's see how it will go if we add a Redis cache middleware.

The next step is to create a cache mechanism, as we already mentioned, to store retrieved data for a 'while' and avoid unnecessary API calls.

Attention! The usage of cache is only recommended when the data we are about to receive is subject to frequent changes. For example, if we made a call to an API whose call-to-call data could be subject to change, then using cache would not be appropriate.

In our example, our cache middleware is going to be a function that is called before the request for the data.

//Cache midleware
function cache(req, res, next) {
  const { username } = req.params;
  redisClient.get(username, (error, cachedData) => {
    if (error) throw error;
    if (cachedData != null) {
      res.send(setResponse(username, cachedData));
    } else {
      next();
    }
  });
}


This function is asking for a key, in our case a GitHub username, from the Redis server in order to get the cached data for that key. If the key cannot be found, the request continues by asking for data from the GitHub API.

In order to enable this mechanism, we must add it in the route before the request handler like below.

app.get("/repos/:username", cache, getPublicReposNumber);


Now, let's run the application again with the npm run start command. (Pay attention to the total time of the request.)

Request time after cache is implemented

Request time after cache is implemented

We can see that is a big difference while using cache. It took only four milliseconds, while it took 1 second without it. The reason is that we have already this data retrieved before and we have stored them in memory with Redis as key-value pair. Every time we will call this endpoint until 1 hour, the server will not make a real request to the GitHub API and will serve us the data that it has in memory.

The expiration time, which we are using for our Redis keys is different from one application to another, and it consists of multiple criteria. Feel free to experiment with this.

As we can see from the example above, using cache could be beneficial for our application. Redis is blazingly fast; it’s a NoSQL database, and in order to save your cloud database calls and eventually saving some euros out there, you can, of course, opt for caching. But, there are also some other reasons to use Redis outside of high performance.

  1. High Availability.
  2. Horizontal and Vertical Scalability.
  3. Native Solution.

There are also some disadvantages of using Redis, some of them are:

  1. Requires Client Support.
  2. Limited Multi-Key Operation Support.
  3. Only Supports One Database.

In other words, Redis is cool and fast, and it depends on your preferences to use it!

Conclusion

It is important to note that I have only scratched the surface in this article and there is a lot more that Redis has to offer! I highly recommend you checking out its official documentation. This is the link to the GitHub with the complete code of our application. If you have any questions, feel free to leave a comment. Also, if this helped you, please like and share it with others.


Further Reading

 

 

 

 

Top