Scaling Node.js Applications Part 1: Cloning

Scaling Node.js Applications Part 1: Cloning

How can you scale Node.js Applications according to the X-Axis of Scale Cube? How to fork your Node.js processes and work with clusters with PM2.

Introduction

Node.js is an asynchronous event-driven, highly scalable runtime Javascript environment. Developers who use Node.js are free from the worries of dead-locking the process since there are no locks. Because nothing blocks, scalable systems are very reasonable to develop in Node.js.

Node.js being designed without threads doesn't mean you can't take advantage of multiple cores in your environment. Child processes can be spawned by using Node.js child_process.fork() API, and are designed to be easy to communicate with. Built upon that same interface is the cluster module, which allows you to share sockets between processes to enable load balancing over your cores.

In this article, we're gonna learn how to break our applications and data up into even tinier bits, single instances that are scalable.

Let's examine the scale cube and discuss the three directions in which an application can be scaled.

The Scale Cube

scaling cube.png

The Scaling Cube is seen in the above diagram. The bottom left corner of the cube represents the least scaled application possible, a single instance monolith. This means you have one application running in a single instance on a single computer. As we scale our applications we move closer to the top right corner of the scale cube. Applications on this side of the diagram could represent multi-instance applications that have been distributed into microservices and contain partitioned database clusters. Applications that are closer to the top right corner of the cube should be able to handle massive amounts of traffic.

The top right corner is where you should typically aim, although it also depends on the needs of your application's business requirements.

Here's what the 3 axises indicate:

  • X-axis: Creating cloned application instances
  • Y-axis: Decomposing into micro-services by functionality
  • Z-axis: Splitting using data ownership

Scaling the X-axis: Cloning

To scale along the x-axis means to clone your app. This means running the exact same application in another instance and then handling the traffic between the two instances instead of one.

For example, let's say you have a business or a server that provides advice to clients. As word gets out about your business, you'll be providing a lot of advice. This could lead to long lines and long wait times among your clients. How are you gonna deal with all this traffic?

Well, one option is to clone your booth. That is to open more advice booths, and split the traffic between them. Each booth provides the exact same service, and they each operate independently on a part of the traffic. We just scaled our advice booth across the x-axis. We can scale our application, even the monolith, by running separate instances of the same application and then splitting the traffic between those instances. We can scale further along the x-axis by running multiple instances of our application on multiple machines.

Let's take a look at how we can fork a node process into multiple instances.

Forking Processes using child_process.fork()

Node.js is designed to clone your application and then run it using multiple instances simultaneously. This process is called forking. Let's take a look at how we can fork our advice server.

Firstly, let's introduce the advice server. Our main server is located in app.js

App.js

You can start the server on port 3000 by typing node app.js 3000. Then, when you go to http://localhost:3000 from the browser, you will see the following screen.

Example server reply.png

Note: The process id will be different depending on the machine you are running.

Now, we only have one instance of one process running, we're only gonna be able to handle so much traffic. Hence the solution is to fork this process. Let's create a file called index.js and write the following code in it.

Fork child processes.png

Later, when we type node index.js in the terminal, we will see the following screen.

Forked.png

When you type http://localhost:3001, http://localhost:3002 and http://localhost:3003 respectively in the browser, you will see the following screen.

Note: As you can see, all instances have different process ids.

They're all running the same code. This is a small example of how we can use the fork method available to us within the child_process module. Here, we took our application, our app.js file, and forked it into three separate instances that use the same code. There is another module called cluster that we can use to fork our application into a pool of processes.

Cluster Module

In Node.js, a single application instance only uses one processor because Node.js is single-threaded. Forking your application into multiple instances is required to take full advantage of your hardware. A cluster is a group of node instances that all work together.

Let's create a cluster to take advantage of every CPU that is available to us. Change the content of index.js as follows.

Cluster fork.png

Later, when we type node index.js in the terminal, we will see the following screen.

Forked for cluster.png

Note: If you come over to the browser and hit http://localhost:3000, you can see that you're hitting the same worker process. That's because you're not getting enough traffic to really use the rest of the processes.

Load Testing with loadtest

So now we'll simulate some traffic and let's watch this traffic spread out.

Install loadtest using sudo npm install -g loadtest , then send 300 requests to http://localhost:3000 requests by typing loadtest -n 300 http://localhost:3000

Loadtest.png

Then, when you check the console logs, you can see logs like below.

Received requests

Note: You can notice that when we have 300 requests, we're actually spreading the load out across eight different processors and taking full advantage of our machine.

Zero Downtime Architecture

One of the advantages of running multiple processes is that your app never has to go down, it can always be available to your users, which is called zero downtime

Sometimes our apps go down, this could be due to some mysterious and obscure bug, high traffic or sometimes we just need to update the code and restart the process. In a cluster when a single instance fails, the traffic will use the remaining worker instances, and the main process can detect worker failures and automatically restart those workers.

When you need to update your app you no longer need to tell your customers that the website will be down due to maintenance. You can simply program your cluster to restart each instance with the updated code.

Let's edit index.js file as below.

zero downtime.png

Note: We added two different parts to the file. First, we added a cluster event that is fired when any worker exited and added a callback to start a new worker for this event. Second, we've checked the URL of incoming requests to the HTTP Server and added a part that kills the worker process if the URL is /kill

Every time when we try to kill a worker, one pops up in its place and no matter how many kill events I issue we still have our website running. So I can still browse http://localhost:3000. It never goes down, it always stays up. If we look at the logs here we can see that we've killed multiple worker processes but we've started a new one each time that worker has died, so this is pretty cool, this is called zero downtime.

Working with clusters with PM2

In the real world, there are already many tools that we can use to help us manage clusters in production. The tool I want to look at is called PM2. PM2 is a Node.js process manager. It will allow you to manage zero downtime clusters in production. Let's install PM2 in terminal by typing sudo npm install -g pm2

Do you remember our advice server in the app.js file above? We will run three instances of the app.js and node by typing pm2 start app.js -i 3

Clusters run using PM2.png

Note: After starting instances using PM2, You can actually clear and still use this terminal. PM2 runs in the background. When you use pm2 start app.js -i -1, PM2 will actually automatically select the number of instances that it should run for your current processor.

For more info, please check the documentation

Conclusion

We saw how to make scalability improvements in the x-axis relative to the scale cube. Thus, we will be able to serve more users. With these methods, you will have a little better understanding of the power of Node.js

Cites