Introduction
Let's say you have a new project idea in your mind which will go viral once it is published. However, you couldn't decide which backend framework to use. Well, in this article, we will test 3 popular Node.js frameworks (Express.js, Koa.js, Fastify) and compare them regarding their response performances. So, you can decide which one to use. We will first set our backend servers with 2 simple endpoints. The first endpoint will return an empty response. The second one will calculate the Fibonacci Series depending on the input that we will provide and then return the sequence. The configuration files used in this blog are available in this repository. Before benchmarking, let's dig into our frameworks a bit more in detail.
Express.js
Express.js is one of the most famous Node.js based web frameworks. It makes the development process easier compared to Node.js without losing its features. Among these 3 frameworks, Express.js is the oldest one. It takes advantage of being the oldest and being widely used with its large community. Express.js has a large bundle of features to support. It uses middleware-based architecture which provides modularity, reusability, flexibility, and ease of testing. With middleware-based architecture, Express.js can be used as a plug-in system and can expand the range of features. However, besides that good plug-in system, Express.js' asynchronous system relies on a callback-based system. This might lead to callback hell.
Koa.js
Koa.js is actually developed by the same team which developed Express.js. They call it the next-level Node.js framework. It is more modern, minimal, flexible, and expressive with a more robust foundation for backend services. The development team aimed to address the limitations of Express.js and enhance its strengths. Compared to Express.js, Koa.js eliminates callbacks and uses generators to handle asynchronous calls. Generators were a key component in earlier versions of Koa.js. It provided a more efficient and elegant way of asynchronous handling. This gives the ability to developers to write cleaner and more maintainable code. However, the recent versions of Koa.js switched to async/await, which is an even more modern and concise way to handle asynchronous operations. Compared to the other 2 frameworks, Koa.js uses “The context object” called “ctx” to handle its request and response objects at once.
Fastify
Fastify’s primary target is performance by optimizing various aspects of the application. It is the youngest among the other frameworks. It also uses middleware-based architecture and supports middlewares like Express.js and Koa.js which helps with providing modularity and allows developers to plug in different functionalities just like Express.js. However, in Fastify’s case, this architecture is designed to minimize overheads and maximize speed. The same aspect about minimizing overheads is followed with routing as well. The routing system in Fastify is created with low overhead and this allows applications to handle a large number of requests. For developers who want to code with TypeScript, Fastify has TypeScript support out of the box. It is important to mention that Fastify supports asynchronous handling without sacrificing its performance as well.
Importance of performance benchmarking in web frameworks
Performance benchmarking of web frameworks is a crucial aspect of evaluating and improving your system. They can have a direct impact on the overall user experience. Web frameworks try to optimize and improve user experience, scalability, and resource usage. However, it might still not be enough. So, benchmarking gives a view about evaluating and improving web frameworks. It helps developers make informed decisions, optimize resource utilization, and enhance the overall user experience. Your web systems can work well with light loads but what happens if thousands of people start using your website and the load increases? So, selecting the right web framework is crucial here.
Test Bed
For our testing environment we started Ubuntu 22.04 with 2 CPUs, and 4GB Memory (c5.large) on AWS EC2.
We will use Ddosify’s open-source load engine to start a load test. Ddosify Engine is a performance testing tool that focuses on backend load testing. The engine is written in Golang and it is fully open-source. You can use it on your terminal. You can set the number of requests, the content (body, header), the duration of the test, the load type (such as linear, incremental or waved). It also supports scenario based load tests using a simple JSON file, dynamic variables (parameterization), assertion, test data (CSV). Check our GitHub repository for more info.
Test Setup
As we mentioned before, there are 2 endpoints in our test system. The main purpose of the first endpoint which has an empty response is to test the frameworks’ performance without any computation. With the second endpoint, where we calculate the Fibonacci sequence, we want to test and see if there is a performance gap between the frameworks with some extra computations.
Calculating Fibonacci Numbers
We will use the function below to calculate Fibonacci for every framework.
function fibonacciSequence(num) {
let result = [];
let num1 = 0;
let num2 = 1;
let nextTerm;
for (let i = 1; i <= num; i++) {
result.push(num1);
nextTerm = num1 + num2;
num1 = num2;
num2 = nextTerm;
}
return result;
}
Express.js
const express = require("express");
const app = express();
const port = 8000;
app.get("/", (req, res) => {
res.end();
});
app.get("/:number", (req, res) => {
const num = req.params.number;
res.send({ result: fibonacciSequence(num) });
});
app.listen(port, "0.0.0.0", () => {
console.log(`Express server listening at ${port}`);
});
Koa.js
const Koa = require("koa");
const Router = require("koa-router");
const app = new Koa();
const router = new Router();
const port = 8000;
router.get("/", async (ctx) => {
ctx.body = "";
});
router.get("/:number", async (ctx) => {
const num = ctx.params.number;
ctx.body = fibonacciSequence(num);
});
app.use(router.routes()).use(router.allowedMethods());
app.listen(port, "0.0.0.0", () => {
console.log(`Koa server listening at ${port}`);
});
Fastify
const fastify = require("fastify")();
const port = 8000;
fastify.get("/", function (request, reply) {
return "";
});
fastify.get("/:number", (request, reply) => {
const num = request.params.number;
return fibonacciSequence(num);
});
fastify.listen({ port: port, host: "0.0.0.0" }, (err, address) => {
if (err) {
fastify.log.error(err);
process.exit(1);
}
fastify.log.info(`Fastify server listening at ${address}`);
});
Even though the syntax between these three frameworks are pretty similar. There are some minor differences. For example, with Express.js, we send the response with res.end or res.send functions whereas with Koa.js we use the context object, and assigning a value to its body returns the response. With Fastify, we can return the response directly like with a normal function.
Test Configs for Ddosify Engine
For test purposes, we will hit the two endpoints mentioned before (empty response and Fibonacci Series) with each framework. Then, we will load on our systems with 10, 100, and 1000 requests per second by using Ddosify engine. Ddosify engine includes different parameters for different test cases. We will create a basic test using:
ddosify -t http://localhost:8000/ -d <Duration> -n <IterationNum>
Since our test case includes 10, 100, and 1000 requests per second. We will replace duration with 10 (seconds) and IterationNum with 10 * request number. So, we can get the desired cases for our empty endpoint.
ddosify -t http://localhost:8000/ -d 10 -n 10000
For the second endpoint, we will use Ddosify engine’s random number feature which produces numbers between 1 and 1000. So, we can create Fibonacci series with random numbers and compare the results.
ddosify -t http://localhost:8000/{{_randomInt}} -d 10 -n 10000
Ddosify engine includes different flags and parameterization options as well. You can check it out from our GitHub repository. By using Ddosify engine, we are also able to see the average response time of our requests as well as their success rates. It also shows the time spent on different sections of the process such as connection, request write, server processing, and so on. It also shows the response status code for a better explanation, especially for errors. As an example, we run Ddosify engine on one of our endpoints with the given CLI command and got the stdout below. We will run the other tests similarly and share the results with you in next the next section.
ddosify -t http://localhost:8000/{{_randomInt}} -d 10 -n 10000
Analysis and Results
Results for Endpoint 1
Table 1- Average response times for empty endpoint.
As we can see from the table above, with empty response endpoints the difference between these 3 frameworks is considerably small. As the request number goes high from 10 to 1000, they all slow down by around ~18%. It might not be an accurate testing method here, since most of the time the server endpoints will make some calculations and then return the result. However, in case you have an endpoint that returns the answer directly without any process, all of these 3 web frameworks should perform well under given circumstances. It is worth mentioning that the response success rate is 100% for all of the frameworks.
The success rate is calculated depending on the responses from the server. If the assertion feature is not being used and the server returns a response, regardless of the status_code, it counts as “Success”. However, if the request doesn’t get any response and gets timeout, then it is marked as “Fail”. When we look at the table below, we can see that the success rate of Express.js at 1000 requests per second is 84%. This means around 160 requests out of 1000 got a timeout error.
Results for Endpoint 2
Table 2 - Average response times for fibonacci sequence endpoint.
Table 3 - Success rates of fibonacci sequence endpoint.
When we calculated the Fibonacci sequence, even though it didn’t make much of a difference with 10 requests per second, we can clearly see that it started going up with 100 requests per second compared to the empty endpoint. At 1000 requests per second, there is a high margin on the average response time compared to previous results. Up to 100 requests, again all three frameworks performed around the same. However, once the requests reached 1000 requests per second, we can see that Express.js performs ~10% better on response time. It might not be correct to consider only response times here because as the number of requests goes high, the success rate drops down. We can see that Express.js has the highest success rate here. It is then followed by Fastify and Koa.js.
To sum up, according to the results from our tests, Express.js performs relatively the best in regard to response times and success rates. This might be a result of being in the marketplace for the longest time and thus being optimized better for our test cases. However, with more complex projects and using middleware agents with callback functions, the performance of Express.js may decrease. The other two frameworks could handle these kinds of scenarios better than Express.js since they are more modernized frameworks. You should also keep in mind that these results might vary in different systems. Our test system is comparably small and starts failing with 1000 requests per second. Our final words; these three frameworks all perform well, with Express.js performing the best, and can definitely be considered as an option for your project. To further enhance your performance testing capabilities, sign up for the Ddosify Cloud platform, which offers a global-scale, no-code solution for accurate and seamless performance testing.
Share on social media: