Introduction
Let's start with an exciting use case. You have implemented your big idea and created a fascinating website or mobile app, and you are ready to tell the world about how cool your app is. Everything went well, everyone wants to use your app. Then suddenly your backend starts to reject new registers because it receives more requests than it can handle. First of all, good for you, it looks like you built a product that people want to use. Secondly, shame on you, because you didn't test the maximum concurrent requests capacity of the backend.
This is an extreme use case for demonstrating the importance of load testing before going to production. Your requirements would be different, maybe your company will start a marketing campaign at a specific date and you want to be sure that the system is able to handle X amount of concurrent new registers. This article shows how to simulate user authentication flow at a given concurrency, and eventually, you will be able to find the upper limit of your backend API.
Test Scenario
The user scenario consists of 3 sequential actions,
- The user registers the system with its email, username, and password
- The user logins the system with its username and password
- The user sends a GET request to fetch its private account information
The first two actions are a common flow for all apps in the world. We added the third action to demonstrate authenticated user behavior. In our use case, the user fetches it is account information once they logged in.
The Environment
We will use the Ddosify test API as a backend service. It has all the endpoints that we need.
POST /account/registerendpoint waits for email, username, and password and returns HTTP 201 Created.POST /account/loginendpoint accepts username and password and checks if there is a user with these credentials, then sends HTTP 200 OK along with a JWT Token.GET /account/userendpoint expects an Authorization header key with a valid access token value to respond private account information of the user.
We will use Ddosify Open Source Load Engine to create and run this test scenario at high IPS (Iteration Per Second). Ddosify can be installed on almost any operating system and via Docker as well. Check out the installation section to find the proper installation method for your host machine. We'll go with brew since we are running these tests on macOS.
Install Ddosify with brew
brew install ddosify/tap/ddosify
Test the installation by passing the -version flag
$ ddosify -version
Version: v0.13.0
Git commit: 5fca361
Built 2023-01-26T12:49:12Z
Go version: go1.18.10
OS/Arch: darwin/arm64
If you see the version number and other details about the version, then you are free to go. If you face any issue you can use Ddosify Discord Channel, this is the fastest way to resolve any problem related to Ddosify.
Inspecting the Endpoints
Before writing a complex test scenario, it is always a good idea to split it into smaller pieces. We have 3 endpoints we would like to hit sequentially. Let's send one request to these endpoints and inspect what we receive. We'll use the --debug flag to inspect request headers, request body, response headers, and response body. Since Ddosify sends only 1 request in debug mode, we don't need to worry about the iteration_count and duration parameters right now.
Inspect Register Endpoint
$ ddosify -t https://testserver.ddosify.com/account/register/ -m POST --debug
⚙️ Initializing...
🐛 Running in debug mode, 1 iteration will be played...
🔥 Engine fired.
🛑 CTRL+C to gracefully stop.
STEP (1)
-------------------------------------
- Environment Variables
- Request
Target: https://testserver.ddosify.com/account/register/
Method: POST
Headers:
Body:
- Response
StatusCode: 400
Headers:
Content-Type: application/json
Connection: keep-alive
Cross-Origin-Opener-Policy: same-origin
X-Frame-Options: DENY
X-Content-Type-Options: nosniff
Server: nginx/1.23.3
Referrer-Policy: same-origin
Date: Thu, 12 Jan 2023 19:26:45 GMT
Strict-Transport-Security: max-age=31536000
Content-Length: 115
Vary: Accept
Allow: POST, OPTIONS
Body:
{
"email": [
"This field is required."
],
"password": [
"This field is required."
],
"username": [
"This field is required."
]
}
When sending the request, we passed the -m flag with the POST value to pass the proper HTTP Method for this request. As shown on the Debug result, the server returned the HTTP 400 status code and a detailed message on the body. Obviously, we need to send the necessary user information (username, email, password) on the request body to successfully create a new record. But it is a good way to show how Debug mode works on Ddosify. We can easily pass request payload with the -b flag but there is another method to achieve this. Ddosify supports configuration file as an input to let you build and pass complex test scenarios. Since we'll have 3 actions work sequentially, it is a good idea to start creating our test scenario config file.
{
"debug": true,
"steps": [
{
"id": 1,
"name": "Register",
"url": "https://testserver.ddosify.com/account/register/",
"method": "POST",
"headers": {
"Content-Type": "application/json"
},
"payload_file": "./register_payload.json"
}
]
}
{
"username": "test_user_name",
"email": "test@dummy.com",
"password": "testpassword"
}
We have created two files register.json and register_payload.json. The first file is the config file that contains our test scenario logic. The second file contains the JSON payload of the Register request. We bound this payload file to the Register request with the payload_file key on register.json. We have also added the Content-Type: application/json header to the request since we are sending a JSON payload. Let's run our scenario and see what happens.
$ ddosify -config register.json
<.. truncated ..>
- Request
Target: https://testserver.ddosify.com/account/register/
Method: POST
Headers:
Content-Type: application/json
Body:
{
"email": "test@dummy.com",
"name": "Test User",
"password": "testpassword",
"username": "test_user_name"
}
- Response
StatusCode: 201
Headers:
X-Content-Type-Options: nosniff
<.. truncated ..>
Body:
And that's it! As shown on the Debug result we've successfully created a new user. The Register part of our test scenario is half-ready. To make it fully ready we have to send a different email, username, and password for each iteration. We'll come back to it later, for now, let's create the other user actions.
If you are using the docker version of Ddosify, you need to bind the configuration files to the docker container. Assume that register.json is located under current directory ($PWD). We must bind the current directory path ($PWD) to Docker container's /ddosify_config path.
Bind the current directory and start the Ddosify container:
$ docker run -it --rm -v $PWD:/ddosify_config ddosify/ddosify:v0.11.0
Run Ddosify inside the container
$ cd /ddosify_config/
$ ddosify -config register.json
Inspect Login Endpoint
{
"debug": true,
"steps": [
{
"id": 1,
"name": "Login",
"url": "https://testserver.ddosify.com/account/login/",
"method": "POST",
"headers": {
"Content-Type": "application/json"
},
"payload_file": "./login_payload.json"
}
]
}
{
"username": "test_user_name",
"password": "testpassword"
}
$ ddosify -config login.json
<.. truncated ..>
- Request
Target: https://testserver.ddosify.com/account/login/
Method: POST
Headers:
Content-Type: application/json
Body:
{
"password": "testpassword",
"username": "test_user_name"
}
- Response
StatusCode: 200
Headers:
X-Frame-Options: DENY
<.. truncated ..>
Body:
{
"tokens": {
"access": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ0b2tlbl90eXBlIjoiYWNjZXNzIiwiZXhwIjoxNjczNTgyNzgyLCJpYXQiOjE2NzMzODIwODIsImp0aSI6IjhiNjFmNjgyZDcxZjRiYmViMjBlZDc3NmQ5N2IwZjM2IiwidXNlcl9pZCI6ImI5ZjI0MjNhLWYyYTgtNGMwZC1iZjZmLWY0ODFlMGUyYmJlZSJ9.9lN55gfCAUka37gGOK-IygDgCccavRl_db77LPOvqaQ",
"refresh": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ0b2tlbl90eXBlIjoicmVmcmVzaCIsImV4cCI6MTY3NDI0NjA4MiwiaWF0IjoxNjczMzgyMDgyLCJqdGkiOiIyNjk3NjAxZDNmNDU0NzdjYjBjZTcwOGNjZjk5OTBhZSIsInVzZXJfaWQiOiJiOWYyNDIzYS1mMmE4LTRjMGQtYmY2Zi1mNDgxZTBlMmJiZWUifQ.IqZUEamOPzEqDZC_6-KdRxR4oOk8v5NM-DA4dTuCu94"
},
}
We have created a new Ddosify configuration file called login.json which includes the request configuration of the Login action along with the login payload file reference. We sent the user's credentials (username and password) that we created previously on Register action. As shown in the Debug result, the server responded with an HTTP 200 OK status code and the response body contains the access token which we can use to send requests to the endpoints that require authentication.
Inspect User Detail Endpoint
{
"debug": true,
"steps": [
{
"id": 1,
"name": "User Detail",
"url": "https://testserver.ddosify.com/account/user/",
"method": "GET",
"headers": {
"Authorization": "Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ0b2tlbl90eXBlIjoiYWNjZXNzIiwiZXhwIjoxNjczNTgzMDg2LCJpYXQiOjE2NzMzODIzODYsImp0aSI6IjFmYWFlMzM5YTYyMjQ2ZGRhNTk1MTZlN2NkYzMyMWU4IiwidXNlcl9pZCI6ImI5ZjI0MjNhLWYyYTgtNGMwZC1iZjZmLWY0ODFlMGUyYmJlZSJ9.X_x63UOFSbz7s8jiR44YUZjPVXRAcpKSLfO8skLCmG0"
}
}
]
}
$ ddosify -config account.json
<.. truncated ..>
- Request
Target: https://testserver.ddosify.com/account/user/
Method: GET
Headers:
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ0b2tlbl90eXBlIjoiYWNjZXNzIiwiZXhwIjoxNjczNTgzMDg2LCJpYXQiOjE2NzMzODIzODYsImp0aSI6IjFmYWFlMzM5YTYyMjQ2ZGRhNTk1MTZlN2NkYzMyMWU4IiwidXNlcl9pZCI6ImI5ZjI0MjNhLWYyYTgtNGMwZC1iZjZmLWY0ODFlMGUyYmJlZSJ9.X_x63UOFSbz7s8jiR44YUZjPVXRAcpKSLfO8skLCmG0
Body:
- Response
StatusCode: 200
Headers:
X-Content-Type-Options: nosniff
<.. truncated ..>
Body:
{
"email": "test@dummy.com",
"id": "b9f2423a-f2a8-4c0d-bf6f-f481e0e2bbee",
"username": "test_user_name"
}
Just like the previous steps, we created a new configuration file (account.json) which injects the access token (received in the Login step) into the Authorization header. The Debug result shows that we have successfully fetched the private user data from the endpoint that requires authentication.
Creating Authentication Scenario File
In the previous sections, we successfully hit the endpoints with different Ddosify configuration files. Now we will combine all these configurations into one Ddosify config file to create a realistic user flow. To achieve this we should consider below requirements:
- On every iteration (new user) we have to pass different email, username, and password
- User's email, username, and password should be the same on all steps in an iteration.
- We should capture the authentication token returned on the Login request. Then we need to use this token on the next request which is user data fetching.
- There should be a wait time between requests to simulate real user behavior.
For the first requirement, we can use dynamic variables to create random data. To use the same randomly generated data on all the steps we can assign them to environment variables. Since the environment variable is created at the beginning of the iteration, random variables will be created for each iteration at once and can be accessible by scenario steps.
Our initial environment looks like as below:
"env": {
"username": "{{_randomUserName}}",
"email": "{{_randomEmail}}",
"password": "{{_randomPassword}}"
}
To capture the access token on the Login action we can use the correlation feature of Ddosify. This feature allows us to extract values from the response body or response header. We can assign the extracted value to a new variable and then we can use this variable as an environment variable in the following steps. In our scenario, the response of the Login endpoint is in JSON format. So we can use the json_path to extract the access token from the response body.
"capture_env": {
"access_token": {"from": "body", "json_path": "tokens.access"}
}
The only requirement we have to handle is the think time between the requests to simulate real user behavior. This is the easiest one since we can just add the sleep property for the first two steps to add think time after finishing that step. Sleep property can be an exact digit in milliseconds or can be a range to add a random wait time between the given range. In our example, we would like to add think time between 1 second to 2.5 seconds.
Here are the final configuration file and new payload files that satisfy all requirements.
{
"iteration_count": 50,
"duration": 10,
"load_type": "linear",
"debug": true,
"env": {
"baseUrl": "https://testserver.ddosify.com/account",
"username": "{{_randomUserName}}",
"email": "{{_randomEmail}}",
"password": "{{_randomPassword}}"
},
"steps": [
{
"id": 1,
"name": "Register",
"url": "{{baseUrl}}/register/",
"method": "POST",
"headers": {
"Content-Type": "application/json"
},
"payload_file": "./register_payload.json",
"sleep": "1000-2500"
},
{
"id": 2,
"name": "Login",
"url": "{{baseUrl}}/login/",
"method": "POST",
"headers": {
"Content-Type": "application/json"
},
"payload_file": "./login_payload.json",
"sleep": "1000-2500",
"capture_env": {
"access_token": { "from": "body", "json_path": "tokens.access" }
}
},
{
"id": 3,
"name": "Account",
"url": "{{baseUrl}}/user/",
"method": "GET",
"sleep": "1000-2500",
"headers": {
"Authorization": "Bearer {{access_token}}"
}
}
]
}
As the above configuration file shows, we created random user data with the dynamic variable feature, and we have assigned these variables to environment variables to use the same generated data on the scenario steps. To capture the access token we used capture_env feature on the Login step then we used the captured variable on the header section of the Account step. Lastly, we have used sleep time after the Register step and Login step to create a more realistic user simulation.
{
"username": "{{username}}",
"email": "{{email}}",
"password": "{{password}}"
}
{
"username": "{{username}}",
"password": "{{password}}"
}
Instead of using static string values for user data, we changed payload files to use the data generated in the environment section of the auth_flow.json test configuration file.
As the debug option is still enabled, let's run the whole scenario in Debug mode and test our final Ddosify configuration file.
$ ddosify -config auth_flow.json
<.. truncated ..>
STEP (1) Register
-------------------------------------
- Environment Variables
password: faqkebowz
baseUrl: https://testserver.ddosify.com/account
username: Connor.Kuhlman
email: sarah.thomas@hotmail.com
- Request
Target: https://testserver.ddosify.com/account/register/
Method: POST
Headers:
Content-Type: application/json
Body:
{
"email": "sarah.thomas@hotmail.com",
"password": "faqkebowz",
"username": "Connor.Kuhlman"
}
- Response
StatusCode: 201
Headers:
<.. truncated ..>
Body:
STEP (2) Login
-------------------------------------
- Environment Variables
email: sarah.thomas@hotmail.com
password: faqkebowz
baseUrl: https://testserver.ddosify.com/account
username: Connor.Kuhlman
- Request
Target: https://testserver.ddosify.com/account/login/
Method: POST
Headers:
Content-Type: application/json
Body:
{
"password": "faqkebowz",
"username": "Connor.Kuhlman"
}
- Response
StatusCode: 200
Headers:
<.. truncated ..>
Body:
{
"tokens": {
"access": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ0b2tlbl90eXBlIjoiYWNjZXNzIiwiZXhwIjoxNjczNzM1MzE0LCJpYXQiOjE2NzM1MzQ2MTQsImp0aSI6IjhkYjBlZDY2YzJhNzQ4ZWY4ZDk1ZjhiMzI3OGI3OTExIiwidXNlcl9pZCI6IjgyZWM5YmFiLWEzODUtNDQ3MS04YjRlLTViM2MwMGExOTU4ZiJ9.q4SOCIn8i-6JTP51h9Jm3VXtI4YCT_yOn9cHRXhtsHw",
"refresh": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ0b2tlbl90eXBlIjoicmVmcmVzaCIsImV4cCI6MTY3NDM5ODYxNCwiaWF0IjoxNjczNTM0NjE0LCJqdGkiOiJmYjIzYzY0YTc5NmY0NTQzYjU5YTA5NmU2ZGVlZTU0OCIsInVzZXJfaWQiOiI4MmVjOWJhYi1hMzg1LTQ0NzEtOGI0ZS01YjNjMDBhMTk1OGYifQ.XpT6wdI9D-wyAhyLGRGMtEhgtacKfB6U_2BfOuQF2dg"
}
}
STEP (3) Account
-------------------------------------
- Environment Variables
baseUrl: https://testserver.ddosify.com/account
username: Connor.Kuhlman
email: sarah.thomas@hotmail.com
password: faqkebowz
access_token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ0b2tlbl90eXBlIjoiYWNjZXNzIiwiZXhwIjoxNjczNzM1MzE0LCJpYXQiOjE2NzM1MzQ2MTQsImp0aSI6IjhkYjBlZDY2YzJhNzQ4ZWY4ZDk1ZjhiMzI3OGI3OTExIiwidXNlcl9pZCI6IjgyZWM5YmFiLWEzODUtNDQ3MS04YjRlLTViM2MwMGExOTU4ZiJ9.q4SOCIn8i-6JTP51h9Jm3VXtI4YCT_yOn9cHRXhtsHw
- Request
Target: https://testserver.ddosify.com/account/user/
Method: GET
Headers:
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ0b2tlbl90eXBlIjoiYWNjZXNzIiwiZXhwIjoxNjczNzM1MzE0LCJpYXQiOjE2NzM1MzQ2MTQsImp0aSI6IjhkYjBlZDY2YzJhNzQ4ZWY4ZDk1ZjhiMzI3OGI3OTExIiwidXNlcl9pZCI6IjgyZWM5YmFiLWEzODUtNDQ3MS04YjRlLTViM2MwMGExOTU4ZiJ9.q4SOCIn8i-6JTP51h9Jm3VXtI4YCT_yOn9cHRXhtsHw
Body:
- Response
StatusCode: 200
Headers:
<.. truncated ..>
Body:
{
"email": "sarah.thomas@hotmail.com",
"id": "82ec9bab-a385-4471-8b4e-5b3c00a1958f",
"username": "Connor.Kuhlman"
}
As shown in the Debug result, the Environment Variables sections are filled with the data generated by dynamic variables. Also, the Account step has the access_token variable which is captured in the Login step. We achieved to create a Ddosify configuration file to simulate user authentication flow. The current configuration says to Ddosify engine run the scenario only 1 time. To start the load test, we have to remove the debug: true field on the auth_flow.json file. Once it is removed, Ddosify respects the iteration_count and duration values to send requests according to the given load_type. For our file, this scenario will be iterated 50 times over 10 seconds. Since the load_type is "linear" Ddosify will run 5 iterations per second. This means our backend server will receive 5 Register -> Login -> Fetch flow per second.
Here is the result of our load test
$ ddosify -config auth_flow.json
⚙️ Initializing...
🔥 Engine fired.
🛑 CTRL+C to gracefully stop.
✔️ Successful Run: 0 0% ❌ Failed Run: 0 0% ⏱️ Avg. Duration: 0.00000s
✔️ Successful Run: 0 0% ❌ Failed Run: 0 0% ⏱️ Avg. Duration: 0.00000s
✔️ Successful Run: 0 0% ❌ Failed Run: 0 0% ⏱️ Avg. Duration: 0.00000s
✔️ Successful Run: 0 0% ❌ Failed Run: 0 0% ⏱️ Avg. Duration: 0.00000s
✔️ Successful Run: 0 0% ❌ Failed Run: 0 0% ⏱️ Avg. Duration: 0.00000s
✔️ Successful Run: 0 0% ❌ Failed Run: 0 0% ⏱️ Avg. Duration: 0.00000s
✔️ Successful Run: 1 100% ❌ Failed Run: 0 0% ⏱️ Avg. Duration: 5.28926s
✔️ Successful Run: 3 100% ❌ Failed Run: 0 0% ⏱️ Avg. Duration: 5.68370s
✔️ Successful Run: 6 100% ❌ Failed Run: 0 0% ⏱️ Avg. Duration: 6.57709s
✔️ Successful Run: 8 100% ❌ Failed Run: 0 0% ⏱️ Avg. Duration: 6.84142s
✔️ Successful Run: 12 100% ❌ Failed Run: 0 0% ⏱️ Avg. Duration: 7.58740s
✔️ Successful Run: 18 100% ❌ Failed Run: 0 0% ⏱️ Avg. Duration: 8.33634s
✔️ Successful Run: 18 100% ❌ Failed Run: 0 0% ⏱️ Avg. Duration: 8.33634s
✔️ Successful Run: 20 90% ❌ Failed Run: 2 10% ⏱️ Avg. Duration: 8.59492s
✔️ Successful Run: 28 68% ❌ Failed Run: 13 32% ⏱️ Avg. Duration: 8.92337s
✔️ Successful Run: 30 63% ❌ Failed Run: 17 37% ⏱️ Avg. Duration: 8.96106s
✔️ Successful Run: 30 60% ❌ Failed Run: 20 40% ⏱️ Avg. Duration: 8.96106s
RESULT
-------------------------------------
1. Register
---------------------------------
Success Count: 41 (82%)
Failed Count: 9 (18%)
Durations (Avg):
DNS :0.0431s
Connection :0.0501s
Request Write :0.0001s
Server Processing :2.3935s
Response Read :0.0001s
Total :2.5485s
Status Code (Message) :Count
201 (Created) :29
400 (Bad Request) :12
Error Distribution (Count:Reason):
9 :connection timeout
2. Login
---------------------------------
Success Count: 38 (76%)
Failed Count: 12 (24%)
Durations (Avg):
DNS :0.0599s
Connection :0.0536s
Request Write :0.0001s
Server Processing :3.8256s
Response Read :0.0001s
Total :4.0416s
Status Code (Message) :Count
200 (OK) :28
400 (Bad Request) :10
Error Distribution (Count:Reason):
12 :connection timeout
3. Account
---------------------------------
Success Count: 50 (100%)
Failed Count: 0 (0%)
Durations (Avg):
DNS :0.0096s
Connection :0.0174s
Request Write :0.0001s
Server Processing :2.2237s
Response Read :0.0001s
Total :2.2734s
Status Code (Message) :Count
200 (OK) :28
401 (Unauthorized) :22
We can easily analyze the response times, status codes and success/fail criteria for each step. Ddosify marks the requests as failed if it doesn't receive the response or if it can't send the request. For example, the Login step result states that 9 requests have timed out, that why Ddosify marks 9 requests as Failed. Returned status codes do not have an impact on success/fail logic. Also, we have received HTTP 400 (Bad Request) for the Register endpoint on some requests because we sent existing user data for these requests and the server doesn't let us create new users.
✔️ Successful Run: 30 60% ❌ Failed Run: 20 40% ⏱️ Avg. Duration: 8.96106s
The last line of real-time messages states the result of our test from an iteration perspective. Ddosify marks the whole Iteration (Run) as failed if any of the steps failed. Otherwise (if all the steps are completed successfully) Ddosify marks the Iteration as Successful. Avg. Duration shows the sum of all the average duration of steps. In our example, the average response time for the Login step is 2.5 seconds, the Register step is 4 seconds, and the Account step is 2.27 seconds. The total value is equal to the Avg. Duration value. It is also easy to say that the Register step is the most time-consuming action on this user flow.
Conclusion
We demonstrated how to create a Ddosify test configuration file to test the performance of a backend server, including the private API endpoints that can be accessible via authentication. Through this article we have used different features of Ddosify Open Source Load Engine, like correlation, environment variables, debug mode, etc. There are other concepts that would help you to create more complex scenarios. Take a look at the Ddosify Readme on Github to learn details about them.
You can find the final Ddosify configuration files at Ddosify Blog Examples repository. If you need assistance, you can join our Discord channel.
Share on social media: