In the java world, every processing by design is a thread (CPU thread) based on blocking operation with imperative codes. Thus the original approach for java web server is thread per request by following the servlet specification. Then rise the limitation of thread-based programming. CPU can only process that much thread at a time. Roughly around 10 thousand of threads.
In today’s business, the need to serve millions of throughput has become a critical requirement in some business applications connected through the internet. If we use the thread per request approach, the CPU needs to have a lot of threads active to be able to process the request. But cloud hosting normally provides 1-2 CPUs or even just a fraction of 1 CPU. Thus, asynchronous event-loop non-blocking web server was created to handle the need for concurrency with a small number of threads and scale with fewer hardware resources.
I always wanted to try building a java web service application using an asynchronous web server, but till now still had no luck, since at my workplace we don’t really serve that much traffic.
Therefore I did just a quick trial to compare the basic performance between requests per thread per request engine and an asynchronous web server. I use spring web MVC and spring webflux for easy and quick comparison. Spring web MVC is based on a servlet web server while webflux is based on asynchronous netty.
Below are the framework version, library, and tools I used for running the comparison :
- JDK : 18.0.1 64bit temurin eclipse adoptium
- org.springframework.boot:spring-boot-starter-parent version : 2.7.2
- io.gatling:gatling-maven-plugin version : 4.2.4
- io.gatling.highcharts:gatling-charts-highcharts : 3.8.3
- visualvm : 2.1.4
I run the test on a machine with Intel 12th Gen core i7 (20 CPU threads).
Project Structure
First, create a parent project to store the common library’s version.
<project ....>
<modelVersion>4.0.0</modelVersion>
<packaging>pom</packaging>
<modules>
<module>web</module>
<module>webflux</module>
<module>gatling</module>
</modules>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.7.2</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>org.example</groupId>
<artifactId>spring-web-vs-webflux</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<maven.compiler.source>18</maven.compiler.source>
<maven.compiler.target>18</maven.compiler.target>
<java.version>18</java.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
</dependencies>
</project>
Then we create a simple sub-project for spring web mvc :
<project ....>
<parent>
<artifactId>spring-web-vs-webflux</artifactId>
<groupId>org.example</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>web</artifactId>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<excludes>
<exclude>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
</project>
with basic main class :
package org.example.web;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
/**
* @author Bayu Utomo
* @date 16/8/2022 9:51 pm
*/
@SpringBootApplication
public class SpringWebApplication {
public static void main(String[] args) {
SpringApplication.run(SpringWebApplication.class, args);
}
}
And one endpoint :
package org.example.web;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
/**
* @author Bayu Utomo
* @date 16/8/2022 10:03 pm
*/
@RestController
public class MainController {
@GetMapping("/test")
public ResponseEntity<String> getPerson() {
return ResponseEntity.ok().body("Test OK!");
}
}
Next, we create another simple sub-project for spring webflux :
<project ....>
<parent>
<artifactId>spring-web-vs-webflux</artifactId>
<groupId>org.example</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>webflux</artifactId>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-webflux</artifactId>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<excludes>
<exclude>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
</project>
The main class :
package org.example.webflux;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
/**
* @author Bayu Utomo
* @date 16/8/2022 9:51 pm
*/
@SpringBootApplication
public class SpringWebfluxApplication {
public static void main(String[] args) {
SpringApplication.run(SpringWebfluxApplication.class, args);
}
}
The controller :
package org.example.webflux;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
/**
* @author Bayu Utomo
* @date 16/8/2022 10:03 pm
*/
@RestController
public class MainController {
@GetMapping("/test")
public ResponseEntity<String> getPerson() {
return ResponseEntity.ok().body("Test OK!");
}
}
Both webflux and web MVC will have all application properties to their default. As you can see, webflux provides the same annotation mechanism for controller routing without the need to implement Mono and the functional way of specifying the routing.
We don’t have JSON processing here since I want to isolate the comparison purely on the HTTP engine processing itself.
Last we create Gatling sub-project for the load test :
<project ....>
<parent>
<artifactId>spring-web-vs-webflux</artifactId>
<groupId>org.example</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>gatling</artifactId>
<dependencies>
<dependency>
<groupId>io.gatling.highcharts</groupId>
<artifactId>gatling-charts-highcharts</artifactId>
<version>3.8.3</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>io.gatling</groupId>
<artifactId>gatling-maven-plugin</artifactId>
<version>4.2.4</version>
<configuration>
<simulationClass>gatling.BasicSimulationTest</simulationClass>
</configuration>
</plugin>
</plugins>
</build>
</project>
And then we create one simulation class :
package gatling;
import io.gatling.javaapi.core.*;
import io.gatling.javaapi.core.loop.Repeat;
import io.gatling.javaapi.http.*;
import static io.gatling.javaapi.core.CoreDsl.*;
import static io.gatling.javaapi.http.HttpDsl.*;
/**
* @author Bayu Utomo
* @date 17/8/2022 9:44 am
*/
public class BasicSimulationTest extends Simulation {
HttpProtocolBuilder httpProtocol = http
.baseUrl("http://localhost:{givePortNumberAsNeeded}")
.acceptHeader("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8") // 6
.doNotTrackHeader("1")
.acceptLanguageHeader("en-US,en;q=0.5")
.acceptEncodingHeader("gzip, deflate")
.userAgentHeader("Mozilla/5.0 (Windows NT 5.1; rv:31.0) Gecko/20100101 Firefox/31.0");
ScenarioBuilder scn = scenario("BasicSimulation")
.repeat(100)
.on(exec(http("spring_{giveSpecificNameAsNeeded}")
.get("/test")
.check(status().is(200)))
);
{
setUp(
scn.injectOpen(atOnceUsers(10000))
).protocols(httpProtocol);
}
}
As you can see, we inject 10000 users. And each user will call the endpoint 100 times. Therefore we will have 1.000.000 HTTP requests. We not going to use any pause or sleep time or duration of execution here. Therefore we going to have the quickest request thrown to the server.
Running the Test
The execution test result can be seen below.
The spring web MVC :
The spring webflux :
The Gatling test was completed in a slightly longer time for webflux. But noted, it doesn’t mean webflux running slower.
As you can see from the table above, the web servlet could complete more requests (OK) as it has more threads (200 threads for spring default setting) to handle the incoming request. While the webflux able to handle less. But if you see the response time, webflux in general able to give a faster response time compared to web MVC. Because context switching in the CPU thread is slower compared to event-loop programming. Even with fewer threads, the gap is just 5% between web servlet and webflux. Also, all the KO was due to the connection being refused means the queue was full. The throughput is higher for web MVC due to more threads that can grab requests from the queue. But when the CPU is switching between threads, it causes a longer processing time to complete the request compared to the asynchronous webflux. Webflux has lower throughput due to incoming requests being immediately refused since the queue was full. Increase the queue and perhaps webflux could handle more throughput.
The startup time taken for both is roughly similar.
Spring web MVC startup time :
Spring webflux startup time :
Below is the graph showing JVM utilization for spring web MVC :
Below is the graph showing JVM utilization for spring webflux :
From the JVM utilization table above, webflux has pretty stable CPU spikes compared to spring web MVC. Webflux also has less used heap size compared to spring web MVC.
Webflux also only uses way fewer threads (20 CPUs thread * 2 = 40) compared to web MVC (219).
For web servlet MVC, a lot of NIO thread created for processing the request :
Webflux uses a minimal thread for request processing :
Repeating the Tests
I repeat the test execution for the second time with the same specification as the first test. And the third time by reducing the number of threads for web servlet MVC to roughly the same amount of threads with webflux. Below is the summary of the results :
Overall webflux was able to handle almost the same throughput with fewer resources needed compared to web servlet. Response time for webflux was also faster than web-servlet. If webflux was configured with a bigger queue size it might be able to reduce the number of connections refused.
The third test shows if we have a machine with a high CPU number (remember I use 12th gen intel), having fewer threads will give better performance to the web servlet. But also will require more heap memory since more requests will be stored in the queue. But on cloud hosting, CPU is very limited. So having fewer threads for the web servlet will not necessarily mean faster or better than webflux (asynchronous web server). A further test needs to be conducted for this.
From the third test, the number of NIO threads was reduced :
Summary
Overall, using an asynchronous web server does not necessarily mean faster/higher throughput than RPS/servlet-based web server. It does mean being able to handle roughly the same amount of throughput with fewer resources needed.
TechEmpower has better benchmark tests and they show that other asynchronous web servers such as vert.x have way better/higher throughput performance. It just that for spring webflux, even though it is built on top of netty (the same with vert.x), but performance seems lower than what normally an asynchronous web server could do.
As you can see, spring-tomcat is faster than spring webflux, tho not that much.
But asynchronous web server has a catch. The programming style is no longer imperative as what java is designed for. It becomes difficult to debug the async code. Let’s wait until Java Loom become officially adopted by many TPR-based web server and we will see again the benchmark result.
