In this article, I will demonstrate how to write a database application that is 10 times smaller than normal(*). The storage requirement will be about 32 MB instead of the usual(*) ~300 MB taking both the application, third-party libraries and the Java runtime into account. As a bonus, the required RAM to run the application will also be reduced by 25%.
(*) These are the storage requirements for the following full JDKs (excluding the application and third-party libs):
jdk.8.0_191 360 MB jdk-9.0.4 504 MB adoptopenjdk-11 298 MB
Using an ORM that Supports Microservices
Most traditional ORMs do not honor Java module encapsulation. Often, this entails shipping off a lot of unnecessary code.In this article, I will use the open-source Stream-based Java ORM Speedment, which, in its latest version, supports the Java Platform Module System (JPMS). This enables us to generate an optimized custom Java Runtime Environment (JRE, the parts from the JDK that is needed to run applications) with only the modules explicitly used by our application.
Read about the new features of Speedment 3.2 in this article.
The Application
The entire application we wish to deploy in this article resides as an open-source project on GitHub under the sub-directory “microservice-jlink”. It connects to a public instance of a MySQL “Sakila” database (containing data about films) hosted in the cloud and lists the ten longest films that are rated “PG-13” on the console. The data model is preconfigured to fit the data structure of this database. If you want to create your own application using another database, visit the Speedment initializer to configure a project for that database specifically.The
main method of the application looks like this:public final class Main {
public static void main(String[] args) {
final Speedment app = new SakilaApplicationBuilder()
.withPassword("sakila")
.build();
final FilmManager films = app.getOrThrow(FilmManager.class);
System.out.println("These are the ten longest films rated as PG-13:");
films.stream() // 1
.filter(Film.RATING.equal("PG-13")) // 2
.sorted(Film.LENGTH.reversed()) // 3
.limit(10) // 4
.map(film -> String.format( // 5
"%-18s %d min",
film.getTitle(),
film.getLength().orElse(0))
)
.forEach(System.out::println); // 6
}
}
First, we pass the database password to the Speedment builder (Speedment never stores passwords internally). The builder is pre-configured with the database IP-address, port, etc. from a configuration file.Then, we obtain the
FilmManager which later can be used to create Java Streams that corresponds directly to the “film” table in the database.At the end, we:
- Create a
Streamof theFilmentities - Filter out
Filmentities that have a rating equal to “PG-13” - Sorts the remaining films in reversed length order (longest first)
- Limits the stream to the first 10 films
- Maps each film entity to a
Stringwith film title and film length - Prints each
Stringto the console
The application itself is very easy to understand. It shall also be noted that Speedment will render the Java Stream to SQL under the hood as shown hereunder:
SELECT
`film_id`,`title`,`description`,`release_year`,
`language_id`,`original_language_id`,`rental_duration`,`rental_rate`,
`length`,`replacement_cost`,`rating`,`special_features`,`last_update`
FROM `sakila`.`film`
WHERE (`rating` = ? COLLATE utf8_bin)
ORDER BY `length`IS NOT NULL, `length` DESC LIMIT ?,
values:[PG-13, 10]
This means that only the desired film entities are ever pulled in from the database.
When running directly under the IDE, the following output is produced:
These are the ten longest films rated as PG-13: GANGS PRIDE 185 min CHICAGO NORTH 185 min POND SEATTLE 185 min THEORY MERMAID 184 min CONSPIRACY SPIRIT 184 min FRONTIER CABIN 183 min REDS POCUS 182 min HOTEL HAPPINESS 181 min JACKET FRISCO 181 min MIXED DOORS 180 min
This looks perfect.
Modularizing the Project
To use modules, we need to run under Java 9 or greater and there has to be amodule-info.java file in our project:module microservice.jlink {
requires com.speedment.runtime.application;
requires com.speedment.runtime.connector.mysql; // (*)
}
The module com.speedment.runtime.application is the basic module that is always needed by any Speedment application.(*) Depending on the database type, you have to replace the MySQL module with the corresponding module for your database. Read all about the various database connector modules here.
Building the Project
As mentioned earlier, the complete project is available on GitHub. This is how you get it:git clone https://github.com/speedment/user-guide-code-samples.git
Change directory to the relevant sub-project:
cd user-guide-code-samples cd microservice-jlink
Build the project (you must use Java 9 or higher because of the module system):
mvn clean install
A Custom JRE Build Script
The project also contains a custom JRE build script calledbuild_jre.sh containing the following commands:#!/bin/bash SPEEDMENT_VERSION=3.2.1 JDBC_VERSION=8.0.18 OUTPUT=customjre echo "Building $OUTPUT..." MODULEPATH=$(find ~/.m2/repository/com/speedment/runtime -name "*.jar" \ | grep $SPEEDMENT_VERSION.jar | xargs echo | tr ' ' ':') MODULEPATH=$MODULEPATH:$(find ~/.m2/repository/com/speedment/common -name "*.jar" \ | grep $SPEEDMENT_VERSION.jar | xargs echo | tr ' ' ':') MODULEPATH=$MODULEPATH:$(find . -name "*.jar" | xargs echo | tr ' ' ':') $JAVA_HOME/bin/jlink \ --no-header-files \ --no-man-pages \ --compress=2 \ --strip-debug \ --module-path "$JAVA_HOME\jmods:$MODULEPATH" \ --add-modules microservice.jlink,java.management,java.naming,java.rmi,java.transaction.xa \ --output $OUTPUT
This is how the script works:
After setting various parameters, the script builds up the module path by adding the jars of the
speedment/runtime and speedment/common directories. Even though we are adding all of them, the module system will later figure out which ones are actually used and discard the other ones. The last line with MODULEPATH will add the JAR file of the application itself.After all the parameters have been set, we invoke the
jlink command which will build the custom JRE. I have used a number of (optional) flags to reduce the size of the target JRE. Because the JDBC driver does not support JPMS, I have manually added some modules that are needed by the driver under the --add-modules parameter.Building the Ultra-Compact JRE
Armed with the script above, we can create the ultra-compact custom JRE for our cloud database application with a single command:./build_jre.shThe build only takes about 5 seconds on my older MacBook Pro. We can check out the total size of the JRE/app with this command:
du -sh customjre/
This will produce the following output:
32M customjre/A staggering result! We have a full-fledged JVM with garbage collect, JIT compiler, all libraries (except the JDBC driver) and the application itself packed into only 32 MB of storage!
We can compare this to the JDK itself in its unreduced size which is often used as a baseline for cloud instances.
du -sh $JAVA_HOME
This will produce the following output on my laptop:
298M /Library/Java/JavaVirtualMachines/adoptopenjdk-11.jdk/Contents/Home/
And this figure does not even include the application or any third-party libraries. So, we have reduced the storage requirements with a factor of perhaps 10!
Modules Actually Used
To see what modules that made it through the reduction process, we can issue the following command:cat customjre/release
This will produce the following output on my machine (reformatted and sorted for clarity):
JAVA_VERSION="11.0.5" MODULES=" com.speedment.common.annotation com.speedment.common.function com.speedment.common.injector com.speedment.common.invariant com.speedment.common.json com.speedment.common.jvm_version com.speedment.common.logger com.speedment.common.mapstream com.speedment.common.tuple com.speedment.runtime.application com.speedment.runtime.compute com.speedment.runtime.config com.speedment.runtime.connector.mysql com.speedment.runtime.core com.speedment.runtime.field com.speedment.runtime.typemapper com.speedment.runtime.welcome java.base java.logging java.management java.naming java.prefs java.rmi java.security.sasl java.sql java.transaction.xa java.xml microservice.jlink "
So, all of Java's modules that were unused (such as
javax.crypto) were not included in the custom runtime.Running the Application
The application can be run using the custom JRE like this:customjre/bin/java --class-path ~/.m2/repository/mysql/mysql-connector-java/8.0.18/mysql-connector-java-8.0.18.jar -m microservice.jlink/com.speedment.example.microservices.jlink.Main
The file
mysql-connector-java-8.0.18.jar was automatically downloaded by Maven to its local repository when the project was first built (i.e. mvn clean install). Because the MySQL JDBC driver is not compatible with the Java Platform Module System yet, we had to glue it on manually.When run, the program produces the same output as it did above but from a runtime that was 10 times smaller:
These are the ten longest films rated as PG-13: GANGS PRIDE 185 min CHICAGO NORTH 185 min POND SEATTLE 185 min THEORY MERMAID 184 min CONSPIRACY SPIRIT 184 min FRONTIER CABIN 183 min REDS POCUS 182 min HOTEL HAPPINESS 181 min JACKET FRISCO 181 min MIXED DOORS 180 min
Memory Usage
A perhaps more important issue is how much application memory (RSS) that is being used by the cloud application in total. A quick look at this reveals that the heap memory usage is also reduced:Standard JDK
Pers-MBP:speedment pemi$ jmap -histo 38715 num #instances #bytes class name (module) ------------------------------------------------------- 1: 25836 3036560 [B (java.base@11.0.5) 2: 2055 1639408 [I (java.base@11.0.5) 3: 4234 511568 java.lang.Class (java.base@11.0.5) 4: 21233 509592 java.lang.String (java.base@11.0.5) 5: 196 270552 [C (java.base@11.0.5) 6: 4181 245400 [Ljava.lang.Object; (java.base@11.0.5) 7: 4801 153632 java.util.concurrent.ConcurrentHashMap$Node (java.base@11.0.5) 8: 3395 135800 java.util.LinkedHashMap$Entry (java.base@11.0.5) … 1804: 1 16 sun.util.resources.cldr.provider.CLDRLocaleDataMetaInfo (jdk.localedata@11.0.5) Total 137524 7800144
Custom JRE
Pers-MBP:speedment pemi$ jmap -histo 38783 | head num #instances #bytes class name (module) ------------------------------------------------------- 1: 22323 1714608 [B (java.base@11.0.5) 2: 4229 511000 java.lang.Class (java.base@11.0.5) 3: 19447 466728 java.lang.String (java.base@11.0.5) 4: 1776 424408 [I (java.base@11.0.5) 5: 69 264656 [C (java.base@11.0.5) 6: 4044 240128 [Ljava.lang.Object; (java.base@11.0.5) 7: 4665 149280 java.util.concurrent.ConcurrentHashMap$Node (java.base@11.0.5) 8: 3395 135800 java.util.LinkedHashMap$Entry (java.base@11.0.5) … 1726: 1 16 sun.util.resources.LocaleData$LocaleDataStrategy (java.base@11.0.5) Total 102904 5727960
Heap Improvement
The heap usage was reduced from 7,800,144 to 5,727,960 bytes (a reduction of over 25%)!NB: Before I ran the
jmap command, I let the application suggest an explicit Garbage Collect and wait for some seconds to even out any differences caused by potential earlier invocations of the Garbage Collector.Overview
Here is a chart that shows the difference in storage requirements (lower is better):
Here is another chart that shows the difference in RAM usage (lower is better):
Modifying the Code
If you want to modify the code, you need to rebuild the app after your changes with:mvn clean install
and then remove the old
customjre and create a new one:rm -rf customjre/ ./build_jre.sh
Creating Your own Database Application
If you want to connect to your own database and want to write your own application logic, you can easily select what tables and columns you want to use and then generate your own java domain model and application builder automatically using the Speedment Tool: |
| The Speedment tool as used in the project demonstrated in this article. |
The tool can be added to your project in the
pom.xml file and invoked by mvn speedment:tool. Visit the Speedment Initializer to generate your own custom pom.xml file and application template.The process can be streamlined by automatic Maven build scripts that will identify any application dependencies and automatic generation of Docker instances that can be deployed instantly following an automatic build. I will write more about this in the coming articles.
Conclusions
The Java Platform Module System (JPMS) allows the building of highly optimized JREs suitable for cloud deployment.It is possible to reduce both storage and RAM requirements.
Traditional ORMs do not honor full Java module encapsulation
Speedment open-source Stream ORM supports JPMS and can be used to build highly efficient database cloud applications.
Resources
Basics about JPMS modulesSpeedment on GitHub
The Speedment Initializer capable of generating project pom.xml templates
Excellent article, it helps overcoming the current intertia of the developer community in adopting the JPMS
ReplyDelete