Minborg

Minborg
Minborg

Friday, April 24, 2020

Java/Cloud: How to Quickly Create a Kubernetes-ready REST Microservice

Java/Cloud: How to Quickly Create a Kubernetes-ready REST Microservice 

It is safe to say that the Microservice + Cloud combination is all the rage these days. Microservices are being developed more than ever, in turn resulting in an increase in the number of application deployments. During the past decade, containerization and orchestration tools such as Docker and Kubernetes were developed, making the microservice pattern really easy to adopt.

This article will teach you how to generate a completely functional microservice with an exposed REST API capable of interacting with a MySQL database and deploy it to your local Kubernetes cluster. The learnings here can be applied to almost any database type like Oracle, SQL Server, DB2, and so on.

If you ever get stuck during the article, feel free to refer to the final version of the source code, which can be found in this GitHub repository.

Speed(ment) is Key 

As developers, one of the things we strive for in our everyday work is shorter development time.

At this point, we can already identify two aspects of our microservice that will make our development time longer than needed:

  1. We need to create a persistence layer 
  2. We need to expose a REST API


What if I were to tell you that a tool exists which can handle these things, without you having to write a single line of code?

Speedment is a Java ORM Toolkit and Runtime designed to allow developers to create super fast applications super fast. Speedment uses the native Java Streams API for database interaction, making it extremely easy to use for newcomers, seasoned veterans and anyone that falls in between. Speedment comes with a graphical tool, giving developers the ability to generate a Java representation of their database within seconds.

Furthermore, Speedment’s bundle system allows developers to easily expand the base functionality of the base Toolkit. One such enhancement is the Spring plugin which enables developers to generate a completely functional CRUD REST API in order to interact with their database.

In the remainder of the article, you will learn how to use the Speedment Tool to generate a working REST microservice and deploy it to a Kubernetes cluster. If you’re interested in Speedment as a whole, detailed documentation with examples can be found in the online manual.

Getting Started

Being distributed via Maven, Speedment is installation free but requires Java 8 or later. To get started, head over to the Speedment Initializer where you’ll be able to download a project template with all of the dependencies needed to create your microservice. These are the settings we will be using in the example:

If your setup is different, e.g. different database, make sure you apply the appropriate changes in the initializer.

Once you’re done configuring the project, click the Download button and unpack the downloaded zip-file. To launch the Speedment Tool, execute the following command from a terminal:

mvn speedment:tool


If this is your first time running the Speedment Tool, you’ll be asked to connect to your database. We’re running the vanilla MySQL Sakila sample database on our local machine, so the auth information would look like this:


Once you’ve filled in the required fields, click the Connect button. If the auth information you’ve provided was correct, you will be presented with the following screen:


Generating the Microservice

When you’ve connected to the database via the Speedment Tool, you can start configuring the various options that are available. There are a lot of options you can play around with, but for the purposes of this article, we will be focusing on the options that are needed to expose a REST API.

To enable REST API generation, click on the Project node in the tree view, and check the Enable REST option:



We’ve also enabled the Generate REST documentation option to automatically generate the OpenAPI documentation for our REST API. This option is not mandatory, but it will allow us to test our API more easily in the end.

The next step is optional, but it will make our REST routes a bit more aesthetically pleasing. Head over to the database schema node in the tree view and set the value of REST Endpoint to a front slash (/). By default, the schema name is included in the generated REST routes and this modification removes it.



Next, we’re going to enable the generation of REST controllers for the following tables:
  • Actor
  • Category
  • Customer
  • Film
  • Staff
  • Store

The steps to enable the controller generation are identical regardless of the table. For that reason we will only demonstrate them on the Actor table.

 Click the Actor table in the tree view and enable the Generate @RestController option. This, in turn, will enable several REST related options for that table. The options we are interested in, which you should enable, are:
  • REST Enable LIST
  • REST Enable GET
  • REST Enable CREATE
  • REST Enable UPDATE
  • REST Enable DELETE

We will also rename the REST Endpoint from /actor to /actors (again only for aesthetic purposes). By default, the REST Endpoint is named the same as the table it is associated with. In our case, the renaming makes sense, because when we visit the /actors endpoint, a list of actors will be retrieved, rather than a single actor.

Go ahead and repeat these steps for the other tables listed above. After you’re done, click the Generate button. This will generate a Java representation of your database along with the necessary REST configurations and controllers.

Running the Microservice


If we were to run our application right now as it is, it will most likely crash. This is because we haven’t specified the password our application should use to connect to the database.

When we generated our application, a bunch of Speedment-specific application properties were exposed. One such property is the spring.speedment.password property, which we can use to set the password Speedment will use to connect to our database.

There are a couple of ways to specify application properties. We’re going to be defining them in the application.properties file, which you should create in your application’s resources folder.

This is what our application.properties file looks like:

 # Application properties file - START
spring.application.name=speedment-spring-app

spring.speedment.password=sakila
# Application properties file - END

The default password for the Sakila database is sakila, but if your database has a different password, make sure those changes are reflected in the application.properties file.

Once we have everything configured, we can run our application. This is done by executing the following command from the project’s root folder:

mvn spring-boot:run


If you’ve enabled the Generate REST documentation option, you can visit http://localhost:8080/swagger-ui.html to access the REST API documentation:



You can execute your requests manually or directly from Swagger UI. If we were to visit http://localhost:8080/actors in our browser, we should get a JSON response with a list of actors stored in our database:
[
   {
      "actorId": 1,
      "firstName": "PENELOPE",
      "lastName": "GUINESS"
   },
   {
      "actorId": 2,
      "firstName": "NICK",
      "lastName": "WAHLBERG"
   },
   {
      "actorId": 3,
      "firstName": "ED",
      "lastName": "CHASE"
   },
... TRUNCATED ... 
]

Before Deployment

Before we start with the deployment process of our microservice, make sure you have the following dependencies installed on your local machine: Docker, kubectl, Minikube and Skaffold.

Dockerizing our Microservice

Before we can actually deploy our microservice to a Kubernetes cluster, we need to convert it into a format that Kubernetes can actually work with. Kubernetes is a container orchestration tool, so this is where Docker comes in to aid us with the container creation process.

In the root of your project, create a Dockerfile with the following contents:

FROM openjdk:11-slim-buster

EXPOSE 8080

ARG JAR_LOCATION=target
ARG JAR_NAME=speedment-spring-app
ARG JAR_VERSION=1.0.0

ADD ${JAR_LOCATION}/${JAR_NAME}-${JAR_VERSION}.jar app.jar

ENTRYPOINT ["java", "-jar", "app.jar", "--spring.speedment.host=sakila"]

The exported arguments (JAR_LOCATION, JAR_NAME, JAR_VERSION) may be different for your project, depending on the information you provided in the pom.xml file. From the root of your project, execute the following command:

mvn install


This will create a target folder with a JAR file containing your microservice. Make sure the name and the version of the file match with the information you put in the Dockerfile.

Creating the Deployment Configurations


We will be deploying two images to our Kubernetes cluster: the Sakila database and our microservice. The Sakila database already has a Docker image publicly available: restsql/mysql-sakila. However, we need to build an image for our microservice. This is where the Dockerfile we created earlier will come in handy. Later on, we will be using a tool called Skaffold to create an image for our microservice and use it in the deployment process.

Start by creating a folder called k8s in the root of your project. This is where you’ll store all of your Kubernetes Deployment and Service configurations. We will keep our microservice and database configurations separate, so create two folders called storage and app in the k8s folder.



We now proceed with the configurations for the Sakila database. In the storage folder, we will create two YAML files - sakila-deployment.yml and sakila-service.yml. The sakila-deployment.yml file will store our deployment configuration for the Sakila database. As this is not a Kubernetes tutorial we will only be providing the final configurations. This is what the sakila-deployment.yml file should look like in the end:


apiVersion: apps/v1
kind: Deployment
metadata:
  name: sakila
  labels:
    storage: sakila
spec:
  replicas: 1
  selector:
    matchLabels:
      storage: sakila
  template:
    metadata:
      labels:
        storage: sakila
    spec:
      containers:
      - name: sakila
        image: restsql/mysql-sakila
        ports:
        - containerPort: 3306

And this is the final sakila-service.yml file:

apiVersion: v1
kind: Service
metadata:
  name: sakila
  labels:
    storage: sakila
spec:
  selector:
    storage: sakila
  ports:
  - name: database
    port: 3306
    targetPort: 3306

The deployment and service configurations for our microservice are nearly identical. In the app folder, create a YAML file called speedment-spring-app-deployment.yml with the following contents:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: speedment-spring-app
  labels:
    app: speedment-spring-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: speedment-spring-app
  template:
    metadata:
      labels:
        app: speedment-spring-app
    spec:
      containers:
      - name: speedment-spring-app
        image: speedment-spring-app-example
        ports:
        - containerPort: 8080

In the same folder create another YAML file called speedment-spring-app-service.yml with the following contents:

apiVersion: v1
kind: Service

metadata:
  name: speedment-spring-app
spec:
  selector:
    app: speedment-spring-app
  ports:
  - name: http
    port: 8080
    targetPort: 8080
  type: NodePort

These are all of the configurations we need in order to deploy our database and application. After adding the deployment configurations, our file structure should look like this:


Starting the Cluster

We are nearly done with the preparations for the deployment. There is one last thing we need to do - start our local Kubernetes cluster. This is done by executing the following command:

minikube start


The Deployment

In order to deploy our database and application we will be using Skaffold. In the root of your project create a file called skaffold.yml with the following contents:


apiVersion: skaffold/v2alpha3
kind: Config
build:
  artifacts:
  - image: speedment-spring-app-example
    docker:
      dockerfile: Dockerfile
deploy:
  kubectl:
    manifests:
    - k8s/app/*
    - k8s/storage/*

With this file completed, we are finally ready for deployment. From the project root, execute the following command:

skaffold dev --port-forward=true


When we execute this command, two things will happen:

  1. A Docker image will be created from the Dockerfile we created earlier
  2. Deployments and services will be created from the configurations we created earlier

Once your microservice starts up, you can use it the same way you used it earlier. The only difference now is that it is running from a Kubernetes cluster.

Note: It takes around 30-60 seconds for the Sakila database to boot up completely. Since our application starts a lot quicker than the Sakila database, it will most likely crash and restart a couple of times before the database is ready.

Summary

Creating applications in a time-efficient manner can sometimes be hard. We have explained how to generate a microservice from a database and deploy it to a Kubernetes cluster, so hopefully, you’ve learned something that will reduce your development time.

We hope you’ve enjoyed reading this article as much as we enjoyed writing it. The final version of the source code from this article can be found here.

Authors

Per Minborg
Mislav Miličević

Resources

The Speedment Initializer capable of generating project templates
Speedment OpenSource on GitHub
Speedment Online Manual
Github Repository with the final version of the source code



Wednesday, March 4, 2020

Java/Spring: How to Generate an Entire Swagger Documented CRUD REST API With Speedment

As developers, one of the most cumbersome tasks we often face in our day-to-day lives is writing good and understandable documentation. It doesn’t matter if our documentation is only a few lines long explaining the core functionality of a feature or if it’s a full-blown essay demonstrating the ins and outs of a system. What matters is that the message we’re trying to convey through our documentation is precise and understandable.

In our previous article, we covered the topic of automatic REST API generation. More precisely, we demonstrated how to generate an entire CRUD REST API for your database using Speedment’s revamped Spring Integration plugin.

Today, we’ll be taking this knowledge a step further and demonstrate how to generate interactive documentation for your REST API in a single click.

If you didn’t get a chance to use the Speedment Spring plugin, we highly suggest you read our previous article as it contains the information necessary to follow this guide.

Do You Like Java Streams?

If the answer to this question is either ‘Yes!’, ‘Absolutely!’ or perhaps ‘Heck yeah!’, then Speedment is the right tool for you. Speedment is a Java ORM toolkit and runtime which uses pure Java Streams as an interface between your application and the database.

Alongside the already familiar Streams API, Speedment provides end-users with a graphical tool in order to generate a Java representation of your database in a matter of seconds, allowing them to completely stay in a Java-only environment.

If you’re interested in learning more about Speedment, head over to the documentation page where you’ll find a bunch of guides and examples. For the remainder of this article, we’ll be focusing on the new update to Speedment’s Spring plugin.

Before we Begin

In order to generate the REST API documentation, Speedment uses a combination of the OpenAPI specification and Swagger UI.

The preparation steps will differ depending on if you’re starting from scratch or not, but the end result will be the same regardless of your starting point.

If you have followed the guide in our previous article, where we explain how to generate a REST API using Speedment, you’ll only need to add a couple of dependencies to your project’s pom.xml file:
 
<dependencies>
    ...
    <dependency>
        <groupId>io.springfox</groupId>
        <artifactId>springfox-swagger2</artifactId>
        <version>2.9.2</version>
    </dependency>

    <dependency>
        <groupId>io.springfox</groupId>
        <artifactId>springfox-swagger-ui</artifactId>
        <version>2.9.2</version>
    </dependency>
   ...
</dependencies>

On the other hand, if you’re starting from scratch, head over to the Initializer where you’ll be able to generate a Speedment project with Spring support. Once you reach the Initializer, you’ll be presented with plenty of options to configure your project. One configuration option that is particularly important is the Plugins section of the Initializer.

To enable Spring support in your new Speedment project, tick the checkbox next to the "Spring" option. Once you’re happy with your project configuration, go ahead and click the Download button at the bottom of the Initializer.



When you’re ready, you can launch the Speedment Tool by executing the following command from the root folder of your project template:

mvn speedment:tool

If you’ve installed the plugin correctly, you’ll see some Spring Boot specific options which can be used to configure your REST API and documentation.

If this is your first time using Speedment, you may want to familiarize yourself with the workflow by following the “Hello Speedment” quick start guide.

Swagger Automata

For the following example, we’ll be using Sakila, a MySQL Sample Database. You can download it as a standalone instance or as a Docker container.

When you open the Speedment Tool and successfully connect to your database, you will be presented with a user interface containing the metadata information about your database and some options that you can configure:



If you click the “Generate” button found in the top banner, a Java representation of your database will get generated. To generate the documentation for your REST API, you must enable the “Generate REST documentation” option found in the project view (which is accessed by selecting the top node in the tree).

Once enabled, additional configuration options will become available allowing you to further customize your generated documentation:



The next time you regenerate your Spring project, some OpenAPI specific configurations will get generated. In order to see and use the generated documentation, you’ll need to run your Spring application. To do so, execute the following command:
 
mvn spring-boot:run


Once your Spring application is up and running, you can find your generated Swagger documentation at the following endpoint - http://localhost:8080/swagger-ui.html



Depending on how you configured your project, you might see different results in the generated documentation. For instance, if you disable REST API generation for a certain table, the next time you regenerate your project, the endpoint for that table will not be available in the documentation.

With the generated Swagger documentation, you’re able to instantly learn what REST endpoints your application has registered, what HTTP methods are available for each endpoint and execute HTTP requests for those endpoints directly from the Swagger UI:



If you’re not sure what is required in the request body, you can find the request body models at the bottom of the documentation, under the “Models” section:



Note: When connecting to the Swagger endpoint, if you get presented with the following prompt, make sure your Spring entry point is in the correct package (must be above or in the same package that the Swagger configuration is located in):



This is usually a sign that your Swagger configuration was not scanned by Spring.

Summary

Writing good and understandable documentation can be a long and tedious process. With the new update to Speedment’s Spring Boot plugin, users are able to generate interactive documentation for their REST API in a matter of seconds.

Resources

Article "How to Generate an Entire Database CRUD REST API with Speedment"
The Speedment Initializer capable of generating project templates
Speedment on GitHub

Authors

Per Minborg
Mislav Miličević

Tuesday, February 11, 2020

Java14: Join Database Tables with Java 14's new Record

Java14 Records in Joins

Did you know that you can join database tables into a Java Stream with Java 14's preview record feature? Read this short article and find out how it is done using the Speedment Stream ORM. We will start with how to set up your project.

Setup

Download Java 14. Go to the Speedment Initializer and download your project skelaton (including pom.xml). Modify the following lines in your pom.xml file:
 
<maven.compiler.source>14</maven.compiler.source>
<maven.compiler.target>14</maven.compiler.target>

...

    <plugin>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.8.1</version>
        <configuration>
            <release>14</release>
            <compilerArgs>
                --enable-preview
            </compilerArgs>
        </configuration>
    </plugin>


Make sure that you have the latest version of your ide (e.g. IDEA 2010.1) that supports the new Java 14 features.

Speedment Joins

Speedment allows dynamically JOIN:ed database tables to be consumed as standard Java Streams. In this article, we will use the exemplary Sakila database that contains films, actors, languages etc. Download Sakila here or grab a Docker version here

Tables, views and joins can easily be turned into standard Java streams with Speedment. This is how it can look like in Java 14:

            
    var speedment = new SakilaApplicationBuilder()
            .withPassword("sakila")
            .withBundle(JoinBundle.class)
            .build();

    var joinComponent = speedment.getOrThrow(JoinComponent.class);

    var films = speedment.getOrThrow(FilmManager.class);

    // Define a Java 14 "record" that can hold a Film and a Language
    record FilmLanguage(Film film, Language language) {}

    var join = joinComponent.from(films.getTableIdentifier())
            .leftJoinOn(Language.LANGUAGE_ID).equal(Film.LANGUAGE_ID)
            // Provide the constructor of the Java 14 "record"
            // to be used to construct Film/Language composites
            .build(FilmLanguage::new);

    join.stream()
            .forEach(filmLanguage -> System.out.format(
                    "%s is in %s%n",
                    filmLanguage.film().getTitle(),
                    filmLanguage.language().getName())
            );


This will produce the following output:

ACADEMY DINOSAUR is in English
ACE GOLDFINGER is in English
ADAPTATION HOLES is in English
...

Code Breakdown

The from() method takes the first table we want to use (Film). The innerJoinOn() method takes a specific column of the second table we want to join. Then, the equal() method takes a column from the first table that we want to use as our join condition. So, in this example, we will get matched Film and Language entities where the column Language.LANGUAGE_ID equal Film.LANGUAGE_ID.

Finally, build() will construct our Join object that can, in turn, be used to create Java Streams. The Join object can be re-used over and over again.

Note how the constructor of the record FilmLanguage is provided in the build() method. Note also how a film and language entity can be obtained from the record (e.g. filmLanguage.film()). This is a big improvement over previous Java version where we had to provide rather lengthy custom classes or use tuples with accessor like get0() and get1() rather than the much more descriptive film() and language()

JOIN Types and Conditions

We can use innerJoinOn(), leftJoinOn(), rightJoinOn() and crossJoin() and tables can be joined using the conditions equal(), notEqual(), lessThan(), lessOrEqual(), greaterThan() and lessOrEqual().

What's Next?

Download open-source Java 14 here.
Download Speedment here.
Read all about the JOIN functionality in the Speedment User's Guide.

Tuesday, December 10, 2019

Java/Spring: How to Generate an Entire Database CRUD REST API with Speedment


As each year passes it is becoming nothing but obvious that the Spring Framework is one of the most widely used web development frameworks for Java. With the next decade just around the corner, Spring’s most popular module, Spring Boot, has just received a major update.

The new Spring Boot version "2.2.0" and the year "2020" are almost a perfect match. Thus, the Speedment team has now filled the gap by revamping the Spring Boot plugin.

If this is your first time hearing of Speedment, you’re in for a treat, as we will be demonstrating how the Speedment Spring Boot plugin allows developers to effortlessly build Spring database applications with Speedment, an ORM with a powerful Stream-based query API, as its backbone.

About Speedment

Speedment can be described as an ORM that the Java community deserves - a toolkit for those who like fast development and even faster applications. By leveraging the Java Stream API as its primary means of communication with your database of choice, Speedment makes developers with previous Java experience feel right at home. Alongside the easy-to-use API, Speedment provides developers with a graphical tool that generates the domain model for them in a matter of seconds.

If you’re interested in Speedment as a whole, detailed documentation with examples can be found here. The rest of this article will focus on the Spring Boot plugin.

Spring Boot: The Quickest Start 

Speedment Spring Boot plugin has been around for a while, but starting with Speedment 3.2.2 a major feature was introduced - the ability to generate an entire CRUD REST API directly from your database model.

Depending on whether you’re starting from scratch or have an existing project, the steps will differ, but rest assured that the integration process is effortless regardless of the state of your project.

If you wish to integrate Spring Boot into an existing Speedment project, you are required to add the following 2 clauses to your pom.xml:

<plugin>
    <groupId>com.speedment.enterprise</groupId>
    <artifactId>speedment-enterprise-maven-plugin</artifactId>
    <version>${speedment.version}</version>
                
    <configuration>
        <components>
            <component>com.speedment.enterprise.plugins.spring.SpringGeneratorBundle</component>
        </components>
        <appName>${project.artifactId}</appName>
        <packageName>${project.groupId}</packageName>
    </configuration>
    ... // Database connector dependency           
</plugin>
<dependencies>
 ...
<dependency>
    <groupId>javax.annotation</groupId>
    <artifactId>javax.annotation-api</artifactId>
    <version>1.3.1</version>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
    <version>2.2.1.RELEASE</version>
</dependency>
<dependency>
    <groupId>com.speedment.enterprise.plugins</groupId>
    <artifactId>spring-runtime</artifactId>
    <version>${speedment.version}</version>
</dependency>
...
</dependencies>

Once added, the next time you regenerate your domain model, a Speedment-specific Spring configuration should get generated. The generated configuration handles database connection properties and bean registrations for various managers generated by Speedment.

If you’re starting from scratch, head over to the project initializer where a Spring-Boot-ready project will be generated for you. How you will configure the project is up to you, but to include the Spring Boot plugin in the generated project, make sure that the checkbox next to Spring in the Plugins section is checked. Once you’re satisfied with the project configuration, click on the Download button and you will receive a zipped-up project ready for use.


CRUD Made Easy

Once you install the plugin, some Spring Boot specific options will be available in the Speedment tool which can be used to configure your REST API. You can launch the Speedment tool by issuing the following command:
mvn speedment:tool

If this is your first time using Speedment, you may want to familiarize yourself with the workflow by following the “Hello Speedment” quick start guide.

By default, the Speedment tool won’t generate a REST API. To generate a CRUD REST API for a specific table, select the table in the tree view and check the “Generate @RestController” option along with the “REST Enable Create/Update/Delete” option. By clicking “Generate”, a set of REST mappings that implement CRUD operations for a specific table will get generated.



That’s it! No manual coding is needed. Just start the generated Spring by issuing the command:
mvn spring-boot:run

Creating Entities Using REST 

To create a new entity using your generated REST API, you must execute a POST request to the route specified in the tool. By default, the name of this route is the same as the name of your table, prefixed by the name of your schema. If you uncheck the “REST Endpoint” option, you may specify a custom route name. The request body should contain key-value pairs in a JSON object where “REST Field Name” values from the tool are keys.

If we were to have a table called ‘person’ with the columns ‘id’, ‘name’, and ‘age’, we would send the following request to create a new ‘person’ entity:

POST localhost:8080/db/person
{
    “id”: 1,
    “name”: “Jane”,
    “age”: 25
}

curl -d '{“id”: 1,“name”: “Jane”,“age”: 25}' -H "Content-Type:application/json" -X POST localhost:8080/db/person

If a column is auto-generated (or is using a sequence), you may exclude it from the POST body. If you wish to enforce the exclusion of a certain column from the POST body, click on the specific column in the tool tree view and uncheck “Include in Create Body” and regenerate your domain model. On the other hand, if you want to enforce that a certain column is present in the POST body when a request is executed, check “Required in Create Body” and regenerate your domain model.



Retrieving Entities Using REST


To retrieve our newly created entity, we must execute a GET request on the same path used for entity creation:

GET localhost:8080/db/person

curl localhost:8080/db/person/1


The executed request will return a JSON Array of all existing entities. In our case, it would return an array with just our only entity:

[
    {
        “id”: 1,
        “name”: “Jane”,
        “age”: 25
    }
]
Advanced concepts such as filters and sorters are covered in great detail in the official Speedment documentation.

Updating Existing Entities Using REST 

Updating an existing entity using your REST API is done in a similar fashion as creating an entity. Instead of a POST request, we execute a PATCH request and the route is extended by an entity identifier. What our entity identifier depends on the Primary Key column of that table. Since our PK column is a numeral type, our entity identifier will be an integer.

To update the entity we created in our previous example, we would execute the following request:

PATCH localhost:8080/db/person/1
{
    “name”: “Mike”,
    “age”: 43
}

curl -d '{“name”: “Jane”,“age”: 25}' -H "Content-Type:application/json" -X PATCH localhost:8080/db/person/1
By default, all columns are included in the PATCH body (except the Primary Key column), but they are not mandatory. If you wish to enforce the exclusion of a certain column from your request body, click on the specific column in the tool tree view and uncheck “Include in Update Body” and regenerate your domain model. On the other hand, if you want to enforce column presence in the request body, check “Required in Update Body” and regenerate your domain model.


Deleting entities using REST

Deleting an entity using your REST API is quite straightforward - we execute a DELETE request on the same path used for entity updates.

To delete the entity we’ve created and updated in the previous examples, we would execute the following request:


DELETE localhost:8080/db/person/1

curl -X DELETE localhost:8080/db/person/1

Summary

Starting new projects can sometimes be a hustle. As developers, we want to avoid these inconveniences as much as possible and dive straight into coding. With Speedment's new Spring Boot plugin, developers can get ahead of the game by automatically generating all necessary configurations and REST controllers directly from the database.

We hope you’ve enjoyed this brief guide to the new CRUD features the Speedment Spring Boot plugin provides. If you’re interested in a more detailed explanation of the new features, head over to the official documentation where you will find in-depth explanations and examples of the new CRUD features.

Authors

Per Minborg
Mislav Miličević

Wednesday, November 27, 2019

Java: Release of Speedment 3.2 - Now Enabling Lightweight Database Microservices

Several traditional ORMs do not fully honor Java module encapsulation (JPMS). This often means a lot of unnecessary files and classes are used when deploying microservices. The latest major release 3.2 of open-source Speedment solves this problem by introducing module system support allowing efficient deployment of cloud applications and providing a more stringent API.

In this article, you will learn what’s new in Speedment 3.2 and how you can deploy cloud database applications that are 10 times smaller and using much less memory.

Support for Java Platform Module System (JPMS)

The biggest feature in Speedment 3.2 is undoubtedly native support for JPMS. Every module in Speedment now contains its own module-info.java file and honors strict encapsulation.

As a developer, it is completely optional to use the module system and thanks to its multi-release JARs, Speedment can still be run under Java 8. You can elect to migrate to JPMS now, later or never.

Other ORMs, such as Hibernate, might support parts of JPMS but does not honor strict encapsulation (e.g. requires --add-opens to be added manually, thereby bypassing protection from accessing internal/protected classes).

Optional Modules Allows Smaller Microservices

The Speedment modules have been internally restructured to reduce inter-module coupling. As a result, some of the modules are now optional, allowing even smaller microservices to be deployed. For example, the various database connectors are now individually pickable and the JoinComponent is also optional.

Size Reduction

The need for several internal modules (such as “lazy” and “mutable-stream”) has been eliminated and some of the others have been optimized and reduced in size.

Strong Encapsulation

Thanks to the module system, internal classes are now fully hidden and are even protected from deep-reflection. This strengthens the API (because only intentionally visible classes and methods are accessible) and therefore allows for future migration of internal classes without affecting the public API.

Cloud Deployment Example

It is possible to create a custom JRE + application + speedment libraries that is 10 times smaller and that is using 25% less memory compared to running the application under a standard JDK. If you have a large number of microservices running in the cloud this adds up to a huge difference.

The following example is further described in my article “Java: How to Create Lightweight Database Microservices”. The database application connects to a public MySQL cloud instance of the “Sakila” database with films, actors, etc. It retrieves the ten longest films and prints them on the console in length order.

The custom JRE still has all the bells and whistles of a real JVM like garbage collect, JIT-compiler, etc. It is just the unused modules and tools that have been removed.

final Speedment app = new SakilaApplicationBuilder()
    .withPassword("sakila")
    .build();

final FilmManager films = app.getOrThrow(FilmManager.class);

System.out.println("These are the ten longest films rated as PG-13:");

films.stream()
    .filter(Film.RATING.equal("PG-13"))
    .sorted(Film.LENGTH.reversed())
    .limit(10)
    .map(film -> String.format(
        "%-18s %d min",
        film.getTitle(),
        film.getLength().orElse(0))
    )

    .forEach(System.out::println);
The application will produce the following output:

These are the ten longest films rated as PG-13:

GANGS PRIDE        185 min
CHICAGO NORTH      185 min
POND SEATTLE       185 min
THEORY MERMAID     184 min
CONSPIRACY SPIRIT  184 min
FRONTIER CABIN     183 min
REDS POCUS         182 min
HOTEL HAPPINESS    181 min
JACKET FRISCO      181 min
MIXED DOORS        180 min

It turns out that the storage requirement for the standard open JDK 11 is 300 MB compared to the custom JRE which only occupies 30 MB (even including the application and the Speedment runtime). Thus, it is possible to reduce the storage requirements by about 90%. When examining heap usage with jmap, it was concluded that the RAM usage was also reduced by about 25%.

How to Get Speedment 3.2

New users can download Speedment 3.2  using the Initializer.

Existing users can just update the speedment version in their pom.xml file and re-generate the domain model by issuing the following command:

mvn speedment:generate

That’s it. Now your old Speedment application will run under the new version.

If you want to use the module system, add the following module-info.java file to your Java 8+ application’s root:
module your.module.name {
    requires com.speedment.runtime.application;
    requires com.speedment.runtime.connector.mysql; // (*)
}

(*) Depending on the database type, you have to replace the MySQL module with the corresponding module for your database. Read all about the various database connector modules here.

Resources

Basics about JPMS modules
The complete Speedment release note history can be found here
Speedment on GitHub
The Speedment Initializer capable of generating project templates

Java: How to Create Lightweight Database Microservices

The number of cloud-based Java database applications grows by the minute. Many organizations deploy hundreds if not thousands of microservice instances. However, most applications carry an astounding amount of unnecessary overhead with respect to the runtime environment. This, in turn, makes the application slower and more expensive to run.

In this article, I will demonstrate how to write a database application that is 10 times smaller than normal(*). The storage requirement will be about 32 MB instead of the usual(*) ~300 MB taking both the application, third-party libraries and the Java runtime into account. As a bonus, the required RAM to run the application will also be reduced by 25%.

(*) These are the storage requirements for the following full JDKs (excluding the application and third-party libs):
jdk.8.0_191        360 MB
jdk-9.0.4          504 MB
adoptopenjdk-11    298 MB

Using an ORM that Supports Microservices

Most traditional ORMs do not honor Java module encapsulation. Often, this entails shipping off a lot of unnecessary code.

In this article, I will use the open-source Stream-based Java ORM Speedment, which, in its latest version, supports the Java Platform Module System (JPMS). This enables us to generate an optimized custom Java Runtime Environment (JRE, the parts from the JDK that is needed to run applications) with only the modules explicitly used by our application.

Read about the new features of Speedment 3.2 in this article.

The Application

The entire application we wish to deploy in this article resides as an open-source project on GitHub under the sub-directory “microservice-jlink”. It connects to a public instance of a MySQL “Sakila” database (containing data about films) hosted in the cloud and lists the ten longest films that are rated “PG-13” on the console. The data model is preconfigured to fit the data structure of this database. If you want to create your own application using another database, visit the Speedment initializer to configure a project for that database specifically.

The main method of the application looks like this:

public final class Main {

    public static void main(String[] args) {

        final Speedment app = new SakilaApplicationBuilder()
            .withPassword("sakila")
            .build();

        final FilmManager films = app.getOrThrow(FilmManager.class);

        System.out.println("These are the ten longest films rated as PG-13:");

        films.stream()                          // 1
            .filter(Film.RATING.equal("PG-13")) // 2
            .sorted(Film.LENGTH.reversed())     // 3
            .limit(10)                          // 4
            .map(film -> String.format(         // 5
                "%-18s %d min",
                film.getTitle(),
                film.getLength().orElse(0))
            )
            .forEach(System.out::println);      // 6

    }
}
First, we pass the database password to the Speedment builder (Speedment never stores passwords internally). The builder is pre-configured with the database IP-address, port, etc. from a configuration file.

Then, we obtain the FilmManager which later can be used to create Java Streams that corresponds directly to the “film” table in the database.

At the end, we:
  1. Create a Stream of the Film entities
  2. Filter out Film entities that have a rating equal to “PG-13”
  3. Sorts the remaining films in reversed length order (longest first)
  4. Limits the stream to the first 10 films
  5. Maps each film entity to a String with film title and film length
  6. Prints each String to the console

The application itself is very easy to understand. It shall also be noted that Speedment will render the Java Stream to SQL under the hood as shown hereunder:
SELECT 
    `film_id`,`title`,`description`,`release_year`,
    `language_id`,`original_language_id`,`rental_duration`,`rental_rate`,
    `length`,`replacement_cost`,`rating`,`special_features`,`last_update`
FROM `sakila`.`film` 
WHERE (`rating` = ? COLLATE utf8_bin) 
ORDER BY `length`IS NOT NULL, `length` DESC LIMIT ?,
values:[PG-13, 10]

This means that only the desired film entities are ever pulled in from the database.

When running directly under the IDE, the following output is produced:

These are the ten longest films rated as PG-13:
GANGS PRIDE        185 min
CHICAGO NORTH      185 min
POND SEATTLE       185 min
THEORY MERMAID     184 min
CONSPIRACY SPIRIT  184 min
FRONTIER CABIN     183 min
REDS POCUS         182 min
HOTEL HAPPINESS    181 min
JACKET FRISCO      181 min
MIXED DOORS        180 min

This looks perfect.

Modularizing the Project

To use modules, we need to run under Java 9 or greater and there has to be a module-info.java file in our project:

module microservice.jlink {
    requires com.speedment.runtime.application;
    requires com.speedment.runtime.connector.mysql; // (*)
}
The module com.speedment.runtime.application is the basic module that is always needed by any Speedment application.

(*) Depending on the database type, you have to replace the MySQL module with the corresponding module for your database. Read all about the various database connector modules here.

Building the Project

As mentioned earlier, the complete project is available on GitHub. This is how you get it:

git clone https://github.com/speedment/user-guide-code-samples.git

Change directory to the relevant sub-project:
cd user-guide-code-samples
cd microservice-jlink

Build the project (you must use Java 9 or higher because of the module system):

mvn clean install

A Custom JRE Build Script

The project also contains a custom JRE build script called build_jre.sh containing the following commands:

#!/bin/bash
SPEEDMENT_VERSION=3.2.1
JDBC_VERSION=8.0.18
OUTPUT=customjre
echo "Building $OUTPUT..."
MODULEPATH=$(find ~/.m2/repository/com/speedment/runtime -name "*.jar" \
  | grep $SPEEDMENT_VERSION.jar | xargs echo | tr ' ' ':')
MODULEPATH=$MODULEPATH:$(find ~/.m2/repository/com/speedment/common -name "*.jar" \
  | grep $SPEEDMENT_VERSION.jar | xargs echo | tr ' ' ':')
MODULEPATH=$MODULEPATH:$(find . -name "*.jar" | xargs echo | tr ' ' ':')

$JAVA_HOME/bin/jlink \
--no-header-files \
--no-man-pages \
--compress=2 \
--strip-debug \
--module-path "$JAVA_HOME\jmods:$MODULEPATH" \
--add-modules microservice.jlink,java.management,java.naming,java.rmi,java.transaction.xa \
--output $OUTPUT

This is how the script works:

After setting various parameters, the script builds up the module path by adding the jars of the speedment/runtime and speedment/common directories. Even though we are adding all of them, the module system will later figure out which ones are actually used and discard the other ones. The last line with MODULEPATH will add the JAR file of the application itself.

After all the parameters have been set, we invoke the jlink command which will build the custom JRE. I have used a number of (optional) flags to reduce the size of the target JRE. Because the JDBC driver does not support JPMS, I have manually added some modules that are needed by the driver under the --add-modules parameter.

Building the Ultra-Compact JRE

Armed with the script above, we can create the ultra-compact custom JRE for our cloud database application with a single command:

./build_jre.sh
The build only takes about 5 seconds on my older MacBook Pro. We can check out the total size of the JRE/app with this command:

du -sh customjre/

This will produce the following output:
 32M customjre/
A staggering result! We have a full-fledged JVM with garbage collect, JIT compiler, all libraries (except the JDBC driver) and the application itself packed into only 32 MB of storage!

We can compare this to the JDK itself in its unreduced size which is often used as a baseline for cloud instances.
du -sh $JAVA_HOME

This will produce the following output on my laptop:

298M /Library/Java/JavaVirtualMachines/adoptopenjdk-11.jdk/Contents/Home/

And this figure does not even include the application or any third-party libraries. So, we have reduced the storage requirements with a factor of perhaps 10!

Modules Actually Used

To see what modules that made it through the reduction process, we can issue the following command:
cat customjre/release

This will produce the following output on my machine (reformatted and sorted for clarity):
JAVA_VERSION="11.0.5"
MODULES="
com.speedment.common.annotation
com.speedment.common.function
com.speedment.common.injector
com.speedment.common.invariant
com.speedment.common.json
com.speedment.common.jvm_version
com.speedment.common.logger
com.speedment.common.mapstream
com.speedment.common.tuple
com.speedment.runtime.application
com.speedment.runtime.compute
com.speedment.runtime.config
com.speedment.runtime.connector.mysql
com.speedment.runtime.core
com.speedment.runtime.field
com.speedment.runtime.typemapper
com.speedment.runtime.welcome
java.base
java.logging
java.management
java.naming
java.prefs
java.rmi
java.security.sasl
java.sql
java.transaction.xa
java.xml
microservice.jlink
"

So, all of Java's modules that were unused (such as javax.crypto) were not included in the custom runtime.

Running the Application

The application can be run using the custom JRE like this:
customjre/bin/java --class-path ~/.m2/repository/mysql/mysql-connector-java/8.0.18/mysql-connector-java-8.0.18.jar -m microservice.jlink/com.speedment.example.microservices.jlink.Main

The file mysql-connector-java-8.0.18.jar was automatically downloaded by Maven to its local repository when the project was first built (i.e. mvn clean install). Because the MySQL JDBC driver is not compatible with the Java Platform Module System yet, we had to glue it on manually.

When run, the program produces the same output as it did above but from a runtime that was 10 times smaller:
These are the ten longest films rated as PG-13:
GANGS PRIDE        185 min
CHICAGO NORTH      185 min
POND SEATTLE       185 min
THEORY MERMAID     184 min
CONSPIRACY SPIRIT  184 min
FRONTIER CABIN     183 min
REDS POCUS         182 min
HOTEL HAPPINESS    181 min
JACKET FRISCO      181 min
MIXED DOORS        180 min

Memory Usage

A perhaps more important issue is how much application memory (RSS) that is being used by the cloud application in total. A quick look at this reveals that the heap memory usage is also reduced:

Standard JDK

Pers-MBP:speedment pemi$  jmap -histo 38715
 num     #instances         #bytes  class name (module)
-------------------------------------------------------
   1:         25836        3036560  [B (java.base@11.0.5)
   2:          2055        1639408  [I (java.base@11.0.5)
   3:          4234         511568  java.lang.Class (java.base@11.0.5)
   4:         21233         509592  java.lang.String (java.base@11.0.5)
   5:           196         270552  [C (java.base@11.0.5)
   6:          4181         245400  [Ljava.lang.Object; (java.base@11.0.5)
   7:          4801         153632  java.util.concurrent.ConcurrentHashMap$Node (java.base@11.0.5)
   8:          3395         135800  java.util.LinkedHashMap$Entry (java.base@11.0.5)
…
1804:             1             16  sun.util.resources.cldr.provider.CLDRLocaleDataMetaInfo (jdk.localedata@11.0.5)
Total        137524        7800144

Custom JRE

Pers-MBP:speedment pemi$  jmap -histo 38783 | head
 num     #instances         #bytes  class name (module)
-------------------------------------------------------
   1:         22323        1714608  [B (java.base@11.0.5)
   2:          4229         511000  java.lang.Class (java.base@11.0.5)
   3:         19447         466728  java.lang.String (java.base@11.0.5)
   4:          1776         424408  [I (java.base@11.0.5)
   5:            69         264656  [C (java.base@11.0.5)
   6:          4044         240128  [Ljava.lang.Object; (java.base@11.0.5)
   7:          4665         149280  java.util.concurrent.ConcurrentHashMap$Node (java.base@11.0.5)
   8:          3395         135800  java.util.LinkedHashMap$Entry (java.base@11.0.5)
…
1726:             1             16  sun.util.resources.LocaleData$LocaleDataStrategy (java.base@11.0.5)
Total        102904        5727960

Heap Improvement

The heap usage was reduced from 7,800,144 to 5,727,960 bytes (a reduction of over 25%)!

NB: Before I ran the jmap command, I let the application suggest an explicit Garbage Collect and wait for some seconds to even out any differences caused by potential earlier invocations of the Garbage Collector.

Overview

Here is a chart that shows the difference in storage requirements (lower is better):


Here is another chart that shows the difference in RAM usage (lower is better):


Modifying the Code

If you want to modify the code, you need to rebuild the app after your changes with:
mvn clean install

and then remove the old customjre and create a new one:
rm -rf customjre/
./build_jre.sh 

Creating Your own Database Application

If you want to connect to your own database and want to write your own application logic, you can easily select what tables and columns you want to use and then generate your own java domain model and application builder automatically using the Speedment Tool:

The Speedment tool as used in the project demonstrated in this article.

The tool can be added to your project in the pom.xml file and invoked by mvn speedment:tool. Visit the Speedment Initializer to generate your own custom pom.xml file and application template.

The process can be streamlined by automatic Maven build scripts that will identify any application dependencies and automatic generation of Docker instances that can be deployed instantly following an automatic build. I will write more about this in the coming articles.

Conclusions

The Java Platform Module System (JPMS) allows the building of highly optimized JREs suitable for cloud deployment.
It is possible to reduce both storage and RAM requirements.
Traditional ORMs do not honor full Java module encapsulation
Speedment open-source Stream ORM supports JPMS and can be used to build highly efficient database cloud applications.

Resources

Basics about JPMS modules
Speedment on GitHub
The Speedment Initializer capable of generating project pom.xml templates

Friday, November 15, 2019

Become a Master of Java Streams - Part 6: Creating a New Database Application Using Streams

Have you ever wanted to develop an "express" version of your database application? In this Hands-On Lab article, you will learn a truly easy and straightforward method. The entire Java domain model will be automatically generated for you. You just connect to your existing database and then start developing using Java streams. You will be able to create, for example, a new web application for your existing database in minutes.

This article is the last article in the series on How to Become a Master of Java Streams.

Part 1: Creating Streams
Part 2: Intermediate Operations
Part 3: Terminal Operations
Part 4: Database Streams
Part 5: Turn Joined Database Tables Into a Stream
Part 6: Creating a Database Application Using Streams

So far, you got to experience Speedment in the articles and through the exercises. For brevity, we did not include any descriptions on how to start from scratch but rather wanted you to get a glimpse of what using Java Streams with databases could look like. In this article, we’ll show you how to leverage Speedment for applications running against any of your databases. Setup only takes a few minutes but will save you vasts amounts of time due to the expressiveness of Streams and the provided type-safety.

Getting Started 

To help you configure your project, Speedment provides a project Initializer. Once you fill out the details of your project, it provides you with a zip-file containing a pom.xml with the needed dependencies and a Main.java starter.



The Speedment Initializer can be used to configure a Speedment project. 

Once you have clicked “download”, unzip the file and open the project in your IDE as a Maven project. In IntelliJ, the easiest way to do that is to choose File -> Open and then select the pom.xml-file in the unzipped project folder.

If you rather want to use Speedment in an existing project, configure your project via the Initializer to make sure you get all needed dependencies. Then simply merge the provided pom.xml with your current one and reimport Maven.

As you may recall from the previous articles, Speedment relies on an automatically generated Java domain model. Hence, before we can write our application, we need to generate the required classes. This is done using the Speedment Tool which is started by running mvn speedment:tool in the terminal or by running the same target via the IDE:s built-in Maven menu.

Firstly, you will be asked to register for a free license and connect to your database. A free license can be used for all open-source databases (unlimited use) and commercial databases (up to 500 MB and doesn’t require any billing information).
A free license can be used with all open-source databases (unlimited) and commercial databases (up to 500 MB and does not require billing information.) 

Once you complete the registration, you will be asked to provide credentials for your database (make sure you selected the correct DB-type in the initializer). Either use a local database of your own or run some tests with the Sakila database we used in the exercises.

Sakila Database Credentials 
Type: MariaDB
Host: 35.203.190.83
Port: 3306
Database name: sakila
User: sakila
Password: sakila
Fill out the database credentials to connect to your data source. (Note: Speedment never stores your database password). 

 A click on the “Connect”-button will launch the Speedment Tool. It presents the database structure to the left-hand side and settings for the selected table or column on the right-hand side. In this case, the default settings are sufficient meaning we can go ahead and press “Generate” (If your application doesn’t require all the tables and/or columns you can disable these before generating).

The Speedment Tool visualizes the data structure and allows customizations of the generated code.

Next, Speedment will analyze the database metadata and generate the entire Java domain model. Once this process is completed you are ready to write your application. If you check out the Main.java-file, you will find a project starter containing something like this:

public class Main {
    
    public static void main(final String... args) {

        Speedment app = new MyApplicationBuilder()
            .withUsername("your-dbms-username")
            .withPassword("your-dbms-password")
            .build();

        app.stop();

    }


}

From here, you are ready to build your application using the examples we have provided in the previous articles. Thereby, we can close the circle by fetching a Manager for the Film table (a handle to the content of the film table) by typing:

FilmManager films = app.getOrThrow(FilmManager.class);

Using the Manager we can now query our connected database as we have shown: 



List<Film> filmsTitleStartsWithA = films.stream()
  .filter(Film.TITLE.startsWith("A"))
  .sorted(Film.LENGTH)
  .collect(Collectors.toList());
 
filmsTitleStartsWithA: [
   FilmImpl { filmId=15, title=ALIEN CENTER, …, rating=NC-17, length = 46,
   FilmImpl { filmId=2, title=ACE GOLDFINGER, …, rating=G, length = 48,
… ]

Exercises 

This week there is no associated GitHub repo for you to play with. Instead, we encourage you to integrate Speedment in a new or an existing database application to try out your newly acquired skills.


Extra Exercise

When you are ready with your project, we encourage you to try out HyperStream, especially if you have a large database and want to increase the reading performance.

HyperStream goes beyond Stream and adds in-JVM-memory capabilities which boost application speed by orders of magnitude. You only need to add a few lines of code in your existing pom.xml and your Main.java file:

    .withBundle(InMemoryBundle.class) // add to the app builder

    ...

    // Load data from database into materialized view
    app.getOrThrow(DataStoreComponent.class) .load();

Read more in the user-guide. The Stream API remains the same but performance is vastly increased.

Conclusion

During the past six weeks, we have demonstrated the usefulness of the Java Stream API and how it can be leveraged for writing type-safe database applications in pure Java. If you wish to learn more about Speedment, check out the user guide which also contains a more thorough guide on Java Streams.

Lastly - thank you for taking interest in our article series, it has been truly great to see that many of you have been following along with the provided exercises. Happy coding!

Authors

Per Minborg
Julia Gustafsson

Resources

Further reading about Speedment Stream JOINs 
Speedment Manual 
Speedment Initializer 
Speedment on GitHub