Minborg

Minborg
Minborg

Wednesday, February 3, 2016

Plugin Your Own Components with Java 8

Components are Good!


Spire and Duke is plugging in...
A couple of years ago when I worked in the telecom industry, I spent time cleaning up Java code that mixed the concept of abstraction (interfaces) and implementation (concrete classes implementing one or several interfaces).

Another related problem I have come across several times, is the use of Singletons, used to hold various implementations. To be fair, I have created more than a couple of those singletons myself, but nowadays, I almost never use them.

In this post, I will show why and how to avoid them and at the same time gain some additional advantages like separation of interfaces and implementations using Components and a ComponentHandler.

The ComponentHandler that is shown in the examples a bit further down is derived from the open-source project Speedment that I am contributing to. Speedment is great if you want to write database applications in Java 8 using standard Java 8 Streams. Read more about Speedment here.

Why Are Singletons Bad?

The purpose of a singleton is to guarantee that just a single instance of a class or interface is present at any single time. This is of course not a bad thing in it self. On the contrary, there are many situations when we want this property, for example for logging or for other system components like queues or factories. However, the way people chose to enforce the singleton property is more often than not, simply bad.

In my previous post Enforcing Java Singletons is Very Hard, I presented a number of Singleton Idioms that can be used to enforce the singleton property and in the same post, I showed that it is hard to really enforce the singleton patterns (albeit "cheating" with reflections) and also talked about some other disadvantages of singletons.

Here is one of the idioms from the post, called the Enum Singleton Idiom which I first learned about in Joshua Bloch's book "Effective Java" many years ago.

public enum FooEnumSingleton {

    INSTANCE;

    public static FooEnumSingleton getInstance() { return INSTANCE; }

    public void bar() { }

}

This provides a relatively solid and simple singleton, but there are some drawbacks. Because all enums inherit from the built-in Java class Enum, we can not inherit from other base classes. Furthermore, we can not override methods when we want to test our singleton in JUnit tests, for example if we want to "harnes" our class, call some methods and then inspect its state or functionality. Additionally, there is no support or control over the singletons Lifecycle Management. When we first reference the singleton above, the class loader will create and initialize it, pretty much outside our control. If another programmer calls our singleton, it will load no matter what. Now, if our singleton depends on other singletons, which each depends on yet another singleton and so on, our code will likely break. Perhaps not in the first version where things were easy, but it is just a matter of time until it undoubtedly will fail.

I will talk more about lifecycles in the next section.

Component Lifecycles

The Lifecycle of a component or class can be certain stages the component will pass during the time it exists. For example, a component may be created, added, started, stopped and and then eventually, removed from a system. Using the right tools, a component can easily be setup to react to its own lifecycle events.

The Component Handler

Often, it is very convenient to setup some kind of Component Handler that keeps track of all our singleton components. This way, we can have them all collected at one place and treat them in a consistent way. The Component Handler can also be made to automatically call the component's lifecycle methods whenever they are added, changed or removed from the Component Handler.

Now, do not fall for the temptation to make the Component Handler a singleton under the excuse that "now it is just one singleton". Admittedly, it is better to have just one singleton than having 100 singletons, but it is generally even better to just let our Component Handler be just a normal object and keep track of the Component Handler instance in our code. Especially if we are deploying our application in a web server, where there might be many instances of the "Singletons" under different class loaders.

The Class Mapper

The classical (apologies for the bad word-joke) "Class Mapper" is a good starting point when we want to implement a Component Handler. Take a look at this example:

public interface ClassMapper {

    /**
     * Puts a new mapping and returns the previous mapping for the given class,
     * or null if no mapping existed.
     *
     * @param <K> key class
     * @param <V> item type
     * @param keyClass to use
     * @param item to add
     * @return the previous mapping that existed, or null 
     *         if no mapping existed
     */
    <K, V extends K> K put(Class<K> keyClass, V item);

    /**
     * Returns the mapping for the given class, or null if no mapping exists
     *
     * @param <K> The class type
     * @param clazz the class to use
     * @return the mapping for the given class, or null if no mapping exists
     */
    public <K> K get(Class<K> clazz);

}

and a corresponding (almost trivial) implementation:

public class ClassMapperImpl implements ClassMapper {

    private final Map<Class<?>, Object> map;

    public ClassMapperImpl() {
        this.map = new ConcurrentHashMap<>();
    }

    @Override
    public <K, V extends K> K put(Class<K> keyClass, V item) {
        requireNonNull(keyClass);
        requireNonNull(item);
        @SuppressWarnings("unchecked")
        // We know that it is a safe cast
        // because V extends K
        final K result = (K) map.put(keyClass, item);
        return result;
    }

    @Override
    public <K> K get(Class<K> clazz) {
        @SuppressWarnings("unchecked")
        // We know that it is a safe cast
        // because the previous put()
        // guarantees this
        final K result = (K) map.get(clazz);
        return result;
    }

}

The good thing with this is that now we have a small rudimentary Component Handler. Suppose that we have an interface like this:

public interface JokeComponent {

    String nextJoke();

}
The nextJoke() method is supposed to return a funny joke, and we want a pluggable concept so we are planning to implement several versions of the JokeComponent. We start by constructing a very simple JokeComponent that just returns a single joke regardless how many times we call the nextJoke() method:

public class StaticJokeComponent implements JokeComponent {

    @Override
    public String nextJoke() {
        return "I went to buy some camouflage trousers the other day,"
                + " but I couldn’t find any.";
    }

}
Now we can test our Component Handler as shown in the following program:

public class Main {

    public static void main(String[] args) {
        final ClassMapper componentHandler = new ClassMapperImpl();
        setupComponents(componentHandler);

        crankOutTenJokes(componentHandler);
    }

    private static void setupComponents(ClassMapper componentHandler) {
        // Plug in the selected basic JokeGenerator
        componentHandler.put(JokeComponent.class, new StaticJokeComponent());
    }

    private static void crankOutTenJokes(ClassMapper componentHandler) {
        // Get the current JokeGenerator that is plugged into the componentHandler
        JokeComponent jokeGenerator = componentHandler.get(JokeComponent.class);
        // Tell the world who is making the jokes
        System.out.println(jokeGenerator.getClass().getSimpleName() + " says:");
        // print ten of its joke
        IntStream.rangeClosed(1, 10)
                .mapToObj(i -> jokeGenerator.nextJoke())
                .forEach(System.out::println);
    }

}

This will produce the following jokes:

StaticJokeComponent says:
I went to buy some camouflage trousers the other day, but I couldn’t find any.
I went to buy some camouflage trousers the other day, but I couldn’t find any.
I went to buy some camouflage trousers the other day, but I couldn’t find any.
I went to buy some camouflage trousers the other day, but I couldn’t find any.
I went to buy some camouflage trousers the other day, but I couldn’t find any.
I went to buy some camouflage trousers the other day, but I couldn’t find any.
I went to buy some camouflage trousers the other day, but I couldn’t find any.
I went to buy some camouflage trousers the other day, but I couldn’t find any.
I went to buy some camouflage trousers the other day, but I couldn’t find any.
I went to buy some camouflage trousers the other day, but I couldn’t find any.
Even though the joke might be a bit fun the first or second time, we start to lose interest a bit further down the list... Time for improvement by writing a new JokeComponent that doesn't tell the same joke all the time.

public class RandomJokeComponent implements JokeComponent {

    private final Random random = new Random();
    private final List<String> jokes = Arrays.asList(
            "What's blue and doesn't weigh much? Light blue.",
            "What do you get when you cross a joke with a rhetorical question?",
            "What do you call a fish with no eyes? A fsh.",
            "Two men walk into a bar... the third one ducks.",
            "A farmer rounded up her 196 cows and got 200 of them."
    );

    @Override
    public String nextJoke() {
        return jokes.get(random.nextInt(jokes.size()));
    }

}
Now we can plug in this JokeComponent instead and see what happens. Modify the code so it looks like this instead:

    private static void setupComponents(ClassMapper componentHandler) {
        // Plug in the selected basic JokeGenerator
        // Use the RandomJokeGenerator this time instead ---+
        //                                                  V
        componentHandler.put(JokeComponent.class, new RandomJokeComponent());
    }
Now we get another output:
What's blue and doesn't weigh much? Light blue.
Two men walk into a bar... the third one ducks.
A farmer rounded up her 196 cows and got 200 of them.
What do you call a fish with no eyes? A fsh.
What do you get when you cross a joke with a rhetorical question?
A farmer rounded up her 196 cows and got 200 of them.
What do you get when you cross a joke with a rhetorical question?
What do you call a fish with no eyes? A fsh.
What's blue and doesn't weigh much? Light blue.
What do you call a fish with no eyes? A fsh.
This is a small step forward for the humor society, but we could as well write yet another implementation of the JokeComponent that, whenever nextJoke() is called, goes out on the internet and retrieves a funny story from some site. The user code outside the component would remain exactly the same.

Note that the Component Handler can handle any number of interfaces and the interfaces can look completely different and contain different methods, so you could set it up to also handle RidleComponent and perhaps also BlooperComponent that could, for instance, return URL:s to funny films on the net.

The ComponentHandler with Lifecycles


It is fairly easy to expand the Component Handler so it can handle Component Lifecycles. Let us define a simple Lifecycle Component like this.

public interface Component {

    void added();

    void removed();
}
So, every time we add a Component to the Component Handler, we want the Component Handler to automatically invoke the Component::add method and every time it is removed we want it to call the Component::removed method. That way, we know that all component in the Component Handler will be initialized (by the added() method) properly and also that they will clean up stuff (if needed by the removed() method) when they are removed from the Component Handler. Evidently, every component that we want to add to the Component Handler must implement the Component interface. First, we improve the ClassMapper interface so it takes a generic parameter for the values to extend.

public interface ClassMapper<T> {

    <K extends T, V extends K> K put(Class<K> keyClass, V item);

    public <K extends T> K get(Class<K> clazz);

}

Then we could write a ComponentHandler as depicted here under:

public class ComponentHandler implements ClassMapper<Component> {

    private final Map<Class<? extends Component>, Component> map;

    public ComponentHandler() {
        this.map = new ConcurrentHashMap<>();
    }

    @Override
    public <K extends Component, V extends K> K put(Class<K> keyClass, V item) {
        requireNonNull(keyClass);
        requireNonNull(item);
        item.added();
        @SuppressWarnings("unchecked")
        // We know that it is a safe cast
        // because V extends K
        final K result = (K) map.put(keyClass, item);
        if (result != null) {
            result.removed();
        }
        return result;
    }

    @Override
    public <K extends Component> K get(Class<K> clazz) {
        @SuppressWarnings("unchecked")
        // We know that it is a safe cast
        // because the previous put()
        // guarantees this
        final K result = (K) map.get(clazz);
        return result;
    }

}

I have made some simplifications in the code, but the general outline becomes obvious. Whenever we add a Component, its added() method will be called and whenever we replace a component with a new one, the old's removed() method is called.

In the example below, we are to make a Queue for Strings that is a pluggable Component in the new ComponentHandler we just wrote. We could define a StringQueueComponent as a Component like this:

public interface StringQueueComponent extends Component {

    boolean offer(String msg);

    String poll();
}
We could start with a very straightforward implementation using one of the existing Queue implementation already in the Java framework.

public class MemoryStringQueueComponent implements StringQueueComponent {

    private final Queue<String> queue = new ConcurrentLinkedQueue<>();

    @Override
    public boolean offer(String msg) {
        return queue.offer(msg);
    }

    @Override
    public String poll() {
        return queue.poll();
    }

    @Override
    public void added() {
        System.out.println(getClass().getSimpleName() + " added");
    }

    @Override
    public void removed() {
        queue.clear();
        System.out.println(getClass().getSimpleName() + " removed");
    }

}
and then we may come up with an alternative Queue that places the strings on a file as shown in this outline (I have not shown the actual code for file handling because it is quit bulky, but the principle is obvious).

public class FileStringQueueComponent implements StringQueueComponent {

    // File variable goes here

    @Override
    public boolean offer(String msg) {
        // Append the string to the file
    }

    @Override
    public String poll() {
        // Check if the file has grown
        // if yes, read the next line and return it
        // if no, return null
    }

    @Override
    public void added() {
        // Open the file and trunkate it if contains an old queue.
        // Set a pointer to the current file location
    }

    @Override
    public void removed() {
        // Close the file and remove it from the file system
    }

}
So now, when we put the new ComponentHandler to use as depicted below:

public class Main {

    public static void main(String[] args) {
        ComponentHandler componentHandler = new ComponentHandler();
        setupComponents(componentHandler);
        ///
        putStuffInTheStingQueue(componentHandler);
        printStuffInTheStringQueue(componentHandler);

        // Select a new queue component (content of the old queue is lost)
        componentHandler.put(StringQueueComponent.class, new FileStringQueueComponent());

        putStuffInTheStingQueue(componentHandler);
        printStuffInTheStringQueue(componentHandler);

    }

    private static void putStuffInTheStingQueue(ComponentHandler componentHandler) {
        StringQueueComponent queue = componentHandler.get(StringQueueComponent.class);
        queue.offer("A");
        queue.offer("B");
        queue.offer("C");
    }

    private static void printStuffInTheStringQueue(ComponentHandler componentHandler) {
        StringQueueComponent queue = componentHandler.get(StringQueueComponent.class);
        System.out.println("Stuff in " + queue.getClass().getSimpleName());
        String s;
        while ((s = queue.poll()) != null) {
            System.out.println(s);
        }
    }

    private static void setupComponents(ComponentHandler componentHandler) {
        componentHandler.put(StringQueueComponent.class, new MemoryStringQueueComponent());
    }

}
it will produce the following output (given a real implementation of the FileStringQueueComponent):

MemoryStringQueueComponent added
Stuff in MemoryStringQueueComponent
A
B
C
FileStringQueueComponent added
MemoryStringQueueComponent removed
Stuff in FileStringQueueComponent
A
B
C

Conclusion


By using a Component Framework, we gain many advantages over singletons :

  • We can control the lifecycle of our components.
  • The ability to test our components becomes much better.
  • We get a clear separation between interfaces an implementations.
  • We collect all our component in one place and access them in a standardized way.
  • We can ensure that our component is not called before it is initialized.
  • Our code will be less likely to fail in a multi-classloader scenario like a web server.


If you want to take a look at a real existing component handler, have a look at Speedment's  component handler on GitHub. For performance reasons, this component handler is more optimized so that we can obtain components without the overhead of looking them up in a Map.


Thursday, January 14, 2016

Be Lazy with Java 8

Background

One of the most distinguished feature of us programmers is that we are inherently lazy. Not in a bad way that we do not want to work, but in a better way: We do not want to do the same thing twice and we do not want to do it at all if we do not have to. In fact, not writing code is often the better alternative in the cases you can reuse something else instead.

The same thing is true for our applications. Often, we want them to be lazy so that they only do what is absolutely necessary and nothing more.

I have used the Lazy class presented here in the open-source project Speedment that makes database applications really short and concise.

Read more on how we can make our applications lazy in this post.

Implementing Lazy Initialization

In this post, the goal is to show a Lazy class that can be used for objects with a relatively long life expectancy and where there might be any number of calls (from zero to the millions) to a particular method. We must also ensure that the class is thread safe. Lastly, we want to have maximum performance for different threads calling the class many times.

Here is the proposed class:

public final class Lazy<T> {

    private volatile T value;

    public T getOrCompute(Supplier<T> supplier) {
        final T result = value;  // Read volatile just once...
        return result == null ? maybeCompute(supplier) : result;
    }

    private synchronized T maybeCompute(Supplier<T> supplier) {
        if (value == null) {
            value = requireNonNull(supplier.get());
        }
        return value;
    }

}


The Lazy class can be used in many applications. Immutable classes are especially good candidates for lazy initialization. For example, Java's built-in String class employs lazy initialization in its hashCode() method. Here is one example how we can use the Lazy class:
public class Point {

    private final int x, y;
    private final Lazy<String> lazyToString;

    public Point(int x, int y) {
        this.x = x; 
        this.y = y;
        lazyToString = new Lazy<>();
    }

    @Override
    public String toString() {
        return lazyToString.getOrCompute( () -> "(" + x + ", " + y + ")");
    }

}

Looking back on the Lazy class again, we see that it only contains a single “holder” field for its value (I will explain why the field is not declared volatile later on) (EDIT: the field must be volatile to guarantee our requirements). There is also a public method getOrCompute() that allows us to retrieve the value. This method also takes a Supplier that will be used if and only if the value has not been set previously. The Supplier must produce a non-null value. Note the use of a local variable result, allowing us to reduce the number of volatile reads from two to one where the Lazy instance has been initialized already. Before I explain the features of this particular implementation, we need to revisit the Java Memory Model and particularly variable visibility across threads. If you want, you can skip the next chapter and just accept that Lazy works as it supposed to do. However, I do encourage you to read on.

The Java Memory Model and Visibility

One of the key issues with the Java Memory Model is the concept of visibility. If Thread 1 updates a variable someValue = 2 then when would the other threads (e.g. Thread 2) see this update? It turns out that Thread 1’s update will not be seen immediately by other threads. In fact, there is no guarantee as to how quickly a change in this variable will be seen by other threads at all. It could be 100 ns, 1 ms, 1 s or even 10 years in theory. There are performance reasons for isolating the java memory view between threads. Because each thread can have its own memory view, the level of parallelism will be much higher than if threads were supposed to share and guarantee the same memory model.

Some of the benefits with relaxed visibility are that it allows:

  • The compiler to reorder instructions in order to execute more efficiently
  • The compiler to cache variables in CPU registers
  • The CPU to defer flushing of writes to main memory
  • Old entries in reading processors’ caches to be used

The Java keywords final, synchronized and volatile allows us to change the visibility of objects across threads. The Java Memory Model is quite a big topic and perhaps I will write a more elaborate post on the issue later on. However, in the case of synchronization, a thread that enters a synchronization block must invalidate its local memory (such as CPU registers or cache entries that involves variables inside the synchronization block) so that reads will be made directly from main memory. In the same way, a thread that exists a synchronization block must flush all its local memory. Because only one thread can be in a synchronization block at any given time, the effect is that all changes to variables are effectively visible to all threads that enters the synchronization block. Note that threads that do not enter the synchronization block does not have any visibility guarantee.

Also, If a field is declared volatile, reads and writes are always made via main memory and in order. Thus, updates to the field are seen by other threads at the cost of performance.

Properties of the Lazy Class

The field value is declared volatile and in the previous chapter we just learned that there are guarantees for visibility in that case and also, more importantly, guarantees of exact timing and in-order execution. So, if Thread 1 calls the Supplier and sets the value, Thread 2 might not see the update. If so, Thread 2 will enter the maybeCompute() method and because it is synchronized it will now, in fact, see Thread 1's update and it will see that the value was already set. From now on, Thread 2 will have a correct view of the value and it will never enter the synchronization block again. This is good if Thread 2 is expected to call the Lazy class many times. If another Thread 3 is created much later, it will most likely see the correct value from the beginning and we avoid synchronization altogether. So, for medium to long lived objects, this scheme is a win! We get thread isolation with no synchronization overhead.

When Are Lazy Appropriate to Use?

Lazy is a good choice if we want to defer calculation to a later time and we do not want to repeat the calculation. If we, on the other hand, know in advance that our toString() method is always going to be called many times, then we would not use the Lazy class. Instead, we could just calculate the toString() value once and for all eagerly in the constructor and store its value for later re-use.

Conclusion

The Lazy class is a very simple, yet powerful means of deferred calculation and a nice tool for performance optimization. The Lazy performs exceptionally well under the circumstances it was constructed for, with no thread synchronization overhead whatsoever for medium and long lived objects.

The Lazy class, as shown, is used in the open-source project Speedment in a number of applications including SQL rendering where, for example, the columns for a table remains the same during the JVM lifetime. Speedment is a tool that allows access to databases using standard Java 8 streams.

Be lazy and “steal” the Lazy class here so that your applications can be lazy too...

Friday, December 18, 2015

My Previous Post on Video

Escape Analysis Aftermath


My post on Java Escape Analysis has triggered a lot of interest. I made a follow up post on the effect of inlining on Escape Analysis.

A company that provides Java programming training filmed the post and had one person summarize the content. The company is called Webucator.

Have a look at the presentation here.



Keep on hackin'



Wednesday, December 16, 2015

Java 8: The JVM Can Re-capture Objects That Have Escaped

Background


In my previous post, I wrote about Escape Analysis and how the JVM can allocate non-escaping objects on the stack rather than on the heap. I immediately got a very interesting question from Caleb Cushing asking if Objects that actually can escape could be optimized anyhow, provided that that escaped object is reasonably contained by the caller.

Read this post and find out the answer!


A Simple Example

Let's assume that we have the following simple Person class:

public class Person {

    private final String firstName;
    private final String middleName;
    private final String lastName;

    public Person(String firstName, String middleName, String lastName) {
        this.firstName = requireNonNull(firstName);  // Cannot be null
        this.middleName = middleName;                // Can be null
        this.lastName = requireNonNull(lastName);    // Cannot be null
    }

    public String getFirstName() {
        return firstName;
    }

    public Optional<String> getMiddleName() {
        return Optional.ofNullable(middleName);
    }

    public String getLastName() {
        return lastName;
    }

}

Now, if we call the method Person::getMiddleName, it is obvious that the Optional object will escape the method because it is returned by the method and becomes visible to anyone calling the method. Thus, it will be classified as GlobalEscape and must be allocated on the heap. However, this is not necessarily the case. The JVM will sometimes be able to allocate it on the stack, despite the fact that it escapes the method. How is that possible?


What is Escape Analysis (EA)?

Before you read on, I encourage you to read my previous post because it will be more easy to understand what is going on. The post describes the fundamental aspects of EA.

How Can GlobalEscape Objects Still Live on the Stack?

It turns out that the C2 compiler is able to do EA not only over single methods, but over larger chunks of code that is inlined by the compiler. Inlining is an optimization scheme where the code is "flattened" to eliminate redundant calls. So, one or several layers of calls are flattened to a sequential list of instructions. The compiler then evaluates EA, not on the individual methods, but on the entire inlined code block. So, even though an object might escape a particular method, it might not be able to escape the larger inlined code block. 

A Demonstration of Inlined Escape Analysis

public class Main2 {

    public static void main(String[] args) throws IOException {

        Person p = new Person("Johan", "Sebastian", "Bach");

        count(p);
        System.gc();
        System.out.println("Press any key to continue");
        System.in.read();
        long sum = count(p);

        System.out.println(sum);
        System.out.println("Press any key to continue2");
        System.in.read();

        sum = count(p);

        System.out.println(sum);
        System.out.println("Press any key to exit");
        System.in.read();

    }

    private static long count(Person p) {
        long count = 0;
        for (int i = 0; i < 1_000_000; i++) {
            if (p.getMiddleName().isPresent()) {
                count++;
            }
        }
        return count;

    }

}

The code above will create a single instance of a Person and then it will call that Person's getMiddleName() method a large number of times. We will do it in three steps where the first step is just for warming up and then GC away all the objects that were created. The two following steps will not remove anything from the heap and we will be able to examine the heap between each step.We can use the following JVM parameters when we run the code:

-server
-XX:BCEATraceLevel=3
-XX:+PrintCompilation
-XX:+UnlockDiagnosticVMOptions
-XX:+PrintInlining
-verbose:gc
-XX:MaxInlineSize=256
-XX:FreqInlineSize=1024
-XX:MaxBCEAEstimateSize=1024
-XX:MaxInlineLevel=22
-XX:CompileThreshold=10
-Xmx4g
-Xms4g


After the first run, we get the following heap usage (after the System.gc() call cleaned up all our Optionals)

pemi$ jps | grep Main2
74886 Main2
 num     #instances         #bytes  class name
----------------------------------------------
   1:            95       42952184  [I
   2:          1062         101408  [C
   3:           486          55384  java.lang.Class
   4:           526          25944  [Ljava.lang.Object;
   5:            13          25664  [B
   6:          1040          24960  java.lang.String
   7:            74           5328  java.lang.reflect.Field

The two following steps gave:

pemi$ jmap -histo 74886 | head

 num     #instances         #bytes  class name
----------------------------------------------
   1:            95       39019792  [I
   2:        245760        3932160  java.util.Optional
   3:          1063         101440  [C
   4:           486          55384  java.lang.Class
   5:           526          25944  [Ljava.lang.Object;
   6:            13          25664  [B
   7:          1041          24984  java.lang.String
pemi$ jmap -histo 74886 | head

 num     #instances         #bytes  class name
----------------------------------------------
   1:            95       39019544  [I
   2:        245760        3932160  java.util.Optional
   3:          1064         101472  [C
   4:           486          55384  java.lang.Class
   5:           526          25944  [Ljava.lang.Object;
   6:            13          25664  [B
   7:          1042          25008  java.lang.String

No new Optionals were created between step two and step three and thus, EA was eventually able to eliminate the creation of the Optional instances on the heap even though they escaped the initial method where they were created and returned. This means that we can use an appropriate level of abstraction and still retain performant code.

Conclusions

Escape Analysis can work on several layers in our code. EA can optimize away heap allocation even though objects escapes one or several methods.  As with EA in general, we do not get a guarantee that we will get the optimizations we are expecting in all cases.

The open-source project Speedment that I am contributing to, often returns Streams containing entities or Optionals. The fact that EA works on several layers makes the application code run faster. The JVM is able to inline code from the Speedment library into the application code itself and then, using EA, temporary return objects are never allocated on the heap. So, Speedment developers can enjoy a nice API while still retaining high performance and low latency
.



Wednesday, December 2, 2015

Do Not Let Your Java Objects Escape

Background

I am working on the Open Source project Speedment and for us contributors, it is important to use code that people can understand and improve. It is also important that performance is good, otherwise people are likely to use some other solution. 

Escape Analysis allows us to write performant code at the same time as we can use good code style with appropriate abstractions.


This is Escape Analysis

Escape Analysis (also abbreviated as "EA") allows the Java compiler to optimize our code in many ways.  Please consider the following simple Point class:

public class Point {

    private final int x, y;

    public Point(int x, int y) {
        this.x = x;
        this.y = y;
    }

    @Override
    public String toString() {
        final StringBuilder sb = new StringBuilder()
                .append("(")
                .append(x)
                .append(", ")
                .append(y)
                .append(")");
        return sb.toString();
    }

}

Each time we call the Point::toString method, it looks like a new StringBuilder object is created. However, as we can see, the StringBuilder object is not visible from outside the method. It cannot be observed neither from outside the method nor by another thread running the same piece of code (because that other thread would se its own version of the StringBuilder).

So, after calling the method some million times, there might be millions of StringBuilder objects lying around? Not so! By employing EA, the compiler can allocate the StringBuilder on the stack instead. So when our method returns, the object is automatically deleted upon return, as the stack pointer is restored to the previous value it had before the method was called.

Escape analysis has been available for a relatively long time in Java. In the beginning we had to enable it using command line options, but nowadays it is used by default. Java 8 has an improved Escape Analysis compared to previous Java versions.


How It Works

Based on EA, an object's escape state will take on one of three distinct values:
  • GlobalEscape: An object may escape the method and/or the thread. Clearly, if an object is returned as the result of a method, its state is GlobalEscape. The same is true for objects that are stored in static fields or in fields of an object that itself is of state GlobalEscape. Also, if we override the finalize() method, the object will always be classified as GlobalEscape and thus, it will be allocated on the heap. This is logical, because eventually the object will be visible to the JVM's finalizer. There are also some other conditions that will render our object's status GlobalEscape.
  • ArgEscape: An object that is passed as an argument to a method but cannot otherwise be observed outside the method or by other threads.
  • NoEscape: An object that cannot escape the method or thread at all.


GlobalEscape and ArgEscape objects must be allocated on the heap, but for ArgEscape objects it is possible to remove some locking and memory synchronization overhead because these objects are only visible from the calling thread.

The NoEscape objects may be allocated freely, for example on the stack instead of on the heap. In fact, under some circumstances, it is not even necessary to construct an object at all, but instead only the object's scalar values, such as an int for the object Integer. Synchronization may be removed too, because we know that only this thread will use the objects. For example, if we were to use the somewhat ancient StringBuffer (which as opposed to StringBuilder has synchronized methods), then these synchronizations could safely be removed.

EA is currently only available under the C2 HotSpot Compiler so we have to make sure that we run in -server mode.


Why It Matters

In theory, NoEscape objects objects can be allocated on the stack or even in CPU registers using EA,  giving very fast execution.

When we allocate objects on the heap, we start to drain our CPU caches because objects are placed on different addresses on the heap possibly far away from each other. This way we will quickly deplete our L1 CPU cache and performance will decrease. With EA and stack allocation on the other hand, we are using memory that (most likely) is already in the L1 cache anyhow.  So, EA and stack allocation will improve our localization of data. This is good from a performance standpoint.

Obviously, the garbage collects needs to run much less frequently when we are using EA with stack allocation. This is perhaps the biggest performance advantage. Recall that each time the JVM runs a complete heap scan, we take performance out of our CPUs and the CPU caches will quickly deplete. Not to mention if we have virtual memory paged out on our server, whereby GC is devastating for performance.

The most important advantage of EA is not performance though. EA allows us to use local abstractions like Lambdas, Functions, Streams, Iterators etc. without any significant performance penalty so that we can write better and more readable code. Code that describes what we are doing rather than how it is done.

A Small Example

public class Main {

    public static void main(String[] args) throws IOException {
        Point p = new Point(100, 200);

        sum(p);
        System.gc();
        System.out.println("Press any key to continue");
        System.in.read();
        long sum = sum(p);

        System.out.println(sum);
        System.out.println("Press any key to continue2");
        System.in.read();
        
        sum = sum(p);

        System.out.println(sum);
        System.out.println("Press any key to exit");
        System.in.read();

    }

    private static long sum(Point p) {
        long sumLen = 0;
        for (int i = 0; i < 1_000_000; i++) {
            sumLen += p.toString().length();
        }
        return sumLen;

    }

}


The code above will create a single instance of a Point and then it will call that Point's toString() method a large number of times. We will do it in three steps where the first step is just for warming up and then GC away all the objects that were created. The two following steps will not remove anything from the heap and we will be able to examine the heap between each step.

If we run the program with the following parameters, we will be able to see what is going on within the JVM:

-server
-XX:BCEATraceLevel=3
-XX:+PrintCompilation
-XX:+UnlockDiagnosticVMOptions
-XX:+PrintInlining
-verbose:gc
-XX:MaxInlineSize=256
-XX:FreqInlineSize=1024
-XX:MaxBCEAEstimateSize=1024
-XX:MaxInlineLevel=22
-XX:CompileThreshold=10
-Xmx4g
-Xms4g

And yes, that is a huge pile of parameters but we really want to be able to see what is going on.

After the first run, we get the following heap usage (after the System.gc() call cleaned up all our StringBuilders)

pemi$ jps | grep Main
50903 Main
pemi$ jmap -histo 50903 | head
 num     #instances         #bytes  class name

----------------------------------------------
   1:            95       42952184  [I
   2:          1079         101120  [C
   3:           485          55272  java.lang.Class
   4:           526          25936  [Ljava.lang.Object;
   5:            13          25664  [B
   6:          1057          25368  java.lang.String
   7:            74           5328  java.lang.reflect.Field

The two following steps gave:

pemi$ jmap -histo 50903 | head
 num     #instances         #bytes  class name
----------------------------------------------
   1:       2001080       88101152  [C
   2:           100       36777992  [I
   3:       1001058       24025392  java.lang.String
   4:         64513        1548312  java.lang.StringBuilder
   5:           485          55272  java.lang.Class
   6:           526          25936  [Ljava.lang.Object;
   7:            13          25664  [B


pemi$ jmap -histo 50903 | head
 num     #instances         #bytes  class name
----------------------------------------------
   1:       4001081      176101184  [C
   2:       2001059       48025416  java.lang.String
   3:           105       32152064  [I
   4:         64513        1548312  java.lang.StringBuilder
   5:           485          55272  java.lang.Class
   6:           526          25936  [Ljava.lang.Object;
   7:            13          25664  [B

As can be seen, EA was eventually able to eliminate the creation of the StringBuilder instances on the heap. There were only 64K created compared to the 2M Stings. A big improvement!

Conclusions

The advantages of Escape Analysis are nice in theory but they are somewhat difficult to understand and predict. We do not get a guarantee that we will get the optimizations we are expecting in all cases but it seems to work reasonably well under common conditions.

Check out open-source Speedment and see if you can spot the places where we rely on Escape Analysis.

Hopefully, this post contributed to shed some light on EA so that you opt to write good code over "performant" code.

I would like to thank Peter Lawery for the tips and suggestions I got from him in connection with writing this post.

Read more on Objects in general here


Thursday, November 12, 2015

Easily Create Database Content with Java 8

Database Connectivity Now and Then

Spire and Duke adding stuff
to a database.
I remember back in the old (Java) days, when we were sitting up late nights and experimented a lot with Java and databases. In the beginning of the Java era, there was not much support for database connectivity and so we had to basically write your own database classes and handle ResultSets, Connections and SQLExceptions by ourself.

Nowadays, we expect the simple things just to happen! Suppose that we have an existing database and we want to add or update information in it using a Java application. How can we do that in a simple way without having to write a lot of boilerplate code?

I have contributed a lot to the Java 8 Open Source project Speedment, that can be used to very easily extract Java code from existing database schemas and start coding applications directly.

Let's take it for a spin.

Example

Let's say we have a MySQL database table that is supposed to contain data on various countries. The table could look something like this:

mysql> explain country +------------+-------------+------+-----+---------+----------------+ | Field      | Type        | Null | Key | Default | Extra          | +------------+-------------+------+-----+---------+----------------+ | id         | int(11)     | NO   | PRI | NULL    | auto_increment | | name       | varchar(45) | YES  | UNI | NULL    |                | | local_name | varchar(45) | YES  |     | NULL    |                | | code       | int(11)     | YES  |     | NULL    |                | | domain     | varchar(10) | YES  |     | NULL    |                +------------+-------------+------+-----+---------+----------------+ 5 rows in set (0.00 sec)

Let us further pretend that we have the task of populating the table with a few countries and see how that can be solved.

Setting up the Speedment Project

In order to set up a project with Speedment, we need to include a few lines in our POM.xml file, connect to the database and generate code. Read more on how to do this here!.

Also, check out this film how easy it is:

Initializing the Database Connection

Now that we have our database domain model generated automatically, we can start with the actual coding for inserting data into the database. First, we need to setup our Java 8 database project like this:

// Setup Speedment speedment = new JavapotApplication().withPassword("javapot").build(); Manager<Country> countries = speedment.managerOf(Country.class);

The JavapotApplication class was generated automatically from the database schema and contains all meta data (like columns and tables) of the database. Note that we manually need to provide the password since this is not stored in the meta data model (for security reasons). The countries variable is a "handle" for the table we are about to work with.

There is really no "magic" going on with the generation. We can see all the generated Java files in clear text and we can change them or introduce our own versions if we want.

Inserting Data in the Database

Once the setup is made, is is very easy to insert data rows in the database like this:

try { countries.newInstance() .setName("United States") .setLocalName("United States") .setCode(1) .setDomain(".us") .persist(); countries.newInstance() .setName("Germany") .setLocalName("Deutschland") .setCode(49) .setDomain(".de") .persist(); // Needless to say, you can call the setters in any order. countries.newInstance() .setDomain(".uk") .setCode(44) .setName("United Kingdom") .setLocalName("United Kingdom") .setDomain(".uk") .persist(); countries.newInstance() .setName("Sweden") .setLocalName("Sverige") .setCode(40) // Intentionally wrong, should be 46!! .setDomain(".se") .persist(); } catch (SpeedmentException se) { // Handle the exception here }

The newInstance() method creates a new empty Country object and then we just use the setters to initialize the country. After all parameters are set, we call the persist() method to store the object in the database. If there is an error during the database insert, a SpeedmentException will be thrown, allowing you to examine why (for example if you are trying to insert two countries with the same name).  I intentionally picked the wrong country call code for Sweden (förlåt Sverige) so that we can learn how to update data in our database too.

Updating Data in the Database

If you want to update an existing row in the database you can do it like this:

countries.stream() .filter(Country.NAME.equal("Sweden")) .findAny() .ifPresent(c -> c.setCode(46).update());

This will create a Stream with all the counties that have the name "Sweden" (which, evidently, is only one country) and then it will try to find that country and if it is present, it will take that country and set the code to 46 (which is the correct calling code for Sweden) and then it will update the selected country in the database. It is important to understand that even though our country table might contain a large number of countries, it will only include those countries in the stream that are satisfying an equivalent query of "select * from country where name='Sweden' " in this case.

Now our database looks like this:
mysql> select * from country; +----+----------------+----------------+------+--------+ | id | name           | local_name     | code | domain | +----+----------------+----------------+------+--------+ |  1 | United States  | United States  |    1 | .us    | |  2 | Germany        | Deutschland    |   49 | .de    | |  3 | United Kingdom | United Kingdom |   44 | .uk    | |  4 | Sweden         | Sverige       |   46 | .se    | +----+----------------+----------------+------+--------+ 4 rows in set (0.00 sec)

Success! Mission accomplished!

Contributing 

Read more about Speedment Open Source on www.speedment.org and that's the place to be if you want to learn more things like how the API looks like and how you use Speedment in your projects. Speedment is here on GitHub. You can contribute by submitting comments on gitter or download the source code and create pull requests with your own code contributions.

Conclusions

With Java 8, you can easily write database applications with almost no extra manual code work. There are tools that automatically can extract your domain model from a database schema.

These days, we Java programmers can can put more time on the actual problem (and perhaps get some more well deserved sleep) instead of fiddling around with basic database functionality.



Thursday, October 15, 2015

Java 8, Query Databases Using Streams

Mapping Streams
When I wrote my first Java database application back in the late 90's, I had to do everything myself. There was a lot of code, error capturing and object conversions going on and the code rapidly became very difficult to maintain and, I have to admit, error prone.

Even today, when you work with databases, it is often cumbersome to bridge between the two fundamentally different worlds of an object oriented language like Java and a relational database. Often, you need write your own mapping layer or you can use an Object Relational Mapper (ORM) like Hibernate. It is often convenient to use an ORM, but sometimes it is not so easy to configure it in the right way. More often than not, the ORM also slows down your application.

Recently, I have been contributing a lot to a new Open Source project named "Speedment" that we contributors are hoping will make life easier for us Java 8 database application developers.

What is Speedment Open Source?

Speedment Open Source is a new library that provides many interesting Java 8 features. It is written entirely in Java 8 from start. Speedment uses standard Streams for querying the database and thanks to that, you do not have to learn any new query API. You do not have to think at all about JDBC, ResultSet and other database specific things either.

Speedment analyses an existing database and generates code automatically. That way, you do not have to write a single line of code for database entities. The generated code even contains auto-generated JavaDocs. This means that you do not have to write entity classes like "User" or "Book" in your Java code, instead you just create (or use an existing) the database, connect Speedment to it and Speedment will analyze the database structure and generate the entity classes by analyzing the database structure.

Spire

Because Speedment's Open Source mascot is a hare, I use hares in many of my examples. Let us suppose that we have an existing (MySQL) table called "hare" that looks like this:






mysql> explain hare;
+-------+-------------+------+-----+---------+----------------+
| Field | Type        | Null | Key | Default | Extra          |
+-------+-------------+------+-----+---------+----------------+
| id    | int(11)     | NO   | PRI | NULL    | auto_increment |
| name  | varchar(45) | NO   |     | NULL    |                |
| color | varchar(45) | NO   |     | NULL    |                |
| age   | int(11)     | NO   |     | NULL    |                |
+-------+-------------+------+-----+---------+----------------+
4 rows in set (0.01 sec)

Then Speedment will generate a corresponding entity (JavaDocs removed for brevity) :

public interface Hare extends Entity<Hare> {
   
    public final static ReferenceComparableField<Hare, Integer> ID = new ReferenceComparableFieldImpl<>("id", Hare::getId, Hare::setId);
    public final static ReferenceComparableStringField<Hare> NAME = new ReferenceComparableStringFieldImpl<>("name", Hare::getName, Hare::setName);
    public final static ReferenceComparableStringField<Hare> COLOR = new ReferenceComparableStringFieldImpl<>("color", Hare::getColor, Hare::setColor);
    public final static ReferenceComparableField<Hare, Integer> AGE = new ReferenceComparableFieldImpl<>("age", Hare::getAge, Hare::setAge);
    
    Integer getId();
    String getName();
    String getColor();
    Integer getAge();

    Hare setId(Integer id);
    Hare setName(String name);
    Hare setColor(String color);
    Hare setAge(Integer age);
    
    /** Graph-like traversal methods eliminating JOINs */
    Stream<Carrot> findCarrotsByOwner();
    Stream<Carrot> findCarrotsByRival();
    Stream<Carrot> findCarrots();
}

I will explain the find*() methods in a separate post. They can be used instead of SQL joins.

Queries

Here is an example how it could look like when you are querying the Hare table :

List<Hare> oldHares = hares.stream()
    .filter(AGE.greaterThan(8))
    .collect(toList());

Smart Streams

Now, it looks like the code above will stream over all rows in the "hare" database table, but it is actually not. The Stream is "smart" and will, upon reaching its Terminal Operation collect(), analyze the filter Predicate and conclude that it is actually the "hare.age" column that is compared to 8 and thus, it will be able to reduce the stream of hares to something corresponding to "select * from hare where age > 8". If you are using several filters, they are of course further combined to reduce the stream even more.

Just to show the principle, here is another stream with more operations:

long noOldHares = hares.stream()
    .filter(AGE.greaterThan(8))
    .mapToInt(Hare::getAge)
    .sorted()
    .count();

When this Stream reaches its Terminal Operation count(), it will inspect its own pipeline. It will then conclude that it can reduce the AGE Predicate just like in the previous example. It will furthermore conclude that the operations mapToInt() and sorted() do not change the outcome of count() and thus, these operations can be eliminated all together. So the statement is reduced to "select count(*) from hare where age > 8".

This means that you can use Java 8 streams while you do not have to care so much about how the streams are translated to SQL.

How to Download and Contribute

Read more about Speedment Open Source on www.speedment.org and that's the place to be if you want to learn more things like how the API looks like and how you use Speedment in your projects. Speedment is here on GitHub. You can contribute by submitting comments on gitter or download the source code and create pull requests with your own code contributions.

Conclusions

Looking back on some of my old projects at the dawn of the Java era, one of my database classes (with over 100 lines) could now be reduced to a single line of Java 8 code. That is Moores law, but inverted! In 14 years (=7 Moore cycles) the number of rows halved about 7 times. That is progress!

Speedment and Java 8 is the new super-awesome alternative to traditional ORMs.