Day 22, Java Holiday Calendar 2016, Use Enums as Method Parameters

Day 22, Java Holiday Calendar 2016, Use Enums as Method Parameters

Today's tips is about using Enums as parameters to indicate method behavior. Let us here the fairy tail of Prince Sort and it will be more apparent why this can be a good thing.

Once upon a time there was a Prince with a sort method like this:

void sort(); // 1

Everything was good until the Prince realized he needed to sort in either direction. And so, he went about and added a boolean like this:

void sort(boolean descending); // 2

Everything was good until the Prince wanted to indicate that sorting could be done in arbitrary order. And so, he went about and changed the boolean to Boolean:

void sort(Boolean descending); // 3, null means any order

Everything was good until the Prince wanted to indicate that sorting could be done in random order too. And so, he went about and changed the Boolean to an int:

void sort(int flag); // 4, 0 = asc, 1 = desc, 2 = unspec, 3 = random

Now, the most beautiful Princess appeared out of thin air and she wore a wonderful dress and she held the most prominent PhD in computer science. The Princess told the Prince "thou shalt introduce Enums in thy methods". Here is my humble gift to you:

enum Order {
    ASC, DESC, UNSPECIFIED, RANDOM;
}

so, he went about and changed the int to the Order Enum like this:

void sort(Order order);

The Prince was delighted and promised that from now on, he would introduce Enums already in step 2 and they both lived happily ever after!

Follow the Java Holiday Calendar 2016 with small tips and tricks all the way through the winter holiday season. I am contributing to open-source Speedment, a stream based ORM tool and runtime. Please check it out on GitHub.

Day 21, Java Holiday Calendar 2016, Concatenate Java Streams

Day 21, Java Holiday Calendar 2016, Concatenate Java Streams

Today's tip is about concatenating streams. The task of the day is to construct a concatenated stream that lazily consumes a number of underlying streams. So, dumping the content from the various streams into a List and then stream from the list or using the Stream.builder() will not do.

As an example, we have three streams with words that are relevant to the US history and constitution:

        // From 1787
        final Stream preamble = Stream.of(
            "We", "the", "people", "of", "the", "United", "States"
        );

        // From 1789
        final Stream firstAmendment = Stream.of(
            "Congress", "shall", "make", "no", "law", 
            "respecting", "an", "establishment", "of", "religion"
        );

        // From more recent days
        final Stream epilogue = Stream.of(
            "In", "reason", "we", "trust"
        );

Creating a concatenated stream can be done in many ways including these:

        // Works for a small number of streams
        Stream.concat(
            preamble,
            Stream.concat(firstAmendment, epilogue)
        )
            .forEach(System.out::println);

        
        // Works for any number of streams
        Stream.of(preamble, firstAmendment, epilogue)
            .flatMap(Function.identity())
            .forEach(System.out::println);

Both methods will produce the same result and they will also close the underlying streams upon completion. Personally, I prefer the latter method since it is more general and can concatenate any number of streams. This is the output of the program:

We
the
people
of
the
United
States
Congress
shall
make
no
law
respecting
an
establishment
of
religion
In
reason
we
trust

Follow the Java Holiday Calendar 2016 with small tips and tricks all the way through the winter holiday season. I am contributing to open-source Speedment, a stream based ORM tool and runtime. Please check it out on GitHub.

Day 20, Java Holiday Calendar 2016, Breakout of the Java Heap

Day 20, Java Holiday Calendar 2016, Breakout of the Java Heap

Today's tip is about storing things off heap. As we all know, Java will occasionally clean up the heap (i.e. invoke its Garbage Collector or GC for short) and remove objects that are no longer used. As the heap grows, so will the time it takes to clean it up. Eventually, the GC will take seconds or minutes and we have hit "the GC wall". Historically, this was a problem for heaps above some 10 GB of data but nowadays we can have larger heaps. How big depends on a vast number of factors.

One way of reducing GC impact is to store data off heap where the GC is not even looking. This way, we can grow to any data size without caring about the GC. The drawback is that we have to manage our memory manually and also provide a means of converting data back and forth between the two memory regions. In the general case, this is a bit tricky but if we limit ourselves to the primitive types like int, long, double and the likes, it is fairly easy.

Consider the following OffHeapIntArray:

public final class OffHeapIntArray implements Iterable<Integer> {

    private final IntBuffer buffer;
    private final int length;

    public OffHeapIntArray(int length) {
        if (length < 0) {
            throw new IllegalArgumentException();
        }
        this.length = length;
        /* Allocates memory off heap */
        this.buffer = ByteBuffer.allocateDirect(length * Integer.BYTES)
            .asIntBuffer();
    }

    public int get(int index) {
        return buffer.get(index);
    }

    public void set(int index, int value) {
        buffer.put(index, value);
    }

    public int length() {
        return length;
    }

    @Override
    public PrimitiveIterator.OfInt iterator() {
        return new Iter();
    }

    @Override
    public void forEach(Consumer<? super Integer> action) {
        for (int i = 0; i < length; i++) {
            action.accept(i);
        }
    }

    @Override
    public Spliterator.OfInt spliterator() {
        return Spliterators.spliterator(iterator(), length,
            Spliterator.SIZED
            | Spliterator.SUBSIZED
            | Spliterator.NONNULL
            | Spliterator.CONCURRENT
        );
    }

    public IntStream stream() {
        return StreamSupport.intStream(spliterator(), false);
    }

    private final class Iter implements PrimitiveIterator.OfInt {

        private int currentIndex;

        public Iter() {
            currentIndex = 0;
        }

        @Override
        public int nextInt() {
            if (hasNext()) {
                return get(currentIndex++);
            }
            throw new NoSuchElementException();
        }

        @Override
        public void forEachRemaining(IntConsumer action) {
            while (currentIndex < length) {
                action.accept(get(currentIndex++));
            }
        }

        @Override
        public boolean hasNext() {
            return currentIndex < length;
        }

        @Override
        public Integer next() {
            return nextInt();
        }

    }

}

As can be seen, it stores all the int elements off heap and allows us to do a number of things like this:

    public static final void main(String... args) {
        final OffHeapIntArray array = new OffHeapIntArray(10);
        array.set(0, 100);
        array.set(1, 101);
        array.set(9, 109);

        System.out.println("** Iterate **");
        for (int val : array) {
            System.out.println(val);
        }

        System.out.println("** Stream **");
        array.stream()
            .filter(i -> i > 0)
            .forEach(System.out::println);

    }

This will produce the following output:

** Iterate **
100
101
0
0
0
0
0
0
0
109
** Stream **
100
101
109

It is possible to create more advanced off heap classes like OffHeapMap, OffHeapArrayList etc. that store all data off heap in a similar way. Speedment Enterprise is using a more advanced version of this technology to be able to store large databases as in-JVM-memory objects. In fact, it is possible to store more data than the available RAM since byte buffers can be mapped onto files. This opens up the path to mammoth JVMs with terabytes of data available at ultra-low latency.

Follow the Java Holiday Calendar 2016 with small tips and tricks all the way through the winter holiday season. I am contributing to open-source Speedment, a stream based ORM tool and runtime. Please check it out on GitHub.

Day 19, Java Holiday Calendar 2016, Speed up Your Enums

Day 19, Java Holiday Calendar 2016, Speed up Your Enums

Today's tips is about Enum performance. A large number of Java programmers think Enums have a very fast method called .values() that returns all the Enums. Heck, it is even an array being returned so it should really be super fast? Well, not really...

Suppose we have declared an enum like this:

public enum Car {

    TESLA, VOLVO, TOYOTA;

}

then it turns out Java will generate an equivalent to the following under the hood:

    public static Car[] values() {
        return (Car[])$VALUES.clone();
    }

This means each time we are calling values(), we are creating a copy of the internal VALUES array. The reason for this is that an array cannot be protected from being overwritten by user code. Thus, a new copy must be provided for each call to guarantee state integrity. The JVM can mitigate the problem under certain circumstances but if we want to be absolutely sure no objects are created, we must implement our own equivalent to values(). This can be done like this:

public enum Car {
    TESLA, VOLVO, TOYOTA;

    private static final List<Car> VALUE_LIST = Stream.of(values())
            .collect(collectingAndThen(toList(), Collections::unmodifiableList));

    public static List<Car> valuesAsList() {
        return VALUE_LIST;
    }

}

Note that the internal VALUE_LIST is unmodifiable and that this is important, or else we cannot expose it directly via the valuesAsList() method.

Now we can use our modified Enum like this with no performance overhead :

Car.valuesAsList().forEach(System.out::println);

Follow the Java Holiday Calendar 2016 with small tips and tricks all the way through the winter holiday season. I am contributing to open-source Speedment, a stream based ORM tool and runtime. Please check it out on GitHub.

Day 18, Java Holiday Calendar 2016, Easily Create Database Content

Day 18, Easily Create Database Content

Today's tips is about creating database content. There are a number of ways to do this, ranging from writing our own entity beans combined with using JDBC directly to fully automating the entire process.

Suppose we already have a database table like this:

mysql> explain country
+------------+-------------+------+-----+---------+----------------+
| Field      | Type        | Null | Key | Default | Extra          |
+------------+-------------+------+-----+---------+----------------+
| id         | int(11)     | NO   | PRI | NULL    | auto_increment |
| name       | varchar(45) | YES  | UNI | NULL    |                |
| local_name | varchar(45) | YES  |     | NULL    |                |
| code       | int(11)     | YES  |     | NULL    |                |
| domain     | varchar(10) | YES  |     | NULL    |                
+------------+-------------+------+-----+---------+----------------+
5 rows in set (0.00 sec)

Then we could add the Speedment plugin and dependency to our POM and launch the Speedment graphic tool that will analyze the database and generate code automatically for us.

After generation we can do this:

Initialization:

final MyApplication app = new MyApplicationBuilder()
    .withPassword("myPwd729") // Replace with the real pwd
    .build();

final CountryManager countries = app.getOrThrow(CountryManager.class);

Insert DB Content:

countries.persist(
    new CountryImpl()
        .setName("Sweden")
        .setLocalName("Sverige")
        .setCode(40)       // Intentionally wrong, should be 46!!
        .setDomain(".se")
);

Update DB Content:

countries.stream()
    .filter(Country.NAME.equal("Sweden"))  // Filter out Sweden
    .map(c -> c.setCode(46))               // Update code to 46
    .forEach(countries.updater());         // Apply the database updater

Read more on Speedment on GitHub here.

Follow the Java Holiday Calendar 2016 with small tips and tricks all the way through the winter holiday season. I am contributing to open-source Speedment, a stream based ORM tool and runtime. Please check it out on GitHub.

Day 17, Java Holiday Calendar 2016, Parallel Streams in Custom Thread Pools

17. Parallel Streams in Custom Thread Pools

Today's tips is about executing parallel streams in custom thread pools. One of the main drivers for developing the Java 8 streams was their ability to abstract away parallelism. In theory, we could take any stream and make it parallel with the .parallel() method. In reality... not so easy...

One of the caveats with parallel streams is that they, by default, execute on the common Fork Join Pool. So, if there are a lot of things going on in our application, those parallel tasks will compete with all other tasks in the same pool.

This is something that can be fixes easily. A nice thing with all parallel streams is that we can submit the parallel tasks to any Thread Pool, not just the common Thread Pool. In the example below, I am showing how to stream database content in parallel using Speedment, a stream based ORM tool and runtime. This way, complex database operations may execute much faster since the tasks can be spread out on several threads.

However, the same principle would work with any type of stream source such as the ones we get from Java's built-in Collections like List or Set.

Do this:

    final ForkJoinPool forkJoinPool = new ForkJoinPool(3);
    forkJoinPool.submit(() -> 
        
        users.stream() 
            .parallel()
            .filter(User.EMAIL.isNotEmpty())
            .forEach(slowOperation()); 
            
    );

    try {
        forkJoinPool.shutdown();
        forkJoinPool.awaitTermination(1, TimeUnit.HOURS);
    } catch (InterruptedException ie) {
        ie.printStackTrace();
    } 

This will create a new Fork Join pool with three threads in which the parallel stream will be executed. This way, we limit the number of parallel task to three and they will operate separate from the common Thread Pool. Well, not entirely separate, because tasks in all pools compete for the same limited CPU resources. But that is another story...

Follow the Java Holiday Calendar 2016 with small tips and tricks all the way through the winter holiday season. I am contributing to open-source Speedment, a stream based ORM tool and runtime. Please check it out on GitHub.

Day 16, Java Holiday Calendar 2016, Hacking Java Classes

16. Hacking the Existing Java Classes

Today's tips is about hacking the existing Java classes. But before we start, I would like to issue a disclaimer: I am only showing this to illustrate certain possibilities. This post contains code that  is against best-practice and also a number of license terms for commercial JVMs and I would discourage anyone from using this for purposes other than just experimenting in dark cellars at late nights.

The Task

There is a lot of magic going around with auto-boxing and instance caching with wrapper classes like java.lang.Integer. Suppose we could hack the Integer class so it will keep track of how many of its instances are created and how many instances are garbage collected and that we could add a way of retrieving this statistics in some form? How do we go about?

Open Sesame

The JVM has a command line parameter named "-Xbootclasspath/p:" that can be used to load Java classes before the standard java classes are loaded. This way, we can inject our own versions of Java classes before the real ones are loading.

This is useful also for other classes that are not a part of the standard Java library. If there is a bug in a third party product, we can correct the bug and just load the correct version before the app loads. It also enables us to instrument classes so they, for example, can collect usage statistics or introduce or prevent certain things from happening.

Hacking the Integer class

Create a custom class named Integer and place that in a package named java.lang. Retrieve the standard source code of Integer form the JDK and copy that into the custom java.lang.Integer class.

Add the following at the beginning of the copied Integer class:

    public static final AtomicLong INSTANCE_CREATE_COUNTER = new AtomicLong();
    public static final AtomicLong INSTANCE_FINALIZE_COUNTER = new AtomicLong();

Modify the constructor like this:

    public Integer(int value) {
        this.value = value;
        INSTANCE_CREATE_COUNTER.incrementAndGet();
    }

Finally, add the following method:

    @Override
    protected void finalize() throws Throwable {
        INSTANCE_FINALIZE_COUNTER.incrementAndGet();
    }

That's it! We have hacked the Integer class. The modified class can now be used like this:

package org.darkside.hack;

public class Main {

    public static void main(String[] args) {

        printIntegerInstanceInfo();
        Integer i = 456;
        printIntegerInstanceInfo();
        System.gc();
        sleep(1000);
        printIntegerInstanceInfo();

    }

    public static void printIntegerInstanceInfo() {
        long create = Integer.INSTANCE_CREATE_COUNTER.get();
        long finalized = Integer.INSTANCE_FINALIZE_COUNTER.get();
        System.out.format(
            "The JVM has created %d instances and finalized %d instances of Integer."
                + " Thus, %d instances are on the heap.%n",
            create,
            finalized,
            create - finalized
        );
    }

    public static void sleep(long ms) {
        try {
            Thread.sleep(ms);
        } catch (InterruptedException ie) {

        }
    }

}

Running the Code

The code can be run like this:

java -Xbootclasspath/p:hack_java_class-1.0.0-SNAPSHOT.jar org.darkside.hack.Main

We have to change the name of the .jar file depending how we named our artifact. Make sure that the modified Integer class and the Main class resides in the same jar. We can also have the modified classes in a separate jar but then we need to add the other parts using the normal class path commands.

Wrap up

The notion here is "Don't Try This at Home". That said, we can learn a lot by injecting custom classes into the JVM.

There are many better ways of keeping track of how many instances that are created and destroyed by the JVM. The task in this post is just for the sake of reasoning.

Be careful out there!