Java: Aggregate Data Off-Heap

Java: Aggregate Data Off-Heap

Explore how to create off-heap aggregations with a minimum of garbage collect impact and

maximum memory utilization.

Creating large aggregations using Java Map, List and Object normally creates a lot of heap memory overhead. This also means that the garbage collector will have to clean up these objects once the aggregation goes out of scope.

Read this short article and discover how we can use Speedment Stream ORM to create off-heap aggregations that can utilize memory more efficiently and with little or no GC impact.

Person

Let’s say we have a large number of Person objects that take the following shape:

public class Person {
    private final int age;
    private final short height;
    private final short weight;        
    private final String gender;
    private final double salary;
    …
    // Getters and setters hidden for brievity
}

For the sake of argument, we also have access to a method called persons() that will create a new Stream with all these Person objects.

Salary per Age

We want to create the average salary for each age bucket. To represent the results of aggregations we will be using a data class called AgeSalary which associates a certain age with an average salary.

public class AgeSalary {
     private int age;
     private double avgSalary;
     … 
    // Getters and setters hidden for brievity
}

Age grouping for salaries normally entails less than 100 buckets being used and so this example is just to show the principle. The more buckets, the more sense it makes to aggregate off-heap.

Solution

Using Speedment Stream ORM, we can derive an off-heap aggregation solution with these three steps:

Create an Aggregator

var aggregator = Aggregator.builderOfType(Person.class, AgeSalary::new)
    .on(Person::age).key(AgeSalary::setAge)
    .on(Person::salary).average(AgeSalary::setAvgSalary)
    .build();

The aggregator can be reused over and over again.

Compute an Aggregation

var aggregation = persons().collect(aggregator.createCollector());

Using the aggregator, we create a standard Java stream Collector that has its internal state completely off-heap.

Use the Aggregation Result

aggregation.streamAndClose()
    .forEach(System.out::println);

Since the Aggregation holds data that is stored off-heap, it may benefit from explicit closing rather than just being cleaned up potentially much later. Closing the Aggregation can be done by calling the close() method, possibly by taking advantage of the AutoCloseable trait, or as in the example above by using streamAndClose() which returns a stream that will close the Aggregation after stream termination.

Everything in a One-Liner

The code above can be condensed to what is effective a one-liner:

persons().collect(Aggregator.builderOfType(Person.class, AgeSalary::new)
    .on(Person::age).key(AgeSalary::setAge)
    .on(Person::salary).average(AgeSalary::setAvgSalary)
    .build()
    .createCollector()
).streamAndClose()
    .forEach(System.out::println);

There is also support for parallel aggregations. Just add the stream operation Stream::parallel and aggregation is done using the ForkJoin pool.

Resources

Download Speedment here

Read more about off-heap aggregations here

Day 20, Java Holiday Calendar 2016, Breakout of the Java Heap

Day 20, Java Holiday Calendar 2016, Breakout of the Java Heap

Today's tip is about storing things off heap. As we all know, Java will occasionally clean up the heap (i.e. invoke its Garbage Collector or GC for short) and remove objects that are no longer used. As the heap grows, so will the time it takes to clean it up. Eventually, the GC will take seconds or minutes and we have hit "the GC wall". Historically, this was a problem for heaps above some 10 GB of data but nowadays we can have larger heaps. How big depends on a vast number of factors.

One way of reducing GC impact is to store data off heap where the GC is not even looking. This way, we can grow to any data size without caring about the GC. The drawback is that we have to manage our memory manually and also provide a means of converting data back and forth between the two memory regions. In the general case, this is a bit tricky but if we limit ourselves to the primitive types like int, long, double and the likes, it is fairly easy.

Consider the following OffHeapIntArray:

public final class OffHeapIntArray implements Iterable<Integer> {

    private final IntBuffer buffer;
    private final int length;

    public OffHeapIntArray(int length) {
        if (length < 0) {
            throw new IllegalArgumentException();
        }
        this.length = length;
        /* Allocates memory off heap */
        this.buffer = ByteBuffer.allocateDirect(length * Integer.BYTES)
            .asIntBuffer();
    }

    public int get(int index) {
        return buffer.get(index);
    }

    public void set(int index, int value) {
        buffer.put(index, value);
    }

    public int length() {
        return length;
    }

    @Override
    public PrimitiveIterator.OfInt iterator() {
        return new Iter();
    }

    @Override
    public void forEach(Consumer<? super Integer> action) {
        for (int i = 0; i < length; i++) {
            action.accept(i);
        }
    }

    @Override
    public Spliterator.OfInt spliterator() {
        return Spliterators.spliterator(iterator(), length,
            Spliterator.SIZED
            | Spliterator.SUBSIZED
            | Spliterator.NONNULL
            | Spliterator.CONCURRENT
        );
    }

    public IntStream stream() {
        return StreamSupport.intStream(spliterator(), false);
    }

    private final class Iter implements PrimitiveIterator.OfInt {

        private int currentIndex;

        public Iter() {
            currentIndex = 0;
        }

        @Override
        public int nextInt() {
            if (hasNext()) {
                return get(currentIndex++);
            }
            throw new NoSuchElementException();
        }

        @Override
        public void forEachRemaining(IntConsumer action) {
            while (currentIndex < length) {
                action.accept(get(currentIndex++));
            }
        }

        @Override
        public boolean hasNext() {
            return currentIndex < length;
        }

        @Override
        public Integer next() {
            return nextInt();
        }

    }

}

As can be seen, it stores all the int elements off heap and allows us to do a number of things like this:

    public static final void main(String... args) {
        final OffHeapIntArray array = new OffHeapIntArray(10);
        array.set(0, 100);
        array.set(1, 101);
        array.set(9, 109);

        System.out.println("** Iterate **");
        for (int val : array) {
            System.out.println(val);
        }

        System.out.println("** Stream **");
        array.stream()
            .filter(i -> i > 0)
            .forEach(System.out::println);

    }

This will produce the following output:

** Iterate **
100
101
0
0
0
0
0
0
0
109
** Stream **
100
101
109

It is possible to create more advanced off heap classes like OffHeapMap, OffHeapArrayList etc. that store all data off heap in a similar way. Speedment Enterprise is using a more advanced version of this technology to be able to store large databases as in-JVM-memory objects. In fact, it is possible to store more data than the available RAM since byte buffers can be mapped onto files. This opens up the path to mammoth JVMs with terabytes of data available at ultra-low latency.

Follow the Java Holiday Calendar 2016 with small tips and tricks all the way through the winter holiday season. I am contributing to open-source Speedment, a stream based ORM tool and runtime. Please check it out on GitHub.