Thursday, August 1, 2019

Why Declarative Coding Makes You a Better Programmer

Declarative solutions with functional composition provide superior code metrics over legacy imperative code in many cases. Read this article and understand how to become a better programmer using declarative code with functional composition.

In this article, we will take a closer look at three problem examples and examine two different techniques (Imperative and Declarative) for solving each of these problems.

All source code in this article is open-source and available at https://github.com/minborg/imperative-vs-declarative. In the end, we will also see how the learnings of this article can be applied in the field of database applications. We will use Speedment Stream as an ORM tool, since it provides standard Java Streams that correspond to tables, views and joins from databases and supports declarative constructs.

There is literally an infinite number of example candidates that can be used for code metrics evaluation.

Problem Examples

In this article, I have selected three common problems we developers might face over the course of our job days:

SumArray

Iterating over an array and perform a calculation

GroupingBy

Aggregating values in parallel

Rest

Implementing a REST interface with pagination

Solution Techniques

As implied at the beginning of this article, we will be solving said problems using these two coding techniques:

Imperative

An Imperative Solution in which we use traditional code styles with for-loops and explicitly mutable states.

Declarative

A Declarative Solution where we compose various functions to form a higher-order composite function that solves the problem, typically using java.util.stream.Stream or variants thereof.

Code Metrics

The idea is then to use static code analysis applied to the different solutions using SonarQube (here SonarQube Community Edition, Version 7.7) ) so that we may derive useful and standardized code metrics for the problem/solution combinations. These metrics would then be compared.

In the article, we will be using the following code metrics:

LOC

“LOC” means “Lines-Of-Code” and is the number of non-empty lines in the code.

Statements

Is the total number of statements in the code. There could be zero to many statements on each code line.

Cyclomatic Complexity

Indicates the complexity of the code and is a quantitative measure of the number of linearly independent paths through a program's source code. For example, a single “if” clause presents two separate paths through the code. Read more here on Wikipedia.

Cognitive Complexity

SonarCube claims that “Cognitive Complexity breaks from the practice of using mathematical models to assess software maintainability. It starts from the precedents set by Cyclomatic Complexity, but uses human judgment to assess how structures should be counted and to decide what should be added to the model as a whole. As a result, it yields method complexity scores which strike programmers as fairer relative assessments of maintainability than have been available with previous models.” Read more here on SonarCube’s own page.

More often than not, it is desirable to conceive a solution where these metrics are small, rather than large.

For the record, it should be noted that any solution devised below is just one way of solving any given problem. Let me know if you know a better solution and feel free to submit a pull request via https://github.com/minborg/imperative-vs-declarative.

Iterating over an Array

We start off with an easy one. The object with this problem example is to compute the sum of the elements in an int array and return the result as a long. The following interface defines the problem:

public interface SumArray {

    long sum(int[] arr);
}

Imperative Solution

The following solution implements the SumArray problem using an imperative technique:

public class SumArrayImperative implements SumArray {

    @Override
    public long sum(int[] arr) {
        long sum = 0;
        for (int i : arr) {
            sum += i;
        }
        return sum;
    }

}

Declarative Solution

Here is a solution that implements SumArray using a declarative technique:

public class SumArrayDeclarative implements SumArray {

    @Override
    public long sum(int[] arr) {
        return IntStream.of(arr)
            .mapToLong(i -> i)
            .sum();
    }
}
Note that IntStream::sum only returns an int and therefore we have to apply the intermediate operation mapToLong().

Analysis

SonarQube provides the following analysis:






The code metrics for SumArray are shown in the following table (lower is generally better):

Technique LOC Statements Cyclomatic Complexity Cognitive Complexity
Imperative 12 5 2 1
Functional 11 2 2 0

This is how it looks in a graph (lower is generally better):


Aggregating Values in Parallel

The object with this problem example is to group Person objects into different buckets, where each bucket constitutes a unique combination of the birth year of a person and the country that a person is working in. For each group, the average salary shall be computed. The aggregation shall be computed in parallel using the common ForkJoin pool.

This is how the (immutable) Person class looks like:

public final class Person {

    private final String firstName;
    private final String lastName;
    private final int birthYear;
    private final String country;
    private final double salary;

    public Person(String firstName, 
                  String lastName, 
                  int birthYear, 
                  String country, 
                  double salary) {
        this.firstName = requireNonNull(firstName);
        this.lastName = requireNonNull(lastName);
        this.birthYear = birthYear;
        this.country = requireNonNull(country);
        this.salary = salary;
    }

    public String firstName() { return firstName; }
    public String lastName() { return lastName; }
    public int birthYear() { return birthYear; }
    public String country() { return country; }
    public double salary() { return salary; }

    // equals, hashCode and toString not shown for brevity
}

We have also defined another immutable class called YearCountry that shall be used as the grouping key:

public final class YearCountry {

    private final int birthYear;
    private final String country;

    public YearCountry(Person person) {
        this.birthYear = person.birthYear();
        this.country = person.country();
    }

    public int birthYear() { return birthYear; }
    public String country() { return country; }

    // equals, hashCode and toString not shown for brevity
}
Having defined these two classes, we can now define this problem example by means of this interface:

public interface GroupingBy {

    Map<YearCountry, Double> average(Collection<Person> persons);

}

Imperative Solution

It is non-trivial to implement an imperative solution to the GroupingBy example problem. Here is one solution that solves the problem:

public class GroupingByImperative implements GroupingBy {

    @Override
    public Map<YearCountry, Double> average(Collection<Person> persons) {
        final List<Person> personList = new ArrayList<>(persons);
        final int threads = ForkJoinPool.commonPool().getParallelism();
        final int step = personList.size() / threads;

        // Divide the work into smaller work items
        final List<List<Person>> subLists = new ArrayList<>();
        for (int i = 0; i < threads - 1; i++) {
            subLists.add(personList.subList(i * step, (i + 1) * step));
        }
        subLists.add(personList.subList((threads - 1) * step, personList.size()));


        final ConcurrentMap<YearCountry, AverageAccumulator> accumulators = new ConcurrentHashMap<>();
        // Submit the work items to the common ForkJoinPool
        final List<CompletableFuture<Void>> futures = new ArrayList<>();
        for (int i = 0; i < threads; i++) {
            final List<Person> subList = subLists.get(i);
            futures.add(CompletableFuture.runAsync(() -> average(subList, accumulators)));
        }

        // Wait for completion
        for (int i = 0; i < threads; i++) {
            futures.get(i).join();
        }

        // Construct the result
        final Map<YearCountry, Double> result = new HashMap<>();
        accumulators.forEach((k, v) -> result.put(k, v.average()));

        return result;
    }

    private void average(List<Person> subList, ConcurrentMap<YearCountry, AverageAccumulator> accumulators) {
        for (Person person : subList) {
            final YearCountry bc = new YearCountry(person);
            accumulators.computeIfAbsent(bc, unused -> new AverageAccumulator())
                .add(person.salary());
        }
    }

    private final class AverageAccumulator {
        int count;
        double sum;

        synchronized void add(double term) {
            count++;
            sum += term;
        }

        double average() {
            return sum / count;
        }
    }

}

Declarative Solution

Here is a solution that implements GroupingBy using a declarative construct:

public class GroupingByDeclarative implements GroupingBy {

    @Override
    public Map<YearCountry, Double> average(Collection<Person> persons) {
        return persons.parallelStream()
            .collect(
                groupingBy(YearCountry::new, averagingDouble(Person::salary))
            );
    }
}
In the code above, I have used some static imports from the Collectors class (e.g. Collectors::groupingBy). This does not affect the code metrics.

Analysis

SonarQube provides the following analysis:




The code metrics for GroupingBy are shown in the following table (lower is better):


Technique LOC Statements Cyclomatic Complexity Cognitive Complexity
Imperative 52 27 11 4
Functional 17 1 1 0

The corresponding graph looks like this (lower is generally better):


Implementing a REST Interface

In this exemplary problem, we are to provide a pagination service for Person objects. Persons appearing on a page must satisfy some (arbitrary) conditions and are to be sorted in a certain given order. The page shall be returned as an unmodifiable List of Person objects.

Here is an interface that captures the problem:

public interface Rest {

/**
 * Returns an unmodifiable list from the given parameters.
 *
 * @param persons as the raw input list
 * @param predicate to select which elements to include
 * @param order in which to present persons
 * @param page to show. 0 is the first page
 * @return an unmodifiable list from the given parameters
 */
 List<Person> page(List<Person> persons, 
                   Predicate<Person> predicate,
                   Comparator<Person> order,
                   int page);
}
The size of a page is given in a separate utility class called RestUtil:

public final class RestUtil {
    private RestUtil() {}

    public static final int PAGE_SIZE = 50;
}

Imperative Solution

Here is an imperative implementation of the Rest interface:

public final class RestImperative implements Rest {

    @Override
    public List<Person> page(List<Person> persons, 
                             Predicate<Person> predicate, 
                             Comparator<Person> order, 
                             int page) {
        final List<Person> list = new ArrayList<>();
        for (Person person:persons) {
            if (predicate.test(person)) {
                list.add(person);
            }
        }
        list.sort(order);
        final int from = RestUtil.PAGE_SIZE * page;
        if (list.size() <= from) {
            return Collections.emptyList();
        }
        return unmodifiableList(list.subList(from, Math.min(list.size(), from + RestUtil.PAGE_SIZE)));
    }
}

Declarative Solution

The following class implements the Rest interface in a declarative way:

public final class RestDeclarative implements Rest {

    @Override
    public List<Person> page(List<Person> persons,
                             Predicate<Person> predicate, 
                             Comparator<Person> order,
                             int page) {
        return persons.stream()
            .filter(predicate)
            .sorted(order)
            .skip(RestUtil.PAGE_SIZE * (long) page)
            .limit(RestUtil.PAGE_SIZE)
            .collect(collectingAndThen(toList(), Collections::unmodifiableList));
    }
}

Analysis

SonarQube provides the following analysis:




The following table shows the code metrics for Rest (lower is generally better):

Technique LOC Statements Cyclomatic Complexity Cognitive Complexity
Imperative 27 10 4 4
Functional 21 1 1 0

Here, the same numbers are shown in a graph (again lower is generally better):


Java 11 Improvements

The examples above were written in Java 8. With Java 11, we could shorten our declarative code using LVTI (Local Variable Type Inference). This would make our code a bit shorter but would not affect code metrics.

@Override
public List<Person> page(List<Person> persons,
                         Predicate<Person> predicate, 
                         Comparator<Person> order, 
                         int page) {
    final var list = new ArrayList<Person>();
    ...
Compared to Java 8, Java 11 contains some new collectors. For example, the Collectors.toUnmodifiableList() which would make our declarative Rest solution a bit shorter:

public final class RestDeclarative implements Rest {

@Override
public List<Person> page(List<Person> persons,
                         Predicate<Person> predicate, 
                         Comparator<Person> order, 
                         int page) {
    return persons.stream()
        .filter(predicate)
        .sorted(order)
        .skip(RestUtil.PAGE_SIZE * (long) page)
        .limit(RestUtil.PAGE_SIZE)
        .collect(toUnmodifiableList());
}
Again, this will not impact the code metrics.

Summary

Averaging the code metrics for our three exemplary problems yields the following result (lower is generally better) :



Given the input requirements in this article, there is a remarkable improvement for all code metrics when we go from imperative to declarative constructs.

Use Declarative Constructs in Database Applications

In order to reap the benefits of declarative constructs in database applications, we have used Speedment Stream. Speedment Stream is a Stream-based Java ORM tool that can turn any database table/view/join into Java streams and thereby allows you to apply your declarative skills in database applications.

Your database applications code will get much better. In fact, a pagination REST solution with Speedment and Spring Boot against a database might be expressed like this:

public Stream<Person> page(Predicate<Person> predicate, 
                           Comparator<Person> order, 
                           int page) {
    return persons.stream()
        .filter(predicate)
        .sorted(order)
        .skip(RestUtil.PAGE_SIZE * (long) page)
        .limit(RestUtil.PAGE_SIZE);
}

Where the Manager<Person> persons is provided by Speedment and constitutes a handle to the database table “Person” and can be @AutoWired via Spring.

Conclusions

Choosing declarative over imperative solutions can reduce general code complexity massively and can provide many benefits including faster coding, better code quality, improved readability, less testing, reduced maintenance costs and more.

In order to benefit from declarative constructs within database applications, Speedment Stream is a tool that can provide standard Java Streams directly from the database.

Mastering declarative constructs and functional composition is a must for any contemporary Java developer these days.

Resources

Article Source Code: https://github.com/minborg/imperative-vs-declarative
SonarQube: https://www.sonarqube.org/
Speedment Stream: https://speedment.com/stream/
Speedment Initializer: https://www.speedment.com/initializer/

9 comments:

  1. > public class SumArrayDeclarative implements SumArray {

    @Override
    public long sum(int[] arr) {
    return IntStream.of(arr)
    .mapToLong(i -> i)
    .sum();
    }
    }

    This is not a declarative solution. You are doing the computations still right inside the function. Just doing it in some ugly and not clear way. Some streams and magic for such a simple task. The imperative code is much better, much clearer because it does not use additional streams magic and does not required the reader to know that magic.

    The declarative approach is something like this

    public class ArraySum implements Number {
    private final int[] arr;

    public ArraySum(final int[] arr) {
    this.arr = arr;
    }

    @Override
    public long longValue() {
    long sum = 0;
    for (int i : arr) {
    sum += i;
    }
    return sum;
    }
    }

    Number sum = new ArraySum(arr);

    In this example I'm just returning a object, a Number which is a sum of array elements. I'm declaring that it is the sum of array elements. No computations is done at that moment. I'm just declaring.

    ReplyDelete
    Replies
    1. Thanks for your comment.

      The solution devised above is indeed a declarative solution where the portion "IntStream.mapToLong(i -> i)" is the declarative part where lower level functions are composed to yield a higher order function. The operation ".sum()" is then applied to said construct to compute some value.

      Streams are in no way "magic". On the contrary, it is a well established and proven part of Java for many years.

      I am curious to see evidence supporting the statement that imperative code is much better given that standard code metrics indicates just the opposite. Which other code metrics do you mean support coding with imperative style?

      The fact that the exemplary problems were structured as an interface and the solutions implementing such interfaces does not have any bearing on the solution being imperative or declarative or of some other sort.

      It is true that the example you show breaks up declaration of a class and computation into two parts. However, this is not the generally used definition of "declarative constructs".

      Thanks again for your comment.

      Delete
  2. import java.util.Arrays;

    class Main {
    public static void main(String[] args) {
    System.out.println(sum(0,new long[]{1,2,3,4,5}));
    }

    private static long sum(long head, long[] tail) {
    return tail.length == 1 ? head + tail[0] : sum(head + tail[0], Arrays.copyOfRange(tail, 1, tail.length));
    }
    }

    ReplyDelete
    Replies
    1. Thanks for this alternate solution!

      After fixing a zero-size array bug above and adapting to the problem interface, the recursive code looks like this:

      @Override
      public long sum(int[] arr) {
      return sum(0, arr);
      }

      private static long sum(long head, int[] tail) {
      return tail.length == 0
      ? head
      : sum(head + tail[0], Arrays.copyOfRange(tail, 1, tail.length));
      }

      However, the code will only work for small arrays. Larger arrays will produce a StackOverflowError because the thread's stack is limited. Also, the code will exhibit worse code metrics compared to both the perviously devised Declarative and Imperative solutions in the article.

      Interesting solution anyhow and good to learn how this solution stacks up against others.

      Delete
  3. class Main {
    public static void main(String[] args) {
    System.out.println(sum(0,new long[]{1,2,3,4,5},0));
    }

    private static long sum(long head, long[] tail, int index) {
    while (index != tail.length) {
    head = head + tail[index++];
    }
    return head;
    }
    }

    ReplyDelete
    Replies
    1. Thanks again for yet another proposal. This code works perfectly well and it has the following code metric properties:

      LOC: 15
      Statements: 4
      Cyclomatic Complexity: 3
      Cognitive Complexity: 1

      So, it is slightly worse than the two solutions devised in the article but nevertheless interesting to see.


      I suspect that the performance of this solution is not as good as the solutions provided in the article but that is another story and perhaps an issue for another article.

      Delete
    2. This is the evaluated code:

      public class SumArrayOther implements SumArray {

      @Override
      public long sum(int[] arr) {
      return sum(0, arr, 0);
      }

      private static long sum(long head, int[] tail, int index) {
      while (index != tail.length) {
      head = head + tail[index++];
      }
      return head;
      }
      }

      Delete
  4. Which other code metrics do you mean support coding with imperative style?

    You can not measure "readiblity and maintainablity" with existing tools. Those are way more important for 99.99999999999% of all code written. Someone not as "clever" as you is going to come along behind you and you do not want to be on the hook for something they can not maintain. All a manager will hear is "this is opaque I do not know what it does, why it does it or how, I need to spend even more time re-aquiring the requirements that this is supposed to solve, figure out if it actually meets them, if so, how and how to modify it. Which is met with "just re-write it to be clearer" and you get a silent reputation of writing crap code. 37+ years of reading and writing code tells me this.

    ReplyDelete
    Replies
    1. Thanks for you comments.

      Apparently, you have almost as many years as I do in terms of code experience. That is good!

      It is important to remember that the article never states any code metrics for "readability" and "maintainability" and thus, it is never formally claimed that these parameters are always better/worse with any particular coding style. As you rightfully point out, there are no known serious tool that can measure said properties.

      The article do however claim that "Declarative ... constructs ... CAN provide many benefits including faster coding, better code quality, improved readability, less testing, reduced maintenance costs and more". Seen over a greater population and a greater example set, it is very likely that this is the case. We are many working here in the office for example that think so.

      The article do claim and, in fact proves, that the code metrics of LOC, Statements, Cyclomatic Complexity and Cognitive Complexity (as defined) are improved with Declarative constructs for the given examples. This is not an opinion but rather a fact.

      It is highly unlikely that "reliability and maintainability" are only trumped by any other arbitrary requirement (such as performance and dependability) with a probability of 1E-12 as you state. I'll guess you mean that "readability and maintainability is important" but failed to capture that semantically. We can all agree on that. My personal opinion is that declarative code is easier to read and maintain but I know that there are people that disagree with this.

      People coming along and using "clever" streams are very common. As I mentioned in previous comments, Streams are in no way obscure and has been a part of Java since Java 8 (released March 2014) it should be expected that any Java programmer worth her salt should be familiar with Streams and lambdas.

      In my opinion, based on creating and maintaining one of the most popular Java ORMs on the planet, declarative constructs generally presents better performance, shorter development time, less errors, shorter code, simpler unit tests and more. However, there are cases when imperative coding could be the better choice and there are also some downsides with declarative coding, for example stack trace readability. On the other hand, you are, in my opinion, much less likely to ever read a stack trace.

      I am curious to learn if you have references to articles or papers that points to the contrary of this article.

      Thanks again for your comment

      Delete

Note: Only a member of this blog may post a comment.