Tuesday, April 22, 2014

Java: Evolution of Handling Null References

Using null in programming is playing with fire. It's powerful and sometimes the right choice, but the infamous NullPointerException can sneak in quickly. That's why I advocate for every software project to include a section about handling null in the coding guidelines.

The first time I've seen a Java exception was in the 90s in the web browser's status bar when surfing the net: a NPE caused by an applet. I had no idea what it meant and what I had done wrong. Today I know that the programmer let a null reference slip in where it wasn't expected to happen.

When looking at open source software written in Java I often come across code where null references are used but not documented. Sometimes preconditions are in place. When stepping deeper it gets really tricky to figure out which variable is now allowed to be null and which is not. For the author it was obvious at the time of writing the code... but software becomes better when written and looked over by many, and thus it is important to make it clear for everyone, everywhere.

Software is constantly shipped with NPEs detected later on. Redeployments and bugfix releases are expensive - and it's a pity because this kind of bug could be eliminated almost completely.


Here's my personal evolution of dealing with null in Java.

Level 1: No Plan

Using it wherever, no information about it in the Javadoc.
Fixing NPEs as they occur, seeing no problem in that.
public class Foo {
    
    public String getText() {
        return null;
    }
    
}

Level 2: Document Null - Sometimes

Realizing that NPEs are a problem that could and should be avoided. Detecting them late is expensive, especially after deployment.
Starting to document variables, arguments and return values that can be null... incomplete.
public class Foo {
    
    /**
     * @return the text, or null
     */
    public String getText() {
        return null;
    }
    
}

Level 3: Add "Null" to Method Names

Realizing it's still a problem. Having Javadoc is nice, but useless when not read.
Starting to name methods with null such as getFooOrNull() to force it to be seen.
public class Foo {
    
    /**
     * @return the text, or null
     */
    public String getTextOrNull() {
        return null;
    }
    
}

Level 4: Code Annotations

Using Jetbrains' null annotations: Annotating all variables with @Nullable and @NotNull. This is a huge step forward. The crippled method name pattern 'OrNull' is obsolete. The code is automatically documented.
public class Foo {
    
    @Nullable
    public String getText() {
        return null;
    }
    
}
And the best part: the IDE checks the code and warns on a mistake.


Using these annotations strictly I don't remember causing a single NPE in years. The drawback is more typing, more text on the screen. But then we use static typing, and not defining null is just incomplete.

Level 5: Guava's Optional

The concept: Instead of using the null reference directly, wrap it in another object that permits null. The getter methods on it myOptional.get(), myOptional.orNull() and myOptional.or(alternative) force the user to think about what to do when it's null. The API becomes extremely clean.

If you don't know about it yet, read the Guava page about using and avoiding null.

It took me a couple of days to get used to this. And I did produce a handful of bugs initially because I've used Optional.of() instead of Optional.fromNullable() by mistake.

Although step 4 with annotations already got rid of NPEs and improved the code confidence, this was another big step forward. The API of the Optional class is clean, and the API of code using it is consistent and clear. A developer only has to learn the concept once, and because it's from Guava, sooner or later everyone will know it.

The Guava team uses @Nullable annotations for the few places where null is permitted. Anywhere else there are no annotations.

Level 6: Java 8 has Optional built in

Oracle has a nice article on it. The API is slightly different, and I have not made the transition to Java 8 yet.



Finger Pointing

The bugtracker for the Glassfish software currently has 1309 matches for NullPointerException. Wow. (Go to the issue navigator, select the 11 projects on the left starting with "glassfish", and add the query NullPointerException, hit enter. The software is session-based, can't paste a link...)

The Grizzly project has 69.

This is a codebase that's still on level 1 regarding null. Exactly 2 years ago I had recommended to improve the Javadoc and start using null annotations. The task was accepted, got priority "Major" assigned, but is still open and things haven't changed.

I had also mentioned in that task the undocumented method int getQueueLimit() which would return the magical -1 for 'no limit'. Nowadays - being a level 5 guru ;-) - I'd instantly turn this into a self-documenting Optional. This forces users to think about the exceptional case. No horrible bugs from computations with -1 can occur. Users of it would then just do something like getQueueLimit().or(Integer.MAX_INT) or whatever suits their case - short, clear and safe.



My Recommendations

1) Avoid Null when possible.

Consider the null object pattern.

Return empty collections, not null.

Throw UnsupportedOperationException() instead of returning null.
Don't do this:
public Object foo() {
     return null; //todo
}
It is done sometimes as a quick way to satisfy the compiler. It's the no-impl pattern.

Do this instead:
public Object foo() {
     throw new UnsupportedOperationException(); //todo
}
If you return null then it (null) can go a long way until it throws a nasty NullPointerException somewhere. By throwing UOE directly the cause is clear in the stack trace.

2) Use Guava's Optional.

Wherever you would use null, use Optional. 
Except in very low level code.

3) Use Jetbrains's Null Annotations.


If allowing null then use the @Nullable annotation.This is a must. It automatically tells the user and the IDE that null is possible/permitted.

Use the @NotNull annotation everywhere else. This is optional. The Guava people do not, I do.

To get the annotations using Maven:

        <dependency>
            <groupId>com.intellij</groupId>
            <artifactId>annotations</artifactId>
            <version>12.0</version>
        </dependency>


4 comments:

  1. It's worth pointing out that Optional is very similar to the Option class in Scala, which is part of the core library and not an add-on.

    And I'd argue that the JSR-303 validation annotations are better to use than the JetBrains ones, because being a JSR they are (in theory) better supported. Plus you get to use the JSR-303 Bean Validation stuff too, and we even go further than that and have aspects that wrap our public methods to do validation of all the input and output objects using the JSR-303 Validators to ensure that the data into and out of our APIs are safe.

    ReplyDelete
  2. "Use the @NotNull annotation everywhere else. This is optional. The Guava people do not, I do."

    That is technically incorrect.

    Guava people do not need @Nonnull because it's using @ParametersAreNonnullByDefault on a package level, see http://jsr-305.googlecode.com/svn/trunk/javadoc/javax/annotation/ParametersAreNonnullByDefault.html

    When you put @ParametersAreNonnullByDefault on your package (I do, and I hope you do too), a good IDE (I use Eclipse, but I hope this also works on IntelliJ) will inform you when putting @Nonnull is not necessary.

    BTW I have nothing against intellij, but the de facto Maven dependency for these annotations would be "com.google.code.findbugs:jsr305", which is also the one used by Guava that you mentioned and used, so I was kinda curious why you used intellij's instead of this one:

    http://search.maven.org/#search%7Cgav%7C1%7Cg%3A%22com.google.code.findbugs%22%20AND%20a%3A%22jsr305%22

    ReplyDelete
  3. JDK 8's Optional is a step backwards from Guava's, because you can't use it with a foreach statement; it is not an Iterable, and will not give you anything that's an Iterable (Guava's has .asSet() for that). A shame, but at least they did give it filter, map and flatMap.

    So for anything 'imperative', you'll need to check and then get, which is where errors happen.

    ReplyDelete
  4. Or use a language where assigning null to an object variable results in a compile error, like Ceylon or Kotlin.

    ReplyDelete