Wednesday, December 10, 2014

Graphical Visualizations in JavaDoc

Having code documentation is crucial for a software. It helps the developers to understand the functions and APIs. Code that is well understood is easier to improve, and therefore more likely to live on, instead of being thrown away in favor of a complete rewrite. It serves as the specification in case the code contradicts the text, and helps identifying possible bugs.

Sometimes explaining a situation only by words is difficult and results in lengthy, hard to comprehend blathering. What options for visualizations are there?


Option 1: Images

Although most Javadoc is plain text, it permits the use of HTML formatting. Images can be embedded. Put them into a doc-files folder and link it:

/**
 * Visualization:
 * <p><img src="doc-files/my-uml-diagram.png"/></p>
 */


There is some cost involved in keeping this up to date; the next programmer touching the code will need to have access to the originals, and have the graph software (UML, image editor, ...) installed. This takes time.

Also, chances are your IDE will not show the image. 

Javadoc history and today's use

When Javadoc was created in the early 90s, source code comments were nothing new. The pioneering feature was the auto-generation of a technical documentation directly from the source code. Prior to such documentation generators, the technical documentation used to be (badly) maintained separate from the source code. Ouch. HTML was brand new, and the build putting those files on a web server was cutting edge technology.

Now, Javadoc is over 20 years old. At the time it was created, virtually all business software was closed source. Today, all software libraries we are using at the company are open source.
Software projects reorganized from lengthy release cycles to release-early-release-often. And software is written in many smaller, modular pieces instead of monolithic systems.

Fact is, I rarely look at Javadoc in the web browser. Most often I see it directly in the IDE. Whether it's one of our own ~40 software projects, or a library checked out from GitHub or from Maven central, or code from the JDK itself. It's just so much better to have a complete view; up to date source code and Javadoc.


Option 2: ASCII Art

An alternative to images is the good old ascii art. I personally use and recommend asciidraw.

Here's an example:

/**
 * <pre>
 *
 *                 +---------gn-hypo-+                +--------------gn-+
 *                 |                 | +------------> |                 |
 *                 | Osquitar        |                | Óscar           |
 *                 |                 |       +------> |                 |
 *                 +-----------------+       |        +-----------------+
 *                                           |
 *                                           |
 *                                           |
 *                 +---------gn-hypo-+       |
 *                 |                 | +-----+
 *                 | Osqui           |                +--------------gn-+
 *                 |                 | +-------+      |                 |
 *                 +-----------------+         +----> | Osgar           |
 *                                                    |                 |
 *                                                    +-----------------+
 *
 * </pre>
 */


The advantages:
  1. It's quick, create a drawing within minutes.
  2. It's super simple, no special knowledge required.
  3. No "file originals" nor special software required.
  4. Every developer can maintain it.
  5. Any IDE or editor is guaranteed to display it in line.
  6. Because of the technical limitations, one is forced to keep it simple and focus on the main components.

Conclusion

You may want to consider ascii visualizations for your Javadoc the next time you're struggling with explaining a circumstance. It works great for us.



Monday, September 22, 2014

Java SLF4J: Dynamic Log Level

In short

Problem: SLF4J doesn't support to set the log level at runtime.
Solution: Use Lidalia Extensions to SLF4J

The whole story

My Java projects that aren't older than 2008 all use the same logging setup: SLF4J for the interface, Logback for the implementation. It's a widely accepted standard with benefits.

Today I had the need to dynamically set the log level at run time. Just like in Log4j logger.log(priority, message). But there's no such or similar method in the API. With one eyebrow raised I started typing into Google:


Ah, good, apparently I'm not the only one. There must be a simple solution. Stackoverflow already has the matching question. But this time the answer is a surprise:
"There is no way to do this with slf4j."
That's not a statement a programmer comes across often. I shared my reaction with this popular user comment:
"WTF? SLF4J is heralded as the future of java's logging, and this simply use case is not supported?"
The SLF4J API team has arguments not to include these methods.

I wasn't too much interested in the architectural design decisions. I just needed a quick and clean solution. And here is a nice one:

Lidalia Extensions to SLF4J "An extension to SLF4J allowing logging at a level determined at run, rather than compile, time via a Level enum."

It's small, available from Maven central, adds exactly the missing functionality, and works.

        <dependency>
            <groupId>uk.org.lidalia</groupId>
            <artifactId>lidalia-slf4j-ext</artifactId>
            <version>1.0.0</version>
        </dependency>


Example:

import uk.org.lidalia.slf4jext.Level;
import uk.org.lidalia.slf4jext.Logger;
import uk.org.lidalia.slf4jext.LoggerFactory;

public class Example {

    private static final Logger logger = LoggerFactory.getLogger(Example.class);

    public static void main(String[] args) {
        Level level = Level.valueOf(args[0]);
        logger.log(level, "Logged at a level configured at runtime");
    }
}





Tuesday, May 20, 2014

If your project is still hosted on SourceForge, I automatically assume it's dead.


SourceForge was the first free project hosting platform for open source software. It was immensely popular. It was the hub during the early open source years on the web - it was the GitHub of today.

Because of its historical popularity, there is still a large amount of projects
hosted there. Some are popular, some are dead - just like in any source code
repository. Many developers moved their code to more modern platforms since then.
At first to Google Code, now to GitHub.


Here are 3 reasons why you should move away from SourceForge.

1) SourceForge feels dead. Many active projects moved away. 

When I look for a software, I don't mind too much where it's hosted. Maturity,
stability, community size and active development are the factors I look for.

I don't advocate moving your whole project with source code and bugtracker every
couple of years to the next hip kid on the block.

But the fact that so many moved on means that what's left is probably abandoned. When I search for a software, and a GitHub and a SourceForge project comes up, I tend to click the GitHub first. Also, because of my search history, I probably get to see the GitHub links first. I'm certainly not alone here.

2)  Developers have GitHub accounts

Open Source needs contributors and users. And the easiest way for developers to pitch in or file a bug report is when they already know the system and have an account. If it's complicated, I pass.

3) Scam ads: this stinks!

Here's the reason for my blog post.

Once again I was downloading a (totally outdated and abandoned, but familiar and working) software that is still on SourceForge, and then I got to see this ad:



It leads to a suspicious looking website, with Microsoft Partner logo and Windows logo, offering Windows driver updates. The kind my mother would click, thinking it's from Microsoft.


And here's the profile on web of trust: https://www.mywot.com/en/scorecard/driverupdate.net


The site even manages to get a top rating on WoT by spamming good ratings, even though all the important comments say it's a scam. WoT, please fix.
"they say I need over a dozen drivers updated. Confirming with the manufacturer, I am up to date. ..." --pmheart6
"why do all the positive comments use poor english with typical chinese mistakes?? A dishonest sales trap. It's difficult to uninstall and must be done manually." --KITRVL



SF changed ownership in 2012, and as the Wikipedia article writes "More recently additional revenue generation schemes, such as bundleware models have been trialled, with the goal of further improving sourceforge's revenue."

This looks to me like beating a dead horse. There's only downhill from here.

Please spare other people from such frustration. Even if the platform was good once, now is the time to move the code away. To BitBucket or Google Code or GitHub or wherever. Thanks.

Tuesday, April 29, 2014

Java: .equals() or == on enum values?

Both do pretty much the same thing: checking whether 2 enum values are the same. And if you search your code base, you probably find both kinds. There are some subtle differences. What are they - and which syntax should I use?

Same same - but different

Enum values are guaranteed singletons - and thus the == reference comparison is safe. The enum itself just delegates to Object.equals(), which does exactly that.



So if they execute the same code, why should I bother?

Learning the hard way - actual bug

I produced a bug which broke the build. Luckily we did have a unit test in place that caught it. But still I would have seen it earlier, had I used the "correct" syntax.

So which one is it?

And the winner is: ==

It provides compile-time safety. While MyColor.BLACK.equals(acceptsAnything)gives a warning at most, the check MyColor.BLACK == something won't even compile if something is not an instance of MyColor.



This makes it refactoring-safe. If you go - like me - and change the type of something later on, you'll notice instantly.

Another difference is the support for null. The check with == allows null on both sides. It doesn't throw, however, it may also "hide" a null value where there shouldn't be one.
The check with equals() allows null only on the right side, and throws a NullPointerException if the left side is null. That may be desired sometimes. Anyway, I vote for not letting null slip into this comparison at all. It should be checked/handled before, instead of being smart. (See my article about null handling.)
So the comparison with == is "safer" also at run-time in that it never throws a NPE, but is possibly hiding a bug when the left side should never have been null.

Then there's the argument of performance. It's irrelevant. I'm not going there.

And the visual aspect. Which one looks nicer?
When I can have compile-time safety, I don't care about the looks. I was used to .equals() simply because that's how Strings are compared. But in retrospect that's a pretty lame explanation.
On this StackOverflow question, which is exactly about the topic,  Kevin Bourrillion from Guava commented that "== may appear incorrect to the reader until he looks at the types" and concludes that "In that sense, it's less distracting to read ".equals()". Au contraire! When I see CONSTANTS around == I instantly know they're either enums, or primitives, of matching types. If not, the code is either red because it doesn't compile, or with a yellow warning from my intelligent IDE saying that .equals() should be used. For example Strings or primitive wrappers like Integer.

After that incident at the company we've decided to go with reference equality comparison ==.

Tuesday, April 22, 2014

Java: Evolution of Handling Null References

Using null in programming is playing with fire. It's powerful and sometimes the right choice, but the infamous NullPointerException can sneak in quickly. That's why I advocate for every software project to include a section about handling null in the coding guidelines.

The first time I've seen a Java exception was in the 90s in the web browser's status bar when surfing the net: a NPE caused by an applet. I had no idea what it meant and what I had done wrong. Today I know that the programmer let a null reference slip in where it wasn't expected to happen.

When looking at open source software written in Java I often come across code where null references are used but not documented. Sometimes preconditions are in place. When stepping deeper it gets really tricky to figure out which variable is now allowed to be null and which is not. For the author it was obvious at the time of writing the code... but software becomes better when written and looked over by many, and thus it is important to make it clear for everyone, everywhere.

Software is constantly shipped with NPEs detected later on. Redeployments and bugfix releases are expensive - and it's a pity because this kind of bug could be eliminated almost completely.


Here's my personal evolution of dealing with null in Java.

Level 1: No Plan

Using it wherever, no information about it in the Javadoc.
Fixing NPEs as they occur, seeing no problem in that.
public class Foo {
    
    public String getText() {
        return null;
    }
    
}

Level 2: Document Null - Sometimes

Realizing that NPEs are a problem that could and should be avoided. Detecting them late is expensive, especially after deployment.
Starting to document variables, arguments and return values that can be null... incomplete.
public class Foo {
    
    /**
     * @return the text, or null
     */
    public String getText() {
        return null;
    }
    
}

Level 3: Add "Null" to Method Names

Realizing it's still a problem. Having Javadoc is nice, but useless when not read.
Starting to name methods with null such as getFooOrNull() to force it to be seen.
public class Foo {
    
    /**
     * @return the text, or null
     */
    public String getTextOrNull() {
        return null;
    }
    
}

Level 4: Code Annotations

Using Jetbrains' null annotations: Annotating all variables with @Nullable and @NotNull. This is a huge step forward. The crippled method name pattern 'OrNull' is obsolete. The code is automatically documented.
public class Foo {
    
    @Nullable
    public String getText() {
        return null;
    }
    
}
And the best part: the IDE checks the code and warns on a mistake.


Using these annotations strictly I don't remember causing a single NPE in years. The drawback is more typing, more text on the screen. But then we use static typing, and not defining null is just incomplete.

Level 5: Guava's Optional

The concept: Instead of using the null reference directly, wrap it in another object that permits null. The getter methods on it myOptional.get(), myOptional.orNull() and myOptional.or(alternative) force the user to think about what to do when it's null. The API becomes extremely clean.

If you don't know about it yet, read the Guava page about using and avoiding null.

It took me a couple of days to get used to this. And I did produce a handful of bugs initially because I've used Optional.of() instead of Optional.fromNullable() by mistake.

Although step 4 with annotations already got rid of NPEs and improved the code confidence, this was another big step forward. The API of the Optional class is clean, and the API of code using it is consistent and clear. A developer only has to learn the concept once, and because it's from Guava, sooner or later everyone will know it.

The Guava team uses @Nullable annotations for the few places where null is permitted. Anywhere else there are no annotations.

Level 6: Java 8 has Optional built in

Oracle has a nice article on it. The API is slightly different, and I have not made the transition to Java 8 yet.



Finger Pointing

The bugtracker for the Glassfish software currently has 1309 matches for NullPointerException. Wow. (Go to the issue navigator, select the 11 projects on the left starting with "glassfish", and add the query NullPointerException, hit enter. The software is session-based, can't paste a link...)

The Grizzly project has 69.

This is a codebase that's still on level 1 regarding null. Exactly 2 years ago I had recommended to improve the Javadoc and start using null annotations. The task was accepted, got priority "Major" assigned, but is still open and things haven't changed.

I had also mentioned in that task the undocumented method int getQueueLimit() which would return the magical -1 for 'no limit'. Nowadays - being a level 5 guru ;-) - I'd instantly turn this into a self-documenting Optional. This forces users to think about the exceptional case. No horrible bugs from computations with -1 can occur. Users of it would then just do something like getQueueLimit().or(Integer.MAX_INT) or whatever suits their case - short, clear and safe.



My Recommendations

1) Avoid Null when possible.

Consider the null object pattern.

Return empty collections, not null.

Throw UnsupportedOperationException() instead of returning null.
Don't do this:
public Object foo() {
     return null; //todo
}
It is done sometimes as a quick way to satisfy the compiler. It's the no-impl pattern.

Do this instead:
public Object foo() {
     throw new UnsupportedOperationException(); //todo
}
If you return null then it (null) can go a long way until it throws a nasty NullPointerException somewhere. By throwing UOE directly the cause is clear in the stack trace.

2) Use Guava's Optional.

Wherever you would use null, use Optional. 
Except in very low level code.

3) Use Jetbrains's Null Annotations.


If allowing null then use the @Nullable annotation.This is a must. It automatically tells the user and the IDE that null is possible/permitted.

Use the @NotNull annotation everywhere else. This is optional. The Guava people do not, I do.

To get the annotations using Maven:

        <dependency>
            <groupId>com.intellij</groupId>
            <artifactId>annotations</artifactId>
            <version>12.0</version>
        </dependency>


Tuesday, March 11, 2014

Software Versioning and Bugfixes

There are many ways how to give version identifiers to software builds.

The goals of software versioning, as I understand them:
  1. Give each build (release, snapshot) a unique identifier.
  2. Apparent order: it must be obvious which is newer when comparing 2.
  3. Apparent changes: contains breaking changes, new functionality, or just bug fixes.
  4. Possibility to release updates later on that "fit in between".
I like the increasingly popular semantic versioning. It accommodates all these goals.

Apparent Order

1.9.0 -> 1.10.0 -> 1.11.0
This is important for me as a user, it lets me select the newest version quickly.

Apparent Changes

Going from 1.9.0 to 2.0.0 may mean that some of my use cases need to be adjusted, some things don't work the same anymore. I might prefer to stay on the "1" branch.

Going from 1.9.0, 1.10.0 contains more functionality, it seems pretty safe. Experience taught me that new features bring new bugs - even if I don't make use of the new stuff.

Going from 1.9.0 to 1.9.1 means there are just bug fixes. It is almost a safe bet to upgrade this. A quick look over the change-list helps me to decide where I might be affected. The risk is very low that this introduces new bugs.

Release Updates

When after 1.9 the version 1.10 has been released, and then bugs are found in 1.9.0 that should be fixed, not everyone will be happy with a 1.11 that now contains new functionality and bug fixes. Hence the 3 numbers.

H2 Database thinks otherwise

Unfortunately this software library that I use, like and value follows a different version system. Just like Google Chrome, they have been publishing new versions with what is effectively a single continuous version number.

For a recent history see http://www.h2database.com/html/changelog.html
And longer: http://mvnrepository.com/artifact/com.h2database/h2

There are no bug fix releases. Each new version brings new features, improvements, bug fixes, and new bugs, which becomes clear when you follow the issues. But to learn in which version they were introduced, and so on, you need to dig and read the bug tracker.

For a web browser, this release cycle OK. Give me the latest, I can cope with some level of annoyances and crashes now and then.

The database however is the back bone of many applications. It is the piece of software that one relies on, that is the worst to break. And as such, I really wish it would maintain two code branches. One for latest developments, and one that lags behind where just fixes are applied.

Whenever I consider updating this library in my code, I hesitate. Is it really necessary? Most times I answer no and move on. Updating means reading all recent changes, and running tests.

I only use this database for read-only scenarios, not as a live database with active changes.

My suggestion to H2 and in general

  1. Move code from Google Code to GitHub
  2. Change to semantic versioning
  3. Create a branch that lags behind, and let contributors do the work of maintaining it (applying just the fixes)