An Open Source “Fork” You Might Have Missed: slf4j and log4j

Everyone in the world started using Log4J a few years back.  From where I sit, I don’t think anyone paid much attention to Sun’s logging framework, and the world is still waking up to the idea of slf4j.   Admit it, you use Log4J, and you don’t pay much attention to the project.   It isn’t some fancy new-fangled NoSQL database, it is a logging library, and it isn’t something that occupies a significant slice of your attention.

I was recently looking at the two projects: log4j and slf4j and I was struck by how stagnant log4j has become.   Nothing seems to happen with this project.  On the other hand, slf4j seems to get things done with a minimum amount of procedure.   Some dude rolls up to the slf4j project write on the mailing list: “Hey I need something for OSGi compatibility” a couple days later, it happens.  No vote thread, no argument about process.   It just happens.   Need to check the code out for slf4j?  Easy, like any other sensible open source project around, they use Github.    But, wait, who’s Github account is that?   Isn’t that the log4j guy?

…and this is what people don’t realize about slf4j and log4j.   There was a “schism” in 2005. The person who was the major force behind log4j from 2000 to 2005, Ceki Gulcu, up and left log4j in 2005 because the community was somewhat paralyzed by “process”.    Depending on what your own defition of “fork” is, you can think of slf4j as the successor to log4j.  So what happen?  I certainly wasn’t paying attention, let’s check the email archives.

Reading the Archives: The slf4j schism of 2005

You can see Ceki announcing slf4j in April of 2005 here.    Then in May of 2005 there is this massive essay festival on log4j-dev about whether or not log4j “should officially support slf4j, yadda yadda yadda” here.   While the tone was nice and political, Ceki chimed in with an email that spoke to the real reason why slf4j was started:

I don’t want to be dismissive but these are just a bunch of excuses. Sure, the objections are all reasonable and all, but at the end of the day they boil down to excuses preventing forward movement.

Fortunately, this is open source where we can take our marbles and play elsewhere

Harsh, huh?  But, something about this makes me want to move everything over the slf4j immediately.   This was almost six years ago, and log4j hasn’t gone anywhere.  The last time Ceki showed up on the log4j mailing lists was in April 2007, in a no-holds barred argument over “Log4J 1.3”   Read it here.  This thread is a very interesting education in open source politics, inertia, and what happens when the community starts to invoke arbitrary rules about compatibility and stability to govern innovation.  It appears that Ceki moved on, started working on logback and didn’t look back, he’s got a great little response (again this is almost four years ago now) that sums up his experiences:

The SLF4J vote was just the straw that broke the camel’s back. After putting many many hours of work into log4j, it became increasingly painful to waste time in arguments, where opinions got asserted by the one writing the longest email. Not fun.

This sort of pattern plays itself out over and over again.  A creative individual starts a project, migrates it to some open-source foundation, a team of interested contributors starts to participate and, predictably, the project becomes mired in Process.   You create this process whereby anyone with an email client can tie everything up, and it happens all the time.   Maybe someone has already come up with a name for this process, I’ll call it “Project death by open source volunteer”.

Avoiding Limits on Innovation

What’s ironic about that April 2007 thread is that it looks like a bunch of people participating who were in opposition to these changes subsequently disappeared from the project almost entirely.  This suggests that the people participating in that thread weren’t invested, and it is something I’ve seen quite a bit in mature, lurker-heavy open source projects.    Often the people “participating” in the mailing list discussions are not the ones making significant code contributions.   Another response from Ceki talks about limiting participation in the slf4j project to people who are “invested” and who “have a clue”, and it suggests that he was frustrated by constant bickering and discussion.

I’ve always felt that someone needs to develop a web site that archives open source mailing lists and overlays individual-specific contribution metrics atop a particular message.   It would be interesting to be able to read a developer mailing list for a popular open source projects and filter out all of the messages from people who haven’t contributed much code.   If the mailing list software added such statistics in automatically it might help change the dynamics of the discussion.

My working theory is that if you created such a tool, you’d find that most successful open source projects are dominated by a few individuals make most of the changes, and you’d also find a very large number of people “participating” in the mailing list without making actual code contributions.

This archive history makes the perfect case that open source is much more about individual contribution than community contribution.   This isn’t to rule out the importance of developing a “healthy” community, but I am saying that great open source is quite often developed by a group of interested “individuals” working with whatever rules and interests make sense to them at any given time.   Slf4j’s logback provides a model for what log4j 2.0 could have been had people not put odd, community-driven limits on innovation.