Rails, You Have Turned into Java. Congratulations!

Rails isn’t the easy framework it once was, age has brought along a good deal of complexity both in the way Rails applications are deployed and in the number of technologies you need to assemble together to get anything meaningful done.

Update (2/20/13): Hello all, one additional note.   I’m surprised to be saying this, but I think the key difference between Rails and Java is that the Java “ecosystem” has this imperfect, somewhat ad-hoc standards group called the JCP.  This means that Hibernate and MyBatis or Spring and Oracle aren’t just pulling another framework out of thin air, they have to agree on standards and APIs.   This slows things down a bit, but, in the long run, this makes things more interchangeable.   It’s odd for me to write this because I’ve railed against the JCP for years, but I do think that Rails is a picture of what happens to “frameworks” in the absence of standards.   Call me old for saying it, but I did.  So there.

Frameworks atop Frameworks atop Opaque Hosting Platforms

Sure, Rails itself is straightforward, but the frameworks you slap on top of it can quickly become burdensome abstractions:  RefineryCMS, Devise, Omniauth, Carrierwave, Unicorn, Rack Rewrite, Fog, New Relic, Foreman, AMQP, and Honeybadger, not to mention the extra magic that Heroku gems throw into the mix (backups and other fun).

Why is the site slow today?  I don’t know maybe AWS is having problems?  Nope.  Ok, maybe the Unicorn configuration is screwy?  No.   Alright, everything works except Refinery take a look at New Relic.   That didn’t help.   Ok, let’s restart the application.   Great, that fixed it.  Why?  Not sure.  Was it a worker thread that was stuck?  Who knows, and who cares the site works now…

Maybe I’m reacting more to Heroku’s recent unpredictably and mysterious performance issues.   Maybe, I’m not fully dissatisfied with Rails or Ruby, but I can’t help but notice that Rails has become heavier these days.

Is it the complexity that Refinery brings to the table?  Maybe.  Refinery does what it does, but it is something of a beast.   If you follow the rules closely, you’ll end up developing a Rails application with an excessive number of Engines, which is really just another way of saying that you’ll be fumbling through a messy file explosion.  (What view is that engine in?   Hold on, let me click through five levels of folders….)    There are those of you who will shout “Sinatra, you should use Sinatra?”  But that’s really not what I came to Rails for.

I know I have unrealistic expectations.  I want it all, and I want it to be easy: I want straightforward defaults, but I also want customizations to be things I can understand.    Take the intersection of Devise, Refinery, and Omniauth as an example.  All three of these frameworks are first-class, don’t get me wrong.   (I lied, Devise and Omniauth are first-class, and Refinery is an awful mess.) Making them work seamlessly together is an obtuse nightmare  – a documentation desert littered with wreckage.

Frameworks that Eat Frameworks don’t like it when you Customize the Frameworks they Eat

The problem is really that once you start making use of Rails and all of the frameworks built atop of Rails you end up creating an opaque mess of customization.   One framework, say Refinery depends on something like Devise.   Customize anything and good luck upgrading.

Who really knows what happens during the login process?  Oh, I know, we’ll use RubyMine to step through this unpredictable mess.     I need to make a simple change to how routes work… oh look Routes have become incomprehensible because you now have 12 difference Refinery engines.

You’ll find yourself staring at incomprehensible mega-frameworks maintained by developers who are unapologetic about how little they care for writing documentation.     You’ll also end up with the most common pattern from Hell: the single Rails application that runs an entire business.

I’m starting to think that Rails has turned into the new Java, and that it is time to return to Java and ask it a simple question…

…Java, have you learned to “easy” yet? 


Loukides: “The winner in the case isn’t just Google; it’s all software developers”

Mike Loukides sums up the end of this phase of the Oracle v. Google trial in the title of his piece on Radar: “The End of a Fishing Expedition”.

First, Mike is right, as he very often is, but this sentence gives me pause:

The winner in the case isn’t just Google; it’s all software developers, who don’t have to worry as much about creative interpretations of copyright law, and are free to develop compatible implementations of an API.

Yes, Alsup’s decision is something of a victory for Open Source and people interested in compatibility, but Alsup also mentioned something in the summary that I’m still trying to understand:

This order does not hold that Java API packages are free for all to use without license.

While this was a win for Dalvik, Java is still confined in this annoying “box”. OpenJDK is open source, but the language and is still gated by this TCK. I have yet to see any analysis from people I trust (aka groklaw) on this topic. Java still isn’t free, but, at least, there’s no barrier to people creating an alternative implementation (they just can’t call it Java).

It will be interesting to see what an Oracle scorned does to the licensing terms it uses to distribute Java. While the APIs are not copyrightable, my lawyer friends have told me that there’s not much limit to what Oracle could put into a EULA that accompanies the JDK. (Just look at how Apple uses the EULA for the iOS SDK for proof that anything is possible.) If they are rational and want the platform to succeed they won’t do this, but I’m worried that now that Java isn’t as lucrative as they thought it was that they might just discard it (and ruin it in the process).

I also fear the appeal. I’m a pessimist, if this case makes it to the Supremes all bets are off.

I do agree with Mike, Alsup is something of a modern hero.

The Day After the Verdict, I Open a Sun Time Capsule

I never published this interview because I didn’t find it notable. Don’t get me wrong, Tim Bray is a brilliant engineer and a great interview subject, but the interview itself was a bit meandering. The interview wasn’t worth publishing more because it was too long, it wasn’t focused. If anything the fact that the thing wasn’t published was more a reflection on my failure to conduct a more interesting interview than Bray’s content – again, Bray is made of awesome.

In 2008, I was focused on the struggle between Apache Harmony and Sun Microsystems and I was really frustrated with what I felt was “doublespeak” coming from Sun at the time. Right before I sat down with Bray, I had a quick talk with Geir M. which is somewhat off-the-record. He was at 10gen at the time IIRC, and I was asking him about the TCK dispute. When I sat for the interview, it was my goal to try to get Bray to make news about TCK licensing, you see that I failed at that goal, but four years later this video does have a few moments relevant to the Oracle v. Google trial.

This long interview covers many topics (maybe too many?), and it doesn’t start to get interesting until the end. You’ll see that I’m asking questions you wouldn’t expect me to ask Bray because I’m trying to zero in on the Sun and Harmony issue. I also figured that this was my chance to ask about bigger issues. I’m asking some fairly general questions about layoffs because there was a disconnect between Sun’s public message and what I was seeing on CNBC. The company was simultaneously struggling and making increasingly more audacious claims, this trend would continue later that year at JavaOne 2008.

Interesting parts of this interview:

I ask him to respond to a quote from his blog on the 18th of July in which he said he “didn’t want to be a share cropper on Massa Steve’s plantation” quote.

“It’s a great device, I’m probably going to get one. It is triumph of industrial design. Despite the fact that these puppies have been able to access the internet for a decade. the iPhone is the first device where people have been able to access the internet at scale and at volume.”

At the end of the day, I’m disappointed that there is a single vendor that controls the platform. You don’t have to ask anyone’s permission to do anything to do something on the web.

Q: “What is Sun doing to make it more realistic to have an open market on a mobile platform?”

“we are substantially in the business for selling the backend network infrastructure. Now a lot of us at Sun think they would do better and they would make more money if they were radically to open the network. We think that it is quite plausible that the operators could open the billing systems…”

“Even assuming that the network operator gets half of that. I’ve been saying that in public for years. Inevitably I think it will happen, let’s radically open up and we’ll see an explosion of creativity. Google seems to be trying to do that with the Android platform.”

“Android looks pretty slick and it looks like a more [modern environment]… I have a couple of issues with Android, It seems to have stalled. There is no hardware yet.”

“I’m a little disappointed in the licensing around Android. If you look at the licensing on Android they can ship devices that have the same walled garden.”

I asked this question, “If Android is released and there is massive uptake, do you see any pressure on Sun to call Harmony a real implementation of Java?” He quite deftly refuses to take me up on the offer to make news. But he does say something along the lines of “It is Google after all, but the phone companies are bigger than Google.”

Two quick asides:

  • This story would have been uploaded earlier, but YouTube kept on removing it. I find something about this to be ironic.
  • Maybe this is ironic: Google used a screen capture from my interview with Bob Lee in its opening statement. It’s unclear to me if this is covered under “fair use”, did they just ignore O’Reilly’s copyright notice (just kidding).

The Debian Java Package Team: Futility Defined

Maybe a year ago, maybe a bit longer, I was motivated to jump in and help get Maven 3 on Debian. Right? Maven 3 in Ubuntu, should be easy right? No, these people are smoking some strong, strong stuff. Evidently, they don’t like the Maven repository because it distributes binaries and distributing binaries is counter to the way linux distributions work.

You think this is crazy? Tough.

So, Central has 400,000 artifacts on it. 400,000 is not a small number, and the only way these Debian folks are going to keep up is if they figure out a way to map all of that into a system that fits into this idea of Linux source distribution. So, I dove in, sent some emails. The only way Maven 3 is going to get packaged is if all of the projects that Maven 3 depends on are packaged in the appropriate source…. zzzzzz. Translation: months and months of busy work just so that some Linux distro can be happy that everything is being compiled from source.

Whatever, back then I was motivated by the idea of “1. Install Ubuntu, 2. Run ‘apt-get install maven3’, 3. Make great victory” So I dove in and started to uncover a vast conspiracy of idiocy at the core of this distribution’s approach to Java libraries.

They have created a series of bash and Perl scripts (Perl, you ask, who brought that guy to the party.) These bash and Perl scripts perform a series of magical incantations that trick Maven into using a new repository. This repository has little to do with Central, it’s a special repository full of special POMs that have different versions. You’d have to be very strange to think this made any sense whatsoever. They even have a whole page dedicated to some idiotic interpretation of the Maven repository specification that shoehorns Maven into how Debian should work: http://wiki.debian.org/Java/MavenRepoSpec.

So, do something sensible and ignore all of this, it just creates problems for people who need to use Maven. Install it manually or just use the binary deb I’ve created. Don’t wait for these people because it will take them years to catch up with the rate at which the Maven community is moving.

I love these posts about Maven

It begins with “I hate maven.”, and it goes on and on. This is a person who is trying to use the Sonar plugin, and who hates Maven so much he can’t bring himself to understand the idea of running a repository manager, or even that he should think about upgrading to a version of Maven that was released two years ago.

Part of my mission is the help people understand Maven, but when I see a chap like this: someone so dead set against investing even a tiny particle of effort in getting things to work. I like to sit back and just watch. I like to see just how bad it can get.

The solution to this dude’s woes are simple: use a repository manager (there are several, they are all free) and upgrade to the latest version of Maven. You can see people in that thread giving him hints, and you can see the resistance.

New Version of the Maven Definitive Guide

Edition 0.7 is out. You can read it on Scribd or online at the Sonatype site. This edition saw a marked improvement in the rendering of the PDF version of the book with the move to the newer docbkx plugin and the integration of the fop-images-pdf library to allow direct embedding of PDF vector art as figures.

Maven: The Definitive Guide

couchdb4j is a Case in Point for Git and Maven

Yesterday, I decided it was time to test out some ideas about storing content in CouchDB. I just wanted to get some preliminary numbers on performance, but I also wanted to see how the thing would scale after loading 10 GB worth of data. So, I went about this by…

  1. Downloading couchdbx which is a one-click distribution of CouchDB for Mac OSX – Once you download this distribution, CouchDB loads as an application, runs on the default port. Downloading couchdbx, unpacking the archive, copying the app to Applications, and then running it. Within 2 minutes, you’ve got a running instance of CouchDB.
  2. Searching for a good Java library… – Right, so even though CouchDB is really all about HTTP and JSON, I still need to find an easy way to call CouchDB from a Java program. I settled on couchdb4j, it has a simple API, and it does everything I need it do.

At this point, I noticed that couchdb4j has a Git repository at: http://github.com/mbreese/couchdb4j. I clone the repository to my local system, I run “mvn clean install”… the tests fail. After some investigation, I realize that the tests are failing because I happen to be running the latest release of CouchDB 0.9.1, and couchdb4j only works with 0.8.

Point One: I didn’t have go fishing around the source code to figure out how to build this sucker. It just worked. Even though the tests fail, I’m up and running in a few seconds. The presence of a pom.xml file is a signal that I don’t have to spend time rifling through someone’s custom build. And, I now know that I’ll be able to make changes to the code easily using m2eclipse.

Once I figure out that I’m probably going to have to get into the source code to make some changes to bring this thing up to speed with CouchDB. I can easily fork the couchdb4j project in GitHub, and create my own fork at http://github.com/tobrien/couchdb4j/tree/master.

Because the CouchDB project maintains simple API documentation on a Wiki, it is easy to make the appropriate changes to couchdb4j to bring it up to speed with the changes in the CouchDB View API and the CouchDB Document API. I know have a fork of the couchdb4j that is 0.9.1 compatible, if other people want to use my changes they can freely clone my repo, or they can pull in the specific changes I made. I haven’t made a pull request, because I don’t know if the changes I made are aligned with the interest of the couchdb4j project.

Point Two: Git made it easier for me to make an instant decision to fork and customize to satisfy my own requirements. I didn’t have to stop and figure out what community dynamics. Because Github makes it so easy to fork and existing repository, I didn’t have to ask permission or chime in with “Hey, is anyone interested in XYZ.” I scratched my own itch, and if the person who maintains the original project finds my changes interesting, he or she can pull them into his own.

Within about an hour, I have an experiment with a customized version of couchdb4j that has been upgrade to 0.9.1 compatibility. Because couchdb4j followed the Maven conventions and because they decided to use Git, it was easy.

A "real" letter and an old book

As a co-author of Harnessing Hibernate, I get to hear about some of the feedback the book has received from time to time. James recently shared a letter he received about the book, a real “letter”: signed, sealed, and delivered by the United States Postal Service. How archaic? Someone decided to read a computer book, type up a letter, and mail a physical artifact. My initial reaction was, “what kind of crazy jackass would bother with the USPS when our email addresses are listed in the…” But, after some reflection, it certainly worked; it gained someone’s attention more so than a simple email. A real letter is much more difficult to ignore than something that shows up in my busy Gmail INBOX, and you have to consider that the person who sent this letter is more invested in his feedback than someone who fires off a random email.

Emails are easy to ignore, and the promises made in an electronic format fade quickly. Committing ink to paper is something else entirely, and you tend to take printed material more seriously than you would an electronic message. Printed letters are much more interesting, and they retain powerful lasting value that emails do not… see this blog post about letters from the recently deceased John Hughes. As effective as it was at gaining my attention, I can’t help but return to my initial reaction. Is this someone with an interesting archaic quirk? Or, is this some email-avoiding conspiracy theorist?

One of the pieces of feedback in the letter was that the book was out of date. Of course the book is out of date, bulk-printing 10,000 copies of a 400 page book about a rapidly evolving open source project is certainly the definition of crazy. Unlike other books I’ve helped to write, Harnessing Hibernate is not free. It doesn’t have a life outside of the printed product, and it is getting more and more irrelevant every single day. Harnessing Hibernate is a lost opportunity, if it were an open title, it would quickly gain an audience and a community willing to help develop the content.

Unintentionally stirring a bee’s nest (by suggesting Maven)

I just had an odd exchange, someone has a great piece of open source software that I totally depend upon, it’s a complex beast of a thing, and I wanted to a.) express gratitude, b.) offer some help with the build. You see the build for this particular system is an Ant build script with a preamble of instructions, and the project itself is this megalith of code in one big src/ directory. Every time I want to use some new component that is in development, I have to download someone’s tarball, uncompress the thing and then fish around for JAR artifacts to upload to central. Some of the JARs that are included have specific Subversion revision numbers in the JAR file name. This makes using the binary artifacts from this particular project a royal pain in the neck.

I follow the project, I’m invested in the code, I thought it might make sense to *ask* if there was *any* interest in migrating the build to Maven on the grounds that it might make it easier for people to contribute and participate. Now note, I didn’t volunteer to switch the build the Maven, I simply “asked if there was any interest”, and not even on a development list. I asked one of the main contributors directly because I didn’t want to ruffle feathers.

What I got in return was this total tirade against Maven. How it isn’t flexible enough, how this particular person wanted to “send an invoice” to whoever was responsible for Maven because he had wasted so much time on it. Ending with the quote: “If our not using Maven as a build system is a problem for those who
do then it’s not our problem but the problem of Maven for not being
flexible enough.”
In other words, my very diplomatic inquiry was met with “#$@! off”.

Not “it’s been a while since I’ve looked at Maven, here are the problems I had, if you can get it working alongside this Ant build, be my guest…”. Or even, “No, I’m not interested in that. I had some problems in the past, and I don’t think it makes sense to distract from the current development.” I didn’t even get a chance to make an argument on the “merits”. So here it is…

The Argument

  1. If I can’t go to your website and figure out how to checkout the source code and build in 5 minutes, your project is a pain in the neck to contribute to. The casual contributor has no incentive to learn how your build system works.
  2. If, in order to use your library, I have to go download some archive, unpack it and then futz around with JAR artifacts, your project is a PITA to use.
  3. I don’t even care if you use Maven, all I really care about is that you publish regular SNAPSHOT builds to a repository. When I see some development release of a JAR that is wrapped in a tarball, contains a README file, and it bundles with other JARs, I end up having to upload all this noise to a repository manager crafting my own groupIds out of thin air.
  4. Yes, I understand, you hate Maven because it called you a bad name two years ago, and you didn’t know anyone qualified at the time to help you. If you had problems in the past, it was probably because you didn’t understand the tool. No offense. I can probably help out there.

If you disagree with some of the assumptions, that’s another matter.

Maven needs more opinion…

When I hear that someone has blogged about some general Maven hatred, I cringe and expect to read a post that consists of 30% incorrect assumptions about how Maven should be used, 50% ignorance of the most basic concepts, and 20% truth.

What can be done:

  • The Maven Users lists needs to become a bit more opinionated for first time users. If someone enters into the discussion asking the following question:

    “I’m attempting to publish a directory full of JARs to my local repository using the Install plugin.”

    The first reaction should be, “No, use a repository manager.” Not, “Let me find seventeen ways to help you do a series of backflips to get Maven to do something it was never intended to do.”. Maven isn’t the general Swiss-army knife tool that many approach it as. While it *can* be made to do anything you want it to do, there are core assumptions that shouldn’t be challenged. Half of the criticism we deal with is from people that were never told: “Don’t try this, don’t try to use Maven for this.”

  • The Maven community needs a better FAQ. I’m of the opinion that the lack of a good FAQ is directly related to the difficulty of the APT format. If Maven had a better FAQ, we’d have less people approaching it with the wrong assumptions.