The Problem was the No One Gave a…


swartz

I called him, we talked for a bit.  But, the interview was never published.   This was during the whole PACER controversy, before the events at MIT.   IIRC I talked to Aaron for a few minutes and we both came to the conclusion that it wasn’t the right time for him to talk to me about the case.    just don’t think he wanted another story about PACER.

It’s a shame I didn’t publish a story on Swartz, but I can guarantee you that, if I had published a story, no one would have read it.   The cynical conclusion I came to after trying to cover important stories like Swartz was that, at the end of the day, very few people have any time to give a #%@# about topics like government transparency.  I think that’s still true today.  That’s a shame.

How Not to Attend Strata: February 2013

This Strata trip was a 20 hour voyage through Silly Valley. I was working on zero sleep, here’s my rambling, sleep-deprived report. This is less a blog post and more an interminable mini-book full of non sequitur.

My Strata 2013 Experience == Surgical Strike (on no Sleep)

The original plan was to fly out Tuesday and fly back on Friday, but my life is never that straightforward. Tuesday’s snow storm in Chicago caused travel delays and I ended up flying out on Wednesday with plans to fly back on Thursday morning. A short trip, yes? But, really, I was on a mission. I’ve got this talk to give, and then I have to get back to work. Spend a few hours at the conference, sleep close to the airport, and fly out the next morning. This is what they call, “parachuting into a conference.” Plus the entire family was at home suffering from this interminable coughing flu pandemic. (Also, how the hell do people with kids travel? I don’t understand this.)

Right, it’s the night before I’m leaving and, of course, my 2 year old decides that this is the night he’s going to wake up at Midnight, with a temperature, screaming for someone to pick him up. I do so, and I’m immediately wide awake in a way that suggests I’m going to be up all night. The kid falls back asleep in five minutes, and there I am with a case of kid-induced insomnia. Great, I’m flying out in nine hours and I’m not sleeping…

Insomnia == Time to O’Reilly-ify my Slide Deck

That’s when I decide that my slides need to be more “O’Reilly-ified” for this particular O’Reilly conference. What does that mean?, you ask. For any other conference you can get away with presenting something compelling with some charts and bullet points, but at an O’Reilly conference the secret is to relate your ideas to something big bordering on the ridiculous. Lefkowitz does this very well at OSCON, and my boring slides from the Strata promo webinar weren’t going to cut it. I presented this black and white deck for the Strata promo webinar that stuck out like a sore thumb, everyone else had these “my slides are all two words or a clever picture” approach.

So, from about Midnight to 6 AM (until the kid starts screaming again) there I am. I found a way to relate my very provocative speech on Big Data to Ouroboros, the Greek symbol of cyclical renewal, the Copernican revolution, and the Roman God Janus. Right? So, I’m not just blathering on for 40 minutes on database this and database this, we’re discussing Big Data as Heliocentrism versus millennia of Ptolemaic entrenchment. After six hours, my slides look far more up to the standard I’m used to for a good OSCON talk. You know, interesting, maybe a little crazy, like a Lefkowitz or a Damien Conway. Give people something to remember, and maybe take your analogy too far and then some.

Must Go to Airport and a Realization that My Slides are Now Crazy

After six hours of this my slides don’t just drag Copernicus and Ptolemy into the mix. I’ve gone all the way to Fontenelle’s popularization of heliocentrism, Gallileo’s trial, and Kepler’s persecution as evidence that Big Data is…..

Yikes, I’ve taken it all too far. I’m up to 60 slides and 20 of them are now explaining some convoluted analogy that doesn’t work. Looks like I’m going to have to do some slide hacking on this flight.

Time to schedule the taxi, placate a screaming two year old, and get ready to go to the airport. …Cab to the airport, rushing through security, ORD Terminal 1, walking through the crazy underground tunnel to concourse C… Grab an egg sandwich, eat it despite the fact that it is still frozen… Some work phone calls, some last minute tasks I forgot to tie up….

Pay Boingo $7 so I can use the internet for five minutes, fire up Photoshop to fix something, dial into a VPN and fix a broken machine, etc. Some volume is running out of space, it can’t wait. Damn, I have no time to go to a conference, this is insane. Stop. Take a few silent minutes to stare out at the plane, worry about flying, and think about the several ways planes can fail in snowy weather. (Did I mention that flying is one of my many irrational fears.)

Keynote and I Become Best Friends on the Flight

The flight was uneventful, maybe a bit turbulent, but I did take three more hours to hack on my slides. Just over Nebraska I’m starting to get nervous. Am I underprepared? Have I thought this presentation through? What the hell am I doing presenting at Strata anyway? I decide that I’m going to mentally rehearse this talk *and* I’m going to finally invest some time into using Keynote as a presenting tool. You know how these tools have presenter notes and rehearsal interfaces and ways for you to annotate your slides? You know, the features that no one ever uses.

I decide to do something very uncharacteristic. I’m going to write a script for my presentation. Maybe not a verbatim script, but I’m going to write out all the transitions so that I know when to advance the slides, how long to spend on slides, etc. How could this possibly go wrong? The flight gives me enough time to run through the majority of the slides (I’m still missing the last 10 or so, BTW. I’m thinking that these will fall into place eventually.)

Arrived at SFO, “Can I have a cop car please?”

Landed at 12:00 PM. I have two hours to get from SFO to Santa Clara, no problem. Terminal 3, run up to the rental car tram thing (which always makes me a bit impatient BTW). I always pick the worst possible rental company so there’s a line and I have to listen to someone have an argument over insurance refusal for five minutes so that I can walk up to the counter and have the same argument over insurance refusal.

On these trips, I always ask for a Crown Victora, why? First, they have enough leg room for someone as tall as me. Second, I have to get to Santa Clara quickly, and, right or wrong, I think that cops are less likely to pull over a cop car. Also, I find that people tend to get out of my way. You see a Crown Vic on the highway pull up behind you, you tend to slow down, mutter an expletive, and hope that you don’t get pulled over. This means that Crown Vics have a built-in safety bubble on the highway.

30 minutes later I’m driving down 101. At this point, most rational people would drive directly to the conference and finish up these slides with time to spare, but I always complicate the straightforward. I meet a colleague for lunch in Palo Alto on the way down. I figure that getting to the conference with enough time to spare would be too easy.

On the way down to Palo Alto I see a billboard advertising a job with a Javascript snippet and a new building I haven’t seen before, the Evernote building. So that’s where my $5/month goes to, huh? Paying for a square millimeter of office space just south of San Mateo?

Palo Alto: Capitol of Silly Valley

101 down to University avenue, drive toward Stanford. I’m meeting someone on High street for a quick lunch. Not business really just a sort of “I’m in town for 20 hours and my slides aren’t finished yet and I haven’t had any sleep and I’m going to be rushing” lunch. Palo Alto has a sort of reality distortion field around it. It’s where people like Larry Page live and where Steve Jobs once lived. There’s this street, University Avenue, which is lined with technology companies (like 10gen), parfumeries, VC firms, and establishments where you could probably drop $500 easy on a box of gourmet chocolates. There’s usually someone juggling a Tesla and, soon, everyone there will be walking around with Glass discussing the latest, fabulous TED talk. It’s that sort of place.

It’s also overly-serious Stanford grad country and there are always a bunch of twenty somethings milling about in suits carrying portfolio briefcases. These people will soon be richer than you can imagine for selling the next startup services company to a startup aggregator associated with a startup incubator. The Crown Vic is a bit out of place on University Avenue, it knows this and starts having loud engine problems. Awesome. I park, meet my colleague for lunch, we talk, he asks me when my talk is, I tell him. “Oh, you have to get down there now….”

Me: “Whatever, I have a crown vic. I’ll make it.”

Great America or Bust, Too Many Redirects

As a man, I don’t use maps until it is too late and I’m already lost, and now I’m just driving around “Santa Something or Other” trying to figure out which way is West. I know one thing, Santa Clara Convention Center is near Six Flags. Look for the signs. I stop, not to ask directions, but to use Google in an effort to locate myself by sextant and iPhone. It turns out that I was close, so I start driving toward what appears to be some sort of stadium under construction? I finally locate the convention center and attempt to park.

And, thus began a 10 minute scavenger hunt style experience following no less than 20 signs directing me to overflow parking. It wasn’t just “turn left dive a few hundred feet and park”, it was an obstacle course of difficult to distinguish signage and cones that I was tempted to slalom through. 10 minutes later I’m in some City of Santa Clara parking facility. I spy someone with a geek backpack and I know this is my chance to follow them to the conference. This same pattern of redirection and signage is repeated on the way from the parking deck to the conference. A series of hastily assembled signs are directing us to climb over bridges and walk on pathways with signs like “Golf Carts Only”. At the end of a brisk half mile walk, I am directed to climb to the top of another parking deck. The guy in front of me commented that this whole thing felt like a trap. Maybe it was…

Oh, you aren’t dressed like a speaker…

I walk into the conference, and locate the speaker’s room. It is now 3 PM and I have an hour left. Speakers check-in at the speaker’s lounge, and the speaker’s lounge was a trip. While not everyone had a suit on, it was immediately clear to me that Strata was an entirely different kind of conference from something like OSCON. OSCON is a geek festival, and while a few speakers might have a suits on (usually vendors or people making an ironic statement), if you show up at OSCON wearing a suit people are going to look at you funny.

I had jeans on. Jeans and a black zip up turtleneck, and when I check-in the person that gave me my pass was like, “Oh, you didn’t look like you were dressed as a speaker…” Don’t get me wrong she was terribly nice, but this wasn’t the greatest morale booster. I don’t know how to take this, does this just mean that I look like a god awful slob? Maybe I do. I am, after all, a god awful slob / idiot who hasn’t slept all night and who now has a strange inferiority complex because it is clear that this speaker lounge is about 90% suits to 10% non-suits. What have a gotten myself into?

Post-doc Opportunities and Business-y Data Topics Abound

I have a few minutes to finish my slide deck: 50 minutes and about 10 slides – the meat of the presentation. Details on Google Spanner, some quotes from Google about BigTable and Megastore, and a quick overview of vendors trying to solve similar problems: Translattice, Akiban, and Drawn to Scale. Hopefully I’ll see someone I recognize because this conference has me feeling like an underdressed stranger. As I’m sitting there, listening to other people’s conversations two things jump out: the guy at my table is talking about post-doc opportunities with someone someone MIT? People behind me are talking about something that has to do with the government? A bunch of suits at another table sound like a eWeek article about storage and “business enablement”.

Real worry, am I even at the right conference? This isn’t a room full of Perl freaks talking about Burning Man. No one is juggling. No, dude, Strata is way different. Maybe I should take my O’Reilly-style, big-idea slides down a notch? Just then, Dean Wampler walks in. Dean and I share a city, but we’ve never met in Chicago. I met Dean once, years ago, at the last Foocamp I’ll probably ever be invited to. Dean’s a prolific writer and developer. He runs the Scala group here in Chicago and write books about functional programming among other topics. He was teaching a Hive tutorial, and was the right person to show up because I wanted to sanity check some assertions about technologies like Impala (a Dremel implementation from Cloudera). Without Dean the presentation would have just been awful, and he doesn’t even know it.

(Something to note about the Big Data space. No one has used enough of these technologies to make definitive comparison between two technologies, if someone comes to you with answers there’s a good chance they are a vendor trying to sell you a story. I know things about Impala but these things change so quickly I needed to double-check some things with Dr. Wampler.)

Bradford Helps with a Slide and an Attendee Insults Me

Of the three vendors I wanted to discuss: Translattice, Akiban, and Drawn to Scale, I had little or no information on Drawn to Scale. Bradford was the co-chair of the track my talk was on, and I had asked him to be my backup in the off chance that my flight was delayed. Now, I’m going out on a limb, but everything I’ve heard about Spire adds up and I do think that Drawn to Scale is a leading candidate for the particular type of database I’m talking about in this session. We talk for 15 minutes, I gather as much information as I need for the presentation, and double check some facts with Bradford. At this point, I’m ready.

With 15 minutes to go, I hear some super cranky business guy behind me talking about my session without knowing that the speaker is sitting four feet away. “The Future of Relational is SQL…what are hell are these crap sessions anyway. I love how these jackasses try to throw in some provocative buzzwords to try to get people….” I don’t turn around, I don’t say anything. I sort of cherish this moment because it gave me that last kick I needed. I could have stopped him and said, “Jump in a lake golf shirt dude, we’re going to talk about Copernicus and Roman Gods, and it’s going to be awesome.” I didn’t say that, but I did take the energy from that interaction and just decide to kick the presentation up a notch or two. If this audience doesn’t respond to slightly crazy big ideas, I’ll go down fighting.

Exhibition Hall: Sensor Lab

Strolling around the exhibition Hall. I never understand these things. Most vendors in the data field tend to hand out one pagers with very predictable charts. “Oh, wow, you offer a 20x performance improvement. That’s great.” When you really feel like saying, “So, these graphs are totally fake right?” The best performing vendors are the ones that look like they want to talk to you the least, “Hey you want a shirt? No. That’s cool. You looking for a database? No. Alright.” The worst vendors are the ones that hire good looking, non-technical picture frame types to talk about how “awesome” everything they do is.

Bump into a bunch of O’Reilly people, some of whom should recognize me but totally don’t (it’s an odd place to have once briefly worked for BTW.) I’m genuinely interested in this sensor lab thing they are doing (but maybe don’t quite understand the immediate application). I’m interested in the technology they are using – specifically the XBee protocol. Not sold on the value of measuring the temperature or ambient noise, but I’m sure someone will convince me with an interesting conclusion on Radar. Implied purpose of the effort: “We really don’t know what this is going to yield, but why not start measuring things.” It looks like it is also something that is being done in conjunction with a company that provides real-time sensor analysis via some system running on AWS. This is the sort of stuff I’m used to seeing at an O’Reilly conference.

One thing that I’m seeing is that there are “Awesome” buttons outside of the talks. This is to measure the relative awesomeness of the talk.

Conference Room J: Getting Setup

Done with my 5 minute exhibition hall visit it was off to the room to give my talk. First impression, this room is bigger than I had expected. Ok, I hope this room doesn’t fill up because that would be terrifying.

The A/V guy starts to help me connect everything up to the display systems. I try to locate myself and make sure I understand exactly where the edge of the stage is. While falling off the stage would be funny, it would also hurt. I do a little bit of planning with the few minutes I have left. Go over some of my slides one last time, and then I plug in my laptop. That’s when the problems start.

The screen doesn’t display properly. I mean all that work I put into getting the Keynote presentation mode working. You know the feature that I spent hours on so I could read a loose script and get my transitions right? You know that interface that allows you to see which slide is next and which reminds you to advance to the next slide if you forget and start to dwell? You know that interface that has a clock on it so you can pace yourself? Forget that, because the AV equipment totally doesn’t work like that… no notes, no anything. Even though a good deal of your presentation’s details were in the presenter notes… well tough. Present it from memory, oh and don’t look up because…

Conference Room J is Standing Room Only, You Have No Notes, and good luck not stuttering

Well look at that, standing room only even though I’m not “dressed like a speaker.” And, there’s no time to regroup from the realization that your lack of notes leaves you incredibly exposed in front of a few hundred attendees. Many of whom I’ve never met, but I recognize from Twitter.

Important piece of background information: I stutter, sometimes badly. Most people I know get that I’m going to seize up every 5000 words and have what looks like a facial seizure for a couple of seconds at least once a day. It’s my defect, but it is also my secret advantage. When you stutter, people are scared to interrupt you, also when speaking fluently takes real effort it forces you to carefully measure your words.

I’m usually unapologetic about my stutter, but in a public speaking situation, I do try to mitigate the stutter as much as possible. Sometimes I lead with a pre-announcement such as: “If a freeze up during this presentation someone needs to reset me, I have a stutter.” I use humor to warn people because I want to save people the uncomfort of laughing at my stutter. (Laughter is the natural reaction if you don’t expect a stammer.)

I can’t say half the worlds in my presentation…

Depending on the day I have a certain set of consonant sounds I know I should avoid, and on the day of my talk, the sudden disruption of my plan meant that I was a bit more nervous than usual. On the fly I had to figure out how to avoid saying words that start with a Hard C or a T. In other words, lack of notes and a clock meant that I was totally off my game. Being more stressed out meant that I was more likely to stutter, and my public speaking subsystem is now developed enough to appreciate that I shouldn’t say certain words.

I started the presentation, and since I changed the majority of this presentation to pictures for which I now had no notes, you’ll notice that the first couple of minutes are a bit uneven. You will also notice that “Codd”, “Copernicus”, and “Ptolemy” play important parts in the first 20 minutes of my session. When the video comes out, you’ll notice something interesting – I think I struggle to say Copernicus once, but I generally avoided words I knew I couldn’t say. This meant that instead of saying “Codd’s seminal paper on Relational Calculus in 1970…” I had to, instead, say “The seminal paper on Relational…” In other words, I had to find a way to avoid speaking these words and let the slides do the work. It felt odd at the time.

Slides with Words Save Me and Questions I can’t answer…

The tail end of the presentation was full of slides with words. Oddly enough this was refreshing because I had access to the details of what I needed to say, I could select important points, and read them. I might stick to slides with words on them from now on.

I finish my presentation and people start asking me questions. Someone from Google comments that he’s perfectly happy with Megastore (something I sort of put down earlier). He asks a question and makes a statement about Google’s use of these technologies, I answer by telling him he’s right, but that it is also true that Google developers are smarter than most. Other people are asking me questions, but someone asks me a very general question about Stonebreaker. “Do you agree with what Stonebreaker wrote?” I do, but for this question I have to get a bit diplomatic. I know one piece he wrote, but I don’t know everything he wrote, I answer the question like Jay Carney at the Whitehouse. “He’s on to something, but I’m not aware of everything he wrote in that particular year.”

Mostly positive questions, one question at the end about discouraging people to use Big Data. I clarify that my talk was really about general trends and was in no way meant to dissuade people from using Hadoop, etc. I back away from most provocative statements by saying things like: “I’m not saying Hadoop is unnecessary”, or “I’m not trying to pick on 10gen, 10gen is great.”

Suddenly the talk is over and I’ve made it through without stammering more than once or twice subtly. I start to relax, but then a big group of people form to ask questions or just say hey. I talk to a group of people about SQL vs. MapReduce, go to the media room to discuss analytics databases. I didn’t expect this many attendees nor this much response. What have I done?

Crazy, I still thought the presentation went poorly, but people convince me otherwise.

Note: There was no awesome button so when someone publishes a summary of the most awesome talks at Strata, do me a favor and leave a comment on that blog entry: “Yeah, well there was no awesome button for O’Brien, conspiracy?”

Stop by the O’Reilly Booth to Say Hello

Wandering around the Exhibition Hall I meet Jack Clark of the Register for the first time. Interesting to me because I read him all the time. He concludes (quite correctly) that I might very well be the most cynical person in the entire exhibit hall. I have to step away, 30 minutes later someone tells me he’s written a piece about the talk. No? Really, my talk. That’s crazy. Just 100 minutes prior someone was giving me a hard time for not being “dressed like a speaker.” Go figure.

Stop by the O’Reilly Booth, which is always interesting, again I once worked for O’Reilly. It was a part-time gig, but I treated it as a serious commitment. I used to cover technology, government, science, and open source. Interviewed a congressman, talked to scientists, and published some interesting news stories. I thought the news experiment was working at the time, but it wasn’t generating enough traffic. (Guess what? people don’t read science journalism.) I was starting to land interviews that were deemed too risky so I had to start getting permission. I didn’t dig that, and I also needed to focus on other work at the time. Writing on the wall was read and I just faded into the background noise.

So when I show up at an O’Reilly booth, it’s often unexpected. Laurel Ruma quickly gets me to sign a release form for the session because she’s seen a couple of tweets that suggest that the talk worked. She also understands that if she doesn’t get me to sign that form now, it’s never happening. I strike up a conversation with Jim Stogdill who I last saw in 2009 at the last Foocamp I’ll ever be invited to. Jim is one of the only individuals I consider sane who is both Paleo and Crossfit and I want to get his opinion on both. He’s convincing, I may commit to this.

Mike Hendrickson overhears the conversation and chimes in, in classic Mike Hendrickson style, agreeing that I could stand to lose a “couple of pounds.” Great not only am I dressed like a god damn slob, now I’m fat. If anyone else said that to me it would earn them an expletive or two, but there’s something about Mike Hendrickson’s blunt honesty that is refreshing. I’ll take it, he’s right.

Mike Loukides and I discuss how I’ve proposed four books recently and have delivered on none of them. I tell him that I’m storing up all my promises so that I can deliver on them all at once – shock and awe style He nods, not buying it at all. He shouldn’t.

They are going out for some post-event meal drink, and I’m going back to my hotel – I haven’t slept, I flew across the country, and I just presented. I’m done.

Exhausted, Fried, and Frightened

I drive back to my hotel, and on the way I take the wrong exit near Oracle. I proceed to drive across a series of meandering islands near the mega-complex headquarters of a company named Yodlee. I am confused, hungry, and unable to figure out how I ended up in this labyrinth of corporate buildings. Finally I break free of the office park, drive by Oracle and return to 101. My hotel (sorry motel) is in San Bruno.

Travelling to Silicon Valley? Here’s a suggestion, don’t go for the bargain hotel on Expedia that happens to be close to the airport. Take it from me, the deal isn’t worth it. I was traveling on my own dime this trip, but even when someone else is paying I still like to travel cheap. Well, the Regency Hotel in San Bruno? Two things: that place is too damn cheap and I’m lucky I made it out of that place alive. The neighborhood is gritty, but there was some the gang dude walking around the parking lot looking into people’s cars. He was the one that convinced me that going out for dinner wasn’t worth the risk. Who needs food? I certainly don’t want to be driving around San Bruno in a Crown Vic at this point.

Bedtime, have to wake up and end this whirlwind tour tomorrow morning, time to get some sleep, but the heat is broken. It’s so cold in the hotel room I’m shivering. There I am, shivering cold, concerned about the gang activity in the parking lot. I move a table in front of the door because there’s someone rapping outside (I kid you not). Somehow I doze off. Thankfully I made it through the night having learned an important lesson: San Bruno is the ghetto.

Leaving California & Learning about Housewares

The rest of the trip was uneventful. Returned the car, navigated security, waiting, waiting, waiting, almost tried to board a flight to Orlando because I got the gates mixed up. End up on the Chicago-bound plane in a middle seat. I’m sitting between a married couple, neither wants to switch with me. I asked one of them if they’d like to sit next to each other, he said, “No I prefer the window seat.” I caught myself before making some possibly accurate joke about how funny it was neither of them wanted to sit next to the other.

They were attending a housewares convention at McCormick Place this weekend. I made the mistake of saying, “Oh, that sounds interesting.” More advice, only say that to a stranger if you are really interested because this Thursday I learned more about Housewares than I ever wanted to know, and it keep on going and going and going. “Oh, kitchen accessories, wow?” This underdressed, fat slob, is too nice to say, “I actually don’t care I’m just trapped on this plane with you.”

Developers: Security is Your Problem, Get Over It

James Turner’s post “Developer Week in Review: The overhead of insecure infrastructure” captures the hubris with which many developers approach not only security issues but a number of issues like performance or deployment. Here’s the paragraph Turner uses to hammer home his point:

“Personally, I’m tired of wasting time playing mall security guard, rather than Great Artist. In a world where we had made security a must-have in the infrastructure we build on, rather than in the code we develop, think of how much more amazing code could have been written. Instead, we spend endless time in code reviews, following best practices, and otherwise cleaning up after our security-challenged operating systems, languages and platform. Last weekend, we honored (at least in the U.S.) those who have given their life to physically secure our country. Maybe it’s time to demand that those who secure our network and computing infrastructures do as good a job …”

“Mall security guard”? Could his attitude toward operations be more patronizing? Boy, would I hate to be the guy responsible for application security in James’ group. Does his idea that developers should be “Great Artists” strike you as a bit grandiose? While I think there is some aspect of “artistry” to development, most of us are creating business applications that babysit relational databases. Calling us artists in the context of this post just makes me think of a “Great Artist” developer laying down some edict about not using insecure operating systems and then storming out of the room…. and the Memorial Day reference, wha? No.

Imagining the meeting with “Mr. Great Artist” lays down the law wrt to an insecure OS…

(Great Artist lead developer makes a declaration that Linux isn’t secure enough for him. He announces this, makes an odd comparison to remembering the dead on Memorial Day, and walks out of the meeting before anyone can respond.)

CTO: “What happened?”

Security Guy: “I don’t know, but wow that was INTENSE. All I did was tell him we found a few more SQL injection attacks and he flipped out. He told us we can’t use Linux anymore because it didn’t meet his security standards and then he just up and left. WTFF?”

Another Security Guy: “Yeah, he started talking about how C is a bad language and how it wasn’t his problem that the operating system doesn’t have the capacity to detect…”

Developer in same group: “Well, maybe he’s having a bad day. Listen, can I just fix these security issues. I think he has a point, but I also think he lacks tact. What can we do to address the security…”

Security Guy: “We’ve tried to set selinux to enforce a number of times, but every time I do that he tells me he doesn’t have time to enumerate the ports and files he needs to access. He told me to ‘figure it out’ last time I asked.”

Developer in same group: “Yes, internally he’s made the same rant a number of times, we’ve been trying to get him to let us use a Java Security Policy to lock down access to resources. He said something about security not being our concern…”

Another developer: “…that was fun. He sort of sneered at us and called me a ‘mall security guard’. And, you know what, screw that! My dad’s a mall security guard right now, it’s the only job he could find this economy… understand that I almost quit right there….”

CTO: “Woah, don’t quit. You are one of the only people on the development team keeping the system secure. Come directly to me next time, I have to find a way to keep the Great Artist occupied with something other than being a pain in the ass.”

Project Manager: Alright, so let’s move without the Great Artist. We’ll turn on selinux and we’ll get the Java Security policy up and running. That should solve the problem then…

Another developer: “…sorry to interrupt, but can I make one point.”

Project Manager: “Of course.”

Another developer: “selinux will help us catch some vulnerabilities at the OS level. The Java security policy is another layer of defense, but there are a series of vulnerabilities that are application specific. Things like insecure versions of Tomcat, that need to be updated. Someone on the development team is going to have to be tasked security on an ongoing basis, and we need to work closely with operations on all of these issues.”

Project Manager: “We didn’t budget for that.”

CTO: “Budget for it.”

Project Manager: “Who’s department does that come from?”

CTO: “Good question….. Ok, I have to sit down with Mr. Great Artist and have a very long talk about attitude. We may have more money in the budget after that conversation.”

Loukides: “The winner in the case isn’t just Google; it’s all software developers”

Mike Loukides sums up the end of this phase of the Oracle v. Google trial in the title of his piece on Radar: “The End of a Fishing Expedition”.

First, Mike is right, as he very often is, but this sentence gives me pause:

The winner in the case isn’t just Google; it’s all software developers, who don’t have to worry as much about creative interpretations of copyright law, and are free to develop compatible implementations of an API.

Yes, Alsup’s decision is something of a victory for Open Source and people interested in compatibility, but Alsup also mentioned something in the summary that I’m still trying to understand:

This order does not hold that Java API packages are free for all to use without license.

While this was a win for Dalvik, Java is still confined in this annoying “box”. OpenJDK is open source, but the language and is still gated by this TCK. I have yet to see any analysis from people I trust (aka groklaw) on this topic. Java still isn’t free, but, at least, there’s no barrier to people creating an alternative implementation (they just can’t call it Java).

It will be interesting to see what an Oracle scorned does to the licensing terms it uses to distribute Java. While the APIs are not copyrightable, my lawyer friends have told me that there’s not much limit to what Oracle could put into a EULA that accompanies the JDK. (Just look at how Apple uses the EULA for the iOS SDK for proof that anything is possible.) If they are rational and want the platform to succeed they won’t do this, but I’m worried that now that Java isn’t as lucrative as they thought it was that they might just discard it (and ruin it in the process).

I also fear the appeal. I’m a pessimist, if this case makes it to the Supremes all bets are off.

I do agree with Mike, Alsup is something of a modern hero.

I’m opening up this thing… Common Java Cookbook

As promised, I’m opening up the license for the cj-cookbook. I’m starting out with Creative Commons Attribution-No Derivatives-Non-commercial 3.0 US. So, I think that this license is a bit Draconian. It essentially means: “Can’t sell it, don’t use it for training, don’t change it, and tell everyone I wrote it.” Doesn’t that seem a bit vain for an open source project? I might relax the license a bit by dropping the NoDerivs clause, but I’m still mulling it over. Does anyone reading this post have any particular feelings about what license this book should be under? Does anyone want to challenge me to release it under ASL 2.0? I have been critical of viral licenses in the past, so it would be ironic of me to release it under the GPL Documentation licenses.

Today I…

I’m sick of writing books behind walls, it’s time to bring it all out into the open.

Two O’Reilly Posts this Morning: PGP-10 and Justifying Twitter Trends

First, I wrote a quick piece pointing people to the Personal Genome Project. Church’s experiment is expanding beyond the initial group of 10 luminaries and is starting to invite more participants. It will be interesting to see how this plays out, from some discussions at last year’s Scifoo, I’m convinced that aggregating everyone’s Genetic sequence into a massive data set would yield a series of benefits that could potentially transform our understanding of disease. The simple act of creating an index of the human genome and comparing that index to the incident rate of various pathologies could yield clues as to the cause of various conditions.

Second, a response to Morozov’s Twitter missive in Foreign Policy. Morozov sounds the alarm that Twitter has the potential to misinform and bemoans the excessive number of tin-hat conspiracy theorists on the platform. Listen, Twitter is a communications platform, and, as such, a bunch of crazies are always going to show up to the party. I don’t think that you discount the entire conversation because of eight example Tweets… nor do I think it is helpful or original to view Twitter as a potential terrorist threat. Maybe Morozov was auditioning for the role of Fox News columnist?

Twist to Graph Twitter Trends (Swine Flu)

This is a follow-up to the two O’Reilly articles from yesterday “Twittering the Swine Flu” and “Tracking and Graphing the Swin Flu with Twitter”.

I’ve been using Twist to graph Twitter trends, here’s an interesting graph of the impact of Swine Flu on Twitter. What I find interesting about Twist is that it shows you a technology that is impossible with Google Search. Google’s Flu Trend service is likely more accurate and able to track local Flu trends. Twist on the other hand is going to provide a quick snapshot of awareness of the general population.

With Twitter, services like Twist are able to perform analytics on the entire data set of Twitter (albeit indirectly). Twist didn’t need to ask for permission to create this valuable graphing service. On the other hand, Google Trends and Google’s Flu Trends had to be generated by the private entity that owns the search data. As services like Twitter and Google become essential tools of government, we should be comparing the openness of these platforms.

Here’s a graph of “swine”, “oprah”, and “obama”. As you can see the Swine Flu story seemed to break yesterday, and the awareness peaked out at just below 1%.

Click on the graph for more detail, or go to http://twist.flaptor.com/

CJCOOK: Updated All Component Versions

I did a quick pass to Common Java Cookbook to update some of the version numbers. Current release version is now 0.12, and you can expect a 0.13 release on Monday that is going to remove most of the references to “Jakarta”. This book uses the following versions of components:

Component Version Notes
Commons Beanutils 1.8.0
Commons Collections 3.2.1
Commons Digester 1.8
Commons HttpClient 3.1 Will update to 4 as soon as the HttpCore stuff is released
Commons JEXL 1.1
Commons JXPath 1.3
Commons Lang 2.4
Commons Logging 1.0.4
Log4J 1.2.15
Commons CLI 1.1
Commons Configuration 1.6
Commons IO 1.4
Commons Math 1.2
Commons Net 2.0
Velocity 1.6.1
Slide 2.1 We’re replacing this with Jackrabbit
Freemarker 2.3.15
Commons Betwixt 0.8
Lucene 1.9.1 We need to upgrade this item.
Component Version Notes

Open Source Writing: Part I: A Few Problems with Publishing…

If you are just tuning in, Common Java Cookbook is an experiment in transparent, open writing. I’m trying to develop this book and make frequent releases every one to three days. The idea behind this book is that open source writing should be no different than open source software. This is the first post in series that explores some of the reasons why I’ve decided to commit myself to open, transparent writing. This post focuses on the problem. What is wrong with the current approach to computer “books”? What is wrong with the current relationship between the author and the publisher? This post focused on some of the problems with the current approach to books about computer programming.

Problem: Driven by the Physical Artifact

While most writing projects are governed by the limitations of the book as a physical artifact, books like Maven: The Definitive Guide and Common Java Cookbook choose to fully embrace the idea that a book is an electronic documentation unaffected by the constraints introduced by the printing process. Most programming books you encounter today have to have a practical deadline after which no changes are introduced. In other words, if you are writing a book that needs to be printed in lots of five thousand and shipped to book stores, your process is always affected by the idea of the book as a static, physical object. You have to “finish” the book by a set deadline. Updating and radical changes to a book which has already been printed tend to decrease book, and (quite often) the original authors retain no rights for redistribution online.

This attachment to the physical object is driven by the economic realities of the publishing industry, but it creates an odd situation when you are writing about a rapidly moving open source project. There is a large disconnect between how we develop open source software and how we write books about open source software. Successful open source projects usually don’t have a set release date, software like Maven is released when it is ready. Imagine how awful open source would be if everyone had to run around like headless chickens to cut a CD for something like Apache HTTPD. Imagine if a Maven release vote were predicated by “People, if we don’t send the Maven ZIP file to the CD factory by next week, they might cancel our contract. Can I get three +1 votes, now.” It just seems odd that we have to dance around publisher deadlines when we are writing books about collaborative, unpredictable, schedule-less open source projects.

Problem: Deteriorating Economic Model

Take, as an example, the Jakarta Commons Cookbook. I wrote this book between 2002 and 2003, and I probably invested about an entire year in the effort. It was my first book, so progress was very, very slow. The book was published, I felt great about the process. I think every first-time author has this initial excitement about having published a book. I didn’t write the book for acclaim, I wrote it because it was my way of giving back to the community. A year passes, and you get the sales figures back and you, the naive author, are impressed that five thousand people bought the book. You get a flood of email from people who have read the book, maybe 10% are fuming mad at typos and the other 90% is just happy to have read the book. The publisher has a totally different view, 5,000 copies is actually viewed as a quarter success, the publisher would have liked to sell 10,000. While you feel great about the idea of a community of 5,000, the publisher is lukewarm about the idea of printing a second edition.

Right right right, 5,000 is a loser? Visualize 5,000 people in a line all holding $20…. If that’s a failure, if that doesn’t justify a second printing, then something is wrong with the model. These days, publishers don’t like to commit to books that are not going to move a significant number of copies. It is becoming more and more difficult to sell a good book to a publisher because as the open source world continues to evolve every topic becomes a niche topic with a limited audience.

Problem: Where’s my community….

When you sell 5,000 copies of a book, you certainly get feedback both good and bad… But, you don’t get the customer relationships. You don’t get a chance to interact, and you certainly don’t establish any sort of persistent HTTP 1.1 connection with your readership. Publishers provide some tools to enable this support: forums, blogs, etc. If you’ve grown used to the “intimacy” and unstructured creative anarchy of open source communities, you’ll feel a bit stifled. Efforts like Jono Bacon’s The Art of Community are an attempt to address this, and publishers like Pragmatic have done a good job of creating that sense of community… But, as an author, you will want to either create that community yourself or (better yet) integrate that community with the community that has already developed around the project you are supporting.

Publishers serve an important curation function they provide the necessary work to ensure that the book meets production standards has come to be expected in a book, but they often don’t do a great job organizing a community. Just like an open source project manages software production, I think authors and open source projects should manage a community of readers. Publishers used to be a necessary intermediary, but as the importance of the book as a physical artifact continues to decrease, I think we’re going to see authors take more initiative and publish works online.