Four Things your Open Source Project Isn’t Doing (But Should)


Choose someone to be responsible for audience measurement and community relations.   Up until now OSS projects have taken a “if we build it they will come” approach to attracting an audience, but there’s so much noise out there these days, it takes real effort to create sites and experiences that lend themselves to easy adoption.

Why should you care, this is open source after all?   Open source projects without an audience tend to wither, and if you aren’t attracting a certain number of new users you can almost bank on the community evaporating over time.  Even your most ardent supports will change jobs or maybe lose interest, having families, get busy, and move on.    You need to work to attract a audience not just for the sake of making some statistic go up, you need to attract an audience and make sure that you measure the health of your audience on an ongoing basis.  Measuring open source projects and gathering audience statistics is something more OSS projects should do.

1. A/B Test the Intro Message – Let’s take the case of a build tool that I used to be interested in, Maven.   Stephen Connolly proposed on the Maven dev list that he wanted to change the message from “Maven is a project management and zzzz….” to “Maven is *the opinionated* build tool.”   If I was involved, I would have jumped in and suggested that he use Optimizely and run an A/B test on that message change.    Test everything and don’t let anyone make subjective guesswork out of something that could be easily measured.  That would have immediately led to a discussion of what the metric for success would be, which leads to #2….

2. Define Metrics for Success – Measure statistics like unique visitors and download numbers and keep a disciplined approach to measuring these numbers over time.  When you run an A/B test using a tool like Optimizely, make sure that you measure the success of the test against these metrics.    Have a report that is generated weekly,  monthly, and annually which captures audience numbers and success metrics (like downloads or mailing list signups).  Share this information with the committers, make them aware of larger trends.

3. Analyze and Refine Analytics (maybe Openly) – Analytics can be dangerous as it is easy to just slap a Google Analytics code on a page and call it done.  If you do this without thinking about canonical host names and making sure that your pages are instrumented correctly you can end up with analytics numbers that are off by a factor of 2x or more.   Analytics is so far from easy, even if you are running it, there’s a good change that you are doing something wrong.   If you run an OSS site, make sure to make analytics (and tending to analytics) a regular task.   Analyze these reports without preconceptions and identify trends in usage.    You likely publish documentation online: do you know what the most important page is?   Do you know how people end up on your site?  What’s the most popular search term?    How many people responded to your launch announcement?

One more thing, if you are an open source project, why not publish your analytics reports publicly as a way to recruit more help?   Transparency is the best publicity.

4. Report on Social Interactions, Monitor Twitter – Pay attention, the worst thing that can happen is someone having a problem with your tool and you missing out on an opportunity to help.    You shouldn’t just rely on contributors to do this in an ad-hoc sort of way, you should have a measured and well-thought out approach to monitoring Stackoverflow, Twitter, LinkedIn, and (possibly) Facebook.      Tools like Hootsuite can help.

I know that some developers read this and think that this is all silly marketing bull.  It isn’t.   What is bull is pretending that attracting an audience doesn’t matter, and that you are just working on OSS because of your own self-interest in code.   In my experience, you are working on OSS because you are trying to attract others with similar scratches to itch.   A bigger community of users results in a healthy pool of potential contributors.   Open source projects, especially those at foundations like Apache and Eclipse, need to get comfortable taking a more aggressive approach at attracting an audience and they need to stop depending on vendors to do this work for them.