Archive for May 2009

The unexpected success of PrettyTable

I’ve been a bit lazy lately with regards to keeping this blog up to date on matters relating to my free software.

PrettyTable, which I released back in February (as blogged here) has been an unexpected smash hit. Not so long after I released it, I thought that since, unlike a lot of my other projects it was of quite general interest and also fairly complete / robust, I’d try something different and put it up on PyPI, the official Python package index and the nearest thing Python has to an equivalent of Perl’s venerable CPAN. I figured this would help the project get a little bit of exposure, but I never expected what happened next!

It wasn’t even a week before I got an email from someone letting me know how much he liked the project. A little later he wrote a blog entry on it. Motivated by this I released an updated second version, which quickly received a bug report from someone who had been using 0.1 “often”! This user kindly tested a fix for this bug which became 0.2.1. More and more people started emailing me to report problems or suggest problems and eventually even contributing code! Everything that is supposed to happen in the “magic pixie dust” view of open source software was happening and I was amazed, since nothing else I’ve ever written has received so much as a single “thank you” email. By the time 0.5 was almost readily I was sending regular emails to an impromptu mailing list of 5 or so consistently interested and helpful people announcing changes and asking for feedback. It was starting to become clear that PrettyTable might grow a larger community than I could easily manage by myself with nothing more than a self-hosted webpage and a manually administered mailing list, and on the advice of one of 5 “friends of PrettyTable” I set up a project at Google code. Google code is very similar in spirit to the older and better known Sourceforge project, providing free facilities like file hosting, documentation wikis, mailing lists and version management repositories to free software projects.

The move to Google code, for reasons that still elude me, really thinned out my burgeoning little community. Only two people from the “friends of PrettyTable” mailing list subscribed to any of the new official mailing lists (although other people I’d had no previous contact with have joined them so there’s not been much of a net loss). This was a surprise since things had been going so swimmingly previously. The project is by no means in danger of collapsing – in fact, someone from the Debian project, which distributes one of the oldest and most respected GNU/Linux distributions, recently contacted me to let me know he was submitting PrettyTable to Debian’s famously large package repositories, which I expect will bring in a ton of interest. I just wish I knew what was so off-putting about the move from a visibly amateur project with hand-manged mailing lists to somewhere with a fairly slick web interface for reporting bugs and other communications. Possibly there exists a misconception that using Google code requires a gmail account or something.

At any rate, I’ve really enjoyed PrettyTable’s rapid growth and the project feels like it is moving in a good direction. I plan to release 0.6 fairly soon, which will be a backward-compatibility breaking release in which the basic API is finalised, in a state which I feel is as clean and Pythonic as it can be. There are a few more features that need to be implemented but I imagine it will be a month or two at most before I have something I am happy to call 1.0, which I am cautiously optimistic will become a somewhat well known and regularly used library.

One question I’d like to have answered is how much of PrettyTable’s unprecedented success is due to me putting it on PyPI and how much of it is due to it simply being better or more interesting than other things I’ve written. I put HTTPeek on PyPI shortly after PrettyTable to try to investigate this, and so far I haven’t heard anything about it from anyone. This is probably not enough evidence to make a decision, though, it might be that HTTPeek simply sucks (it turns out a good chunk of its functionality can also be provided by Firefox extensions, so perhaps that’s why it gets no love). I’ll probably start releasing more stuff on PyPI in future to get a better feel for this. I have quite a few projects under way or at least in my head at the moment, including a web browser with a Tk interface (I’ve already written this to a pretty complete degree, but I did it in Python 3, to get a feel for the changes, but probably need to backport it to 2.6 before releasing it because the lack of good libraries like BeautifulSoup and the Python Imaging Library on 3.0 is holding me back too much) and a few tools related to the sadly under-appreciated FOAF project.

Unplanned downtime, YouTube without Flash

I had some unexpected downtime of this website (but not my mail service) recently which actually went undiagnosed for a few days. The cause was an annoying bug in lighttpd – annoying not so much because it is there, since everybody makes mistakes, but annoying because they actually seem reluctant to admit that there is a problem, even though there clearly is. A few days ago I set up an IPv6 tunnel with the very cool guys at SixXS. I can’t get native IPv6 from my ISP, but tunnel brokers like SixXS can act as an intermediary – my computer wraps IPv6 packets up inside IPv4 packets, send them to an IPv6 capable host in the SixXS system, which passes on the IPv6 packets over the actual IPv6 internet, and then wraps the response back up in IPv4 and sends it back to me. This offers practically all the benefits of IPv6 without me actually having it per se. In particular, I can now accept incoming IPv6 connections to my mail and web servers, and so now do so. It turns out, however, that enabling IPv6 support in Lighttpd has the effect of disabling IPv4 support on all Unix OSes other than Linux (because Linux takes the non-standard and technically wrong approach of having IPv6 sockets accept incoming IPv4 connections). I didn’t notice this immediately, of course, because I could access my pages just fine – via my IPv6 tunnel. Anybody on an IPv4-only connection (and that’s pretty much everybody) wouldn’t have been able to connect. Sorry about this.

Another quick thing: last night I learned, via undeadly.org of an astonishing little thing called HQTube: a Greasemonkey script which, using the well-known mplayer-plugin for Firefox, lets you watch YouTube videos inside your browser using mplayer, even if you don’t have Flash installed. Even more incredible than the fact that this exists and works is that it has been around since at least May 14, 2008 (date of the last update), and people are only just discovering this now. People using BSD operating systems that don’t have Flash support have been jumping through all sorts of hoops for years in an effort to get Flash working, and YouTube is normally a huge part of the motivation to do this. None of the other methods I’ve read about are close to being as easy or robust as this one solution seems to be. It should be much better known than it is.