The show was largely structured around the well known Drake equation, which tries to estimate the number of intelligent civilisations within the Milky Way galaxy, other than our own, which we should in principle be able to make contact with.  It does this by multiplying together estimates of a bunch of relevant terms, namely:

• the average rate of star formation per year in our galaxy,
• the fraction of those stars that have planets,
• the average number of planets that can potentially support life per star that has planets,
• the fraction of the above that actually go on to develop life at some point,
• the fraction of the above that actually go on to develop intelligent life,
• the fraction of civilizations that develop a technology that releases detectable signs of their existence into space,
• the length of time such civilizations release detectable signals into space.

The format of the show was basically to explore the most interesting of these concepts and what we know about them – for instance, how we’re starting to get pretty good at discovering exoplanets, planets outside our solar system, which helps give us an idea of how many stars have planets and what those planets are like, at least in terms of very high level features like size and distance from their sun.  What I found most interesting was the discussion of the 5th term in the Drake equation, which deals with the fraction of planets bearing life where that life eventually evolves to be intelligent.

The show’s discussion of this term was mostly centred around the fact that for intelligence to evolve from unintelligent life takes quite a lot of time, and this time may not always be available.  All kinds of events, ranging from asteroid impacts to strong tectonic activity, can very easily completely or almost completely wipe life off a planet (Earth itself has had 5 major extinction events so far, and some would argue, not unconvincingly, that it is currently going through a 6th, in the form of humans wiping out species at an alarming rate), and if these average duration between these events is shorter than the average time it takes unintelligent life to evolve intelligence, then that suggests that the jump from life to intelligent life will be very rare indeed.

There’s nothing wrong with the above analysis, of course: sufficiently frequent extinction events are both a reality (I recently finished reading reading Bill Bryson’s “A Brief History of Nearly Everything“, which was quite an eye-opener on just how inhospitable Earth is to life on long enough timescales) and a real obstacle to the evolution of intelligence.  But underlying all of the discussion on the show seemed to be an implicit assumption that this was all that was standing in the way: that if there was some particularly lucky life-bearing planet out there which was somehow shielded from asteroids and solar flares and supernovas, and had relatively stable, benign weather and tectonic activity, and basically was left completely unmolested by forces of great destruction, then it would be a matter of certainty that, eventually, intelligence would evolve.  To be fair, I don’t know if the producers of the show or Neil himself believe this, but certainly the show did nothing to explicitly dismiss this notion.

The problem with this is that it’s completely wrong, and yet it is surprisingly often overlooked.  I wouldn’t have noticed this oversight myself if I hadn’t previously read either Steven Pinker’s “The Blank Slate” or his “How the Mind Works” (I forget which it was), which talks about this misconception in considerable detail (I can’t remember whether or not it was in the context of the Drake equation).  Although it’s easy to fall into the trap of thinking that it is, evolution is of course absolutely not some kind of driven, inevitable progression from simple to complex organisms.  It’s about adaption to an environment so as to maximise reproductive success, and unless there is an unintelligent organism somewhere in an environment with the following conditions satisfied:

• intelligence is evolutionarily accessible to the organism (i.e. it already has necessary prerequisites, like a sufficiently complex nervous system, for a few mutations to lead to some kind of intelligence),
• evolving intelligence will give it the organism a significant reproductive advantage over its unintelligent companions,
• and there are no other evolutionary pathways open to the organism which will yield a better ratio of reproductive advantage to “cost” (in terms of energy requirements, etc.) than intelligence,

then intelligence isn’t going to just turn up for the sake of carrying life higher and further.  Those organisms will remain unintelligent, possibly extremely successfully, possibly for an extremely long time, until the next extinction event wipes them out.  Intelligence is not inevitable given sufficiently long time.  It needs a good reason to emerge.

The problem that this situation poses for accurately estimating the 5th term of the Drake equation is that we actually have no idea why humans evolved intelligence.  There are plenty of plausible hypotheses out there, but nothing for certain, and I don’t think there is likely to be anything certain in the near future, given that we know extremely little about the lives of early humans and their ancestors (something else, incidentally, which Bill Bryson’s book gives a good accessible account of) and that we really know extremely little about intelligence (to the extent that there isn’t even a universally agreed upon, objective definition of what intelligence even is).  If we have no idea how we became intelligent, we’re not really in a position to speculate reliably about how likely other organisms are to become intelligent, given the chance.  The 5th term of the Drake equation could, in fact, be arbitrarily close to zero: close enough to zero to completely counteract all the terms in the equation which are very probably quite large.

Of course, it’s by no means a new criticism of the usefulness of the Drake equation to point out that the uncertainty surrounding our best estimates of each of its terms is so great that the final answer can vary by orders of magnitude, and even reach zero.  However, as far as I know, the term relating to the likelihood of the evolution of intelligence is the only one which currently has no reasonable lower bound: you can push it as close to zero as you like and not really reach a point where you can compellingly say “come on, surely it has to be higher than that“.  Which means that no new discovery suggesting that one of the other terms is actually incredibly huge will be sufficient to guarantee a result of more than one.  Which means, somewhat sadly, that perhaps we are much closer to being alone than a lot of people, myself included, have always thought.

On a related note, I recently read this BBC article, which discusses the opinion of one SETI astronomer that we should stop structuring the search for alien intelligence exclusively around the assumption that said intelligence will be biological in nature (which is an implicit assumption – and not the only one – of the Drake equation’s structure) and instead start to consider the possibility that a lot of that intelligence will – in its own version of our own transhumanism movement – have become non-biological in nature; that we should be looking for civilisations of intelligent machines, which are likely to hang out in very different places to intelligent meatbags.  I think this is a fairly persuasive argument.  Eliminating the problem of the mind-blowing slowness of interstellar travel (which is essentially a necessity for a civilisation to be truly long lasting) by figuring out how to transplant our consciousness into machines is probably considerably easier than the alternative of getting around the slowness directly with some sort of sci-fi-esque wormhole stuff.  At the very least, a lot of people who are experts in the relevant field believe that the former may be possible in principle, whereas, as far as I know, the latter is purely speculative.

are capable of diving hundreds of metres under the surface of the ocean and find a wide range of applications in both the commercial world (underwater oil drilling operations being a topical example) and the scientific research community.  They’re fairly pricey too, with even the most basic models – which basically consist of a camera and some light – costing tens of thousands of dollars.  However, a surprisingly large number of hobbyists have managed to build capable “hobby ROVs” for only a few hundred dollars, such as the Seafox, pictured below:

The precise designs of these hobby ROVs varies somewhat, but there are a few things that they all seem to have in common, which contribute to their simple construction and low cost:

• They use standard PVC pipe and fittings as their main structural components
• They use small 12V DC electric boat bilge pump as their main means of propulsion
• They use common, non-waterproof devices like webcams mounted in watertight containers for the onboard electronics

I’ve done some fairly thorough scouring of the web for resources on how to build these things.  As near as I can tell, the hobby ROV movement began, or at the very least was widely popularised by the book “Build Your Own Underwater Robot And Other Wet Projects“, which includes complete designs for two PVC ROVs, the Seaperch and the Seafox (pictured above).  The book is fairly cheap, but there is so much information about these kinds of vehicles online now that you probably don’t need to buy one unless you’re unfamiliar with basic electronics, like using relays – I don’t plan to buy the book unless I run into trouble.  Far and away the best online resource for building hobby ROVs seems to be the Homebuilt ROVs page by Stephen Thone, who has built a number of ROVs, most of them of his own design.  The image of the Seafox above is of the Seafox he built.  Some other decent resources are:

• Hobby Submarines (featuring a slightly modified Seafox)
• Underwater ROV at instructables.com
• How to build an underwater robot, at eHow (the ROV shown here is named “Bob”)

There’s also a Yahoo Group called Robotrov which seems highly regarded in the hobby ROV community, but I haven’t checked it out yet (I’m not partial to Yahoo Groups).

In some ways this is a bit of an odd project for me to undertake, in that I’m not much of a water person.  I’m not afraid of water by any means, but I’m not really into fishing, boating, swimming, diving, etc. like a lot of people who build these things seem to be.  However, while most of my projects are software-based, I have always enjoyed making “real things”, and for a long time I’ve wanted a project I could use to deepen my understanding of electronics (which was probably my first serious hobby when I was a young kid, before I had access to the internet or even computers).  In particular I’ve always wanted to have an excuse to buy and learn to use something like the very popular Arduino boards, but nothing has really grabbed me yet.  ROVs seem like a great candidate for such a project, though, in that they don’t require a lot of space or tooling to construct (I don’t currently have regular access to a large shed or any tools, I don’t know how to weld, etc.), they don’t need to be precisely built (unlike, say, UAVs, which would otherwise be an awesome hobby and for many people are), none of their components require special training or licenses (unlike model rockets), there aren’t any issues with government regulations (unlike the high-altitude balloon projects which are currently extremely popular), and it seems unlikely that simple mistakes could result in catastrophic failures: most ROVs are designed to be slightly positively buoyant, so that a broken tether or flat batteries means the ROV will slowly rise back to the surface of the water, rather than sink to the bottom.

I’ve decided to build a very simple ROV first to get my feet wet (figuratively and literally!), probably just something with the bare minimum number of thrusters to get around, a camera and some lights, before moving onto something more complex.  Rather than strictly following the Seafox design like lots of other people do, I’ve decided to use the “Bob” ROV as a starting point, because its frame/shape strikes me as more efficient than the Seafox, which is really rather chunky.  So today I headed to my local hardware store and bought a bunch of 15 mm PVC pipe and fittings to use for the main frame of the ROV and some 90 mm PVC pipe to use for buoyancy pods, as well as other associated stuff like PVC cement, a hacksaw, sandpaper, some cable ties, etc.  I’ll try my best to make regular posts about the construction process.

After testing the water with a really simple ROV I might have a go at designing something more complicated, from scratch.  I’m particularly interested in building a complex navigation system: GPS doesn’t work under water, since EM radiation at the appropriate frequency only penetrates water a few centimeters deep.  This means ROVs need to rely on things like digital compasses, accelerometers, gyroscopes, sonar and laser rangers to find their way around underwater – lots of fun stuff and plenty of opportunities for learning.

The two directions for NASA are multidimensional, and I think it’s simplistic to simply take a side.  I’m disappointed to see Constellation cancelled, although certainly there were things about it I thought could be better: the DIRECT group’s Jupiter rocket family seems like a better idea to me than NASA’s Ares family, for example.  I think the Obama plan’s focus on privatising the development of launch vehicles makes sense, and stayed up quite late to watch the recent inaugural flight of SpaceX’s Falcon 9, but I wonder about its suitability for manned flight out of Low Earth Orbit: SpaceX have a direct commercial incentive to develop the Dragon capsule and rockets for launching it – ISS ferry missions – but do they have the incentive or knowledge base for something like a Mars mission?  I’m excited about the Obama plan’s push for research on exciting new technology like inflatable spacecraft modules and orbital propellant deposits, but I don’t see why we can’t combine that sort of research with, well, actually going to places and doing stuff.  But I think perhaps the central question in this whole thing, the one which is most important for the long-term future of manned spaceflight, and the one where I am the most baffled by one side of the two camps, is the question of whether we should ever go back to the moon, or simply proclaim “been there, done that!” and head straight for Mars.

I absolutely do not understand the attitude of “we’ve already been to the moon, why go back?”.  Not only does it not make a lot of sense, it also seems to be an attitude reserved singularly for the moon.  Did anybody seriously break this argument out after the first time we reached the north or south pole, or the first time we climbed Mount Everest, or each time we discovered a new island?  Of course they didn’t.  It’s really just a silly argument, and so it’s disappointing that it’s one of the most commonly cited arguments against a return to the moon: probably less common than the old “why spend so much money on space when we have so many problems here on Earth” line, which I feel is effectively dealt with by a combination of pointing out that humanity is actually capable of doing more than one thing at a time, that sometimes the solutions to problems on Earth can be found in space (and this is going to become more true rather than less true in the future as land and resource pressures on Earth mount), and finally that we really can spare the money – the total amount spent so far on the war in Iraq could have paid for a complete repeat of the Apollo program (including R&D costs) with $500 billion left over to spend on the environment, poverty, hunger, education and whatever else you care to mention.

Viewing the fact that we’ve already been to the moon as an argument for never going back confuses space exploration as something done purely for prestige, entirely as a stunt, a game of ticking boxes with no further meaning or purpose.  Admittedly this attitude played a strong role in the original space race, with a desperate desire to show up the commies fuelling most of what NASA did, but in 2010 it’s entirely out-dated and nobody should cling to it anymore.  We’ve been to the moon, yes, but we haven’t yet even scratched the surface of what there is to do on the moon!  We haven’t done a fraction of the meaningful geological research that could be done on the moon: research which can help to improve our understanding of the formation and history of the solar system, including Earth.  We haven’t set up observatories on the far side of the moon to take advantage of a complete lack of light pollution or atmospheric distortion.   We haven’t prospected for valuable Helium 3 in the lunar soil.  We haven’t tried growing edible plants in the lunar soil.  We haven’t tried performing industrial processes on the moon which will work much better and much more cheaply in the reduced gravity.  The list of interesting stuff we haven’t done goes on.  Many would argue that most or even all of the above could be done by teleoperated robots from Earth at a lower cost than it could be done by people.  This is undoubtedly true, but doing it at least partly with people has the benefit of providing much needed practice for the establishment of future long-term colonies on the moon for their own sake.  Seeing permanent off-Earth human presence is essential for guaranteeing the very-long-term survival of the human race, and should always be considered the ultimate goal of manned spaceflight.

Some people who are in favour of off-Earth colonisation would argue that the moon is a bad place to do it.  It’s true that if we just had to pick somewhere in the solar system as a starting point for off-Earth human presence, based entirely on how nice various places are to live, the moon would probably not end up high on anybody’s list.  It has no atmosphere, no magnetic field, very low gravity, not a whole lot of water, and unless you’re in one of a few special places, you can’t rely on solar power easily because night time lasts 14 days.  It makes a lot more sense to head for Mars, and arguably even more sense to head for the atmosphere of Venus (although this idea has a lot less currency amongst space enthusiasts, for some reason).  However, the simple fact is that moving humanity off the Earth is not a simple matter of choosing somewhere to go and then going there.  We have to deal with the practical matters of actually getting and living there.  The moon is 3 days away from Earth using existing technology, meaning that rescue missions and emergency resupplies are actually feasible: contrast this with months of travel for Mars using existing technology.  The radio delay to the moon is a little less than 2 seconds, compared with about 15 minutes for Mars, reducing the psychological feeling of isolation and allowing real time problem-solving assistance from specialist teams on Earth.  Landing on and taking off from the atmosphere-lacking moon is a completely solved problem which we’ve done before, whereas the same problems on Mars are a lot harder and it’s by no means certain the techniques we use for probes and rovers will scale up to vehicles large enough to support a human crew.  Even though Mars is a nicer place to be than the moon, getting to and living on the moon is going to be considerably easier than doing the same on Mars for the forseeable future, and since there are still so many unanswered questions about living off-Earth – what are the long term physical and psychological effects of reduced gravity, of increased radiation exposure, of living so far away from the rest of humanity? – it makes sense to me that we should try to answer these questions in the very near future on the moon, at the same time as we work on getting everything else we can get out of the moon in terms of science, resources and industry.

First up is photo management.  This is something that was done extremely poorly on my old self-hosted site, I had to admit.  I originally had a series of scripts for producing static galleries, pulling in metadata (like descriptions and tags, etc.) from flat text files, but it didn’t work particularly well because I simply didn’t have the energy to do things like make the gallery look nice (not that I didn’t care, but CSS, despite being a fantastic idea in principle, is such a poorly implemented and counterintuitive hack-fest that making a nice image gallery without using tables is far more difficult than it should be) or provide the metadata.  After that I experimented briefly with using a third party PHP-based application which didn’t require the use of a database, but it was a fairly half-hearted effort and I never got a significant number of pictures into it.

The two big “cloud based” photo management apps are, of course, Flickr and Google’s Picasa.  Flickr has by far the largest userbase, and looking at a few “Flickr vs Picasa” articles turned up by a quick Google suggested that Flickr is ahead in a lot of important ways, so I decided to give it a try.  When I first tried to sign up and realised I would be forced to setup a Yahoo account, complete with @yahoo.com email address, I quickly lost my stomach for the idea: I have no interest in using any other Yahoo services and I don’t want yet another unused and unwanted email address cluttering up my online presence.  After further reflection, I decided this was a little hypocritical, since I already use a number of Google services which require a Google account: although, to be fair, Google makes this process a lot easier and much more lightweight.  Anyway, I bit the bullet and got myself an account.

My experience with Flickr so far has been entirely adequate, but it’s not exactly something I’ve fallen in love with.  The interface could use a lot of work, I don’t find it particularly straightforward: or rather, the stuff that everybody wants to do all of the time is not done using more obvious or prominent controls than the stuff that few people want to do some of the time.  While the idea of a Photostream (a sequence of photos arranged in the order you upload them) is fine (indeed, it makes sense from the point of view of keeping up to date with the photos of friends),  I don’t think.  I’m also really irritated that the ability to define hierarchical sets (“sets” are coherent collections of photos in Flickr, things that might be called “albums” elsewhere) is restricted to those who hold a “Pro account”, which costs US$25 per year.  I have no objection to the concept of a paid account status, I realise Flickr have costs to cover, but it seems like the features that you need to pay to get should be limited to things which are in some sense “special” or “excessive”: increasing the number of photos you can upload per month, or the maximum size of photos, or even the number of sets you can define are sensible candidates.  Such a basic and fundamental organisational tool as the hierarchical arrangement of sets should really be a freebie: it’s something just about everybody is going to want.  Anyway, I’ll be sticking with Flickr for the forseeable future.

The second thing I’ve been looking at is keeping a record of which books I read and when.  This is something I never even tried to implement back when I was self-hosting everything, although I’ve had an account at LibraryThing for some time (but rarely used it).  I really like the idea of this kind of thing, but I never kept up to date with it much, mostly because the interface at LibraryThing is so terrible.  For one thing, I found it genuinely hard to remember where to go to edit my metadata about the book (such as when I read it or what tags I’d like to give it), as opposed to editing site-wide metadata about the book, such as who wrote it.  Worse, LibraryThing treats every separate edition of a book as a distinct entity.  There is no way to say “I have read William Gibson’s Neuromancer“.  You are forced by the system to proclaim “I have read the edition of Neuromancer published in 1984 by Ace Books”.  By all means, the ability to express that information if one wants to should be present, but actually forcing it is ridiculous, especially since if your particular edition is not in the system you are forced to enter incorrect data by choosing another edition.  The refusal to combine all editions into a single Platonic concept of the book also really interferes with useful collaborative filtering: you can’t learn things about people who also read Neuromancer, only things about people who also read your edition of Neuromancer, which is not a distinction I can imagine many people are interested in.

Frustrated with the above shortcomings, I’ve checked out GoodReads, an application similar in spirit to LibraryThing but which seems to lack these shortcomings.  It has Platonic concepts of books, but also the ability to specify editions if one is so interested.  I’ve loaded a small, initial subset of everything I can remember reading (basically favourites and stuff I’ve read recently) into my account, and the interface seems clean and intuitive.  It remains to be seen how long I can keep up the habit of chronicling what I read, but it’s nice to have a tool for doing so which is straightforward enough that I can’t really blame it for my failures to do so.

The starting point for all of this was my having to move house a little more than a fortnight ago.  I decided that, after several years of it, I was starting to get sick and tired of the hassle involved in self-hosting (especially when moving time comes), and in fact I was starting to get sick and tired of doing everything computer-related “the hard way” for the sake of things like control, ideology and technological machismo (more on this in later posts).  So before the move, I got a friend who owns a hosting company to set me up with some webspace and began the process of migrating everything across.

I only have FTP, not shell access to my new host, which necessitated a lot of significant changes to the way everything worked.  Previously my website was powered by a self-written collection of hackish Python scripts to translate text files written in Markdown format into HTML files using Cheetah templates, and the blog was powered by my rudimentary own blogging engine Yomiko, a CherryPy application.  Not only was getting all of this set up and operating on a new host without shell access impossible, I have to admit that the inconvenience and fragility of the system probably prevented me from writing anywhere near as much as I would if were more simple.  So I ended up leaping from one extreme to the other and using, like everybody else, Wordpress for my new site.  I’ve avoided using Wordpress in the past primarily out of a distaste for PHP and because the fact that it strictly requires the use of MySQL strikes me as extremely poor design (why not write everything in terms of a database abstraction layer so that people can choose to use PostgreSQL or SQLite if they prefer them?), but I have to admit that if you can get past all of that its a very quick and easy way to throw up a website.  I don’t think it will extend to handling the “Writing” part of my website, at least not very conveniently, so I may look at using a Wiki application for that, even though it will break visual consistency.

The biggest question mark still hovering over this whole setup is how I will go about porting my old blog posts into Wordpress.  I would naturally very much like to avoid having to do this by manually copying, pasting and backdating everything, but without the ability to run scripts on the new host I’m not entirely sure how to go about this.

That’s all for now.  Hopefully all this decadent point-and-click WYSIWYG AJAX goodness will encourage me to start writing more frequently again.

I liked the idea of a blog entry that was autobiographical but still mainly technical in nature, and thought about writing my own version of “Programming languages I have known”, but I realised that this would, in fact, be pretty boring. I don’t know that many programming languages, the ones I do know are entirely mainstream and historically-uninteresting, and I taught myself comparatively late (I think I was 15 when I bought “Sam’s Teach Yourself C in 24 Hours”, which I just now found free online by Googling it). Then I thought about just writing something not on languages but on the various computers I’d used throughout my life, but that would be pretty standard and boring, too: My primary school had two BBC Micros, my household, two relatives and lots of my neighbourhood friends had Commodore 64s (one of these machines had a magnetic tape drive, which took cassettes of the same size as the old music cassettes, everybody else had 5.25 inch floppy drives), and then came the rise of the IBM PC.

Instead, I decided to write about two incidents I can remember from my primary school years in which I displayed a way of thinking which was in some sense algorithmic in nature – thinking like a computer programmer years before I wrote a single line of code, or at least wrote one that I actually understood.

The first of these concerns a game that one of my primary school classes would occasionally play. One student chooses a number between 1 and 100 and keeps it a secret. The rest of the class take turns attempting to guess the number, to which the first student must respond with either “higher” or “lower”. The student who eventually guesses the correct number then gets to choose the next one, and the game repeats. In retrospect, it’s amazing the kind of mindless and repetitive stuff that young kids will find fun. Anyway, I remember quite clearly the time we were playing this game when I realised that there was one clearly optimal strategy. I think I was around 11 or 12 years old at the time. The strategy I had developed was this: the first student to take a guess should guess 50. Depending on whether the response was “higher” or “lower”, the next student should guess 25 or 75, and so on, with each student guessing the number in the middle of the not-yet-disqualified range of possible answers. If you have no prior knowledge about what the secret number is likely to be, this strategy is the best you can do, in the sense that it minimises the average number of guesses required to find the answer. A very similar method can be used to find the zeroes of a scalar function, though of course I had no idea about anything like that at the time (note that the method is not at all optimal for the zero finding problem). I remember that I figured this out by visualising the number line from 1 to 100 and realising that each guess ruled out either all the numbers to the left or the right of it, and observing that by guessing in the middle of the remaining range you were guaranteed to rule out half of it, whereas by guessing anywhere else there was always a chance of ruling out less than half of it. I became quite excited when I realised this and explained it to the rest of the class, assuming that they would concur and cooperate with the strategy. Ever future instance of the game would be over in less than 7 guesses (the logarithm of 100 to base 2 is 6.64)! However, nobody else seemed to believe me and in fact the class as a whole was quite hostile to the idea of any kind of systematic approach to the game. Looking back, I guess the algorithmic approach probably ruined whatever element of “fun” we must have seen in the game, but at the time I was baffled by this resistance.

The second incident happened earlier, I must have been between 8 and 10 if I correctly remember the way year levels were distributed amongst classrooms at my primary school. However, it’s also less impressive of an insight. Around this time my classroom went through something of a football craze (note that Australian football is not the same as European football, which we call “soccer”, nor American football, which we call “rugby”. Actually, I’m not sure rugby and American football are the same, but who cares?). I never really got into this (I’ve never really got into following any kind of sports), but I remember being baffled by how little any of the other kids thought about the outcomes of football matches (I’m talking about matches in the national league, here, not lunch-time games) as being predictable from data. During this craze, a lot of kids collected football cards, and each card would have on the back all sorts of statistics about the corresponding player, things like the average number of points scored or marks taken per match. Kids would have huge folders full of these cards, each one slotted into a position in a plastic sleeve. They were veritable hand-held databases. And yet, given these databases, kids were making their decisions about which team to cheer for based on things like team loyalty or who was on a “winning streak” or simple gut instinct. I remember it being obvious to me at the time that the sensible thing to do was to assign some number of points to each team on the basis of the available statistics and to support whichever team had the most points. I never worked out an explicit scheme for doing this – presumably a team would get something like one point for each game it had won so far in the season, and one point for each average goal scored by each player who was participating in the match. I do remember realising it would be necessary to take off points for each participating player who had an injury. If I ever explained these ideas to anybody, I don’t remember it. No doubt they would have gone down as well as the method of bisection did for guessing secret numbers, anyway. No doubt my plans for a point assignment system were rather simplistic, perhaps too simplistic to have worked very well if I’d put them into practice (I never did, because at that age I didn’t have the computer skills to program it, and doing it by hand would have been tedious, especially since I didn’t really care about the football anyway), but the point stands that I was aware by this age of the principle that numerical data about past events could be used to forecast future events. I have no idea where I got this idea, whether it was just intuition or if I generalised from an actual observation.

Although these are appropriate childhood stories for someone like me to be able to tell now, I don’t actually understand why it is that I can remember them as clearly as I can. They must surely have been fairly inconsequential at the time. Can I remember these things so well because they were part of the birthing process of my now excessively algorithmic way of thinking, or is my distribution of childhood memories unbiased with regard to these sorts of things, and two of those memories just happened by chance to be like this because I thought that way a lot? Memory’s a funny thing.

Anyway, a few months ago I accidentally destroyed my Thinkpad’s motherboard in an incident that I guess I should write up as a cautionary tale about self-repair (probably more so against being impatient, but that’s another story). So I needed a new Linux desktop machine until such time as I could find an affordable replacement Thinkpad mobo. I have a collection of 3 HP “e-Vectra” machines, which I bought from eBay years ago when I was very interested in cluster computing. I really like these machines because they’re easy to find cheap, are very small (less than 30cm/1 foot in either of the long dimensions and not 10cm wide), they stack well (check out the photo in this forum post!) and don’t draw a lot of power (there’s no internal PSU, just a little brick transformer at the end of the cord, like a laptop). They’re not the most powerful machines in the world, with 700 MHz Pentium III’s and 128 MB of RAM, but I maintain that the perception of those sorts of stats as “uselessly low” by most people is largely an illusion generated by bloated mainstream software (this is a problem in both the Windows world and the FOSS world) and by a hardware industry driven almost purely by the game industry. I upgraded the RAM in one of them to 256 MB and went about seeing how it held up as a modern desktop. It wasn’t terrible, and if it weren’t for today’s RAM-intensive web, it probably would have sufficed, but ultimately Firefox froze up just way too often so I had to look for another solution.

The big Windows XP machine actually has two identical hard drives in a RAID array, so I split them apart and installed Linux on the second one, keeping XP on the first. This machine is considerably more modern (2.8 GHz Athlon 64 processor, 2 GB RAM), so no performance issues at all, but having my Linux system and Windows system on the same machine was kind of a hassle. Anytime I wanted to do play a game or print something, I’d need to carefully close down everything I had running in Linux and reboot, only to have to carefully open all that stuff back up again when I was finished. Furthermore, I couldn’t manage to get suspend to RAM working on that machine under Linux at all, so the machine ended up being left on 24/7. Because this machine was originally built for gaming by my brother-in-law, it’s horribly power hungry(although probably not half as bad as the machine he built to replace it, leading to my inheriting this one) so I really didn’t like this.

For the last few days I’ve been experimenting with a novel solution to this situation, and I think that I’ll stick with it because it’s worked out better than I ever expected. I’ve installed Sun’s VirtualBox system on the Windows XP install of the machine and installed Linux inside of a virtual machine. This is pretty much the first time I’ve ever experimented with any kind of virtualisation technology seriously, and I have to say I’m incredibly impressed. I never imagined that the performance of a virtual machine would be good enough to actually use it as a desktop, but I’m writing this right now from inside the virtual Linux install, with mail client and browser running, and music playing smoothly. The performance is just fine. Yes, it’s perceptibly slower than running Linux natively, but only just, and it’s 100% endurable. With the virutal machine running in full-screen mode, there’s pretty much nothing at all to give away the fact that Windows is talking to the hardware underneath it, all I can see is my ion3 X11 desktop. However, when the need arises, I can just minimise the machine and find myself at an XP desktop, ready to play games, to print something out, or to suspend the machine to RAM (which of course works perfectly under Windows). It really is like having immediate access to the very best of both worlds, the mainstream software and superior hardware support of Windows and the, well, everything else of Linux. As an added bonus, I can backup my entire Linux system as a single file, and even migrate a copy of it to any other Windows machine running VirtualBox! Seriously cool stuff.

The only real drawback I’ve found is that, because the virtual machine’s network interface is implemented using NAT behind the host machine’s interface, a few networky things don’t work out of the box. The only thing that hasn’t worked so far is NFS, as described in this forum post. This was easily solved by using sshfs, which just worked. Also I’ll have to set up port forwarding if I want to be able to ssh into the virtual box from anywhere else, but that’s no big thing.

I’m going to wait a few weeks to make sure there really aren’t any hidden problems with this approach, but I’m thinking I’ll probably stick to this arrangement and “re-RAID-ify” the two drives with the XP/virtual Linux image. Viva la virtualisation!

For reason’s I’ll go into another time, I’ve recently been trying to chroot a Python installation into the directory /srv/www on an OpenBSD 4.5 machine. I took the standard approach to this, which is to examine the output of ldd /usr/local/bin/python2.5:

/usr/local/bin/python2.5:
Start End Type Open Ref GrpRef Name
1c000000 3c004000 exe 1 0 0 /usr/local/bin/python2.5
05432000 2547c000 rlib 0 1 0 /usr/local/lib/libpython2.5.so.1.0
05dc3000 25dc7000 rlib 0 1 0 /usr/lib/libutil.so.11.0
0df1e000 2df44000 rlib 0 1 0 /usr/lib/libstdc++.so.47.0
0d543000 2d54d000 rlib 0 1 0 /usr/lib/libm.so.5.0
02a7c000 22a85000 rlib 0 1 0 /usr/lib/libpthread.so.11.1
06a56000 26a8f000 rlib 0 1 0 /usr/lib/libc.so.50.1
08db8000 08db8000 rtld 0 1 0 /usr/libexec/ld.so

and copy each of these files over to the corresponding location in the chroot directory (i.e. stuff from /usr/lib/ goes in /srv/www/usr/lib). But after doing this, testing the setup with chroot /srv/www python2.5 yielded the error: /usr/bin/local/python2.5: can't load library 'libpython2.5.so.1.0'.

I went near mad trying to figure out why this wasn’t working. The libpython file was clearly in the correct location. A PHP installation that I had chrooted into the very same place using the very same procedure worked flawlessly. There seemed to be no scope for rational explanation of why this wasn’t working.

In the end, I managed to solve this problem by copying the file /var/run/ld.so.hints into the corresponding location in the chroot directory. Everything worked perfectly after that. I don’t profess to have any idea why you need to do this (I discovered that it worked after much trial and error, based on random permutations of stuff found on the web), but you do. Hopefully this post saves some other poor hacker from wasting a few frustrated hours.

I note that the ld.so.hints file exists in the same location on recent NetBSD releases, so I assume this advice ports over to NetBSD as well. However, there is no such file in my install of Arch Linux, so things are probably different in Linux land. Linux distros do seem to have a file /etc/ld.so.conf which serves a similar purposes – it was actually reading about this that gave me the idea to look for something similar in my situation.