Why do Christmas lights all go out when one bulb blows? (and how to find the broken one)

The answer is rather simple: traditional Christmas lights (I'm ignoring newfangled LED varieties) were typically connected directly to the mains power supply and wired in series like this:


Only if the filaments of all the bulbs are intact will a current flow around the circuit; if one bulb breaks then the circuit is broken and all the lights go out. The reason the bulbs are wired in this, inconvenient, manner is that it's convenient for the manufacturer.

Although the supply voltage is 230v (or 110v) the bulbs are rated for a much lower voltage. At home I have a string of 20 lights like this with 12v bulbs. This works because of the rules of series circuits. In my home lights there are 20 bulbs each with some unknown resistance R. The total resistance of the circuit is 20R and the entire circuit is a sort of voltage divider.

The current flowing through the entire circuit is I = 230/20R and the voltage across any individual lamp is V= R * 230/20R or 230/20. So my 20 bulbs are each getting 11.5v. That's handy for the manufacturer because they can use cheap, small bulbs that use a low voltage.

BTW Some bulbs have a second piece of wire called a shunt that passes current when the filament breaks. With a shunt the manufacturer can still use series wiring and cheap bulbs, but a blown bulb doesn't stop all the lights from working.

Finding the broken bulb

A really fast way to find which bulb is broken is to perform a binary chop. To do that you need a multimeter (or similar meter to test continuity).

0. Unplug the string of lights from the power.

1. Remove the first and last bulbs and check that they are ok.

2. Remove the bulb in the centre of the string of lights. Using the multimeter check to see if there's an electrical connection between the contacts in the centre bulb socket and each of the end bulb sockets that you remove the bulbs from (you can actually look at the wiring to see which way the wires go and which contact that corresponds to).

3. Pick the half where there's no connection. The broken bulb is there. Remove the bulb that's in the middle of that half of the string and check it. If it's ok proceed to checking the electrical connection between the socket of the bulb you just removed the two nearest bulbs you removed (which will be the middle of the string and one end).

4. Proceed like that following where there's no electrical connection and dividing in half until you find the broken bulb.

This is a technique from computer science and you will find the broken bulb much faster (on average) than if you proceed checking each bulb in turn.

My password generator code

Some people have asked me about the code for my password generator. Here it is:

use strict;
use warnings;

use Crypt::Random qw(makerandom_itv);
use HTML::Entities;

print "<pre>\n  ";
print join( ' ', ('A'..'Z') );
print "\n +-", '--' x 25, "\n";

foreach my $x ('A'..'Z') {
    print "$x|";
    foreach my $y (0..25) {
 print encode_entities(
           chr(makerandom_itv( Strength => 1,
          Uniform  => 1,
          Lower    = >ord('!'),
          Upper    => ord('~')))), ' ';
    }
    print "\n";
}
print '</pre>';

Royal Festival Hall condundrum

When I went to record Shift Run Stop at the Royal Festival Hall a few weeks ago I noticed that the display on the 5th floor lift was not showing 5 but a bit pattern. I snapped a quick photo and decided to look into it later:


And here's a close up of the top of it.


If you look carefully you'll see that there are 8 columns of on or off squares. I transcribed the squares with on = 1 and off = 0 to get the following list: 11111111 11000100 11011000 11101100 00000000 00010100 00101000 01001110 01110100 10001000 10011100 10110000 11000100 11011000 11101100 00000000 00010100 00101000 00111100 01010000 01100100 01111000 10001100 10100000 10110100 11001000 11011100 11110000 00000100 00011000 00101100 01000000 01010100 01101110 10001110 10110100 11001000 11101110 00000010 00010110 00101010 01010000 01111000 10001100 10110010 11001110 11101100 00000000 00100110 00111010 01100000 10000110 10011010 11000000 11010100 11111010 00100000 01001100 01101100 10000000 10010100 10101000 10111100 11010000 11100100 11111000 00001100 00110110.

Apart from the first item which is all 1s all the others have a right-most bit of zero. At first I thought this might be 7-bit ASCII (LSB first), but decoding that just gives a mess. Then I wondered if it was machine code, but I think that's unlikely given the fact that one of the bits is always zero. I don't think this is random data.

Here it is as hex with LSB on the right.

ff c4 d8 ec 00 14 28 4e 74 88 9c b0 c4 d8 ec 00 14 
28 3c 50 64 78 8c a0 b4 c8 dc f0 04 18 2c 40 54 6e
8e b4 c8 ee 02 16 2a 50 78 8c b2 ce ec 00 26 3a 60 
86 9a c0 d4 fa 20 4c 6c 80 94 a8 bc d0 e4 f8 0c 36

And reversed:

ff 23 1b 37 00 28 14 72 2e 11 39 0d 23 1b 37 00 28 
14 3c 0a 26 1e 31 05 2d 13 3b 0f 20 18 34 02 2a 76 
71 2d 13 77 40 68 54 0a 1e 31 4d 73 37 00 64 5c 06 
61 59 03 2b 5f 04 32 36 01 29 15 3d 0b 27 1f 30 6c

So, what could it be? I'm assuming that the display is showing something from either its internal memory or from the memory of its controller and that we are looking at consecutive memory locations (this could, also be incorrect).

Anyone else want to take a stab at this? Anyone know what company made the controller for the display or the lift?

The other thing that's odd is that there are lots of monotonic increasing sequences in the data. e.g. drop the ff and observe:

c4 d8 ec
00 14 28 4e 74 88 9c b0 c4 d8 ec 
00 14 28 3c 50 64 78 8c a0 b4 c8 dc f0 
04 18 2c 40 54 6e8e b4 c8 ee 
02 16 2a 50 78 8c b2 ce ec 
00 26 3a 60 86 9a c0 d4 fa 
20 4c 
6c 80 
94 a8 bc d0 e4 f8 
0c 36

Write your passwords down

Here's my advice on password security based on the collected opinions of others:

1. Write them down and keep them in your wallet because you are good at securing your wallet. (ref)

2. Use different passwords on every web site because if you don't one site hacked = all your accounts hacked. (ref)

3. Use passwords of at least 12 characters. (ref)

4. Use mixed-case, numbers and special characters. (ref)

Research says you need 80-bits of entropy in your password so it needs to be long, chosen from a wide range of characters and chosen randomly. My scheme gives me 104 bits of entropy.

My passwords are generated using a little program I wrote that chooses random characters (using a cryptographically secure random number generator) and then printing them out on a tabula recta. If you were to steal my wallet you would find a sheet of paper that looks like this in it (I have a second copy of that sheet left with a friend in an envelope):


I use that sheet as follows. If I'm logging into Amazon I'll find the intersection of column M and column A (the second and third letters of Amazon) and then read off diagonally 16 characters. That would be my Amazon password (in this case, TZ'k}T'p39m-Y>4d); when I hit the edge of the paper I just follow the edge).

The security of this system rests on the randomness of the generated characters and the piece of paper.

PS Yes, it's a total pain to use long, random, different passwords.

PPS If it's not obvious to people you can add a second factor to this (something only you know) in the form of the algorithm for picking the password from the sheet. For example, instead of using the second and third characters from the site name you could pick any combination. And you could change the letters as well (e.g. for Amazon you could use the last two letters moved on one place in the alphabet; you'd have PO as the key). Also you don't have to read diagonally but could use any scheme that works for you (e.g. a spiral pattern, read vertically, read characters at offsets from the start based on the Fibonacci sequence, etc.).

Plan 28 gets some professional PR

Last week I announced that Doron Swade had joined Plan 28. I'm happy to say this week that we're getting some professional help with our announcements (and more) from global PR firm AxiCom. AxiCom handles clients such as Dell, Panasonic, Ericsson, Fujitsu, Logitech, McAfee, Qualcomm, Salesforce.com and more.

And now, on a pro bono basis, they are handling Plan 28. Here's their official blog announcement of their involvement.

Having professional PR is another big boost for the project because it takes a load off my shoulders and AxiCom can reach people and places I simply can't. I expect that their involvement will help Plan 28 enormously. Expect to see more news stories about the project over the coming months and more announcements about additional support for the project.

As always there's lots more going on, once details are finalized I'll announce. And please remember that Plan 28 still needs your financial support to make it a reality.

Don't write to me asking me to support your crusade against global warming science

I've received yet another email indicating that the author thinks I don't believe man is responsible for global warming. This comes about because of an insidious sort of tribalism that has turned conversations about climate change into a "you're either with us or against us" situation.

For the record, my reading of the scientific literature and my own reproductions of Met Office data convince me that (a) the world is warming and (b) the most likely reason for this is man.

Much of the 'debate' about climate change reminds me of the pro-choice/pro-life non-debates in the US. Once you split down what look suspiciously like faith lines you're no longer doing science at all. Many people seem to mistake my criticism of the quality of source code used by UEA's CRU as indication of some underlying belief on my part.

Poppycock.

To be clear, I think the code I saw from CRU was woeful and had many easily identified bugs. I also think that source code used for scientific papers should be routinely be made available. And, yes, I did find errors in Met Office software. People who discuss those errors often seem to omit the fact that correcting them reduces the error range for global temperatures thus increasing the confidence that the temperature trend is up since the 1970s.

I find it very sad that I can't criticize the one area of climate change science I know something about (software) without suddenly being thought of as 'on the side of the skeptics/deniers'. I'm not on anyone's side. I'll call it like I see it.

Shift Run Stop

Some time ago I recorded a long interview with the fine folks at Shift Run Stop. The interview covered all sorts of topics, but focussed on Plan 28 with detours through Kinect hacking, GAGA-1, Tron and The Geek Atlas.

The podcast comes out this Thursday, but here's a sneak preview.

John Graham-Cumming from shiftrunstop on Vimeo.

Backgrounder document on Plan 28

Doron and I have prepared a short document that describes the background and goals of the project. This is primarily intended for use with third-parties (such as sponsors, institutions and the press), but in the spirit of openness here's a copy that anyone can read.

A brief introduction to the Plan 28 Project by John Graham-Cumming/Doron Swade is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

If you want to understand the Analytical Engine, start with the Difference Engine No. 2

There are large similarities between Charles Babbage's Difference Engine No. 2 and the Analytical Engine. Critically, Babbage designed the Difference Engine No. 2 after the Analytical Engine and it incorporates improvements discovered during the design of the Analytical Engine.

And the printer that's part of the Difference Engine No. 2 is identical to the printer needed for the Analytical Engine. Babbage said that the same printer would be used for both. The memory of the Analytical Engine is very similar to the figure wheels in the Difference Engine No. 2.

Here's Doron Swade demonstrating and explaining the Difference Engine No. 2:



And here's a lovely video of the machine in motion. Now try to imagine the Analytical Engine which will have 8x the number of parts and be vastly bigger.

Babbage books as stocking stuffers

If you're following along with Plan 28 (the project to build Charles Babbage's Analytical Engine) then you might like to do some background reading. Here are four suggestions for stocking stuffers for the coming holiday:

1. Doron Swade's The Difference Engine (also published with the title The Cogwheel Brain).



This is Doron's account of the Difference Engine No. 2 as envisaged by Babbage and as built by the Science Museum in London.

2. William Gibson and Bruce Sterling's The Difference Engine.



A fancy based that imagines what would have happened if the Analytical Engine had been built in Babbage's time.

3. Cultural Babbage



A set of essays inspired by the Difference Engine No. 2 that discuss the cultural significance of Babbage and his life.

4. Charles Babbage's Passages from the Life of a Philosopher



Babbage's autobiography.

More background reading here.

A boost for Plan 28

Up until a couple of weeks ago Plan 28 was a one man show. Although Plan 28 has received enormous press coverage and many people have pledged money, services, material and time, the project was still just me.

I'm happy to say that that's no longer the case.

Doron Swade, the pre-eminent Babbage expert, who, as curator of computing a the Science Museum, masterminded the project to build Babbage's Difference Engine No. 2 has joined me on the project. Doron and I now share responsibility for finishing Babbage's work.

Doron and I met over coffee a few weeks ago to discuss the Analytical Engine and it was clear that both of us had been dreaming of building the physical engine for public display. Happily, Doron had been doing a lot more than dreaming. His deep knowledge of Babbage's engines and his continuing study of Babbage's plans and notes have placed him in the unique position of being the key figure in any attempt to build the world's first digital, programmable, automatic computer.

Much more has been happening behind the scenes that we cannot yet discuss, and the project's success is by no means guaranteed, but Plan 28 has received a major boost in the form of Doron Swade.

PS You can still pledge to the project; your promise of $, € or £ is much needed!

GAGA-1: The Camera Hole

This weekend's work on GAGA-1 was mostly around mounting the camera inside the capsule. The capsule walls are 95mm thick so a hole had to be cut all the way through for the thinnest part of the lens and then part way through for two other parts. A second trench had to be cut into the polystyrene for the part of the camera where the batteries are held.

The other thing I worked on was the positioning and mounting of the computers and where the batteries will sit. Here's a shot inside the box showing the camera pushed into place and flush against the capsule sides. There's a single battery pack in roughly the spot where it will be fixed and the recovery computer on the wall opposite the camera. The two gold connectors are the GSM and GPS antenna SMA connectors.


And here's a show showing the hole pierced through the capsule wall to allow the camera to take photos (yes, I have checked that the capsule wall is not seen in the photos). The recovery computer can be clearly seen at the back. I will be painting the hole the same yellow as the rest of the capsule just to make it look nicer.


The hole was cut with a very sharp, thin knife. A bit messy but the end result is certainly good enough. Here's the camera in the hole.


I insulated the trench with space blanket to keep the camera as warm as possible, but left the lens hole untouched because the walls are very thick there. The black circles are velcro pads used to help keep the camera in place during the flight.

Breaking the Reddit code

A few days ago an entry on Reddit asked for help breaking a code. Because I was laid up in bed yesterday with something chesty and nasty I couldn't help but wonder about the decryption of the message (see also the Fermilab code). At the time no one had broken it.

I managed to break it; here's how.

The original message was written on paper like this:


So I did my own transcription of the message and obtained the following four lines:

SSNTTNNDERPEVEEEHNOTONNAAEWMAEEMUDRITRNTNDOAWNETOHTVEEDMRMRTTFOGT
HUUFSHIIEMAHVOIANRTOARRSJRGEHHIEREELSEANMSTEMEWYEOHAMDEOMITTIECI 
OLCHHIMDBRPPCAPROMRADIMEOSISLTSTYMEIATYOOEDSTHIEVLVEOBECWGEOORYA 
TYERNOAEONLWRSLESKEEHTAEYIODSAAOIHWIUTMNWEONTHATPLVRLAPLIEOAAOUN

There were a couple of things that stood out immediately. Just eyeballing the text it looks like it's English (lots of E's, T's, etc.) and so I ran it through a letter frequency checker and sure enough it looks like English.


So given that, the code was most likely some kind of transposition cipher. I blindly ran through a bunch of classic ciphers using anagramming to try to find likely words. Wasted ages on this and got nowhere. Although I did discover that the last 16 letters can be rearranged to say POPULAR I LOVE ANAL.

Then I went back and looked at the text. There are clues within it. First, it's broken into four separate rows and that's likely significant. Secondly the first row is one character longer. That made me think that character must be the last one in the message.

After much messing around with the order of the rows I discovered that reversing the first and third rows resulted in the word THAT appearing in the first column:

TGOFTTRMRMDEEVTHOTENWAODNTNRTIRDUMEEAMWEAANNOTONHEEEVEPREDNNTTNSS 
HUUFSHIIEMAHVOIANRTOARRSJRGEHHIEREELSEANMSTEMEWYEOHAMDEOMITTIECI 
AYROOEGWCEBOEVLVEIHTSDEOOYTAIEMYTSTLSISOEMIDARMORPACPPRBDMIHHCLO 
TYERNOAEONLWRSLESKEEHTAEYIODSAAOIHWIUTMNWEONTHATPLVRLAPLIEOAAOUN

And, in fact, if you read down the columns from left to right (and add some spaces) you get:

THAT GUY YOUR EFFORTS ON THE ORIGAMI WERE COMMENDABLE HOWEVER VOV
STILL HAVE ONE STRIKE THE NOTE WAS HARD TO READ SO ENJOY TRYING TO READ 
THIS I HEAR I MADE YOUR TIMESHEET WELL I ASSUME IT WAS ME NO NAME WAS 
MENTIONED NO MATTER HOW MANY OTHER PEOPLE HAVE A CRVMPLED PAPER 
PROBLEM DID I MENTION THAT I HATE CONCLUSIONS

Notice that I made a few transcription errors. I suspect that VOV is really YOU and CRVMPLED must be CRUMPLED.

Guess I'll have to get back to Kryptos now.

GAGA-1: CoCom limit for GPS

One problem for high-altitude balloon projects is the CoCom limit on how high and how fast a GPS will operate. To prevent GPS modules from being used in very fast moving weapons (such as ballistic missiles) GPS receivers are not allowed to operate at:

1. Higher than 60,000 feet

2. When traveling faster than 1,000 knots

The second restriction doesn't matter for GAGA-1, but the first does. GAGA-1 will have a maximum altitude (balloon dependent) of more like 100,000 feet.

Different manufacturers implement the CoCom limit in different ways: some use an AND rule (>60,000 ft and >1,000 knots) and others use an OR rule (>60,000 ft or >1,000 knots). For high-altitude ballooning it's ideal if the GPS uses AND. Unfortunately, this information is shrouded mostly in mystery and it's only through actual flights and testing that people have managed to determine which GPS receivers are AND and which are OR.

For GAGA-1 I have two GPS units: one in the Recovery Computer and one in the Flight Computer. The Flight Computer is using a Lassen IQ which is known to work correctly on balloon flights.

The Recovery Computer is using the GM862-GPS which will fail. This is OK because it is used when the balloon has landed to send GPS location via SMS messages. But the failure mode is important.

I've been back and forth with Telit technical support and they claim that the module will simply fail to give me a GPS fix above 60,000 ft but that once the balloon is down again it'll restart automatically. Others claim that code should be included to automatically reset the GPS if it hasn't given a fix for some length of time. I plan to update the code to include an auto-reset after 30 minutes if no fix or no satellites during flight and recovery.

GAGA-1: Capsule insulation and antenna mounting

A bit of physical stuff on GAGA-1 this weekend after the Recovery Computer software last time. I'd previously painted the capsule for high visibility, but hadn't started cutting it or sticking on parts. After the successful test of the Recovery Computer it's time to put some bits on the box!


The three antennae visible on the box (as with the other components) are hot glued in place. I pierced holes in the box using a long metal skewer and a chop stick.

Here's a close up of the top of the capsule.


The top two antennae are for the two GPS modules (one in the Flight Computer and the other in the Recovery Computer). The long thin antenna is for the GSM connection that's part of the Recovery Computer.

The other two parts are a small red straw and a large black straw. The small red straw is simply there to allow the pressure to equalize between the inside and the outside of the capsule. Since the pressure is very low in the stratosphere it would be dangerous to send the box up completely sealed.

The black straw is sealed at the end with hot glue and will be where the external temperature sensor is placed.

I've further insulated the box by lining the interior with sheets of space blanket. This reflects almost all the heat generated inside the box (by the electronics) and should help keep things warm.


This was very fiddly to do as the space blanket material is very thin. I cut sheets out using a stencil and glued them in place. Placing my hand in the box I can feel warmth: the reflected warmth of my own hand.



Finally, here's an interior shot of the lid of the capsule showing where the cables for the antennae and straws poke through.

Notes on Kryptos Part 4

Copy of message I sent to the Kryptos group on Yahoo! for anyone whose working on Kryptos but not in that group.

Given Elonka's notes mentioning that K4 uses a cipher system not known to anyone else I decided to investigate other possible ways of attacking K4. Specifically, I wondered if the BERLIN crib might not be as simple as NYVTT turning letter for letter into BERLIN.

First I assume that this is something that's breakable by hand as was the rest of Kryptos and thus would simply be based on MOD 26 arithmetic of letters and might involve transposition of characters.

So I went to see if there's a word that could be permuted to create some permutation of BERLIN from NYVTT. There is: it is SILENT

NYVTT
ENTSIL
-----
RLINBE

More strikingly this works if you are sliding SILENT from the start of K4, it falls in just the right position to make BERLIN

OBKRUOXOGHULBSOLIFBBWFLRVQQPRNGKSSOTWTQSJQSSEKZZWATJKLUDIAWINFBNYPVTTMZFPKWGDKZXTJCDIGKUHUAUEKCAR
SILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTS
GJVVHHPWRLHETAZPVYTJHJYKNYBTEGYSDWBMOBBWWJKAPOMSOIENXEMLTEJBFNMRLINBEQMYHSHKQDRFENPWAOVYUNSCPOPTJ

Now leading on from this I wonder if the cipher used for K4 consists of permutations of both the key and the ciphertext. Note how BERLIN is permuted within itself and so then I returned to the start of cipher text to see if there's a permutation of SILENT that results in a word (after permutation) starting in position 0. Once again there is:

OBKRUO
ILENTS
------
WMOENG

i.e. the word is WOMEN, assuming that the G is in the word after women. In this case ILENTS is a simple rotate of the word SILENT (just as ENTSIL followed by ENTSIL gives us BERLIN). There are likely other words as well, but this one is strikingly long.

Running through all six possible rotations of SILENT you get:

OBKRUOXOGHULBSOLIFBBWFLRVQQPRNGKSSOTWTQSJQSSEKZZWATJKLUDIAWINFBNYPVTTMZFPKWGDKZXTJCDIGKUHUAUEKCAR
SILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTS
GJVVHHPWRLHETAZPVYTJHJYKNYBTEGYSDWBMOBBWWJKAPOMSOIENXEMLTEJBFNMRLINBEQMYHSHKQDRFENPWAOVYUNSCPOPTJ

OBKRUOXOGHULBSOLIFBBWFLRVQQPRNGKSSOTWTQSJQSSEKZZWATJKLUDIAWINFBNYPVTTMZFPKWGDKZXTJCDIGKUHUAUEKCAR
ILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSI
WMOENGFZKUNDJDSYBXJMASEJDBUCKFOVWFHLEEUFCIADIXSRELXWDDCOMNPAVQFARHDEXZSXXVATWCHIXWVVQROHAMIFIXVSZ

OBKRUOXOGHULBSOLIFBBWFLRVQQPRNGKSSOTWTQSJQSSEKZZWATJKLUDIAWINFBNYPVTTMZFPKWGDKZXTJCDIGKUHUAUEKCAR
LENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSIL
ZFXKMWISTAMTMWBEANMFJYDZGUDIJVROFLGBHXDLBYDWRDRHHEGCCTFHVTOQYJOGQXGXGFRNAOJZVSKBGCULTKXNZCLYRDUIC

OBKRUOXOGHULBSOLIFBBWFLRVQQPRNGKSSOTWTQSJQSSEKZZWATJKLUDIAWINFBNYPVTTMZFPKWGDKZXTJCDIGKUHUAUEKCAR
ENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILE
SODJCZBBZZCWFFHDQQFOPXTCZDJHZYKXLKWEAGJKRBWFXCHKANMBSWYQBSETRSUFGAZGMEHQTXPYLVDKMBKOMTDMPFEHXCKLV

OBKRUOXOGHULBSOLIFBBWFLRVQQPRNGKSSOTWTQSJQSSEKZZWATJKLUDIAWINFBNYPVTTMZFPKWGDKZXTJCDIGKUHUAUEKCAR
NTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILEN
BUCZFSKHYPFPOLGTTJOUONWVIJIXCRTDKAZXJMIAUUFLWSKDJTLRVPHWAIHMAYTVJTIMLUKJCDOOOOMQLRNHVZCCSYNNWSNEE

OBKRUOXOGHULBSOLIFBBWFLRVQQPRNGKSSOTWTQSJQSSEKZZWATJKLUDIAWINFBNYPVTTMZFPKWGDKZXTJCDIGKUHUAUEKCAR
TSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENTSILENT
HTSCYBQGOSYYUKWWMSUTEQPEOIYAVAZCADSGPLYDNDLKMVDMPSBUOYNVQLAVGXJYCCOLBXDSICERHXSPBUGQBYSFLHTMMVGNK

If you look you'll see various words popping out (in the ILENTS set, second row above, there's WOMEN at the beginning and closer to the end an anagram of WATCH).

Perhaps there's a method to choosing which shift of SILENT to use followed by some sort of transposition.

A proper Dr.

I wrote in my bio for The Geek Atlas that (speaking about myself in the third person) "Because he has a doctorate in computer security he's deeply suspicious of people who insist on being called Dr.". I am very suspicious of people who shove their PhD in your face, or who insist on being called Dr. In fact, like a British surgeon, I would much prefer to be called Mr. (which I suppose is a form of snobbery) mostly because it's what I've done after my doctorate that I'm most proud of.

Which brings me to the case of "Dr." Gillian McKeith. I only became aware of her because of the wonderful Ben Goldacre who has taken her to task about her qualifications and claims.

In an old article Goldacre talks about McKeith's qualifications and her legal threats against people who criticize her. He makes the very good point that it's easy to validate real credentials (e.g. if you want to check that I've really got a DPhil from Oxford you just need to call them up or, unlike McKeith, you can read my thesis).

Another person with a PhD from an unaccredited institution is John Gray (who wrote Men are from Mars, Women are from Venus). He refers to himself as Dr John Gray or John Gray PhD.

A striking similarity between McKeith's and Gray's web sites are the special sections explaining their degrees. John Gray has a page on the subject and McKeith explains her degree in detail:

Gillian then spent several years re-training for a Masters and Doctorate (PhD) in Holistic Nutrition from the American Holistic College of Nutrition (USA).
[...]
To obtain a PhD in Holistic Nutrition from the College, it is a pre-requisite that a student must have a Masters Degree and then undertakes study through a number of preliminary courses and a core curriculum, including general nutrition, immune system health, detoxification, herbology, human anatomy, enzymatic nutritional therapy, vitamin and mineral studies, nutrients, relationship of diet and disease, geriatric nutrition and nutrition and mental health. Doctoral students also have to prepare an original and practical dissertation. Gillian studied and completed the PhD course and dissertation over a period of more than 4 years between 1993 and 1997.

The PhD in Holistic Nutrition is a doctorate programme that is approved by the (American) National Association of Nutrition Professionals (NANP), a non-profit organisation which maintains the integrity of the holistic nutrition profession by establishing educational standards, a rigorous code of ethics and registration of nutrition professionals.

Whenever I see these long explanations I can't help remembering the line: "The lady doth protest too much, methinks". Let me, for the record, do a special section describing my qualifications: "John Graham-Cumming, MA (Oxon), DPhil (Oxon)"

Which brings me to the question of what the proper title is for someone who has a PhD that isn't really a PhD (or at least a PhD from an unaccredited institution). Might I suggest Ph. in front of their name for 'Phony'?

GAGA-1 Recovery Computer Ground Test

Today was the first live test of the GAGA-1 Recovery Computer and it, at least initially, didn't go well. The result is much improved, fully working, code in the repository. This is why I'm obsessed with actual testing of the components of GAGA-1.

First the good news: the module ran on 4 AA batteries for 9 hours without showing any problems caused by power. For over 3 hours the module was getting GPS location information and sending SMS messages. This is very reassuring.

Here's the module sitting on the kitchen table ready to go:


Here's the commit message for the code changes:

Significant changes based on live testing of the code on the Telit GM862-GPS module:

1. The module does not support floating point and so the gps2.get() function has been modified to only use integer arithmetic and thus not do conversion of latitude and longitude. This leaves most of the returned elements as strings. Except altitude which is needed for comparisons.

2. The main loop is wrapped in the try/except that will hide any inflight errors (although they will be logged to the log file). I hope this will never be called.

3. The timeout on SMS message sending has been increased from the default to 30 seconds because it can take a while to send the message and this was corrupting the return from the temperature command.

4. Improved handling of timestamps to make it clearer in the SMS messages when a message occurred.

5. Modified the upload script to delete the .pyo files for uploaded .py files so that the module will recompile and not use the previous version.

The floating point was a pain (and yes, it is documented in the Telit documentation). The SMS timeout was showing up in the log file as follows:

32945: Temperature read returned 

+CMGS:  161



OK



#TEMPMEAS: 0,22



OK

The first part (+CMGS: 161) is left over from sending an SMS with the AT+CMGS command and meant that the read buffer hadn't been flushed. Changing the timeout fixes this. The good news here is that my defensive programming style worked well in keeping the module running in under this error condition.

I ran the module for 208 minutes (3 hours, 28 minutes) sitting motionless in the garden reporting position via SMS every two minutes. Here's a graph showing reported altitude, number of satellites, internal temperature and speed. At the beginning I take the module out from the house (room was at 20C) into the garden (temperature was 7C) and at the end I bring it in again.


The gaps in the temperature record are where the problem in number 3 above occurred. The chart starts just as I put the module down in the garden; at the end you can see the number of satellites drop and the apparent speed increase as the module is brought indoors. The module seems to be running about 3C hotter than ambient temperatures.

Things to do on this part:

1. Shorten the leads on the two antennae and install in the capsule.

2. Run a three hour test in a moving car.

3. I am still very worried about the COCOM limit and am waiting for a response from Telit. In the worst case I'm going to add a watchdog to the code so that if there's been no GPS lock for an hour a complete reset of the GPS module is forced.

I guess Hacker News doesn't do meta very well

(which is ironic for something written in a Lisp variant)

Recently, I've grown tired of stories about the TSA on Hacker News.

So I posted and item saying that I was taking a break (the title was "Au revoir Hacker News") with text saying that I was fed up with the TSA stories (and in particular the Ron Paul story in the top slot) and that I was going to take a temporary break. It ended saying I'd see everyone in the New Year once things had blown over.

It was at this link but it was nuked by someone. Not made [dead], simply expunged by a moderator.

Feels a bit uncalled for to me, I'd have been happy for the community to shoot me down.

(Actually it is [dead] so it was the community that killed it off)

I guess I'll be back in the New Year.

The things make got right (and how to make it better)

make is much maligned because people mistake its terse syntax and pickiness about whitespace for signs of being an anachronism. But make's terseness is what makes make fit for purpose, and people who design 'improvements' rarely seem to understand the fundamental zen nature of make.

Here are some things make does well:

1. make's key use is in the expression of dependencies. make has a compact, syntactic cruft-free way of expressing a dependency between a file and other files.

2. Since make is so dependent on handling lists of dependencies it has built-in list processing functionality.

3. Second to dependency management is the need to execute shell commands. make's syntax for including dependencies in shell commands is small which prevents the eye from being distracted from the commands themselves.

4. make is a macro-language not a programming language. The state of a build is determined by the dependency structure and the 'up to dateness' of files. There's no (or little) need for any other internal state.

To see the ways in which make is superior to other similar, more modern, systems this post will compare GNU Make and Rake. I've chosen Rake because I believe its illustrative of what happens when people create new make-like systems instead of just fixing the things that are broken about make.

Here's a simple Makefile showing the syntax used for updating a file (called target) from a list of dependent files by running a command called update.

target: prereq1 prereq2 prereq3 prereq4
    update [email protected] $^

(If you are unfamiliar with make then it's helpful to know that [email protected] is the name of the file to the left of the :, and $^ is the list of files to the right).

Here's the same thing expressed in Rake. The first thing that's obvious is that there's a lot of syntactic noise around the command and the expression of dependencies. What was clear in make now requires more digging to uncover and things like #{t.prerequisites.join(' ')} are long and unnecessarily ugly.

file target => [ 'prereq1', 'prereq2', 'prereq3', 'prereq4' ] do |t|
    sh "update #{t.name} #{t.prerequisites.join(' ')}" 
end

The biggest 'problem' that the Rake syntax fixes in make is that the target and prerequisite names can have spaces in them without difficulty. Because a make list is space-separated and there's no escaping mechanism for spaces it's a royal pain to work with paths with spaces in them.

make's terse syntax [email protected] is replaced by #{t.name} and $^ is #{t.prerequisites.join(' ')}. The great advantage of the terse syntax is that the actual command being executed can be clearly seen. When the command lines are long (with many options) this makes a real difference in debug-ability.

This terseness is better can be seen in an example taken from the Rake documentation:

  
task :default => ["hello"]
  
SRC = FileList['*.c']
OBJ = SRC.ext('o')
 
rule '.o' => '.c' do |t|
  sh "cc -c -o #{t.name} #{t.source}"
end
 
file "hello" => OBJ do
  sh "cc -o hello #{OBJ}"
end
   
# File dependencies go here ...
file 'main.o' => ['main.c', 'greet.h']
file 'greet.o' => ['greet.c']

which rewritten in make syntax is:

  
SRC := $(wildcard *.c)
OBJ := $(SRC:.c=.o)

all: hello

.o.c:
    cc -c -o [email protected] $<

hello: $(OBJ)
    cc -o hello $(OBJ)

main.o: main.c greet.h
greet.o: greet.c

If you want to fix make then it's worth considering the following make problems that don't require an entirely new language:

1. Fix the 'spaces in filenames' problem. Not hard, just needs consistent escaping or quoting.

2. make has a concept of a PHONY target which is a target that isn't a file (used for things like clean and all). These are in the same namespace as file targets. This should be fixed.

3. make can't detect changes in the commands used to build targets. It would be better if make could do this. You can hack that into make but it's ugly.

4. make relies on timestamps for 'up to date' information. It would be better if make used hashes (in some situations, such as when files are extracted from a source code management system, timestamps can be unreliable). This can also be hacked into make if needed.

5. Ensure that non-recursive make is handled in an efficient manner.

Overall I'd urge make reimplementers to do as Paul Graham has done with LISP: his arc language is very LISP-like rather than something brand new.

And one final note: building and maintaining software build systems is inherently hard. Visualizing and getting right the graph of dependencies and handling cross-platform problems isn't easy. If you do come up with something good, please write good documentation for it.

GAGA-1 Recovery Computer

Finally, got some time to work on the GAGA-1 Recovery Computer that uses a combination of a GPS and a GSM module to send position updates via SMS to a cell phone. The complete code is now in the repository in the gaga-1/recovery/ folder.

The recovery computer itself is a Telit GM862-GPS module mounted on a board that supplies power from four AA batteries. It has two external antennas: one for GPS and one for GSM access. Here's a shot of the computer before installation in the capsule (clearly the cables are going to have to be shortened and the power supply cleaned up before the real flight). The GPS antenna is square and the GSM is the long thin bar.


The GM862-GPS has an integrated Python interpreter so the control software is a set of Python modules that handle getting GPS information (and sundry information like temperature and voltage) and sending SMS messages at appropriate times. Here's the key piece of code for the recovery computer:

# The recovery computer runs through a sequence of simple states that
# determine its behaviour.  It starts in the Launch state, transitions
# to the Ascent mode once above a preset altitude, then moves to
# Flight mode once too high for safe SMS usage.  Once below the safe
# altitude it transitions to Recovery mode.

state = ''
set_state( 'LAUNCH' )
sms.init()
gps2.init()

# The rules for the states are as follows:
#
#   Launch: get GPS position every 1 minute and SMS, check for
#   transition to Ascent mode
#
#   Ascent: get GPS position every 2 minute and SMS, check for 
#   transition to flight mode
#
#   Flight: get GPS position every 5 minutes and check for 
#   transition to Recovery mode
#
#   Recovery: get GPS position every 1 minute and SMS

while 1:
    position = gps2.get()

    if state == 'LAUNCH':
        report_position(position)
        if position['valid'] and
            ( position['altitude'] > ascent_altitude ):
            set_state( 'ASCENT' )
    elif state == 'ASCENT':
        report_position(position)
        if position['valid'] and
            ( position['altitude'] > flight_altitude ):
            set_state( 'FLIGHT' )
    elif state == 'FLIGHT':
        if position['valid'] and
            ( position['altitude'] < recovery_altitude ):
            set_state( 'RECOVERY' )
    elif state == 'RECOVERY':
        report_position(position)

    if state == 'LAUNCH' or state == 'RECOVERY':
        delay = 1
    elif state == 'ASCENT':
        delay = 2
    else:
        delay = 5

    MOD.sleep(delay * 600)

That code can be found in gaga-1.py which is the main module executed automatically by the GM862-GPS. The other important modules are logger.py (logs to the serial port for debugging and a file in the NVRAM on the GM862-GPS), at.py (simple wrapper for AT command access on the module), sms.py (module for sending SMS messages) and gps2.py (module to get GPS location).

There's a small Makefile the controls building and uploading of the code to the module (upload is achieved using the upload.pl helper program). The main commands are make all to build the code into compiled Python files, make upload to upload to the GM862-GPS and make test to run a flight simulation.

To test the code I've written modules that pretend to be the Telit Python modules (MDM, MOD, GPS, SER, etc.) and respond realistically to API calls and AT commands from my code. Within these modules I've programmed a simulated flight (an ascent, albeit a fast one, followed by descent) and random appearance of errors coming from the module (such as no GPS fix, no GSM access and other errors).

Here's a log of a simulated flight. You can see times when failures occurred (loss of GPS, can't send SMS: those lines are in red). I've highlighted the altitude in blue for easy reading.

$ make test
342295541: sms.send(+447...,"Transition to state LAUNCH")
342295541: gps2.get() -> 180541.000,5238.7818N,00211.1238W,
 1.2,13.00,3,138.00,0.00,0.00,051110,02
342295541: sms.send(+447...,"52.6464N 2.1854W 13.00m 138.00deg 
 0.00kph 2sats (1711mV, 28C)")
342295601: gps2.get() -> 180641.000,5238.2410N,00211.3171W,
 1.2,2053.03,3,114.00,45.00,24.30,051110,05
342295601: sms.send(+447...,"52.6373N 2.1886W 2053.03m 114.00deg 
 45.00kph 5sats (1919mV, -51C)")
342295601: sms.send(+447...,"Transition to state ASCENT")
342295721: gps2.get() -> 180841.000,5238.1755N,00211.5866W,
 1.2,7093.11,3,241.00,38.00,20.52,051110,03
342295721: sms.send(+447...,"52.6363N 2.1931W 7093.11m 241.00deg 
 38.00kph 3sats (535mV, -11C)")
342295721: Failed to get SMS prompt
342295721: sms.send(+447...,"Transition to state FLIGHT")
342295721: Failed to get SMS prompt
342296021: gps2.get() -> 181341.000,,,,,0,,,,051110,00
342296321: gps2.get() -> 181841.000,5238.5639N,00211.2198W,
 1.2,37093.23,3,46.00,33.00,17.82,051110,06
342296621: gps2.get() -> 182341.000,5238.7426N,00211.8475W,
 1.2,48193.23,3,94.00,43.00,23.22,051110,01
342296921: gps2.get() -> 182841.000,5238.8810N,00211.9387W,
 1.2,34693.21,3,355.00,5.00,2.70,051110,06
342297221: gps2.get() -> 183341.000,5238.1542N,00211.6911W,
 1.2,20293.21,3,26.00,28.00,15.12,051110,04
342297522: gps2.get() -> 183842.000,5238.2393N,00211.8530W,
 1.2,8262.62,3,54.00,24.00,12.96,051110,02
342297822: gps2.get() -> 184342.000,5238.1079N,00211.0661W,
 1.2,12.00,3,3.00,0.00,0.00,051110,08
342297822: sms.send(+447...,"Transition to state RECOVERY")
342297882: gps2.get() -> 184442.000,5238.6368N,00211.9774W,
 1.2,12.00,3,77.00,0.00,0.00,051110,06
342297882: sms.send(+447...,"52.6439N 2.1996W 12.00m 77.00deg 
 0.00kph 6sats (1444mV, -2C)")
342297942: gps2.get() -> 184542.000,5238.6790N,00211.3624W,
 1.2,18.00,3,63.00,0.00,0.00,051110,03
342297942: sms.send(+447...,"52.6446N 2.1894W 18.00m 63.00deg 
 0.00kph 3sats (1246mV, -22C)")
342298002: gps2.get() -> 184642.000,5238.4941N,00211.8801W,
 1.2,17.00,3,256.00,0.00,0.00,051110,01
342298002: sms.send(+447...,"52.6416N 2.1980W 17.00m 256.00deg 
 0.00kph 1sats (3095mV, -51C)")
342298062: gps2.get() -> 184742.000,,,,,2,,,,051110,00
342298062: sms.send(+447...,"No GPS lock (1045mV, 32C)")
342298122: gps2.get() -> 184842.000,5238.9542N,00211.9596W,
 1.2,11.00,3,21.00,0.00,0.00,051110,05
342298122: sms.send(+447...,"52.6492N 2.1993W 11.00m 21.00deg 
 0.00kph 5sats (2742mV, 48C)")
342298182: gps2.get() -> 184942.000,5238.9607N,00211.1014W,
 1.2,14.00,3,167.00,0.00,0.00,051110,08
342298182: sms.send(+447...,"52.6493N 2.1850W 14.00m 167.00deg 
 0.00kph 8sats (819mV, 6C)")

There are a few remaining items:

1. Run a complete, real test of the module using fresh batteries in a moving car and ensure that it correctly logs information and sends SMS. Also, see how long it lasts.

2. Get an answer from Telit on the COCOM limits so that I understand how the GPS fails above the 18km altitude line.

3. Cut down the cables and install in the capsule.

Then it'll be on to the flight computer.

The most common objection to my 'releasing scientific code' post

Is...

And why dismiss so casually the argument that running the code used to generate a paper's result provides no actual independent verification of that result? How does running the same buggy code and getting the same buggy result help anyone?

Or as expressed at RealClimate:

First, the practical scientific issues. Consider, for example, the production of key observational climate data sets. While replicability is a vital component of the enterprise, this is not the same thing as simply repetition. It is independent replication that counts far more towards acceptance of a result than merely demonstrating that given the same assumptions, the same input, and the same code, somebody can get the same result. It is far better to have two independent ice core isotope records from Summit in Greenland than it is to see the code used in the mass spectrometer in one of them. Similarly, it is better to have two (or three or four) independent analyses of the surface temperature station data showing essentially the same global trends than it is to see the code for one of them. Better that an ocean sediment core corroborates a cave record than looking at the code that produced the age model. Our point is not that the code is not useful, but that this level of replication is not particularly relevant to the observational sciences.

This argument strikes me as bogus. It comes down to something like "we should protect other scientists from themselves by not giving them code that they might run; by not releasing code we are ensuring that the scientific method is followed".

Imagine the situation where a scientist runs someone else's code on the data that person released and gets the same result. Clearly, they have done no science. All they have done is the simplest verification that the original scientist didn't screw up in their methods. That person has not used the scientific method, they have not independently verified the results and their work is close to useless.

Is this enough to argue that the code should have been closed in the first place?

I can't see that it is. No one's going to be able to publish a paper saying "I ran X's code and it works", it would never get through peer review and isn't scientific.

To return to the first quote above, running someone else's buggy code proves nothing. But in hiding the buggy code you've lost the valuable situation where someone can verify that the code was good in the first place. Just look at the effort I went to do discover the code error in CRUTEM (which, ironically, is a 'key observational climate data sets' to use RealClimate's words).

The argument from RealClimate can also be stated as 'running someone else's code isn't helpful so there's no point releasing it'. (see comments below to understand why this is struck out) The premise is reasonable, the conclusion not. I say that because there are other reasons to release code:

1. It can be used by others for other work. For example, good code can form part of a library of code that is used to improve or speedup science.

2. The algorithm in a paper can be quickly checked against the implementation to ensure that the results being generated are correct. For example, the CRUTEM error I found could have been quickly eliminated by access to the paper and source code at the same time.

3. Releasing code has a psychological effect which will improve its quality. This will lead to fewer errors on the part of scientists who rely on computer methods.

All Plan 28 Media Coverage

This is the 2010 archive, for 2011 see 2011 Plan 28 Media Coverage

I'll be keeping this page updated as new stories are posted. Articles with * next to them are the most in-depth.

October 3, 2010

The Independent: One of the great inventions that never was – until now? *
The Independent: Start the Engine

October 4, 2010

The Daily Mail: Campaign to build Charles Babbage's steam-powered 'Analytical Engine' - 173 years after he designed it
O'Reilly Radar: The 100-year leap *

October 5, 2010

The Register: Legendary steampunk computer 'should be built' - programmer
THINQ.co.uk: Geek plans to build Babbage compute

October 6, 2010

iProgrammer: Plans to build Babbage's Analytical Engine

October 7, 2010

Fudzilla: Babbage's analytical engine should be constructed

October 8, 2010

e-Katalog: Британский энтузиаст намерен построить паровой компьютер

October 9, 2010

Der Standard: Dampf-Computer von 1837 soll Realität werden

October 11, 2010

elektor: Dampf-Computer soll Realität werden
Der Standard: 1837 entworfener Dampf-Computer wird gebaut

October 12, 2010

sg.hu: Megépülhet az 1837-ben megálmodott gőzzel működő számítógép

October 14, 2010

BBC News: Campaign builds to construct Babbage Analytical Engine *
redOrbit: Campaign To Raise Funds For Analytical Engine Underway
The Daily Telegraph: Campaign launched to build Charles Babbage's Analytical Engine
UPI: Historic computer replica proposed
ubergizmo: Charles Babbage's Analytical Engine could very well be realized

Komputer Swiat: Chcą zbudować komputer na parę

October 15, 2010

Tech.Blorge: Babbage’s Analytical Engine may get built
TG Daily: Campaign launched to build Babbage's steam-powered computer
Geekosystem: World of Steampunk Rejoice: Somebody Might Build Charles Babbage’s Analytical Engine
PC World: Rebuilding a 19th-Century Computer: One Programmer's Quest
Inhabitat: UK Campaign Aims to Build Legendary Steam-Powered Computer
Gizmodo: Charles Babbage’s Failed Computer From 1837 Will Be Built
Discover Magazine: Get Your Steampunk On: This Guy’s Building a Computer From 1837
Geeks are Sexy: 1837 steam powered computer could finally enter production
io9: Bringing back Babbage

sudkurier: Computer unter Dampf
microsiervos: Proyecto para construir una máquina analítica de Babbage en el MundoReal™

October 17, 2010

Escapist Magazine: Campaign Pledges to Finish 19th Century Steampunk Computer

October 18, 2010

PCR: Plan to build Babbage steam computer
Gizmag: Ambitious project to bring world's first general purpose computer to life
ITProPortal: Babbage Steam Computer To Be Built
Hexus: Babbage behemoth battle
TechNewsWorld: Geek Seeks to Bring Babbage's Analytical Engine to Life
PC Pro: Wanted: Assembly programmers for 173-year-old computer *
Techwatch: 170-year-old steam powered computer to be built?
PhysOrg.com: Campaign to build 1837 Babbage's Analytical Engine
GreenMuze: Building Babbage's Analytical Engine

Hirado: Meg akarják építeni Charles Babbage analitikus gépét
hvg: Meg akarják építeni Charles Babbage gőzhajtású számítógépét

October 19, 2010

CBS News: Move to Bring Babbage's Famous Computer to Life

detikinet: Kampanye 'Plan 28', Ajak Orang Ciptakan Prototipe Komputer Jadul
Heise: Spendensammlung für Dampfmaschinen-Computer

October 21, 2010

De Standaard: En als we eens de stoomcomputer van Babbage bouwden?! *
NyTeknik: Nu ska ångdatorn äntligen byggas

October 22, 2010

As it happens (CBC Radio): Prithee, dispatch thine riotous video portraying the farcical antics of cats!

October 25, 2010

Radio New Zealand: Long interview about Plan 28 *

October 26, 2010

Wall Street Journal Europe: The Ultimate Steampunk Project *
TechEye: Steampunk computer to be made of brass and powered by steam

November 2, 2010

Haaretz: אחרי 170 שנה, המחשב הראשון קורם עור וגידים
Wall Street Journal Europe: Geek immortality from €10

November 4, 2010

PÚBLICO: O século XIX dos computadores a vapor está a nascer agora

November 12, 2010

Irish Times: Revisited invention planned as tribute to 'father' of computing *

A Plan 28 completion date

Today in 1871 Charles Babbage died. 11 years from now it will be 150 years from Babbage's death.

He kept working on the Analytical Engine up until his death. Let's make sure that his Analytical Engine is built by October 18, 2021.

Help get the word out about Plan 28.

Plan 28: the first 10 days

So, it's been 10 days since I posted the Analytical Engine PledgeBank and the total number of signatories stands at over 2,700 with many people pledging £50, €100 or even $1,000. The PledgeBank estimator reckons I'll hit 42,435 pledges (84.9% of target) by the target date. Given how much more than $10/£10/€10 many people are pledging I'd imagine that my goal isn't unrealistic, but please keep spreading the news.

As well as pledges of money I've received pledges of professional design and manufacturing help, document digitizing, CAD software and more. These are super-helpful.

In addition there's been a lot of media interest. The best articles are the BBC's Campaign builds to construct Babbage Analytical Engine and The Independent's One of the great inventions that never was – until now?. If podcasts are your sort of thing then you can listen to the excellent session about Plan 28 on TWiT #269.

There's also been some nice tweet-support from BBC broadcaster Maggie Philbin and author William Gibson.

And behind the scenes I've received hundreds of emails of support or with questions. To help field some of the questions I did a reddit AMA and posted a FAQ.

Also behind the scenes there's a proper design well underway for the Plan 28 web site which will transform it from my awful design skills to something respectable.

And, finally, I'll be meeting with the folks at the Science Museum early in November to talk more about Plan 28.

Above all: thanks to everyone who's written about Plan 28 in the media, on Twitter or on a blog, thanks for all the pledges of money or services, and all your kinds emails.

PS Forgot to add that descendants of Charles Babbage have been in contact to express their support for the project.

What Nature didn't say

Nature has a nice article about scientific software which starts by mentioning the hacking of the Climatic Research Unit and the release of software from the hacked files, and then goes on to talk generally about the state of scientific software. My summary would be that it's generally a mess because software engineering has crept up on scientists and now they need to get educated about things that have been common in the commercial software world for years.

Which is pretty much what I said on Newsnight in December 2009.

The article begins:

When hackers leaked thousands of e-mails from the Climatic Research Unit (CRU) at the University of East Anglia in Norwich, UK, last year, global-warming sceptics pored over the documents for signs that researchers had manipulated data. No such evidence emerged, but the e-mails did reveal another problem — one described by a CRU employee named "Harry", who often wrote of his wrestling matches with wonky computer software.

"Yup, my awful programming strikes again," Harry lamented in one of his notes, as he attempted to correct a code analysing weather-station data from Mexico.

Although Harry's frustrations did not ultimately compromise CRU's work, his difficulties will strike a chord with scientists in a wide range of disciplines who do a large amount of coding.

True enough that the messy code from CRU wasn't shown to compromise any of their scientific results. None of the enquiries into "ClimateGate" examined the CRU code. I did show, however, that the code I saw was buggy. (See, Whoops, there's a third bug in that code and Bugs in the software flash the message 'Something's out there'.) In fact, the best that can be said is that CRU's code was buggy and we don't know if those bugs have a material impact on the science.

And another piece of CRU-related code, the code used by the Met Office to produce the CRUTEM3 temperature set was similarly buggy. I first showed that there were errors in the way the station files were generated (see The full response from the Met Office. By the way, I'm still waiting for them to make good on their promise to credit me) and then (with Ilya Goz) showed that the program used to generate the station errors in CRUTEM3 wasn't working (see Something odd in CRUTEM3 station errors and Met Office confirms that CRUTEM3 station errors are incorrectly calculated.)

What's interesting about these bugs is that practices like unit testing or automation (through even make, which has been around since the 1970s) would have helped avoid the problems the Met Office saw. And likely the bugs in the CRU code. It really would be a good idea for commercial best practices to be introduced to scientists.

Nature didn't mention any of that. Pity. Those are real bugs in the real software related to CRU.

I don't know if any of this leads to problems with actual climate science, that would take a real examination of the source code used to produce published papers. But it does concern me that it was so easy to find so many errors. There really could be a nasty surprise lurking in the code.

Plan 28 (Analytical Engine) FAQ

Answers to your questions about the Babbage PledgeBank:

1. Can I pledge more than $10/£10/€10?

Yes, of course! Many people have done so. The system doesn't have a box where you can enter an amount, but you can contact me and let me know, or, even better, publish a public comment with your pledge amount.

2. Can I pay you the money?

Not yet. Currently, I'm working on building up enough pledges to make sure this is viable. Once I have enough you'll receive an email detailing the new Plan 28 organization and how to actual send in the money.

3. Do you really think you can get 50,000 people to pledge?

Yes. Last year I got over 30,000 in the UK alone to sign the Alan Turing petition. This campaign is global and so I am confident that with the right exposure in the media and the help of enthusiastic people over 50,000 will pledge.

4. The Plan 28 web site is ugly.

Yes, I know. This is being actively worked on by a professional designer.

5. I have another idea about this.

Great. Contact me. I'm building up a big list of ideas, volunteers, etc. Would love to hear from you.

6. How can I help?

Sign the pledge. Tell people about this: tweet it, Facebook, email, etc.

7. Why hasn't someone done this before?

Because it takes time. It wasn't until the 1970s that Babbage's contributions were fully understood (see Bruce Collier's thesis), and until the 1980s that his plans had been deciphered by the likes of Allan Bromley. Only then could the Difference Engine No. 2 be constructed. It was finished in 1991. In many ways, The Difference Engine No. 2 was an 'easy' project because Babbage had left complete plans for it. The Analytical Engine is a different matter.

Babbage left multiple plans for the Analytical Engine and was constantly refining its design up until his death. To build the Analytical Engine first requires a research project to figure out which plan to build from. That's why the project has steps involving scanning all of Babbage's papers and a research project on them.

8. What's the best introduction to this project for a newbie?

Probably, the article The 100-year leap.

9. Why are you doing this?

In 1992 I sat on the floor of the Science Museum in London and watched the Difference Engine No. 2 demonstrated. And then I sat outside in the sun near the Natural History Museum and worked through its operation on paper. Since then I have been fascinated by Babbage and the fact that he had invented a real computer in 1837.

In 2000 the Science Museum completed the printer attachment for the Difference Engine No. 2 showing that Babbage's dream would have worked. I simply want to complete the task of honouring Babbage by creating his computer and showing it to the public.

1,000 people sign the Plan 28 pledge

A few moments ago I checked the Plan 28 pledge page to find that it had passed the 1,000 people mark. That took just 6 days. Thank you to everyone who's pledged so far (Peter Zuidhoek was the 1,000 person).

But even more heartwarming than the 1,000 are the comments people have been leaving on the page. You can read them all on the pledge page, but here are a few highlights. Many people have said that they'll pledge more than $10/£10/€10, with some going as high as $1,000. Brilliant.

Worth it just for the first step. Do it for the next generation. -- Matt Doar

Put me down for a hundred quid. -- Aidan Karley

I have been hoping for years that someone would do this. I contributed to the construction of the Difference Engine and am proud to be able to contribute here. -- Raymond Woodward K3VSA

I'll pledge $100 - Roston McGregor

Great idea and willing to pledge more 50$ when the time comes -- Marcus Jaeger

Great endeavor, sign me up for $50 USD. -- Robert Huwar

I'd love to see this in action. I've seen the Difference Engine 2 at the Computer History Museum, and it's absolutely beautiful. I'm happy to pledge $1000. -- Eric Uhrhane

I hope this happens! $100 is yours. -- Christopher MacMurray

I will happily pledge way more than the $10. Best of luck. -- Noah Gildersleeve,

I'd love to see this happen. I pledge $100. -- Laurence Gonsalves

Your pledges and comments are marvelous. Keep 'em coming!

PS If you want to know what this is all about visit Plan 28 or read The 100-year leap.

Related Hacker News discussion

A big boost for Plan 28

Last night I appeared on TWiT with Leo Laporte and John C. Dvorak and guests. We talked about a lot of things late into my night, but mostly about Plan 28.

Leo encouraged listeners to pledge money and made the stunning offer of personally pledging $1,000 to the effort! And so, on top of Leo's pledge the number of people pledging has tripled!


Thanks Leo.

Geek Weekend: Kew Bridge Steam Museum

So, I managed to persuade the people around me that it would fun to go and visit Kew Bridge Steam Museum, and it was brilliant! The museum is one of the places that was on the list for The Geek Atlas but got cut for space. Nevertheless, it's really worth visiting.


On weekends and other special days the folks at the museum set the machines running (including the massive 90 inch engine). I got there early enough to see most of the engines demonstrated and the staff were great. Got a good explanation of the operation of the triple expansion engine used for pumping water for London. Here it is in action:



Another nice engine is the Dancer's End engine that was used for pumping water on the estate of Lord Rothschild and was moved to the museum in the 1970s. And makes a lovely noise as the condenser water can be heard bubbling away. Here it is.



And finally, it's not all steam. There's a collection of Diesel engines. Here's a small one in action. What a brilliant sound.



There's also a steam train and Diesel train that run on a small track and are included in the price. And the ticket is valid for a year, so you can go back and see these wonderful engines in action as often as you can persuade your family to go with you.

1,000 (bad) ideas

Tidying up my messy corner in the basement I unearthed one of my older "ideas books" in which I write down the random ideas that flit through my head quite frequently. Flicking through it I noticed that this book contained idea number 1,000 written on October 3, 2006. Most of these ideas never get implemented but they are fun to go back and read.


Idea 1,000 was In-ear headphones that automatically pause music when removed. The details mention using either a proximity detector to discover when the headphones are in the ear, or a strain gauge to detect the pressure of the ear canal.

The previous idea was Brake lights that show the severity of braking. The idea there was to replace the current 'high-level' brake lights in cars (that are typically a horizontal bar), with a triangle of lights. Under normal braking just the top, horizontal red bar would illuminate. Under heavy braking the triangle would illuminate warning drivers of an emergency situation (particularly useful on motorways at high-speed). Since the red triangle is a common warning symbol this would be instantly recognizable.

I've been filling these books since about 1992. By 2006 I'd had 1,000 (bad) ideas, which works out to about one every five days. Some of them I've implemented (such as l8tr), others I've seen implemented by other people: there's an amusing entry from 1995 about building a networked disk drive service that I wanted to call "I-Drive". The plan was to use FTP as the underlying protocol and make a drive (I:) appear in Windows 95. I: would actually be on the Internet and could be accessed by multiple machines.

The idea I'm most pleased with is my book, The Geek Atlas, because it was (fairly) original. No one else had written "Lonely Planet for Nerds". Overall, perhaps 10 of my 1,000 ideas look like they might be 'good ideas'.

Does anyone else generate ideas like this? And if so, what percentage turn out to be good?

Related Hacker News discussion

The Analytical Engine vs. The ZX81

Many people got started with computing in my age group with the Sinclair ZX81 (which in the US was sold as the Timex Sinclair 1000).


Reading through Allan Bromley's excellent papers on Babbage's Analytical Engine (as described in 1847) I thought it might be fun to compare the two machines.

The ZX81 has 1KB of memory in which programs and data had to be stored. The Analytical Engine as first imagined would have had 50 variables capable of each storing a 30 digit decimal number. That's equivalent to each variable having 100 bits and hence the memory for the Analytical Engine would have been 5000 bits (675 bytes). Later Babbage proposed much larger memory sizes with up to 50 decimal digits per variables and 1000 variables: that would have been a memory of 166 bits per variable or over 20 KB of memory.

Critically, the Analytical Engine's programs would be stored on punched cards and executed directly from them so that entire memory space was for data (not the program). In contrast the ZX81 had to load the program from magnetic tape into its 1KB of RAM.

But the ZX81 was much, much faster than Babbage's machine. It had a clock speed of 3.25 MHz. The Analytical Engine was based around a standard cycle time of 1/7s which is the same as 7Hz. To get an idea of the speed of the machines, here's a comparison of a 100 bit add on the two machines.

On the Analytical Engine it would take about three seconds to fetch the two numbers from memory (the store) and perform the addition. Although this is long Babbage added the ability to do a pipeline of multiple additions with overlapped reads from memory and addition happened in parallel. The ZX81 does not have that ability.

On the ZX81 in Z80 assembler a 100 bit addition (which I've done as 13 8-bit additions here) could be implemented as follows. I've assumed that IX and IY are pointing to the operands in memory and that the result will be stored in the memory addressed by IY.

  LD  A,(IX+0)
  ADD A,(IY+0)
  LD  (IY+0),A
  LD  A,(IX+1)
  ADC A,(IY+1)
  LD  (IY+1),A
  LD  A,(IX+2)
  ADC A,(IY+2)
  LD  (IY+2),A

  [...]

  LD  A,(IX+12)
  ADC A,(IY+12)
  LD  (IY+12),A

Each instruction there takes 19 clock cycles and there are a total of 39 instructions giving 741 clock cycles. With the Z80 running at 3.25MHz that set of instructions takes 0.228 ms. So for that addition the ZX81 would have been 13,000x faster.

Obviously, this is slightly bogus because there are other more complex and slow instructions in the Analytical Engine (such as division) but it gives an idea of the scale.

So, the ZX81 had much less RAM than the Analytical Engine (and who doesn't remember all the trouble of fitting everything into RAM on those machines) and was forced to use it to hold programs, but the Analytical Engine was slow in comparison.

PS If you'd like to see the Analytical Engine built, read this.