Skip to main content

Posts

Showing posts from 2009

Toy decoding: vtech Push and Ride Alphabet Train

So, it's Christmas and you end up visiting people with kids... and they've got a fancy new vtech Push and Ride Alphabet Train. Now, you're the world's worst child minder because you see it and think: how does that work?


Specifically, when you insert one of the 26 alphabet blocks into the side of the train how does it know to say the correct letter? And how does it know which side (letter or word) is facing outwards (so it can say a letter or a corresponding word: "A is for Apple" etc.).

Now it quick examination shows that there are 6 small switches in each block receptacle and that each block has corresponding bits of plastic and holes to make different binary patterns. The top bit (bit 5) seems to be used to indicate which side of the block is showing.

That leaves 5 bits for the alphabet. Of course that means there are 32 possible combinations (actually 31 since 'block not present' indicated by all switches up is important), and 26 letters in the al…

Parsing a JSON document and applying it to an HTML template in Google Go

Here's some simple code to parse a JSON document and the transform it into an HTML document using the Google Go packages json and template.

If you've done anything in a scripting language then you'll probably be surprised by the generation of fixed struct types that have to match the parsed JSON document (or at least match some subset of it). Also because of the way reflection works in Google Go the struct member names need to be in uppercase (and for that reason I've used uppercase everywhere).

import (
"fmt";
"os";
"json";
"template"
)

type Row struct {
Column1 string;
Column2 string;
}

type Document struct {
Title string;
Rows []Row;
}

const a_document = `
{
"Title" : "This is the title",
"Rows" : [ { "Column1" : "A1", "Column2" : "B1" },
{ "Column1" : "A2", "Column2" : "B2" }
]
}`

const a_temp…

Installing Google Go on Mac OS X

I decided to have a go with Google Go since I'm an old fogey C/C++ programmer. Any new innovation in the C/C++ family gets me excited and Google Go has quite a few nice features (garbage collection is really nice to have and channels make me think of all the work I did in CSP).

I decided to go with the 6g compiler since gccgo doesn't have garbage collection implemented yet and hence there's no way to free memory. The only way to get 6g is to mirror its Mercurial repository. So...

Step 1: Install Mercurial

For that I used prebuilt packages from here and got Mercurial 1.4 for Mac OS X 1.5 (no, I haven't upgraded to Snow Leopard yet).

Step 2. Set GOROOT

I just did a quick cd ; mkdir go ; export GOROOT=$HOME/go to get me started.

Step 3. Clone the 6g repository

That was a quick hg clone -r https://go.googlecode.com/hg/ $GOROOT followed by the hard part: compiling it. You need to have gcc, make, bison and ed installed (whcih I do since I do development work on my Mac).

Step 5. S…

Geek Weekend (Paris Edition), Day 4: Institut Pasteur

Leaving my SO in bed at the hotel with a nasty bacterial infection and some antibiotics, I went with timely irony to visit the home and laboratory of Louis Pasteur at the Institut Pasteur. (It's pretty easy to find since it has a conveniently named stop on the Paris metro: Pasteur).


At the Institut Pasteur there's a wonderful museum that covers the life and work of Louis Pasteur (and his wife). It's housed in the building (above) where the Pasteurs lived. There's a single room of Pasteur's science and the rest of the house is Pasteur's home; so a visit is partly scienfitic and partly like visiting any old home. I was mostly interested in the laboratory (although seeing how he lived---pretty darn well!---was also worth it).

Pasteur wrote standing up at a raised table (much like old bank clerks used to use) and his lab is full of specimens that he worked on. There's a nice display about chirality which Pasteur had initially worked on while study tartaric a…

Geek Weekend (Paris Edition), Day 3: The Arago Medallions

The old Paris Meridian (which was in use up until 1914) passes not far from The Pantheon which I visited to see Foucault's Pendulum. It's actual longitude today is 2°20′14.025″.

To mark the old meridian the French decided to install some art work and they commissioned an artist called Jan Dibbets to build something appropriate. What he did was embed brass disks in the streets of Paris marking the meridian and turning the whole city into a sort of treasure hunt.

These Arago medallions (which celebrate the meridian and the life of François Arago) cut through the very heart of Paris. They make a wonderful way to see Paris at going on a treasure hunt. And the meridian goes to the very heart of something important: the meter. The original definition of a meter was based on the length of the Paris meridian from the north pole to the equator. Arago surveyed the meridian and came up with a very precise definition for this fundamental unit of measure.

Here's a photo I took of on…

Parsing HTML in Python with BeautifulSoup

I got into a spat with Eric Raymond the other day about some code he's written called ForgePlucker. I took a look at the source code and posted saying it looks like a total hack job by a poor programmer.

Raymond replied by posting a blog entry in which he called me a poor fool and snotty kid.

So far so good. However, he hadn't actually fixed the problems I was talking about (and which I still think are the work of a poor programmer). This morning I checked and he's removed two offending lines that I was talking about and done some code rearrangement. The function that had caught my eye initially was one to parse data from an HTML table which he does with this code:

def walk_table(text):
"Parse out the rows of an HTML table."
rows = []
while True:
oldtext = text
# First, strip out all attributes for easier parsing
text = re.sub('<TR[^>]+>', '<TR>', text, re.I)
text = re.sub('<TD[^>]+>', '<TD>…

Geek Weekend (Paris Edition), Day 2: Foucault's Pendulum

Not very far from The Curie Museum is the former church and now burial place for the great and good men (and one woman) of France: The Pantheon. Inside the Pantheon is the original Foucault's Pendulum.

The pendulum was first mounted in the Pantheon in 1851 to demonstrate that the Earth is rotating. The pendulum swings back and forth in the same plane, but the Earth moves. Relative to the floor (and to the convenient hour scale provided) the pendulum appears to rotate.


The pendulum is on a 67m long cable hanging from the roof of the Pantheon. The bob at the end of the cable weight 27kg. In the Pantheon the pendulum appears to rotate at 11 degrees per hour (which means it takes more than a day to return to its original position). If it were mounted at the North Pole it would 'rotate' once every 24 hours, the pendulum's period of rotation depends on the latitude diminishing to 0 degrees per hour at the equator (i.e. it doesn't 'rotate' at all).


If you take …

Geek Weekend (Paris Edition), Day 1: The Curie Museum

So, it was off to Paris for the weekend via Eurotunnel and I managed to fit in four places from The Geek Atlas in four days. I was staying in a hotel in the Latin Quarter which is a stone's throw from... The Curie Museum.

Here's Marie Curie's laboratory:


The museum covers the lives and works of two Nobel Prize-winning couples: Pierre and Marie Curie (they discovered Radium and Polonium) and their daughter Irene and her husband Frederic Joliot (they discovered artificial radioactivity: you could make a substance radioactive by bombarding it with alpha particles).

Their Nobel Prizes are on display as is the equipment that they used (including the apparatus for measuring radiation by measuring ionization of air---which itself had been discovered by Becquerel).

Here are the Nobel Prizes:


Although I love the science section of the museum (including the laboratory where they worked with a piece of paper from one of their notebooks with its radioactive thumb print---they weren't …

Der Geek Atlas

The Geek Atlas ist jetzt auch in Deutsch.


Kaufen Sie es hier.

Die lebendige Geschichte der Wissenschaften ist überall um uns herum, man muss nur wissen, wo man hinschauen muss. Mit diesem einzigartigen Reiseführer kann man 128 Orte auf der Welt kennen lernen, die für bedeutsame Ereignisse in Wissenschaft und Technik stehen. Erlebe das Foucaultsches Pendel, das in Paris schwingt; erfahre Interessantes über das größste Wissenschaftsmuseum der Welt, das "Deutsche Museum" in München; besuche einen Ableger des Newtons Apfelbaums am Trinity College in Cambridge und vieles, vieles mehr...

Jeder Ort in Der Geek-Atlas stellt eine außerordentliche Entdeckung oder Erfindung in den Mittelpunkt und befasst sich darüber hinaus auch mit den Menschen und Geschichten, die hinter diesen Erfindungen stehen. Alle Orte werden mit interessanten Fotos vorgestellt und die Themen mit zahlreichen Zeichnungen illustriert. Das Buch ist nach Ländern aufgeteilt, für alle interessanten Orte werden auch -- ne…

Some real data about JavaScript tagging on web pages

Since March of this year I've been running a private web spider looking at the number of web tags on web pages belonging to the Fortune 1000 and the top 1,000 web sites by traffic. Using the spider I've been able to see which products are deployed where, and how those products are growing or shrinking.

The web tags being tracked are those used for ad serving, web analytics, A/B testing, audience measurement and similar.

The spider captures everything about the page, including screen shots, and I'm able to drill in to see the state of a page and all its includes at the time of spidering. Here's shot of Apple with all the detail that the spider keeps.



The first interesting thing is to look at the top 1,000 web sites by traffic and see how many different tags are deployed per page. The average is 2.21, but if you exclude those that have no tags at all then the average is 3.10. Here's the distribution of number of tags against percentage of sites.


And of course, it'…

Monopoly .com Edition

I love Monopoly and have a small collection of Monopoly games from around the world. The oddest one is Monopoly .com Edition which was released in 2000.



In it the streets are replaced with '30 of today's hottest web sites'. These are: Sportsline.com and FoxSports.com, Yahoo! Geocities, Oxygen and iVillage, shockwave.com, games.com and E! Online, Priceline, Expedia, and eBay, weather.com, about.com and cnet.com, ETrade, monster.com and MarketWatch.com, Ask Jeeves, AltaVista and Lycos, and [email protected] and Yahoo! (Yes, there are only 22!)



The railway stations are replaced with telecom companies: MCI WorldCom, Nokia, Sprint and AT&T. The playing pieces are made of pewter and depict Mr Monopoly sitting at computer, the Internet Explorer Hand, a surfboard, a computer screen, a web browser, a PC, an email, a mouse and a microchip. The Mr Monopoly piece is a special token that can take any web site 'offline' making it unavailable for purchase.



The buildings are houses…

Solving the XSS problem by signing <SCRIPT> tags

Last week I talked about JavaScript security at Virus Bulletin 2009. One of the security problems with JavaScript (probably the most insidious) is Cross-site Scripting (which is usually shortened to XSS).

The basic defense against XSS is to filter user input, but this has been repeatedly shown to be a nightmare. Just yesterday Reddit got hit by an XSS worm that created comments because of a bug in the implementation of markdown.

I believe the answer is for sites to sign the <SCRIPT> tags that they serve up. If they signed against a key that they control then injected JavaScript could be rejected by the browser because its signature would be missing or incorrect and the entire XSS problem would disappear.

For example, this site includes Google Analytics and here's the JavaScript:

<script type="text/javascript">
var gaJsHost = (("https:" == document.location.protocol) ?
"https://ssl." : "http://www.");
document.write(unescape("%3C…

Geek Side Trip: CERN

While over in Geneva for the Virus Bulletin 2009 conference I managed to make a side trip to see CERN. It turned out to be a great afternoon because the tour was guided by actual physicists and I took a school trip.

I am a little old for it, but when I organized my trip I was told that I would be added to a group from Steyning Grammar School. There I was with 23 final year A-level students on a whirlwind trip to Switzerland. They were extremely nice kids, and I could easily imagine that teaching such a group would be incredibly rewarding.

The visit started with a talk and a film. This told the story of CERN itself (it's almost 55 years old) and described the operation of the Large Hadron Collider.

Here's what part of the LHC looks like (this is a mock-up). The large blue thing is one of the super-conducting magnets. There are 1,232 of these in the 27km ring, each weighs about 27 tonnes.



After that we were bussed over to where the superconducting magnets used in the LHC are r…

POPFile v1.1.1

The cool team that manages the POPFile project (that I started what seems like years ago...) have just released v1.1.1 with a bunch of improvements (especially for Windows users).

From the release notes:


1. New features

You can now customize Subject Header modification placement (head or tail)
by changing the new option 'bayes_subject_mod_pos'. (ticket #74)

NNTP module now caches articles received with the message number specified.

You can now jump to message header/message body/quick magnets/scores in the
single message view by clicking links on the head of the page. (ticket #77)

You can now filter messages shown in the history using 'reclassified' option.
(ticket #67)


2. Windows version improvements

The minimal Perl has been updated to the most recent 5.8 release. Since this
release of Perl only officially supports Windows 2000 or later POPFile 1.1.1
may not work on Windows 95, Windows 98, Windows Millennium or Windows NT. The
installer will display a warning message explaining th…

JavaScript must die

I've just completed my presentation at Virus Bulletin 2009 which was entitled JavaScript Security: The Elephant running in your browser.

My thesis is that the security situation with JavaScript is so poor that the only solution is to kill it. End users have very little in the way of protection against malicious JavaScript, major web sites suffer from XSS and CSRF flaws, the language itself allows appalling security holes, and as data moves to the cloud the 14 year old JavaScript security sandbox becomes more and more irrelevant.

Here are the slides:

Javascript SecurityView more presentations from jgrahamc.

The Geek Atlas: now on your iPhone

Today, O'Reilly released my book, The Geek Atlas, as an iPhone application. It's the complete text of the book on the iPhone. Since the book is organized as small chapters it's very readable on a small screen.



The neatest feature is that latitude and longitude given for each place in the book is clickable and takes you straight to that location on Google Maps.

And it's only $5.99 or £3.49.

"Hello John. It's Gordon Brown."

Last night the British Prime Minister Gordon Brown issued a long statement about my Alan Turing petition that included a clear apology for his treatment. Unfortunately, I've been in bed nursing the flu so it was only by chance that an amazing sequence of events occurred.

Yesterday evening I realized that I had to check my email (I'd been avoiding it while ill) because of a work commitment on Friday and so I logged in to find a message that read:

John - I wonder if you could call me as a matter of urgency, regarding your petition. Very many thanks!

Kirsty

Kirsty xxxxxxx
10 Downing St, SW1A 2AA
Tel: 020x xxxx xxxx

So, I called back. The telephone number was the Downing Street switchboard and after Kirsty told me that the government was planning to apologize for Alan Turing's treatment she then said "Gordon would like to talk to you".

A few minutes later the phone rang and a soft Scottish voice said: "Hello John. It's Gordon Brown. I think you know why I am ca…

How to trick Apple Numbers into colouring individual bars on a bar chart

Suppose you have a bar chart like this:



and it's made from a data table like this:



And you are really proud of the sales of the XP1000 and want to change the colour of its bar to red. In Apple Numbers you can't do that because the bar colour is based on the data series.

But you can fool Apple Numbers by creating two data series like this:



Then choose a Stacking Bar chart after selecting the two series of data in the data table and you'll get a chart like this:



You can change the colour of any of the series by clicking on the Fill button on the toolbar. And you can extend that beyond two series to colour the individual bars as needed.

Geek Weekend, Day 2: The Brunel Museum

So after yesterday's trip to Bletchley Park I stayed in London and hopped over to a spot not far from Tower Bridge where Marc Brunel and his son Isambard built the first tunnel under a navigable river: the Thames Tunnel. The tunnel was dug out by hand using a tunnel shield (which is the basis of all tunnel building to the present day). Workers stood inside a metal cage pressed against the undug earth and removed boards, dug in a few inches and replaced the boards. Once the digging was done the entire structure was forced forwards a few centimeters and bricklayers would fill in behind.



The tunnel has a rich and varied history and is still in use today (read the Wikipedia link above to learn more). The entrance to the tunnel was through a massive circular tube (a caisson) which the Brunels built above ground and then sunk it into place. The entrance has been closed for about 140 years and is being renovated, but I was lucky enough to be taken into it by the curator of the Brunel…

Geek Weekend, Day 1: Bletchley Park

Left to my own devices to the weekend I decided to embark on a Geek Weekend with visits to two places within easy reach of London. Today I visited Bletchley Park which is simply wonderful for any geek out there.

Bletchley Park is where the cryptanalysts of the Second World War worked in great secrecy (including Alan Turing) to break the Nazi German Enigma and Lorenz ciphers. To break them they used a combination of intimate knowledge of language, mathematics and machines.

Here's a Nazi German Enigma machine:



And here's a look inside one of the rotors inside an Enigma machine to see the wiring:



Two of the code breaking machines have been reconstructed. One is the Turing Bombe, an electromechanical machine made to break the Enigma cipher. Here's a look at the wiring in the back of the Bombe:



The other machine is the Colossus, a binary computer built to decipher Lorenz. Enigma is far more famous than Lorenz, but I have a soft spot for the Lorenz code because of its close rela…

Regular expression are hard, let's go shopping

After looking at a Tweet from Charles Arthur of The Guardian and I decided to hunt down his blog. I typed "Charles Arthur" into Google and the first link was to his blog.

But there was something strange about it. All the letter t's following an apostrophe were highlighted. Here's a screen shot:



Yet, if I typed the exact same URL into Firefox the highlighted t's were not there. Odd. Since the URL was there this had to be something inside the HTTP headers sent when I was clicking through from Google.

I fired up HTTPFox and watched the transaction. Here's a screen shot of the HTTP headers of the GET request for his page. The interesting thing to look at is the Referer header.



It immediately jumped out to me that one of the parameters was aq=t. Looked to me like something on his blog was reading that parameter and using it to highlight. Poking around I discovered that his site is written using WordPress and there's a plugin for WordPress (that hasn't …

In which I resurrect a 13 year old 3.5" floppy disk and reprint my doctoral thesis

This is a follow up to a post from the weekend about playing with my old Sharp MZ-80K. Someone commented that they'd be more impressed if I resurrected a 15 year old floppy disk than a 30 year cassette tape.

I don't have a 15 year old floppy disk to hand, but I do have this one that's 13 years old and according to the label contains a copy of my doctoral thesis. The disk was created in 1996 and the files on it date to 1994 for my doctoral thesis which I completed in 1992.



But would it still read?

The first step was finding a drive. I had an old-ish 3.5" USB disk drive kicking around, so I plugged it into my MacBook Air and fired up Windows XP under VMWare. It happily recognized the drive and the magically it loaded up the floppy disk:



The disk contains a single ZIP file called oxford.zip. Unzipping it and poking around in the directories reveals that it contains my thesis, all the papers I wrote as a doctoral student, my CV and helpful READ.ME files: a gift to my fut…