Monday, February 27, 2012

How to break the 'rapper code'

The Mirror has a story about a code sent by a prisoner that revealed a bribery plot. The code was apparently broken by a professional code breaker. Looking at the picture in the article I figured I'd have a go.

It's actually a pretty easy code. The first step was to transcribe the code so I could play with it and then do a simple frequency analysis. Lo and behold it looks like it's a substitution cipher. The top six most frequent numbers are:
Number   Frequency
10       0.101243339253996
5        0.0959147424511545
27       0.0852575488454707
8        0.0746003552397869
3        0.0728241563055062
16       0.0603907637655417
25       0.0532859680284192
and there's a steep drop off that looks just like the pattern you'd see in English. So those first few numbers likely match the most common letters in English: ETAION. So a quick substitution in that order reveals:
... ... ..NO .O. IE . .. .N..T ..TEIT 
..NA ..I. I... AE.T . .E.O . E .TT. .E AEI. 
OI A. .T ....OIAT. .. EN. .TENI NT.T.IIE.. 
..TEIT AEI. IAEAT.TNA .... I.E.E A...... E 
ON.T.TN.ENA I..O.OA.. IE.ON A.T ...O.T .E.T 
.O. .O.. .T ..A ON .E O. .E.E.T .. .O.ON..O. 
E IO.NE. OT IAEN.ON. .. .AE.ON. .O. ... .TN 
...O. EE.T .. A. E..OT.T A.OI AEI.. ..IA .ON. 
A.O. . .O. ... AE.. . .E.I . ..IA.. . . .TN.. 
EA .OT .E....A.T.I .E..... ..IA .O.. .. .E. 
. .TTA.O. EI E .EIA .TI..A O .E.T E..TE.. 
.E.T ..NAE.A .OA. I.E.EI ..IITI .E ....ONI 
.E.O.. ..NT.I .. .E ..N. .T.I IA.O. .... 
ON ..ONEA..N A.T. .ENA .O.ATTN . ... .E 
IAEAT.TNA .T.AO.OT. ..A A.T. .E.T A. .TA..N 
.. .T.I . ...NTI A. I... .... .EOA..T...T EN. 
Since the commas in the original text look like they might be word breaks I've added spaces where they were and I've output a . when I don't know the substitution.

Doesn't look very revealing so I tried permuting the first few numbers so that 5 = E and 10 = A to get:
... ... ..NO .O. IA . .. .N..E ..EAIE 
..NT ..I. I... TA.E . .A.O . A .EE. .A TAI. 
OI T. .E ....OITE. .. AN. .EANI NE.E.IIA.. 
..EAIE TAI. ITATE.ENT .... I.A.A T...... 
A ON.E.EN.ANT  I..O.OT.. IA.ON T.E ...O.E 
.A.E .O. .O.. .E ..T ON .A O. .A.A.E .. 
.O.ON..O. A IO.NA. OE  ITAN.ON. .. .TA.ON. 
.O. ... .EN ...O. AA.E .. T. A..OE.E T.OI 
TAI.. ..IT .ON. T.O. . .O. ... TA.. . .A.I 
. ..IT.. . . .EN.. AT .OE .A....T.E.I 
.A..... ..IT .O.. .. .A. . .EET.O. 
AI A .AIT .EI..T O .A.E A..EA.. .A.E ..NTA.T 
.OT. I.A.AI ..IIEI .A ....ONI .A.O.. ..NE.I 
.. .A ..N. .E.I IT.O. .... ON ..ONAT..N T.E. 
..T T.E. .A.E T. .ET..N .. .E.I . ...NEI T. 
I... .... .AOT..E...E AN. .A..ENT
The word ITATE.ENT really stands out. Could it be STATEMENT? Updating the number to letter mapping reveals:
... ..M M.NO .O. SA . M. .N..E ..EASE 
..NT ..S. S... TA.E . .A.O . A .EE. .A TAS. 
.OM .O.. ME ..T ON .A O. .A.A.E .. .O.ON..OM 
A SO.NA. OE STAN.ON. .. .TA.ON. .O. ... .EN
M..O. AAME .. T. A..OE.E T.OS TAS.. M.ST .ON. 
T.O. . .O. .M. TA.. . .A.S . M.ST.. . . .EN.. 
AT .OE .A..M.T.E.S .A..... M.ST .O.. M. .A. . 
..NTA.T .OT. S.A.AS ..SSES .A M...ONS .AMO.. 
..NE.S .. .A ..N. .E.S ST.O. .... ON ..ONAT..N 
..T T.E. .A.E T. .ET..N M. .E.S . ...NES T. 
S... .... .AOT..E...E AN. .A.MENT
Now MEANS NE.E.SSA.. stands out. So substitute the right numbers to get:
... C.M M.NO .O. SA . MY .NC.E ..EASE 
..NT R.S. S... TA.E . .AYO . A .EE. .A TAS. 
.OM .O.. ME ..T ON .A O. .ARA.E .Y .O.ON..OM 
A SO.NA. OE STAN.ON. .. .TA.ON. .O. ... .EN 
T.O. . .O. .M. TA.. . .ARS . M.ST.. . C .EN.Y 
AT .OE .A..M.T.ERS YAR.Y.. M.ST RO.. MY .A. 
..NERS .. .A ..N. .E.S STRO. .... ON C.ONAT..N 
..T T.EY .A.E T. RET.RN M. .EYS . ...NES T. 
S... .... .AOT..E..RE ANY .AYMENT 
A few things stand out here: "ANY .AYMENT", "A.REA.Y MA.E C.NTACT", ".AST RES.RT" and also the ".Y" before "ANY MEANS NECERSSARY". Substituting those characters we get:
The rest is left as an exercise for the reader.

The only surprising thing here is the comment from the 'expert':
Dr Olsson, who has worked on hundreds of cases for police around the world, told the trial: ‘The thought process behind the code shows someone who is very able, very intelligent, very skillful.’
Really? Looks like someone who'd read the simplest childhood book on codes to me. Not saying he's an idiot, but hardly very skillful code making.

PS Some people have been asking about the code I wrote to do this. Here's my little program:
use strict;
use warnings;

my @code = ( [qw/6 25 4/], [qw/30 40 13/], [qw/13 12 16 8/], [qw/X 8 
11/], [qw/3 10/], [qw/1/], [qw/13 43/], [qw/40 16 30 9 5/], [qw/_ 9 5
10 3 5/], [qw/11 25 16 27/], [qw/20 40 3 4/], [qw/3 4 40 11/], [qw/27
10 6 5/], [qw/2/], [qw/11 10 43 8/], [qw/2/], [qw/10/], [qw/X 5 5 6/],
[qw/11 10/], [qw/27 10 3 6/], [qw/8 3/], [qw/27 25/], [qw/19 5/],
[qw/30 25 13 _ 8 3 27 5 11/], [qw/19 43/], [qw/10 16 43/], [qw/13 5 10
16 3/], [qw/16 5 30 5 20 3 3 10 20 43/], [qw/_ 9 5 10 3 5/], [qw/27 10
3 6/], [qw/3 27 10 27 5 13 5 16 27/], [qw/24 20 25 13/], [qw/3 4 10 6
10/], [qw/27 4 20 25 40 _ 4/], [qw/10/], [qw/8 16 11 5 _ 5 16 11 10 16
27/], [qw/3 25 9 8 30 8 27 25 20/], [qw/3 10 43 8 16/], [qw/27 4 5/],
[qw/_ 25 9 8 30 5/], [qw/13 10 11 5/], [qw/4 8 13/], [qw/_ 8 80 6/],
[qw/13 5/], [qw/25 40 27/], [qw/8 16/], [qw/11 10/], [qw/8 11/], [qw/_
10 20 10 11 5/], [qw/19 43/], [qw/_ 8 42 8 16 _ 4 8 13/], [qw/10/],
[qw/3 8 _ 16 10 9/], [qw/8 5/], [qw/3 27 10 16 11 8 16 _/], [qw/40
_/], [qw/1 27 10 6 8 16 _/], [qw/9 8 11/], [qw/25 24 24/], [qw/_ 5
16/], [qw/13 43 _ 8 30/], [qw/10 10 13 5/], [qw/40 _/], [qw/27 25/],
[qw/10 30 4 8 5 42 5/], [qw/27 4 8 3/], [qw/27 10 3 6 40/], [qw/13 40
3 27/], [qw/9 8 16 6/], [qw/27 19 8 _/], [qw/1/], [qw/X 8 11/], [qw/48
13 2/], [qw/27 10 9 6/], [qw/2/], [qw/_ 10 20 3/], [qw/40/], [qw/13 40
3 27 _ 25/], [qw/1/], [qw/30/], [qw/19 5 16 _ 43/], [qw/10 27/], [qw/4
8 5/], [qw/19 10 19 40 13 25 27 4 5 20 3/], [qw/43 10 20 11 43 25
40/], [qw/13 40 3 27/], [qw/20 8 17 _/], [qw/13 43/], [qw/11 10 11/],
[qw/1/], [qw/13 5 5 27 4 8 13/], [qw/10 3/], [qw/10/], [qw/9 10 3
27/], [qw/20 5 3 25 20 27/], [qw/8/], [qw/4 10 42 5/], [qw/10 9 20 5
10 11 43/], [qw/13 10 11 5/], [qw/30 25 16 27 10 30 27/], [qw/X 8 27
4/], [qw/3 4 10 6 10 3/], [qw/19 25 3 3 5 3/], [qw/11 10/], [qw/13 40
9 9 8 16 3/], [qw/24 10 13 8 9 43/], [qw/25 X 16 5 20 3/], [qw/25
24/], [qw/11 10/], [qw/9 25 16 _/], [qw/9 5 _ 3/], [qw/3 27 20 8 _/],
[qw/50 9 40 19/], [qw/8 16/], [qw/30 4 8 16 10 27 25 X 16/], [qw/27 4
5 43/], [qw/X 10 16 27/], [qw/24 8 24 27 5 5 16/], [qw/6/], [qw/24 25
20/], [qw/11 10/], [qw/3 27 10 27 5 13 5 16 27/], [qw/30 5 20 27 8 24
8 5 11/], [qw/19 40 27/], [qw/27 4 5 43/], [qw/4 10 42 5/], [qw/27
25/], [qw/20 5 27 40 20 16/], [qw/13 40/], [qw/6 5 43 3/], [qw/1/],
[qw/_ 4 25 16 5 3/], [qw/27 25/], [qw/3 4 25 X/], [qw/_ 25 25 11/],
[qw/24 10 8 27 4 19 5 24 25 20 5/], [qw/10 16 43/], [qw/_ 10 43 13 5
16 27/] );

my %f; 
my $count = 0;

foreach my $row (@code) {
    foreach my $c (@$row) {
        $f{$c} += 1;
        $count += 1;

foreach my $c (sort { $f{$b} <=> $f{$a} } keys %f) {
    my $p = $f{$c}/$count;
    print "$c $p\n";

my %s = (
    '10' => 'E',
    '5'  => 'T',
    '27' => 'A',
    '8'  => 'I',
    '3'  => 'O',
    '16' => 'N' );

foreach my $row (@code) {
    foreach my $c (@$row) {
        if ( defined( $s{$c} ) ) {
            print $s{$c};
        } else {
             print ".";
    print " ";
print "\n";

Friday, February 24, 2012


Recently, I joined a new company called CloudFlare where I was asked to pick a job title. I decided to go with Programmer. It was the first job title I ever had:

(Notice how at my first job there was no email address, but there was one for Telex.)

I choose Programmer for both its simplicity and accuracy. I am working as a programmer helping to speed up and protect web sites, and other programming-related job titles seem overblown. Software Engineer always seems like an overly grand way of saying the same thing. And I'm certainly not a Computer Scientist.

Fundamentally, I make computers do what I want them to do.

Of course, people like to come up with grand titles. I once had the stupid title of Chief Architect. And there's a whole variety of Lead Engineer, Principal Engineer, etc. This is partly done so that in large companies people of different seniorities are distinguished and partly so that people feel important.

It might seem lowly to be a Programmer, but in a world where so much is driven by computers there's nothing shameful in being the person who makes them go.

Mobile subscriber leakage in HTTP headers in the wild

At the end of January mobile phone company O2 admitted that because of a technical failure it had been leaking the mobile phone numbers of people who browsed using their phones on the O2 network. The phone numbers were being leaked to the web sites people visited.

On that day I set up logging on to see if the same thing happened again, and to gather information on in the wild leakage of phone numbers in HTTP headers. The bottom line is that in a month of gathering data I've seen phone numbers sent to my server likely without the person browsing knowing.

The most common headers I've seen are MSISDN and X-MSISDN. Both seem to contain the phone's MSISDN (i.e number). About 1 in every 22,000 visits to the site had the MSISDN header in it with something that looked like a valid phone number.

Another common header was X-UP-SUBNO which AT&T apparently adds. This contains a unique identifier for the subscriber. A typical X-UP-SUBNO in my log files looks like: T_ILL_CHI_1311xxxxxx00010xxxxxxx or T_DLS_DFW_1283xxxxxx00010xxxxxxx (I have replaced some numbers with x to preserve the visitor's privacy). About 1 in every 3,000 visits to had a unique AT&T identifier in it. This doesn't appear to be the phone number.

One mobile visitor in the last month had headers called X-UP-SUBSCRIBER-COS, X-OPWV-DDM-HTTPMISCDD, X-OPWV-DDM-IDENTITY, X-OPWV-DDM-SUBSCRIBER. Between them they were full of base 64 encoded data containing details of what appeared to be the plans the subscriber was on (mobileGenericPlan, Vodacom, mobilePushEnabledPlan). And there were a variety of identifiers for the subscriber, device and 'station id'.

Then there were a bunch of headers apparently added by Sprint PCS containing a CLIENTID encoded with base 64. And Vodafone seemed to be injecting big chunks of base64 in an X-VF-ACR header.

It's hard to see why these headers should be sent all the way to the origin web server.

PS While looking at the headers the most WTF I came across were the headers X_MTI_USERNAME, X_MTI_EMAIL and X_MTI_EMPID which appeared to be giving the username, email address and employee ID(?) of someone at a company called Meyer Tool.

Wednesday, February 22, 2012

The Case for Open Computer Programs

A few moments ago Nature made public a paper that I've written with two co-authors: Professor Darrel Ince and Professor Les Hatton entitled The Case for Open Computer Programs. The paper argues for the routine release of source code for software used in generating results presented in scientific papers.

Here's the abstract:
Scientific communication has always relied on evidence that cannot be included in publications. But the rise of computational science has added a vast new layer of too often inaccessible data sets and computer programs. Research journals are now coping with the issue of reproducibility of computationally derived results, and while it is generally accepted that all data should be made available upon request, the current landscape of regulations regarding the openness of computer programs remains muddled. Here we argue that, with some exceptions, anything less than release of the actual source programs used is intolerable for any scientific results that depend on computation. The vagaries of executing programs in different hardware and software environments ensures that exact reproducibility will always remain uncertain, but withholding code increases the chances that efforts to reproduce results will fail, possibly due to ambiguities or misunderstandings in non-explicit program descriptions, rather than to scientific errors.
I'm not the lead author on this paper, Darrel Ince is, so it's best to send any questions to him. The preferred means is by commenting directly on the paper via the Nature web site so that comments and responses are public.

Alan Turing's reading list (with readable links)

Alex Bellos published a list of books that Alan Turing took out from the school library as a child. I've tracked down as many as possible should you wish to follow his reading.

1. Isotopes by Frederick William Aston (1922 edition).

2. Mathematical Recreations and Essays by W. W. Rouse Ball

3. Alice's Adventures in Wonderland, Game of Logic and Through The Looking Glass by Lewis Carroll.

4. The Common Sense of the Exact Sciences by William Kingdon Clifford.

5. Space, Time and Gravitation and The Nature of the Physical World by Sir Arthur Stanley Eddington.

6. Sidelights on Einstein by Albert Einstein.

7. The Escaping Club by A. J. Evans.

8. The New Physics by Arthur Haas.

9. Supply and Demand by Hubert D. Henderson.

10. Atoms and Rays and Phases of Modern Science by Sir Oliver Lodge.

11. Matter and Motion by James Clerk Maxwell.

12. The Theory of Heat by Thomas Preston.

13. Modern Chromatics, with applications to art and industry by Ogden Nicholas Rood.

14. Celestial Objects for Common Telescopes by Thomas William Webb.

15. The Recent Development of Physical Science by William Cecil Dampier Whetham.

16. Science and The Modern World by Alfred North Whitehead.

One book stands out as totally different from all the others: The Escaping Club by A. J. Evans. I'm putting that on my Kindle.

Tuesday, February 07, 2012

So much for Google's Privacy Settings

Just the other day I deleted most of my online accounts and I set my Google Profile to "not visible in search":

And there on page 3 of Google search results for "John Graham-Cumming" what do we find:

Why, it's a Google+ profile (which I explicitly set to not appear in search results) for a Google+ account that I deleted. Clicking through we discover there's almost no information (because I deleted it), except for the totally brain dead:

Because it's not possible to not show your gender.

It looks like telling Google to not show my profile in search results was useless, the only option was to totally delete it which I have now done.

My foxhole radio

As a child I'd made a crystal radio set and had been meaning to make one again when I came across stories of foxhole or PoW radios made from scavenged parts. These were made during the Second World War (and other wars) from copper wire from a broken motor or alternator and anything from which a diode could be fashioned.

So, I decided to give it a go. Here's my foxhole radio. It's made from: a toilet roll tube, some copper wire, a blue steel razor blade, the tip of a pencil, a piece of coat hanger and an earpiece. Connected to ground and a long wire antenna in the garden it works... just. (PoW's seem to have used the barbed wire of the prisons they were held in as antennas, I used a long piece of bell wire).

The components are:

1. A coil of enameled copper wire wound about 100 times around a toilet roll tube. I used 22 SWG wire.

2. A tuning arm made from a piece of coat hanger than can sweep across the coil to find radio stations. The coil's actual copper is exposed where the arm sweeps by sandpapering off the enamel coating.

3. A crystal earpiece.

4. A diode made from a blued steel razor blade, the point of a pencil and a safety pin. I bought a blued steel antique razor blade on eBay after having trouble with the modern blade that I blued myself.

Modern blade blued on the gas stove (I just held it with pliers in the flame until it turned blue which took just seconds).

And the antique blued steel blade from eBay:

The antique blade worked well (i.e. I was able to find sensitive spots on it with ease, the modern blade was a pain).

Here's the end of a pencil and safety pin combination:

The antenna connects to the bottom end of the sweep arm (to the screw in the middle of the picture) and the ground connects to the right hand side of the coil (to the screw at top right, the screw top left is not used). Using the radio involves finding a radio station by sweeping around and adjusting the pencil lead so that the signal is detected. I found it easiest to sweep around for some crackles and then fiddle with the pencil to find the right spot on the blade. Once found the pencil is left in place and not moved again.

The circuit is essentially the same as this classic design with my razor blade/pencil combination instead of the cat's whisker detector in the picture.

The radio works because the razor blade and lead combination form a point contact diode which is able to detect the AM radio signal.

There are three radio stations that I can hear using the set, the loudest one appears to be Sunrise Radio an AM station catering to London's Asian community. Here's a recording of a few seconds of reception using my radio (it's very quiet, turn your volume right up): MP3.

There are lots of improvements that could be made. Currently, there's no capacitor in the circuit and so it is relying on capacitance in the antenna to make a tuned circuit. Fashioning a capacitor from aluminum foil and newspaper would probably help.

Monday, February 06, 2012

UK Government declines to pardon Alan Turing

There's been a petition to the government and an Early Day Motion asking for Alan Turing to be pardoned. This is something I oppose despite having been behind the 2009 Alan Turing apology campaign.

Last, Thursday, February 2, the Liberal Democrat Peer Lord Sharkey asked in the House of Lords: "To ask Her Majesty's Government whether they will consider granting a posthumous pardon to Alan Turing."

A government minister, Lord McNally, responded for the government declining to pardon Turing:
The question of granting a posthumous pardon to Mr Turing was considered by the previous Government in 2009.

As a result of the previous campaign, the then Prime Minister Gordon Brown issued an unequivocal posthumous apology to Mr Turing on behalf of the Government, describing his treatment as "horrifying" and "utterly unfair". Mr Brown said the country owed him a huge debt. This apology was also shown at the end of the Channel 4 documentary celebrating Mr Turing's life and achievements which was broadcast on 21 November 2011.

A posthumous pardon was not considered appropriate as Alan Turing was properly convicted of what at the time was a criminal offence. He would have known that his offence was against the law and that he would be prosecuted.

It is tragic that Alan Turing was convicted of an offence which now seems both cruel and absurd-particularly poignant given his outstanding contribution to the war effort. However, the law at the time required a prosecution and, as such, long-standing policy has been to accept that such convictions took place and, rather than trying to alter the historical context and to put right what cannot be put right, ensure instead that we never again return to those times.
It's interesting, to me at least, that the issue of a pardon was considered in 2009 as this was not something I had been asking for. The government's response makes clear that they do not consider a pardon appropriate.

Saturday, February 04, 2012

Long range WiFi antenna from Illy coffee can

I have a stack of half finished projects that I'm trying to complete. Happily, the first one was finished late last night: a long range WiFi antenna made using an empty coffee can. Working from the ideas here I made an antenna from the following:

1. An empty 250g Illy Espresso ground coffee can

2. A Type N bulkhead socket and a Type N plug

3. A piece of WiFi antenna extension cable with a reverse SMA connector on the end.

4. A short piece of coat hanger for the active element of the antenna

Here's a shot inside the can showing the antenna element and the Type H bulkhead socket screwed into place. I simply drilled five holes in the can using a HSS drill.

And from the outside:

The active part of the antenna is made by soldering a piece of coat hanger into the Type N bulkhead connector.

For testing I'm using a USB WiFi adapter that has a reverse SMA connector on the side:

The cable was pretty simple to make, all that was needed was connecting the Type N connector on one end:

And the coffee can lid proves handy because the adapter and cable can all be stored inside the can when not in use. I made a little cover for the bulkhead socket using the neck of a balloon that I tied and turned inside out:

Testing using a combination of iwlist and Kismet showed that the antenna was both directional (as would be expected from its design) and more sensitive than the little whip antenna.

From the top room in my house I could see 58 networks using the standard antenna and 97 using the can pointing in one direction. The received power was on average between 3 and 9 dB better with the antenna for distant networks and had much greater range (that's between 2x and 8x better).

About 1/4 mile from my house (actually 423m or 0.26 mile or 1386 feet) is a business hotel that offers its guests free WiFi. Pointing the antenna in its direction I was able to connect to the hotel WiFi (which doesn't have a WEP or WPA password) and then reach a screen where I had to agree to the hotel's terms and conditions for use of their network.

Looks like I've got a backup connection if my home broadband goes down.

PS There are lots of details missing from this blog post because I linked above to a very detailed explanation of the whys and wherefores of building a can antenna.  Look there for critical details on picking a can size (the Illy can is a good size for the frequency of WiFi), the length of the active element, and the position within the can.   The other thing that's worth noting is that I shortened the cable going to the antenna as it was too long: the longer the cable the worse the loss.

Making an old USB printer support Apple AirPrint using a Raspberry Pi

There are longer tutorials on how to connect a USB printer to a Raspberry Pi and make it accessible via AirPrint but here's the minimal ...