Thursday, December 22, 2011

A simple illustration of the use of goroutines and channels in Google Go

For a long time I've been meaning to spend some quality time with Google Go and I finally have the chance. For the little home project I'm working on I needed a way to generate unique IDs. In Go there's a really nice and simple way of doing this: assign a single goroutine as an ID generator and use a channel as the way to grab a new ID.

Here's the code:
    idMaker := make(chan string)

    go func() {
        var counter int64 = 0
        for {
            idMaker <- fmt.Sprintf("%x", counter)
            counter++
        }
    } ()
The first line makes a channel that will be used to communicate strings. In this case, I'd decided to have unique string IDs.

Then there's a go function (in this case an anonymous function) that contains a 64 bit integer counter that is incremented every time an ID is generated. When an ID is generated it is formatted as a hex number and made available on the channel. It just loops around forever providing IDs and only updating the counter when an ID has been used.

Any other part of the program can grab an ID like this:
    id := <-idMaker
Because reading from a channel is atomic and only one part of the program can read from the channel at once this eliminates any headaches about threads or processes needing to share some unique ID generating code. In fact, multiple goroutines can use exactly the same channel to get unique IDs trivially.

This works because Google Go takes its channel synchronization idea from CSP where communication is how synchronization is done. There's no need for any sort of locking around the var counter because it only gets updated in one place (inside the goroutine) and it only gets updated when its value has been consumed by reading from the channel.

Thursday, December 15, 2011

A pre-Christmas Plan 28 update

Despite the lack of public announcements from Plan 28 on the project to build Charles Babbage's Analytical Engine there's plenty going on behind the scenes.

The big news is the Plan 28's registration as a charity in the UK is now complete and has been published by the Charity Commission. Plan 28 is now registered charity number 1145043.

Every charity has to have 'objects' which are a statement of its aims. Plan 28's objects are as follows:
3. Objects

3.1 The Charity's objects ("The Objects") are for the public benefit

3.1.1 The advancement of education, in particular in the fields of science and the history of science, through the construction of Charles Babbage's Analytical Engine; and

3.1.2 The advancement of science, in particular through:
a) Research in the fields of computer science, engineering and mathematics; and
b) The raising of public awareness of Charles Babbage's Analytical Engine and related scientific theories and developments.

3.2 Article 3.1 may be amended by special resolution but only with the prior written consent of the commission.
What this means is that early in the New Year Plan 28 will enter a fundraising phase and all those folks who pledged money will now be able to donate it. It also means that Plan 28 will be eligible for Gift Aid for UK donors.

Here's to getting the Engine project moving in early 2012.

PS A big thank you to the solicitors at Stone King who worked on the charity registration process with me. They've been simply brilliant to work with and I would highly recommend them for other people starting a charity.

Tuesday, December 06, 2011

A back channel confirms that I'm right... sort of

Through a little circuitous back channel I received an unofficial follow up to a blog post about the unused machine code in the GCHQ Code Challenge part 2. The follow up assured me that the unused code was left over after some clean up was done, and that the rest of the data in the file was random filler (as I'd already heard).

But as I'd already determined that it was not actually random (at least at one level) I sent a carrier pigeon out with a question about my follow up blog post.

Listening for secret messages transmitted during The Archers I received a reply indicating that I had indeed 'broken' the encryption on that stream of ASCII data (by guessing the algorithm and reverse engineering the key stream), but that the underlying data was actually random ASCII generated and encrypted using the following Python code:
b = ''.join([chr(randint(0x20, 0x7f)) for i in range(0, 16 * 7)])
c = codec(b, 0x1f, s=5)
The first part generates 112 bytes of random ASCII text and the codec function is apparently the encryption function with the key I had identified (see the 0x1f and 5).

There are a couple of oddities about this code. The randint could generate an 0x7F which is non-printable (and doesn't appear in the decrypted text) and it generates precisely 112 bytes of data (whereas the part 2 actually contains two blocks of 112 and one of 102).

I questioned that by leaving a note in a tree in St. James's Park and received a response by exchanging briefcases at King's Cross Station to the effect that the 102 byte block had simply been truncated to make the code look interesting.

Of course, that could all be completely non-suspicious, but having been told that first it was random filler and then that actually it was blocks of encrypted random printable ASCII filler it wouldn't surprise me if the truth was even more complex (and perhaps rather mundane).

But why go to all this trouble on 'random filler' text? And why update the web site to say "The Challenge Continues"?

I am left slightly baffled, which I'd imagine is just how people working in secret places would like it to be.

Monday, December 05, 2011

Down the GCHQ rabbit hole (or I think there really is a part 4)

Update: Confirmation that this is correct

Early this morning I blogged that I thought there was a hidden part 4 to the GCHQ Code Challenge because of the amount of non-random data in part 2. Through someone who knows someone who knows someone at GCHQ I got the response the data was "a random filler and that they ran out of time to do anything interesting".

Well, I don't believe that's the whole story. There's a distinct pattern worth investigating.

Firstly, here are the three blocks of data that are untouched in part 2:
0150: 7d 1f 15 60 4d 4d 52 7d 0e 27 6d 10 6d 5a 06 56   }..`MMR}.'m.mZ.V
0160: 47 14 42 0e b6 b2 b2 e6 eb b4 83 8e d7 e5 d4 d9   G.B.............
0170: c3 f0 80 95 f1 82 82 9a bd 95 a4 8d 9a 2b 30 69   .............+0i
0180: 4a 69 65 55 1c 7b 69 1c 6e 04 74 35 21 26 2f 60   JieU.{i.n.t5!&/`
0190: 03 4e 37 1e 33 54 39 e6 ba b4 a2 ad a4 c5 95 c8   .N7.3T9.........
01a0: c1 e4 8a ec e7 92 8b e8 81 f0 ad 98 a4 d0 c0 8d   ................
01b0: ac 22 52 65 7e 27 2b 5a 12 61 0a 01 7a 6b 1d 67   ."Re~'+Z.a..zk.g

0200: 37 7a 07 11 1f 1d 68 25 32 77 1e 62 23 5b 47 55   7z....h%2w.b#[GU
0210: 53 30 11 42 f6 f1 b1 e6 c3 cc f8 c5 e4 cc c0 d3   S0.B............
0220: 85 fd 9a e3 e6 81 b5 bb d7 cd 87 a3 d3 6b 36 6f   .............k6o
0230: 6f 66 55 30 16 45 5e 09 74 5c 3f 29 2b 66 3d 0d   ofU0.E^.t\?)+f=.
0240: 02 30 28 35 15 09 15 dd ec b8 e2 fb d8 cb d8 d1   .0(5............
0250: 8b d5 82 d9 9a f1 92 ab e8 a6 d6 d0 8c aa d2 94   ................
0260: cf 45 46 67 20 7d 44 14 6b 45 6d 54 03 17 60 62   .EFg }D.kEmT..`b

0270: 55 5a 4a 66 61 11 57 68 75 05 62 36 7d 02 10 4b   UZJfa.Whu.b6}..K
0280: 08 22 42 32 ba e2 b9 e2 d6 b9 ff c3 e9 8a 8f c1   ."B2............
0290: 8f e1 b8 a4 96 f1 8f 81 b1 8d 89 cc d4 78 76 61   .............xva
02a0: 72 3e 37 23 56 73 71 79 63 7c 08 11 20 69 7a 14   r>7#Vsqyc|.. iz.
02b0: 68 05 21 1e 32 27 59 b7 cf ab dd d5 cc 97 93 f2   h.!.2'Y.........
02c0: e7 c0 eb ff e9 a3 bf a1 ab 8b bb 9e 9e 8c a0 c1   ................
02d0: 9b 5a 2f 2f 4e 4e 00 00 00 00 00 00 00 00 00 00   .Z//NN..........
I say three (rather than two blocks), because as you'll see below there's a real pattern that indicates that we are looking at three blocks of data encrypted using the same method and the same key.

The decryption scheme used for the other parts of part 2 (that actually get decrypted) is based on calculating the following where xn is the nth byte (0 based) of the block to be descrypted: f(xn) = xn xor ( a * n + b ). a and b are constants. The first decryption uses a = 1 and b = 0xAA. The second decryption uses a = 3 and b = 0x32.

Working under the assumption that a similar scheme was used for these three blocks I ran various tests and noticed a peculiar rhythmic pattern where whole runs of random ASCII characters would appear followed by runs of non-printable characters. It occurred to me that this could imply that (a) the underlying data is in ASCII (using printable characters) and that (b) the pattern I was seeing was a pattern in the top bit of key. If the key is following the scheme above it would go through runs of 0s followed by 1s and if the plain text is ASCII the top bit is always 0 and so for that one bit the 'key' comes through.

A quick calculation of just the top bits of the three blocks above showed the following pattern:
0150: 
000000000000000000001111111111111111111111111
0000000000000000000000000011111111111111111111111111
000000000000000
0200: 
000000000000000000001111111111111111111111111
0000000000000000000000000011111111111111111111111111
000000000000000
0270: 
000000000000000000001111111111111111111111111
0000000000000000000000000011111111111111111111111111
00000
Immediately, it's apparent this this data is not random (at least in the top bit), and that it follows a pattern of 0s and 1s alternating with the following counts:
150: 20 (0s), 25 (1s), 26 (0s), 26 (1s), 15 (0s)
200: 20 (0s), 25 (1s), 26 (0s), 26 (1s), 15 (0s)
270: 20 (0s), 25 (1s), 26 (0s), 26 (1s), 15 (0s)
(Note that the last block is short because the data is truncated and ends with 0s). It also might indicate that there are three separate blocks of data encrypted using the same key.

Theorizing that that was generated using the f(xn) function above meant finding a and b that fit the bill. It's not hard to calculate this spotting that the 25 occurs when the 0xFF boundary is passed and that the 20 occurs because we start in the middle of a 26 block. The actual coefficents are a = 5 and b = 0x1F. That generates the following sequence to use as the key:

1f 24 29 2e 33 38 3d 42 47 4c 51 56 5b 60 65 6a 6f 74 79 7e 83 88 8d 92 97 9c a1 a6 ab b0 b5 ba bf c4 c9 ce d3 d8 dd e2 e7 ec f1 f6 fb 00 05 0a 0f 14 19 1e 23 28 2d 32 37 3c 41 46 4b 50 55 5a 5f 64 69 6e 73 78 7d 82 87 8c 91 96 9b a0 a5 aa af b4 b9 be c3 c8 cd d2 d7 dc e1 e6 eb f0 f5 fa ff 04 09 0e 13 18 1d 22 27 2c 31 36 3b 40 45 4a 4f 54 59 5e 63 68 6d 72 77 7c 81 86 8b 90 95 9a 9f a4 a9 ae b3 b8 bd c2 c7 cc d1 d6 db e0 e5 ea ef f4 f9 fe 03 08 0d 12 17 1c 21 26 2b 30 35 3a 3f 44 49 4e 53 58 5d 62 67 6c 71 76 7b 80 85 8a 8f 94 99 9e a3 a8 ad b2 b7 bc c1 c6 cb d0 d5 da df e4 e9 ee f3 f8 fd 02 07 0c 11 16 1b 20 25 2a 2f 34 39 3e 43 48 4d 52 57 5c 61 66 6b 70 75 7a 7f 84 89 8e 93 98 9d a2 a7 ac b1 b6 bb c0 c5 ca cf d4 d9 de e3 e8 ed f2 f7 fc 01 06 0b 10 15 1a 1f

Which has the required sequence of 0s and 1s and begins: 20 (0s), 25 (1s), 26 (0s), 26 (1s), 25 (0s). So, at this stage I've identified a pattern and a way to generate that sequence using the encryption scheme used in part 2.

The surprising thing that happens when you decrypt those three blocks using that key stream is that not only is it 7 bit (which was bound to happen because of the way the key was generated) but the entire thing is printable ASCII. No characters below 0x20 and no 0x7F. Here are the three decrypted blocks in ASCII (I added line breaks):

b;<N~uo?Ik<F6:c<(`;p5:?t|("(|Uac|4I["Z_xZyU{a+5cE}|K?SD.Y85sjvz:
\*^[email protected],Dd=83;?e0bnP3R$ZF:V,L~O 5wS&[km?6x5M;7A+X-(^.?,%Ugu;O4x;"?
<Dh<uy<tTPYcO|ui:9S-5YhY0!vU(k3e`rL.5ms;C`~o`6hW]TA[fqh_k4smCk}{
$a;gY9_y?z76gZ'n0AOi3eY6Li\b8W%(J~cHR)j*2I3`&bu!gV;L9j4pA%^eB::{
0%qjE)RcVax:/xsk}*.=u[\[email protected]/N7aHpA_$5H'LCW76XHtRA*krs|WZxu|U;
d^&!]V"',16;@EJ

The fact that this decrypts as printable ASCII reinforces my belief that this is correct. Of course, it looks like gibberish, but it's printable. There are 89 characters used from the 95 possible. The missing characters are # 1 > G Q l.

It seems unlikely that the sequence of steps I have taken would have resulted in a block of perfectly printable ASCII characters had this not been the correct thing to do.

Perhaps this is a base-89 encoding of some other binary data. Whatever it is it has high-entropy and could be encrypted with some robust scheme, or it could be the result of compression. Of course, it could be that this data is random and was encrypted by GCHQ as I specified as a sort of dead end.

PS If anyone from GCHQ is reading... can you email me a simple 'carry on' or 'stop wasting your time'. Need to sleep...

Is there another GCHQ Code Challenge?

Update: The answer is Yes, sort of.

Last week GCHQ issued a 'code challenge' as a way of attracting candidates via a web site called Can You Crack It?. I did the challenge, which actually consists of three parts and has more to do with assembly language and reverse engineering skills than cryptography (in fact, the only time you come across a cryptographic function, you can completely ignore it).

The cat is completely out of the bag now because some spoilsport published complete details on the web showing how to break it, but I think there may be one more mystery worth investigating.

Part 2 of the challenge involves writing a virtual machine that executes some code to decrypt an in memory buffer containing an HTTP command. Here's a dump of the memory of the virtual machine after execution:
0000: 31 04 33 aa 40 02 80 03 52 00 72 01 73 01 b2 50   1.3.@...R.r.s..P
0010: 30 14 c0 01 80 00 10 10 00 00 00 00 00 00 00 00   0...............
0020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0090: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
00a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
00b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
00c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
00d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
00e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
00f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0100: 32 00 75 0c 31 08 33 32 40 02 80 03 52 00 72 01   2.u.1.32@...R.r.
0110: 73 03 b2 00 c3 b0 00 30 1b c0 01 ff 00 00 00 00   s......0........
0120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0130: 00 00 75 10 01 00 00 00 00 00 00 00 00 00 00 00   ..u.............
0140: cc 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0150: 7d 1f 15 60 4d 4d 52 7d 0e 27 6d 10 6d 5a 06 56   }..`MMR}.'m.mZ.V
0160: 47 14 42 0e b6 b2 b2 e6 eb b4 83 8e d7 e5 d4 d9   G.B.............
0170: c3 f0 80 95 f1 82 82 9a bd 95 a4 8d 9a 2b 30 69   .............+0i
0180: 4a 69 65 55 1c 7b 69 1c 6e 04 74 35 21 26 2f 60   JieU.{i.n.t5!&/`
0190: 03 4e 37 1e 33 54 39 e6 ba b4 a2 ad a4 c5 95 c8   .N7.3T9.........
01a0: c1 e4 8a ec e7 92 8b e8 81 f0 ad 98 a4 d0 c0 8d   ................
01b0: ac 22 52 65 7e 27 2b 5a 12 61 0a 01 7a 6b 1d 67   ."Re~'+Z.a..zk.g
01c0: 47 45 54 20 2f 64 61 37 35 33 37 30 66 65 31 35   GET /da75370fe15
01d0: 63 34 31 34 38 62 64 34 63 65 65 63 38 36 31 66   c4148bd4ceec861f
01e0: 62 64 61 61 35 2e 65 78 65 20 48 54 54 50 2f 31   bdaa5.exe HTTP/1
01f0: 2e 30 00 00 00 00 00 00 00 00 00 00 00 00 00 00   .0..............
0200: 37 7a 07 11 1f 1d 68 25 32 77 1e 62 23 5b 47 55   7z....h%2w.b#[GU
0210: 53 30 11 42 f6 f1 b1 e6 c3 cc f8 c5 e4 cc c0 d3   S0.B............
0220: 85 fd 9a e3 e6 81 b5 bb d7 cd 87 a3 d3 6b 36 6f   .............k6o
0230: 6f 66 55 30 16 45 5e 09 74 5c 3f 29 2b 66 3d 0d   ofU0.E^.t\?)+f=.
0240: 02 30 28 35 15 09 15 dd ec b8 e2 fb d8 cb d8 d1   .0(5............
0250: 8b d5 82 d9 9a f1 92 ab e8 a6 d6 d0 8c aa d2 94   ................
0260: cf 45 46 67 20 7d 44 14 6b 45 6d 54 03 17 60 62   .EFg }D.kEmT..`b
0270: 55 5a 4a 66 61 11 57 68 75 05 62 36 7d 02 10 4b   UZJfa.Whu.b6}..K
0280: 08 22 42 32 ba e2 b9 e2 d6 b9 ff c3 e9 8a 8f c1   ."B2............
0290: 8f e1 b8 a4 96 f1 8f 81 b1 8d 89 cc d4 78 76 61   .............xva
02a0: 72 3e 37 23 56 73 71 79 63 7c 08 11 20 69 7a 14   r>7#Vsqyc|.. iz.
02b0: 68 05 21 1e 32 27 59 b7 cf ab dd d5 cc 97 93 f2   h.!.2'Y.........
02c0: e7 c0 eb ff e9 a3 bf a1 ab 8b bb 9e 9e 8c a0 c1   ................
02d0: 9b 5a 2f 2f 4e 4e 00 00 00 00 00 00 00 00 00 00   .Z//NN..........
02e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
02f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
There are three colors used there to highlight areas: memory in blue was executed, memory in red was both read and written, memory in green was read, written and executed. As you can see the program starts running at 0x0000. There there's small program that decrypts the block of memory between 0x100 and 0x14F and jumps to 0x100.

The decrypted program decrypts the memory at 0x1C0 to reveal the HTTP command that is used to execute part 3 of the GCHQ challenge.

But what about all the rest of the unused memory? No other part of the GCHQ challenge wastes bytes, they are all used for something, but here there's a ton of memory that's filled with data and that isn't referred to anywhere else.

Is this a hidden fourth part of the GCHQ challenge? Perhaps even the real challenge?

If you look into the first block of data that's been decrypted there are two intriguing pieces of additional information. At 0x132 there are three bytes: 75 10 01. Those three bytes are actually valid code in the virtual machine. They mean
75 10   add   ds, 0x10
01      jmpr  r1
This seems like perfectly reasonable code. It moves the data segment forward by 0x10 and then jumps to R1. At the end of execution of the original program the data segment is at 0x1c. Thus moving it forward puts it at 0x2c (i.e. the block starting at 0x02c0 above). An alternative explanation is that based on the initial conditions of the VM that data segment would now be 0x20 (i.e. pointing immediately after the decrypted HTTP command at 0x0200).

Also at the end of running the program R1 contains 0x08. Doing the jump would drop straight into the decryption loop just past the initialization and into the XOR portion. That could be totally valid. The jump would take the instruction pointer to:
10:08 movm r0, [ds:r2]
10:0a xor r0, r3
10:0c movm [ds:r2], r0
10:0e add r2, 01
10:10 add r3, 03
10:12 cmp r2, 00
10:14 jmpe r3
10:15 cmp r0, 00
10:17 movr r0, 1b
10:19 jmpe r0
10:1a jmpr r1
The only other clue is that there's the byte 0xCC in the decrypted memory.

And one more thing. Notice the 10:14 jmpe r3. That never gets taken because r2 is never 00. But if you investigate the circumstances under which r2 would be 00 you find that it's when r3 would be 0x32 (i.e. when that jump would take you right to the 75 10 01 sequence that's been decrypted).

I haven't had enough free time to investigate this further. Perhaps it's a red herring, but it looks awfully suspicious. Especially given that the main loop will take that jump instruction when it's completely exhaused 0x100 bytes of data. This little 'subroutine' then moves the data segment on by 0x10 (i.e. 0x100 bytes) and the decryption will continue until a 00 byte is written. So it looks valid and is designed to cope with not having hit a 00 before the end of a segment.

PS It's been pointed out to me that the Can You Crack It? web site has been altered since the beginning to add the words "The challenge continues".

Wednesday, November 30, 2011

Downton Abbey Series 1 Episode 1 Morse Code

The very beginning of Downton Abbey's first ever episode begins with a close up of a Morse code key being used to send a telegram that will announce the critical news of two deaths. If you're like me you instantly wondered what was being keyed.

Here's what I hear:
dah                   T 
dah dah dah           O

di dah di dit         L
dah dah dah           O
di dah dit            R
dah di dah dit        C

dah dah dit           G
di dah dit?           R
So, it appears that the message begins: "TO LORC GR". I assume that is meant to be "TO LORD GRANTHAM" and that the C is actually D and has been incorrectly transcribed.

Here's the audio as an MP3: Downton Abbey Series 1 Episode 1 Morse Code (first part).

Then over loud music it continues:
dah dit               N
dah dah dit           G
dit                   E
di dah dit            R

di dah di dit         L
di dit                I
di di dit             S
dah                   T

di dah                A
di dah                A
di dah                A

di dah dah dah        J
di dah                A
I think that's a fragment of "NGER LIST AAA JA". Since AAA is a full stop and this telegram is referring to the sinking of the RMS Titanic where Patrick and James Crawley were on the passenger list (as we later learn in the episode) it's likely that this is a fragment of "PASSENGER LIST. JAMES".

Here's the audio as an MP3: Downton Abbey Series 1 Episode 1 Morse Code (over music).

Anyone else hear something different?

PS It's pleasing that the producers must have gone to the trouble of composing the complete telegram and the recording its transmission using Morse code just for that small segment at the beginning of the first episode.

PPS A little Google search reveals that it was G3YXZ [PDF] who was recruited to send the CW and he notes that for television reasons the key was adjusted to increase the gap size thus making sending more difficult (hence, no doubt, the mistake with C and D).

Tuesday, November 29, 2011

A nice bottle of Lorem Ipsum 2009

Going out to lunch with friends, having a nice bottle of wine in a gastropub. Good way to spend the weekend. Then you decide the wine was really nice and you'd take a quick photograph of the label so you don't forget it.

And then you notice the text on the bottle of Domaine Tissier Sancerre Rouge 2009. Yes, it's Lorem Ipsum filler text!

Interestingly it's the version that's popular in English speaking countries and not in France. I called the vineyard in France who were only able to tell me that the bottle of wine was genuine, that there's very little of the 2009 red left, and that it's only available in the UK to the restaurant trade.

So, if you want to get a bottle as a souvenir or Christmas present for the designer in your family you'll have to persuade a restaurant to part with the bottle.

Thursday, November 17, 2011

The Lack of Affordance Problem

Yesterday, I complained about the lack of an eject button on an Apple USB DVD drive. That design philosophy problem goes much deeper than just a single drive. It pervades Apple products, and, although I love them, it drives me spare!

If I could say one thing to Jonny Ive it would be: "In designing to make the every day use of Apple's products a wonderful experience you're missing the moment when your users need you most. It's when everything goes wrong that the user needs the maximum help. It's in those moments (of despair) that a well designed product will surprise and delight the user."

The DVD drive is an example of the opposite. When it's working it's lovely to look at, but when it goes wrong it's as enigmatic as the 2001 monolith.

The mystical incantations necessary to remove the DVD from the drive (such as the hold down the mouse button while rebooting your machine) have to be passed on from the Apple elders in secret whispers (or found by a desperate Google search). Surely, there's a better way? The DVD drive could have had a discreet eject button. And Apple software could be better.

Take the MacBook Air. If it's gone wrong and I need help I'm going to need the serial number. How do I get it? There are (at least) two ways. First it's printed on the bottom of the machine in tiny type printed in almost the same colour as the case. It's as if the designer wanted to make it hard to read. For goodness sake it's on the bottom of the machine, it wouldn't have killed you to use bigger type, or a different colour.

The second way is in software. But knowing that requires more mystical knowledge and reveals something truly nasty. Here's the dialog you get by clicking Apple -> About This Mac.

If you click on the grey text that says Version 10.6.8 the text changes, first to the build number and then to the serial number. There's is no indication that this will happen. The cursor doesn't change, it's normal text that the user wouldn't expect to see change like this. (You can also click More Info... to get the System Profiler where you can find the same information).

Why do this Apple? Why not let me know this is clickable? What's needed is an affordance. Something that lets the user know that this is interactive, that leads the user to the right action.

This situation gets worse with touch screens. Because there's no 'hover' possibility you are left guessing which inscrutable user interface element might be interactive, or what gestures are possible.

I swear that one day the iPhone will do something interesting if you take the Tai Chi "Carry Tiger to Mountain" position.

And don't even get me started on the abomination that is the Apple keyboard USB extension cable fiasco which is a USB cable with a special little indentation in the USB plug so you can't use it for anything else. That just feels like that designers sat around laughing at the purchaser.

And then there's the problem of video connectors.

PS A commentator on Hacker News pointed out that the USB specification prohibits extension cables and so although millions of the things exist, Apple probably did the right thing by making the extension a non-USB cable.

Wednesday, November 16, 2011

How to forcibly eject a CD/DVD from a MacBook Air USB SuperDrive

So you've got a DVD stuck in the external USB SuperDrive that connects to the MacBook Air? And you've tried hitting the magic Eject button on the keyboard to no avail?

And you've tried drutil tray eject at the command-line? And you've tried booting with Option held down and clicking eject? And you've tried plugging the drive into other machines (Macs, Windows PCs, Linux machines) to no avail?

What do you do?

Clearly you hit the physical eject button, right? Every DVD drive has one of those. On the SuperDrive you first look for it on the front:

Hmm. Nothing there. Perhaps it's one of those pinhole things on the back.

Or on one side:

Or, heart in hand, the other side:

I know, I know. Jonny Ive hid it on the bottom out of sight, right? It's so obvious now. Put a pinhole on the bottom where it will only be seen when needed. Form and functionality in one. Sleek lines when in use. An accessible emergency button when needed. Genius.

Nope. But, phew, at least there's an Apple logo there. I was beginning to worry.

So, the easy procedure for removing the disk is as follows. First you pry open the case with some thin available tool:

Then you remove the six screws holding the drive into the case:

Then you turn the drive over in anticipation of seeing your DVD only to be thwarted by another cover and a set of screws:

And then, finally, you peel back the cover to see your DVD and remove it.

Once reassembled the drive complains about the missing disk and promptly goes back to working normally.

And then you wonder to yourself. The drive inside the case is a Sony NEC Optiarc AD-5630A. And every other drive that Sony sells has a physical eject button, even the slot loading ones. Did Apple hide it? Did they ask Sony to not provide a button? I'm guessing that the Cypress Semiconductor microcontroller inside there knows all about doing an emergency eject.

Of course, you stop worrying about that and wonder how the cable has managed to fray on this barely used drive:


PS Some people have pointed out that there may be a secret lever inside the drive accessible from the front by sticking a thin screwdriver inside. I can't find any reliable mechanism (although jamming the DVD in place using a wad of paper and rebooting does work).

Sunday, November 13, 2011

Back from the dead with a power supply repair: my BBC Micro Model B

A couple of weeks ago I plugged in my BBC Micro Model B and it went up in a plume of smoke.

Unfortunately, the 30 year old machine had undergone one of the most common failures. A capacitor that had been sitting there all these years failed, burst open, emitted smoke and leaked all over the circuit board.

Today, I repaired the damage. This blog post is for anyone who faces the same thing.

To fix the damage I replaced three capacitors that are likely to die over time (including the one that had burst) using a kit costing £2.70 from this site and a copy of the BBC Microcomputer Service Manual.

The kit has one electrolytic capacitor (which is polarized and has to be inserted the right way round) and two ordinary capacitors which can be inserted in any way. The three capacitors to be changed are marked C1, C2 and C9 on the circuit board. On my machine it was C2 that had failed.

The first step is to remove the power supply from its case. This is carefully described in the service manual (including two cable ties that need to be cut and replaced). Here's the supply without its case. I've marked the three capacitors to be replaced.

And here's the reverse of the circuit board (clearly hand soldered) and I've marked the six joints that have to be desoldered. I used my normal soldering iron and a solder sucker to remove all three capacitors.

And here's a shot of the three capacitors after they've been removed with a close up of the ruptured C2.

With the capacitors removed you can see the capacitor juice that had leaked onto the PCB. I cleaned it off with some alcohol and Q-tips. You can also see that C2 actually had four holes drilled for it. Two were used for the supplied capacitor and the other two are used in the repair as the new capacitor has leads that are slightly closer together (good of Acorn to plan 30 years ahead!).

Soldering the three in is not hard (make sure the electrolytic is the right way round). Here's my finished circuit board:

And the reverse side showing the three new capacitors in place:

And here are the two cable ties that need to be replaced. One holds the low voltage cables going to the main PCB in place (it's attached to the power supply casing), the other holds the mains cables in place (it's attached to the power switch).

And putting it all back inside the case... it works!

Monday, November 07, 2011

Turning GE Color Effects G-35 Christmas Lights into a 7x7 display with Arduino

Some time ago I saw some GE Color Effects G-35 Christmas lights and was fascinated by the combination of color patterns that they were capable of and the fact that they only had three wires. That meant that something digital was going on.

Happily, a chap called Robert Quattlebaum did the hard work last year of hacking these lights to reveal that they are linked by a simple self-clocked serial protocol that can easily be driven by a microcontroller. That blog post gives the full details of the protocol.

Each lamp can be set to a specific brightness and an RGB color (4 bits per color). And the protocol is clocked at 30µs per bit so it's possible to refresh the entire 50 LED string at a rate of 24Hz. Perfect for hacking fun.

So I got a set and plugged them in just once (still in their packaging) to check that they worked. Here's a video of the lights getting their one real work out before the soldering iron got to them.



The first thing I did was remove the lovely plastic blub covers (these are being saved to brighten up a simple set of Christmas lights) to reveal the LED inside. This picture shows all three states: one has the original plastic cover on, the one in my hand has it removed but the plastic base intact, the bottom one is the naked LED and controller.

Once you've ripped off the protective plastic you are down to a simple LED with three wires going in (+5V, Data and GND) and three coming out. With 50 of these on a string I decided to make a 7x7 display (with one LED 'spare' should I break one).

Here's the finished display. The entire chain of lights has been cut up, mounted, resoldered and is now controlled by an Arduino Pro. All that's needed is a 5V power supply to plug into the power socket.

It's controlled by a small piece of Arduino software that contains a simple frame buffer implementation, code to implement the LED protocol via bit banging, code for scrolling text and for making faces. You can get the source code here. The key file is protocol.cpp which initializes the display and allows simple setting of the color and brightness of each LED. The most interesting part of that code (at least to me) is the initialization of the LED addresses. On power up the LEDs wait to learn their 6 bit address. They wait in turn and have to be programmed for use. I exploit this to make addressing them easy:

To build this I first cut up the entire string of lights to get just the LEDs with a small amount of cable attached to each. Then I prepared a piece of plywood, marked it up, drilled guide holes and then drilled the 49 mounting holes for the lights.

Then I glued in place each of the lights taking care to orient them for easy soldering into the chain. Then just 144 solder joints and little bits of heat shrink later I had the chain reattached and capable of being driven by the GE supplied controller.

The next step was some sort of opaque diffuser to put in front of the display. For this I used a cutting board that I cut to the right size. With the board in various positions it's possible to get different diffusion effects.

The final step in the hardware was to get a picture frame made to measure (I got a cheap, but deep, one in simple white wood from these folks) and then stuck the cutting board inside, used some old nylon nuts as spacers, mounted the LED board and finally used a piece of MDF that was supplied with the frame as a backing. The back is fixed on using some parts from an old servo.

Electronically this modification to the original lights was pretty simple. Having eliminated the original power supply and controller all that was needed was a new controller in the form of an Arduino Pro and a 5V power supply. The supply provided with the lights gave 5V at 3A (which seemed at lot). I measured the actual power usage and the highest current seen was 1.68A with all LEDs on white at maximum brightness. Thus I used an off-the-shelf power supply for the lights (plus Arduino Pro) capable of providing 2.25A.

The Arduino Pro is screwed to the back of the picture frame and there's only one component added: a 10K pull down resistor between GND and pin 7. Pin 7 is driving the serial data through the LEDs and it is low when not being driven with data. You can see that here:

If you're eagle-eyed you might have spotted a capacitor as well. I got this Arduino Pro free because it had a manufacturing fault and I repaired it using an old capacitor that I'd desoldered from the control board of a dead toaster. In that photo the Arduino Pro is connected via an FTDI cable for programming.

The other slightly tricky thing with this project was getting the timing of the serial protocol right. The simple serial protocol uses alternating high/low pulses of 20µs and 10µs (or low/high). In protocol.cpp I've made use of C #define statements to inline code for speed and also third-party code for fast changes to the I/O ports and accurate delays.

When working on this I used the Salae Logic analyzer to view the actual serial protocol in action. Here are two screen shots. The upper one shows the standard GE controller sending the initialization packets to the each of the LEDs in turn (with a gap of 40ms between them), the lower one shows a zoom into a specific packet being sent.

Finally, here's a video of the complete unit running through a simple "Hello World", followed by a few facial expressions and finishing with a "Thanks GE" for such a nice hackable string of Christmas lights.



PS If anyone from GE is reading this... how about making a 220V version of these? All you need to change is the wall wart to one that does 110-220V and you'd have all us Europeans interested in these lights.

PPS Other Arduino-based projects you might enjoy: GAGA-1: High Altitude Balloon Project and Cansole: Video games console in a can. Other non-Arduino projects: Revealing the secrets of the Ikea Lillabo wooden train set and Building a 'Ponyo' boat.

Tuesday, November 01, 2011

My BBC Micro Model B and a plume of acrid smoke

So, after yesterday's post about The Demon Machine I decided it would be fun to play around with the BBC Micro Model B and so I plugged it in and powered up. The familiar two-tone boot sound played and the machine was on.

About 15 minutes after power up there was a fizzing, popping sound from the computer and I ran to cut the power. Too late! A plume of blue/grey acrid smoke poured out of the left hand side of the machine right by the power supply.

I cut the power and moved the machine away from anything that might burn quickly and waited for the smoke to stop. It quickly dispersed and so I did the only thing a self respecting software engineer would do... I decided to turn it back on again.

But not without opening the power supply so I could take a look inside while it was running. Here's the PSU open after the fire:

Warning. I don't recommend that you do this sort of thing yourself. There are high voltages there, plus a bunch of electrolytic capacitors that can do nasty things if they feel like it. I was doing this while wearing my safety glasses.

You'll notice that there doesn't seem to be anything burnt in the supply and when I powered on the machine worked perfectly. Close examination shows that one of the capacitors used for power supply filtering had cracked open and burnt up.

If you look carefully you can see the cracking on the case of the capacitor and fresh capacitor juice running across the circuit board (most visible on the top photograph). A quick Google tells me that this is one of the most common failures on the BBC Micro power supply and easily repaired.

It's interesting to note that the capacitor used is rated by safety agencies because of the potential for failure leading to a short at mains potential and the metallized film is designed to self heal or fail safely. The device is also meant to fail safely at a peak potential of 2.5kV without burning with a flame.
EN132400 (IEC384-14): Active Flammability

The capacitor under test is connected to rated voltage through a transformer and filter. 20 transients are then introduced across the capacitor at random intervals while rated voltage remains applied. The amplitude of the transient is dependent on the class of capacitor. The capacitor may not flame during this test.
Not often you get to see an international safety standard take effect.

Here's the power supply circuit diagram.

The dead capacitor is C1 which sits right across the mains power input. Looks like I'll be getting the soldering iron out once the replacement part arrives.