Monday, March 29, 2010

Squaring two digit numbers in your head

All my life I've done mental arithmetic the 'wrong way': I've calculated from left-to-right, instead of right-to-left. So when I do something like 24 + 35 I'll see the 50 first and then the 9. This even applies when there's a carry and I'll do something like 36 + 56 as 80 + 12. I do the same thing for multiplication as well.

Turns out I'm not so weird after all (well, apart from the finding doing mental arithmetic fun bit). I've been reading Secrets of Mental Math: The Mathemagician's Guide to Lightning Calculation and Amazing Mental Math Tricks and the author, Michael Shermer, is just like me: he works from left-to-right.

He, like me, has found this to be a good system because it lets you discard digits early and not hold some enormous calculation in your head. For example, in the calculation 124 + 353 you can immediately say "four hundred" before doing the rest of the sum. This seems to free up headspace (at least for me) and let's me do the rest of the calculation. I'd do it like this: 124 + 353 = 400 + 24 + 53 = 400 + 70 + 4 + 3 = 477.

The books is filled with tricks for doing all sorts of mental calculations, including a nice section about estimation (I've always been an estimator) and working out things like tips and sales taxes. But the most fun part to me was a trick to let you do two-digit squares in your head really, really fast.

Quick, what's 272. Of course, the brute force way to do that is to calculate 27 x 27 which is a bit of a pain because it involves doing something like 27 x 20 + 27 x 7 = 540 + 189 = 729. But there's a much faster way.

Observe that 272 = 30 x 24 + 32. Since you probably know that 32 = 9 this means you have to calculate 30 x 24 + 9 which is relatively easy because the multiplication involves a multiple of ten which means it's really 3 x 24 and then add a zero.

So the rule is that if you want to square number X you first round it to the nearest multiple of 10, called that X + r, and then calculate X - r (i.e. round the same amount in the opposite direction). You calculate (X + r) x (X - r) and add back the square of the amount you rounded by, r2, which will be 1, 4, 9, 16 or 25.

This works because ( X + r ) x ( X - r ) + r2 = X2 - rX + rX - r2 + r2 = X2.

Example: 672 is 70 x 64 + 32 which is fairly easy to do in your head. And naturally the same trick works no matter how many digits you have, it's just that the multiplication gets harder.

The trick is especially impressive/easy with numbers near 100 because the multiplication becomes dead easy. For example, 962 is 100 x 92 + 42 which you (or at least I) can almost instantly see is 9216.

Thursday, March 25, 2010

Goodbye HP Procurve Access Point 420

Over a year ago I discovered a bug in an HP ProCurve Wireless Access Point 420 that we were using in our office. After being treated badly by HP I finally got support from them by blogging my frustration and ending up on the front page of Google search results for procurve support.

Eventually, weeks later, HP acknowledged the problem with the device. But this story doesn't have a happy ending.

In November 2009 HP informed me that the product was being end of lifed.

Out of curiosity I called HP Procurve Support and asked them about the status of my case and they couldn't even look up my case number. Searching around with my name they did manage to find me and my case was active and open. The latest update was on March 19, 2010 and the case had been escalated to Level 3 Support.

My previous experience with HP support wasn't good, but this time Derek was great. He tracked down the ancient case, updated me with information, updated contact information for me. An excellent experience. Unfortunately, waiting over a year for this to be fixed had become intolerable. (If anyone from HP is reading this, email me and I'll tell you Derek's email address since he deserves a special mention).

I replaced it today with an Airport Extreme. Configuration was a breeze with Apple's Airport Utility. And, here's a little known fact, the Airport Extreme can act as a level 2 bridge which means it can successfully extend our existing network without doing NAT and messing up our Bonjour packets (which were the source of the original HP bug).

And, joy of joys, it can perform WPA2 Enterprise authentication against our RADIUS server.

It's nice that HP is working to track down this bug, but 13 months is a little too long to wait for a fix. Sorry.

Wednesday, March 24, 2010

An appeal for unremarkable women

Today across the blogosphere Ada Lovelace will be honoured by writing about her and other remarkable women in technology. But today I prefer to appeal to and for the unremarkable.

At the day job we've been hiring people continuously for a year for programming jobs. My entire team of 12 is all men, and in the last 12 months we've interviewed two women. Looking into all the candidates who've sent in an application, men overwhelm women with a ratio of 24:1. Even if we just hired people randomnly the chance of hiring a woman would be tiny.

I find this situation tragic. We've hired with blindness to the age of our employees or their backgrounds. In a team of 12 there are people who hail from Mexico, Northern Ireland, Greece (via Canada), and Poland. But not a single woman.

And yet I've known women who love these implacable machines as much as I. I've talked to women programmers about the deepest guts of computers, about algorithms and problems to be solved. Many of those women have recognized something that people outside the industry overlook: much of computer programming is a craft.

The people who made the smart phone in your hand, or built the system that waits patiently in your car for an accident to happen so it can fire the airbags in the right order, or made your web-based email and your music player, or built the software on the Voyager 1 spacecraft that keeps it looking longingly back at Earth as it leaves our grasp: those people are incredibly creative.

They have made beautiful things: beautiful things that everybody sees, and beautiful computer structures hidden out of sight but playing a part in all of our lives. They have struggled against constraints of time, space and resources, and they have made. Although software may seem ephemeral because it has no physical form, these people, men and women, have sculpted electricity to form numbers to form programs and make software that touches the physical world.

So, my appeal when thinking about Ada Lovelace is to all the women who, like me, will never be a world famous genius, who will never look like computer engineer Barbie, but, like the men I've worked with in the computer industry, love what they do. Don't think that computers are just for men to program and debug, but come place your fingers on a keyboard and share with me the joy of making.

And, perhaps, one day, I won't remark a woman in a team of computer programmers, because she'll be unremarkable. Or at least I'll only remark her because she's remarkable for her ability, not her missing Y.

Friday, March 19, 2010

Responding to jacquesm's challenge

The other day on Hacker News a user posted an anonymous comment. Regular Hacker News participant jacquesm wanted to unmask the writer and posted a challenge to unmask the user.

He also emailed me because he thought I might be the anonymous user. I agreed to help him with a little bit of text mining. Jacques had a nice database of all Hacker News submissions and comments and gave me a 250Mb SQL file suitable for loading into MySQL. Unfortunately Jacques' database only had comments up until September 2009 so if the user who wrote the anonymous comment has joined recently then it won't be possible to identify them.

I quickly whipped up a naive Bayesian text classifier in Perl (similar to this one that I wrote about in Dr Dobbs five years ago) with one category per Hacker News user. I took the text of the anonymous comment and fed it to the classifier. The classifier used the complete set of comments for each user to train each category and then scored the anonymous comment against those categories.

Culling the list for users who have commented recently the 10 most likely users (according to a text classification) are (in order of likelihood): sh1mmer, stcredzero, gojomo, patio11, andreyf, anigbrowl, teej, physcab, thorax, run4yourlives.

Now none of those users actually commented on the thread in question. Assuming it was someone who was commenting and then switched to a different account the most likely candidates are (again in order): petercooper and jacquesm.

So did this approach work?

PS As a quick test of my classifier I ran it against one of jacquesm's own comments and got the following people in order of likelihood: jacquesm, geoscripting, kunqiana, jerf, mixmax.

Thursday, March 18, 2010

A fascinating little beastie

Back in 2004 I was living in New York and commuting between New York and Washington, DC on the Acela. I was working in a fairly rural part of Virginia and was lucky enough to accidentally experience a once in 17 years event: the emergence of Magicicada Brood X.


(Picture from Wikipedia)

Now I realize that most people probably don't think that being in a place where millions of large winged insects appear from the ground and make a deafening noise in search of a mate is fun. But Magicicada is so cool that it's hard to miss their emergence. And Magicicada loves prime numbers.

For that reason, Magicicada Brood X (which lives along the Eastern coast of the US) is place number 127 in The Geek Atlas. Your next chance to meet the little beastie is 2021.

This particular brood spends 17 years underground living off the sap inside the roots of trees. Once ready to become an adult it burrows upwards and climbs its tree. Then when high up in the tree it molts and spreads its wings.

Once ready to fly it makes a humming sound which, given that millions emerge all at the same time, fills the air with an incredible din. The female cicadas make a clicking sound and between the humming and clicking the males and females find each other and mate.

They live only a few weeks above ground and new baby cicadas fall to the grown and burrow to the roots to start another 17 year wait.

To get a feeling for what it's like to be around when Magicicada makes its appearance, here's Sir David Attenborough:



There are other cicadas that live on a 7 and 13 year cycle. All these cycles are based around a prime number of years. The hypothesis is that the cicada uses a combination of emergence en masse and a prime numbered cycle to avoid predators. A prime numbered cycle means that the cicada rarely meets a predator and mass emergence would overwhelm any predator around.

A prime numbered cycle avoids predators because it doesn't have any factors. Suppose that the cicada had an 18 year cycle, any predator that peaked every 2, 3, 6 or 9 years could synchronize with the cicada and always meet it and eat it. By having a prime numbered cycle the cicada rarely meets a predator and predators have a hard time synchronizing with it.

For a more detailed look at the prime numbered cycle, you can read the paper Prime Number Selection of Cycles in a Predator-Prey Model.

PS If you've enjoyed this, consider buying my book.

Tuesday, March 16, 2010

London Transport Museum: Acton Depot Weekend

This past weekend the London Transport Museum held an open weekend at its Acton Depot where they keep a collection of trams, trolley cards, buses and underground trains, plus all the associated equipment. They only open the depot twice a year so this was a chance to see some things that are rarely open to the public.

I didn't include this museum in The Geek Atlas but after a visit it's likely a candidate for a volume 2 since it is packed with interesting stuff.

Like a really big collection of old underground signs:


Or shielding used while constructing the tunnels for the London Underground:


And speaking of the Underground, here's a power control panel with meters indicating hundreds of amps and some serious on/off switches:


And a lovely mercury arc rectifier used to turn AC into DC (the Underground uses 630V DC power).


And here are the wheels of a 1930s trolley car:


And here's the control panel from an Otis elevator:


But the highlight was a ride on the prototype Routemaster RM-1 bus. I forgot to photograph it, so here's a picture from Wikipedia:

Thursday, March 11, 2010

My bio

Occasionally I get asked for some sort of official bio. Here's one people can use:
John Graham-Cumming is a computer programmer and author. He studied mathematics and computation at Oxford and stayed for a doctorate in computer security. As a programmer he has worked in Silicon Valley and New York, the UK, Germany and France and currently works at CloudFlare. His open source POPFile program won a Jolt Productivity Award in 2004.

He is the author of a travel book for scientists published in 2009 called The Geek Atlas and has written articles for The Times, The Guardian, The Sunday Times, The San Francisco Chronicle, New Scientist and other publications.

He can be found on the web at jgc.org and on Twitter as @jgrahamc.

If you've heard of him at all, it's likely because in 2009 he successfully petitioned the British Government to apologize for the mistreatment of British mathematician Alan Turing.

He is a licensed radio amateur.
Updated February 24, 2012

Wednesday, March 10, 2010

An Olympic honour for Alan Turing

Over at The Guardian I write:

Last year I led a campaign to obtain an apology for the mistreatment of the British mathematician Alan Turing. Turing's prosecution for homosexuality led to the death of a true genius at the age of only 41 in 1954. On 10 September last year, Gordon Brown issued an apology that recognised Turing's stature as one of the greatest Britons. But Britain has a final opportunity to unapologetically recognise Alan Turing in two years' time, at the 2012 Olympics.

Read the rest here.

Tuesday, March 09, 2010

Did Monbiot try to understand climate science?

In The Guardian's Comment is Free section there's an article by George Monbiot called The trouble with trusting complex science which argues that:

The detail of modern science is incomprehensible to almost everyone, which means that we have to take what scientists say on trust.

He does this in the context of climate change science. I wonder if he actually tried to read the key paper that describes why we know that the global temperature is increasing. The paper is Uncertainty estimates in regional and global observed temperature changes: a new dataset from 1850. Go on, read it. I dare you.

The critical thing you need to be able to understand to understand that paper is... how to calculate an average. That's a GCSE level maths subject; here's a quick page to revise that in case you've forgotten how to average.

Because, you see, the entire process described in that paper involves the following steps:

1. Get temperature data (i.e. thermometer readings) at different places around the world for many, many years
2. Work out the average temperature at each location by averaging the values between 1961 and 1990 on a monthly basis. So you end up knowing things like the average January temperature at Heathrow.
3. Now go back and work out how much the temperature for any given month and year deviates from the average: all that means is subtract the average temperature from the observed temperature for the same month. Now you know how 'different' the temperature is. This is called the anomaly. If it's getting hotter the anomalies will get bigger.
4. Divide the globe up into squares 5 degrees on each side. Find all the thermometers inside each square, find their anomalies for each month and year. Average them to get an average anomaly for that square.
5. Take all the squares in the northern hemisphere, average their anomalies for each month and year. Draw a graph showing the temperature changing. Repeat the for southern hemisphere.
6. Now take the northern and southern hemisphere temperatures for each month and year and average them to get a global temperature anomaly chart.

Child's play? Yes.

I'll admit that the rest of the paper has some harder concepts (standard deviation, anyone?). But I'll wager that the real reason that people don't understand science is not because it's too hard to understand, but because they aren't motivated.

Yes, there are parts of science that require a lot of knowledge, but covering your eyes and not trying to understand is likely where many people go wrong.

Or to put it Monbiot's way:

My heart rebels against this project: I would rather be pelting scientists with eggs than trying to understand their datasets.

Wednesday, March 03, 2010

A welcome bunch of amateurs

Here's me writing in The Guardian's Comment is Free section:

We're all the children of amateurs: amateur parents. There's no government department that will certify you as a parent (thankfully), nor a university department where you get your PhD in being a daddy, nor a professional body ready to strike you off for not following mothering standards. But any parent who's held a newborn child in their arms has unconsciously taken the amateur's oath: "I may not be a professional, but I'm going to do whatever it takes to act like one."

You can read the rest here.

Monday, March 01, 2010

The reason I managed to find errors in the Met Office data and code

It turns out the reason is rather simple. In the evidence being given today to the Parliamentary Science and Technology Committee the Met Office says of their quality control procedures:

Manual inspection, including real-time quality control using GIS software; quality control described in literature for the various regional studies.

Contrast that with what they say about NOAA's procedures for the same sort of data:

A long series of automatic quality control tests based on both statistics and physics (e.g., outlier tests, identical values two months in row, etc.)

Given the amount of data it's unsurprising that 'manual inspection' isn't enough.

Update There's a lovely bit of evidence from Professor Darrel Ince about software quality that I wholeheartedly agree with.