Monday, March 31, 2008

Multi-route (email and phone) self-aware phishing

Today, I received the following email:

This communication was sent to safeguard your account against any
unauthorized activity.

Max Federal Credit Union is aware of new phishing e-mails
that are circulating. These e-mails request consumers to click
a link due to a compromise of a credit card account.

You should not respond to this message.

For your security we have deactivate your card.

How to activate your card

Call +1 (800)-xxx-9629

Our automated system allows you to quickly activate your card

Card activation will take approximately one minute to complete.

Of course, I don't have an account with Max Federal Credit Union and this is obviously a phish. Notice that the English is quite right:

"For your security we have deactivate your card." and "You should not respond to this message." doesn't make sense in context.

What's more interesting is that the message itself warns you about phishing emails and asks you to call an 800 number.

If you call the 800 number an electronic voice reminds you again to never give your PIN, password or SSN in email and then proceeds to ask you for the card number, PIN, expiry date and CVV2. The assumption is that you've been warned twice not to do something in email, so it's OK by phone.

It's painful to see the phisher use the existence of phishing as a way to phish.

Names: Boys vs. Girls

Using data from the 1990 US Census I was amazed to discover that 90% of the US male population has one of 1,219 first names, but 90% of the female population has one of 4,275. There are 3.5x as many female first names as male first names.

The top 10 male first names are: James, John, Robert, Michael, William, David, Richard, Charles, Joseph and Thomas (which account for 23.2% of the male population; 50% of the population have one of only 60 names).

The top 10 female first names are: Mary, Patricia, Linda, Barbara, Elizabeth, Maria, Susan, Margaret and Dorothy (which account for 10.7% of the female population; 50% of the population have on of 139 names).

You can also see that all the variety in names happens between the 80% and 90%. For males 80% of the population is covered by 27% of the names; for females 80% is covered by 19% of the names).

The large numbers of female names appears to be because there are lots of variants of female names compared to male names. A quick run through calculating the Levenshtein distance between names and selecting the 10 closest for each gives an average distance of:

Male: 2.62
Female: 2.01

So female names are more 'similar' than male names, hence the variety created by all these variants.

The other thing we can extract from this data is the prevalence of names beginning with certain letters and weight adjust based on the occurrence of each name.

Things are much more polarized when you look at trailing letters (for example, the trailing letter A is an almost sure sign that it's a woman; the opposite is true of D):

So combining the two it's possible to give a 'maleness' score (the blue part) to each final letter:

BOUTS: The Complete Song Parodies

Back when my site UseTheSource was a "blog" (this was late 90s/early naughties) I wrote a number of song parodies. Someone emailed me and asked where they were.

So, here are my complete song parodies from back in 2001:

March 7, 2001: "Candle in the Wind"

Yahoo! was in trouble with banner ad sales falling, profits disappearing and the then CEO, Tim Koogle, was off to spend more time with his family.

Goodbye Tim Koogle
Though I never Yahoo! at all
You had the faith, to sell banners
While those around you failed
They crashed into the deadpool
And they whispered into your brain
You need to begin to charge
For things you give for free

And it seems to me you lived your life
Like a candle in the wind
Never knowing what to add next
To your list of links
And I would have liked to have told you
But I was just a geek
Your web brand burned out long before
Your stock price ever did

Jerry Yang was tough
The toughest boss you ever had
Softbank created a superstar
And pain was the price you paid
As the whole web died
Oh CNET still hounded you
All they had to say
Was that Google was the site to use

[Repeat chorus]

Goodbye Tim Koogle
From the young man on the DSL link
Who sees you as something more successful
More than just our long lost CEO

[Repeat chorus]

March 22, 2001: "Don't Cry for Me Argentina"

Steve Jobs was back at Apple, the blue iMac was out, Microsoft was in big anti-trust trouble and had just invested $100m in Apple, Steve had bought out NeXT. But the future wasn't yet assured:

This won't be easy,
you'll think it's strange.
When we try to explain what we need
that we now need your help
after all that we've said.

You won't believe us.
All you'll see is Apple you once knew,
although we've crashed down in the dumps
begging for Microsoft cash.

It didn't have to happen.
We should have won
Better software and patents than Bill
Looking down on Windows,
staying far from Sun.
So we chose NeXT.
Running aground, trying computers in blue.
But nothing revived us at all.
You never expected it too.

Don't cry for us, William H. Gates.
The truth is we're dead without you.
We need your dollars
We need Mac Office
You need a rival, for your survival.

And as for Fortune,
and as for Time,
We never invited them in
though it appeared, to the world,
they were all Steve desired.
Even Adobe,
they're making solutions for Windows right now
The answer was here all the time.
We need you, and hope you need us.


Have we begged too much?
There's nothing more we can think of to say to you.
But all you have to do,
is look at us to know,
we're through without you.

April 1, 2001: "I Just Called To Say I Love You"

In the midst of the crash, .coms were going out of business like crazy:

No IPO to celebrate
No friends and family stocks and shares to give away
No big opening
No first day ping
In fact here's just another ordinary day

No Aeron chair
No onsite chef
No working Saturday until the site is done
But what it is, is something blue
Made up of these few words that I must say to you

We just failed to get more funding
We just failed to keep our doors open
We just failed to get more funding
And we need it just to avoid bankruptcy

No free massage
No free soda
No caffeine trip to keep us working every night
No dry cleaning
No stock option
Not even time for us to pack our things and leave

No beanbag room
No Maui trip
No giving thanks to all that NASDAQ did for us
But what it is, though old so new
Grab what you can before your jobs right here are through



April 6, 2001: "Uptown Girl"

Ah, to be in love with a marketing .com girl:

.com girl
She's been living in her .com world
I bet she never had a software guy
I bet her mama never told her why
I'm gonna try for a .com girl
She's been living in her wide web world
As long as anyone in marketing can
And now she's looking for a comp. sci. man
That's what I am

And when she knows what
She wants from her time
And when she wakes up
And makes up her mind

She'll see I'm not a nerd
Just because
I'm in love with a .com girl
You know I've seen her in her online world
She's getting tired of her high tech toys
And all her presents from her VC boys
She's got a choice
.com girl
You know I can't afford to buy her a Porsche
But maybe someday when my stock cashes in
She'll understand what kind of guy I've been
And then I'll win

And when she's walking
She's using her Nokia
And when she's talking
She'll say that she's mine

She'll say I'm not a nerd
Just because
I'm in love
With a .com girl
She's been living in her latte world
As long as anyone in marketing can
And now she's looking for a comp. sci. man
That's what I am
.com girl
She's my .com girl
You know I'm in love
With a .com girl

April 13, 2001: "Gangsta's Paradise"

Linus Torvalds was the flavor of the day as one of the thorn's in Microsoft's side:

As I drive through the Valley of the Silicon Dream
I take a look at my life and realize there's nothing left
'Cause I've been coding and debuggin' so long
That even my manager thinks that my mind has gone
But I ain't never crossed a man that didn't deserve it
Linus treated like a punk, ya know that's unheard of
Ya better watch how ya postin'
And what ya codin'
Or you Dr Tanenbaum'll be lined in chalk
I really hate Minix and FreeBSD
As they croak, I see myself in the pistol smoke
Fool, I'm the kinda hacker script kiddies wanna be like
On the Net in the night, writin' layers of the core code

Been spending most our lives living in a Windows paradise
Been spending most our lives living in a Windows paradise
Keep spending most our lives living in Bill Gates' paradise
Keep spending most our lives living in Bill Gates' paradise

Look at the situation they got me facing
I can't live a normal life, I was raised on the PC
So I gotta be down with the kernel team
Too much crazy Usenet posting got me chasing dreams
I'm a educated fool with Posix on my mind
Speak Swedish in my home and English on the phone
I'm a loc'd out hacker, wrote my life story
And my homies is down so don't arouse my anger
Fool, death ain't nothin' but a heart beat away
I'm livin' life, do or die, what can I say?
I'm 28 now, but will I ever see 29?
The way things is going, I don't know

Tell me why are we so blind to see
That Microsoft's a monopoly


Tell me why are we so blind to see
That Microsoft's a monopoly

Tell me why are we so blind to see
That Microsoft's a monopoly

April 18, 2001: "Copacabana"

Carly Fiorina was fighting for her life as she tried to merge HP and Compaq with Walter Hewlett attempting a proxy fight to stop her in the name of the family:

Her name was Carly, she was a VP
With Lucent and AT&T and a degree from MIT
She went to HP and wowed the board room
And while she tried to be a star, sometimes went a bit too far
And then September 4, Compaq became the score
They were failing and needed each other
Leaning drunks galore!

At the HP, HP/Com-pa-q
The merger that upset the family
At the HP, HP/Com-pa-q
David and William were always the fashion
At the HP... they ran the show

His name was Walter, his dad was famous
He wasn't present for the board, but he wouldn't be ignored
And what she pro-posed, "Dad would've hated"
Then Walter went a bit too far, "Carly: time for au revoir!"
And then the insults flew and careers were smashed in two
There were ads and a lot of bankers, but just who screwed who?

At the HP, HP/Com-pa-q
The merger that upset the family
At the HP, HP/Com-pa-q
David and Walter are today the fashion
At the HP... they run the show

Her name is Carly, she was CEO
But that was 30 weeks ago, when she used to run show
Now she's a VC, but that's our Carly
Still in the suit she used to wear, new blonde highlights in her hair
She sits there so refined, and drinks to Walter's health
She lost her job and she lost the proxy, now she enjoy's her wealth!

At the HP, well just the HP
The toughest job belongs to Walter
At the HP, well just the HP
William and David were always the fashion
At the HP, don't buy the stock...

If you find these funny and can sing... feel free to set them to music and give me a laugh.

Saturday, March 29, 2008

More 11:11 mystical nuttery

Out of the blue I received an email about my post the other day about Benford's Law and 11:11:

every time i look at the clock the number add up to 11.
how does that get explained

OK. Well, it turns out that that's pretty simple to explain: the sum 11 is the most common sum you'll see on a clock. The following graph shows the count for each sum of digits. You'll see that for a 12 hour clock the peak is at 11 and for a 24 hour clock the peak is at 12 with 11 being a close second.

For a 12 hour clock the probability that the sum of digits will be 128/1440 (or about 9% of the time). For a 24 hour clock it's 124/1440 (or about 9% of the time). So it's unsurprising that 11 comes up a lot here.

Another area of 11 craziness is airline seating. This is probably because people get freaked out by flying and look for patterns. Suppose you sit in economy on a British Airways long haul flight. You'll be sitting in a 747, 767 or 777. You then take your seat number add up the digits and then add the letter on using its place in the alphabet (e.g. sitting in 14F then you have 1 + 4 + 6 = 11). Using the British Airways seat maps you can compute the value of for each seat in economy:

On a 767 11 is the most frequently occurring sum, on a 747 it's 10 (with 11 close behind) and on a 777 11 is just beaten out by 12.

Tuesday, March 25, 2008

"Retiring" from anti-spam

Today, I'm "retiring" from anti-spam work. Practically, that means the following:
  • No more updates to The Spammers' Compendium or Anti-spam Tool League Table pages. These remain on line, but are not being maintained.
  • I'm looking for a new leader for the POPFile project.
  • I'm no longer active on any anti-spam mailing lists.
  • I am leaving all anti-spam conference committees.
  • My anti-spam newsletter is no longer being published.
I will, however, be continuing with commercial anti-spam work where I have agreements currently in place with customers. No change to their support, terms or assistance.

The obvious question is why? For me, the interest just isn't there. The battle against spam continues but is now about trench warfare rather than creating new weapons. We'll continue to see innovation, but for any hacker it's the new, new thing that's important. For me, spam is yesterday's news. Watching companies squabble and refuse to cooperate, seeing a decline in quality at anti-spam conferences, and major companies essentially killing their consumer anti-spam means anti-spam just isn't where I want to be.

Of course, there are many really good people fighting spam out there. This post isn't meant to demean them.

Thank you to everyone who has supported what I've done over the last 7 years, and good luck!

Saturday, March 22, 2008

Building a temperature probe for the OLPC XO-1 laptop

I bought an OLPC XO-1 laptop through the G1G1 program and was intrigued to discover the Measure activity.

The measure activity uses the internal audio system to measure a value input on the microphone socket. With nothing connected this application reads the value of the internal microphone and displays a waveform. You can have fun just by whistling, speaking or singing with Measure running.

But since you can measure a voltage input into the microphone socket, it's possible to build sensors and connect them to th OLPC XO-1. On the Measure web site they mention building a simple temperature sensor using an LM35 temperature sensor that looks like this:

The LM35 can measure a temperature between 0 and 155 Celsius just by hooking it up to a 5v supply. It outputs 10mv per degree so a temperature of 20 Celsius corresponds to 0.200v.

Since the OLPC XO-1 has a USB port it's possible to get 5v from the laptop by hacking a USB connector, and connect 5v to the LM35 and then take the signal coming from the LM35 (the middle pin) and connect it to the microphone socket.

I did this by building two parts: a generic adapter which gives me 5v and a signal line out of a standard stereo 3.5mm jack:

The stereo jack is wired up so that the tip is +5v, the base is Gnd and the middle is the signal going to the microphone socket. The USB plug has only two wires connected (for +5v and Gnd), and the jack going to the microphone socket (which is mono) has the connected to the middle of the stereo jack, and the base is Gnd. All the grounds are joined together.

When plugged into the OLPC XO-1 it creates a generic connector for any other projects I might work on:

For the temperature sensor I simply connected the LM35 to a stereo socket with the correct connections to match up with the stereo jack plug. Then I created a probe with an old plastic pen and some waterproofing compound (so that I can do things like shove the probe in a cup of coffee without wetting the contacts on the LM35). Here it is:

Connect the two together and run the standard Measure activity and you can start to look at the output of the sensor and hence the temperature.

But there's a problem. The microphone input can only handle voltages in the range 0.3v to 1.9v (and my measurements of my OLPC XO-1 show this range to actually be 0.4v to 1.9v). So that means as is the probe can be used to measure temperatures in the range 40 Celsius to 155 Celsius. That low end is a bit high for the sorts of experimentation you can do at home (e.g. measure the temperature in the fridge, or a glass of cold water, or even the temperature inside your mouth).

So we need to scale the voltages coming from the sensor to fit better into the range that's readable by the laptop. The standard way to do that is with an operational amplifier which is used to add two voltages together: the voltage coming from the sensor and a reference voltage. Doing this will move the voltage up.

For that I used the LM1458 which in a single 8 pin package contains a pair of operational amplifiers.

Here's the circuit diagram:

The circuit has three parts: a voltage divider, a summing amplifier and an inverting amplifier.

Voltage divider: the reference voltage is created by taking the 5v available from the USB port and passing it through resistors R8 and R9. The voltage at the middle point of these two resistors is determined by the standard formula for a voltage divider of 5v * R9/(R8 + R9) = 5v * 1 / ( 10 + 1 ) = 0.45v. In my actual circuit with 1% tolerance resistors the measured voltage was 0.41v.

Summing amplifier: the middle portion of the circuit takes the two inputs and adds them together (and because of the nature of the circuit inverts the summed value). So its output going into R7 is -ve the sum of the reference voltage and the sensor voltage.

Inverting amplifier: the final part just inverts the voltage so that the output is +ve and in the range that the OLPC XO-1 can read.

One complexity is that this circuit requires +9v, Gnd and -9v to operate. I obtain that with a pair of 9v batteries linked together giving Gnd where the two are connected. Here's the final circuit with appropriate connectors to hook up to my existing probe and laptop adapter:

And here's what it looks like when it's all hooked together:

Now, this wouldn't be any fun without a bit of software and since the Measure activity can only display the voltage being presented (which is now a mixture of the sensor voltage and the reference voltage) what's needed as a new activity.

I found the developer documentation to be very hard to follow and I ended up hacking the existing Measure activity and renaming it Temperature.

The critical code is in the file where it reads self.avg (the value coming from the microphone input via the ADC) and scale it for display. I measured voltages coming from my probe for a couple of known temperatures and worked out a scale factor (The +32768 is because the self.avg ranges from -32768 to 32767):

layout.set_text("Temperature: %.1f C" % (0.00221833*(self.avg+32768)) )

Here's a screenshot of Temperature running on the laptop and measuring the ambient temperature in my office:

You can download my Temperature activity using the browser on your OLPC XO-1 to install it.

Thursday, March 20, 2008

Sleeping with the enemy

I loaded the dish washer:

My SO loaded the dish washer:

Who needs help?

Tuesday, March 11, 2008

First assume all new email is useless

When I download email none of it goes in my Inbox. In fact, I don't have an Inbox. I work on the assumption that all new email is useless.

Many reports tell us that between 80% and 90% of all email is spam, so for starters only 10% to 20% is at all likely to be useful. Then, if you account for being on mailing lists, being CC:ed needlessly and receiving automatic updates such as order confirmations from, you'll see that almost all email is useless. Only a tiny fraction of the mail you receive is useful. And by useful I mean requiring action.

I use Thunderbird and my email folder structure looks like this:

When email arrives it is automatically sorted using POPFile into the folders: Family, GNU Make, Misc, polymail, POPFile and Spam. These six folders are the categories of mail that I receive:

  • POPFile: Since I wrote POPFile I get lots of mail about it and I use this is a general box for other open source projects I work on and anything else about anti-spam
  • polymail: Anything to do with my commercial product polymail and my consulting business
  • GNU Make: Anything to do with GNU Make or the company, Electric Cloud, that I co-founded
  • Family: Anything from my family
  • Misc: Order confirmations, airline tickets, PayPal statements, etc.
  • Spam: spam

POPFile uses Naive Bayesian text classification to automatically sort my email (with just a point and click interface for training) and then six rules (which never need updating) move the incoming mail based on POPFile's classification to one of those folders.

Of course, POPFile can be used to sort mail in any way you choose: my categories are unlikely to be yours. You might use POPFile to sort Work from Home from Spam. At least one journalist I know uses POPFile to sort Interesting from Boring from Spam so that he only gets to read interesting press releases.

When I identify mail that does need action taken I move it to the ACTION folder (which is the closest I've got to an Inbox). Moving mail there is a snap because I use the QuickMove extension for Thunderbird and have ALT-number keys mapped to each folder: one key press and the message is moved into or out of ACTION.

To keep on top of things I publish the number of items in my ACTION folder on my web site. Here's a live view over the last 24 hours. Currently, 9 items need dealing with.

My rules for managing email:

  1. Assume that all new email is useless
  2. Automatically sort email into folders on delivery
  3. Take control of your Inbox: only you put email in it

Monday, March 10, 2008

Why Rails rules: continuous forward motion

Lately I've been playing with Ruby on Rails and I'm impressed. Not by the documentation (I was pulling my hair out trying to map my copy of the Rails book that deals with 1.x to Rails 2.0 installed on my machine). Not by the screencasts, or by DHH being arrogant.

I'm impressed by the fact that Rails keeps you (or at least me) in continuous forward motion.

Yesterday sitting in an airport I decided to learn Rails. I had the two books (one on Ruby, one on Rails) which I'd read before, but I'd never actually coded anything. I had an idea for an application that was CRUD worthy.

Tonight, after a total of 4 hours of programming I have a working application in Rails that allows me to track health care expenses (appointments, bills, insurance reimbursements, payments, ...). Zero knowledge to working application in four hours isn't meant to illustrate my genius, it illustrates that Rails/Ruby is easy to learn and that the combination of generators and scaffolding keep you moving.

I've noticed in the past when working on apps that I'll come up against a difficult bit and go work on something easier ("Oh, I don't want to come up with foo-bar algorithm right now, I'll go design the buttons"). And the easier things are lower value.

Rails keeps me going after the functionality because it puts in place most of the functionality and then lets me evolve it. My application looks horrible (I've wasted no time on the CSS or HTML), but the functionality is there.

Some sleep and a little design work and it'll look like something.

Anyone else like access to a free application for health care expense management?

Monday, March 03, 2008

To the idiotic spammer posting comment spam on this site

Since your name is two Chinese characters I'm going to address you as "Dude".


Lately you've been posting comment spam on my blog for your World of Warcraft Gold. This is a little silly:

1. I'm fairly well known in anti-spam circles, did you really think I was going to let comment spam through on this site?

2. Comment moderation is turned on on this site. So your comment spam goes nowhere when I click the Discard button.

3. There has been a little some collateral damage from your World of Warcraft spamming. I accidentally killed two comments by Hypermechanic and I can't retrieve them. He/she wanted to say something useful about an old post:

You could do that like .
I guess in JAVA or .NET.


Cool I will hunt for it… This is a very sweet look app you have here. Even though you down play your role this is still brilliant.

Thank you for something new and useful.