Skip to main content

Posts

Showing posts from 2008

POPFile v1.1.0 Released

There's a new POPFile out (v1.1.0) and I had almost nothing to do with it. This is the first release where the new (global) POPFile Core Team did all the work. Thanks Brian in the UK, Joseph in the US, Manni in Germany and Naoki in Japan. A truly global effort.

As part of the v1.1.0 release POPFile has moved from SourceForge to its own server and has a totally new web site.

v1.1.0 also includes some great new features: it is the first to use a
SQLite 3.x database and it is the first to offer a Mac OS X installer in addition
to the usual cross-platform and Windows installer versions.

And there are a raft of bug fixes as well which you can read about in the release notes.

Spaces are a pain in painless non-recursive Make

In my book GNU Make Unleashed I published a pattern for doing Make without having a recursive descent into directories. It works well and I know that many people are using it.

But the other day I received an email from Terry V. Bush at VMWare saying that he had trouble with it because of 'the third-party problem'. The third-party problem is my name for the problem that occurs when your beautifully written Make system has to incorporate some wart of source code from a third-party vendor. In Terry's case that third-party has spaces in the path names.

Space is path names are a real bind in Make (that's another topic I cover in GNU Make Unleashed) and Terry really wanted to use my non-recursive Make pattern but needed to handle this ugly third-party.

I'll let him continue the story...

What happens is that if you have a directory name with a space in it your functions fail to find the root. Also, they always walk the entire tree up to the top even after they have found …

The Ultimate Nerd Honeymoon

There's been a major gap in this blog because I'm in the midst of writing a book for O'Reilly. As part of the research on the book I came across the ultimate nerd honeymoon.

In 1812, Sir Humphry Davy, the British chemist and inventor who is best remembered today for Davy Lamp used in mines, married.

In October 1813 Davy and his wife set off on a honeymoon across Europe. First stop was Paris to pick up a medal from Napoleon. But Davy needed a valet to help out, so he took Michael Faraday with him. That way Davy and Faraday could perform experiments along the way.

While in Paris they got together with Joseph Louis Gay-Lussac (of Gay-Lussac's Law) and showed that iodine was an element. And André-Marie Ampère stopped by for a chat.

Off they went to Italy to hang out with Alessandro Volta and also did an experiment setting fire to a diamond using the sun's rays and demonstrated that a diamond is made of carbon.

The honeymoon lasted 18 months.

Advice to a young programmer

I received a mail from an acquaintance who'd come to the realization that his 13-year-old wanted to be programmer, specifically a games programmer. Here's the advice I gave. Perhaps others have things to add:

1. I'm tempted to tell you that the right way to learn to be a programmer is to start with LISP, or the lambda calculus, or even denotational semantics but you can come back to those after a few years getting your feet wet.

2. Lots of programming involves logic (or at least thinking logically) so learning about and enjoying logic is probably a good foundation. You could start by learning about boolean algebra since it's simple and fun and the basis for a lot of what computers do.

3. Since games programmer involves a lot of physics, you should also learn about Newton's Three Laws and Universal Gravitation and play around with things like springs and pendulums.

4. Basic trigonmetry is important to the games programmer. It'll be handy to know about Pythagoras

The Colarie: A new way of measuring calorie intake

Recommended daily energy intake for a man is generally considered to be roughly 2,500 Calories (or kilocalories: 1 Calorie = 1,000 calories) and for a woman it's 2,000. The problem with those figures is that they are rather abstract. If you are trying to count your energy intake it would be much easier to deal with something smaller and easier to understand.

Hence my idea for the Colarie.

1 Colarie is the number of Calories in a single can of non-diet Coca Cola. It's easy to appreciate that a single can of Coke isn't very good for you and so comparing a food stuff to a can of Coke is an easy measure of whether you are eating something that's got too much fat or sugar in it.

The actual Calorie count for a Coke can varies by country. In France there are 139 Calories in a can, in the US there are 155. So I've settled on 147 as a good measure. So 1 Colarie = 147 Calories.

That means a man needs to consume the equivalent of 17 cans of Coke per day; for a woman it&#…

GNU Make Unleashed release

For 4 years I've written the Ask Mr Make column over at CM Crossroads (and I continue to write it). Since there's been great interest in the column, I've put together all 4 years of columns plus additional unpublished material as a book and ebook.

All the material has been rechecked for accuracy, errata have been incorporated and the text re-edited. The result is a 230 page book covering everything from basics of GNU Make to advanced topics like eliminating recursive make, doing arithmetic in GNU Make or dealing with spaces in file names.





The book contains 43 separate articles about GNU Make, plus a complete reference to the GNU Make Standard Library.

You can buy a copy in either form here.

A big thank you to everyone who's commented, emailed, or made suggestions on my GNU Make articles over the years.

POPFile v1.0.1 released plus a glimpse of the future

POPFile v1.0.1 was released today; this is the first ever POPFile release that I didn't do. POPFile is now being managed by a core team of developers: Manni Heumann (in Germany), Brian Smith (in the UK), me (in France), Joseph Connors (in the US) and Naoki Iimura (in Japan). A truly international effort. The actual release binaries were built by Brian Smith who, for a long time, has been the installer guru.

This release contains minor feature improvements and a number of bug fixes. Some of the bugs fixes were for annoying bugs that showed up only occasionally: that makes it a worthwhile upgrade.

Since I pulled back from being involved in every detail of POPFile's evolution the core team has been liberated to work on the project. v1.0.1 is their first release, and it is minor, but much greater things are coming:

1. A native Mac installer

2. A SOHO version of POPFile. Some time ago I did most, but not all, of the work to make a multi-user version of POPFile. That work is be…

Breaking the Fermilab Code

A story appeared on Slashdot about a mysterious fax received at Fermilab written in an unknown code. The full story is here. I looked at it and immediately noticed a few things:

1. The first part looked like ternary (base 3) with digits 1 (|), 2(||) and 3(|||).

2. The last part looked like binary with digits 1(|) and 2(||)

3. The middle bit looked like either a weird substitution code, or I wondered if it might be machine code.

4. In the last part the digit 2 (||) never occurs more than once, perhaps it was actually a separator and the last part is not binary.

The first step was to convert the bars into numbers. Here's a copy of my marked up print out:



The first part has the numbers (or at least I thought):

323233331112132
333231322123312
111331132312233
333212123213113
311333313331111
211333323232211
232313331121231
33231312

Noticing this had 113 digits (which is a prime number) I went off on a wild goose chase around primes, and then around the interpretation of this number in hexadecimal …

Multi-route (email and phone) self-aware phishing

Today, I received the following email:

This communication was sent to safeguard your account against any
unauthorized activity.

Max Federal Credit Union is aware of new phishing e-mails
that are circulating. These e-mails request consumers to click
a link due to a compromise of a credit card account.

You should not respond to this message.

For your security we have deactivate your card.

How to activate your card

Call +1 (800)-xxx-9629

Our automated system allows you to quickly activate your card

Card activation will take approximately one minute to complete.


Of course, I don't have an account with Max Federal Credit Union and this is obviously a phish. Notice that the English is quite right:

"For your security we have deactivate your card." and "You should not respond to this message." doesn't make sense in context.

What's more interesting is that the message itself warns you about phishing emails and asks you to call an 800 number.

If you call the 800 number an ele…

Building a temperature probe for the OLPC XO-1 laptop

I bought an OLPC XO-1 laptop through the G1G1 program and was intrigued to discover the Measure activity.

The measure activity uses the internal audio system to measure a value input on the microphone socket. With nothing connected this application reads the value of the internal microphone and displays a waveform. You can have fun just by whistling, speaking or singing with Measure running.

But since you can measure a voltage input into the microphone socket, it's possible to build sensors and connect them to th OLPC XO-1. On the Measure web site they mention building a simple temperature sensor using an LM35 temperature sensor that looks like this:

The LM35 can measure a temperature between 0 and 155 Celsius just by hooking it up to a 5v supply. It outputs 10mv per degree so a temperature of 20 Celsius corresponds to 0.200v.

Since the OLPC XO-1 has a USB port it's possible to get 5v from the laptop by hacking a USB connector, and connect 5v to the LM35 and then take the si…

The sum of the first n odd numbers is always a square

I was staring at the checked pattern on the back of an airline seat the other day when I suddenly saw that the sum of the first n odd numbers is always a square. For example,

1
1 + 3 = 4
1 + 3 + 5 = 9
1 + 3 + 5 + 7 = 16

And, of course, it occurred to me that it would be nice to be able to prove it. There are lots of ways to do that. Firstly, this is just the sum of an arithmetic progression starting at a = 1 with a difference of d = 2. So the standard formula gives us:

sum_odd(n) = n(2a + (n-1)d)/2
= n(2 + (n-1)2)/2
= n(1 + n - 1)
= n^2

So, the sum of the first n odd numbers is n^2.

But using standard formulae is annoying, so how about trying a little induction.

sum_odd(1) = 1

sum_odd(n+1) = sum_odd(n) + (2n + 1)
= n^2 + 2n + 1
= (n+1)^2

But back to the airline seat. Here's what I saw (I added the numbering, Lufthansa isn't kind enough to do that for you :-):



The other thing I noticed was this:



You can view the square as the sum of…

Tonight, I'm going to write myself an Aston Martin

This is the story of my attempt to 'cheat' in an on-line spot-the-ball competition to win an Aston Martin. It's also the story of my failure, but you get free source code that implements automatic detection of image alteration using copy/paste or tools like the Clone Tool in Photoshop.

First, take a look at this photo:



Notice anything strange? In fact this image has been tampered with to cover up a truck. The truck is completely hidden by foliage. Here's the original:



Wouldn't it be nice to be able to detect that automatically? It is possible. Here's an image automatically generated by my code showing what was moved. All of the red was moved to the blue (or the other way around).



I was motivated to work on this program by greed (or at least my never-ending love of having a little flutter on things). Best of the Best runs spot-the-ball competitions in airports to win very expensive cars. But they also run the same competition online. That meant I could ge…

Interface to SQLite database in 23 lines of Arc

One thing that the first release of Arc was missing was access to any sort of database, but that's easily remedied. Here are 23 lines of Arc code that provide access to a SQLite database:

(= db! 'nil)

(def db+ (name (o host "localhost") (o port 49153))
(let (i o) (connect-socket host port)
(db> o name)
(if (db< i) (list i o))))

(def sql ((i o) q)
(db> o q)
(if (db< i) (readall i 200)))

(def db- (db)
(map close db))

(def db> (o s)
(write s o)
(writec #\return o)
(writec #\newline o)
(flush-socket o))

(def db< (i)
(= db! (read i))
(iso db! 200))

The three functions you need to care about are db+ (get a connection to a named SQLite database), db- (close a connection to a database) and sql (execute a SQL query and return a list (or lists) of rows. There's also db! which contains the status of the last command (200 for OK, or 500 followed by a string explaining the error).

Here's a little Arc session creating a database, putting some data in it an…

My first Arc project: a simple Wiki

The only way to learn a programming language is to write something in it. So, I decided it was time to dig into Arc and my first project is a very simple Wiki.

Here's the source (wiki.arc):

; A wiki written in Arc (arc0)
;
; Copyright (c) 2008 John Graham-Cumming
;
; (load "wiki.arc")
; (wsv)
;
; Then go to http://localhost:8080/show

(load "web.arc")
(load "util.arc")

(= pagedir* "wiki/")

(def histfiles (page)
(sort > (map [coerce _ 'int] (rem [is "current" _] (dir (pagepath page))))))

(def nexthist (page)
(let h (histfiles page)
(if h (++ (car h)) 0)))

(def pagepath (page)
(string pagedir* (page 0) "/" (page 0) (page 1) "/" page ))

(def pagefile (page (o file))
(string (pagepath page) "/" (or file "current")))

(def slurp (page (o file))
(if
(let p (pagefile page file)
(if (file-exists p) (readfile p)))))

(def upperlen (word)
(len (keep upper word)))

(def is-wikilink (word)
(if (alphas wo…

The Arc Challenge Explained

When I first looked at the Arc Challenge code my reaction, like that of many people, was WTH? Here's the code:

(defop said req
(aform [w/link (pr "you said: " (arg _ "foo"))
(pr "click here")]
(input "foo")
(submit)))

Within the context of the Arc web/app server this creates a page called /said which has a form on it:

<form method=post action="x">
<input type=hidden name="fnid" value="JtCw8ju328">
<input type=text name="foo" value="" size=10>
<input type=submit value="submit">
</form>

That form accepts a single parameter called foo and redirects to /x.

When clicking submit the user is taken to a page with a single link on it:

<a href="x?fnid=bHJpJ5G1DH">click here</a>

Following that link brings up a page showing what you typed in the first; here's the output when I typed hello in the form:

you said: hello

So, ho…

Proof that the sum of the squares of the first n whole numbers is n^3/3 + n^2/2 + n/6

A recent thread on Hacker News that I started with a flippant comment turned into a little mathematical puzzle.

What's the sum of the square of the first n whole numbers?

It's well known that the sum of the first n whole numbers is n(n+1)/2. But what's the value of sum(i=1..n) n^2? (I'll call this number S for the remainder of this post).

It turns out that it's easy to prove that S = n^3/3 + n^2/2 + n/6 by induction. But how is the formula derived? To help with reasoning here's a little picture of the first 4 squares stacked up one on top of the other:



If we fill in the blank squares to make a rectangle we have the basis of a derivation of the formula:



Looking at the formerly blank squares (that I've numbered to assist with the thinking) we can see that the columns have 1 then 1+2 then 1+2+3 and finally 1+2+3+4 squares. Thus the columns are sums of consecutive whole numbers (for which we already have the n(n+1)/2 formula.

Now the total rectangle is n+1 squ…

Another use of POPFile: detecting weakly encrypted email

Almost all users use POPFile as a spam filter, most of them also use the fact that POPFile can sort in arbitrary categories of mail. However, some people have pushed POPFile even further... Martin Overton (of IBM) has used POPFile to discover email borne malware, even finding that POPFile could automatically detect mutations. Now, some researchers in Japan have used POPFile to detect weak encryption of email with 80% accuracy.

The researchers were building a system to detect improper sending of personal information by email. Their system first checked for the use of strong encryption (if the mail is strongly encrypted then there's no need to worry about eavesdropping), the system also checked for things like telephone numbers, email addresses and other personal data in non-encrypted mail.

But they also wanted a system to detect poor encryption (such as ROT-13), and for that they turned to POPFile. After a mere 30 emails had been trained in POPFile it was able to distinguish pl…