POPFile v1.0.1 released plus a glimpse of the future

POPFile v1.0.1 was released today; this is the first ever POPFile release that I didn't do. POPFile is now being managed by a core team of developers: Manni Heumann (in Germany), Brian Smith (in the UK), me (in France), Joseph Connors (in the US) and Naoki Iimura (in Japan). A truly international effort. The actual release binaries were built by Brian Smith who, for a long time, has been the installer guru.

This release contains minor feature improvements and a number of bug fixes. Some of the bugs fixes were for annoying bugs that showed up only occasionally: that makes it a worthwhile upgrade.

Since I pulled back from being involved in every detail of POPFile's evolution the core team has been liberated to work on the project. v1.0.1 is their first release, and it is minor, but much greater things are coming:

1. A native Mac installer

2. A SOHO version of POPFile. Some time ago I did most, but not all, of the work to make a multi-user version of POPFile. That work is being completed by the core team and will allow a single POPFile installation to be shared by multiple users.

Thank you to the POPFile Core Team for this great start to a new chapter in POPFile history.

A post (anti-spam-) retirement note

One of the anti-spam companies I was/am involved with, MailChannels, made an interesting announcement recently about a commercial offering for SpamAssassin. What makes the announcement interesting to me is that Justin Mason (who wrote SpamAssassin) is also an advisor to MailChannels.

The program, Traffic Control 3 for SpamAssassin, is a free download and for sites that process less than 10,000 messages per day there's no charge at all (and no need to go and get a license from MailChannels).

Basically, the new product acts as a front-end to SpamAssassin traffic shaping incoming messages so that load is taken off SpamAssassin and the mail server.

Breaking the Fermilab Code

A story appeared on Slashdot about a mysterious fax received at Fermilab written in an unknown code. The full story is here. I looked at it and immediately noticed a few things:

1. The first part looked like ternary (base 3) with digits 1 (|), 2(||) and 3(|||).

2. The last part looked like binary with digits 1(|) and 2(||)

3. The middle bit looked like either a weird substitution code, or I wondered if it might be machine code.

4. In the last part the digit 2 (||) never occurs more than once, perhaps it was actually a separator and the last part is not binary.

The first step was to convert the bars into numbers. Here's a copy of my marked up print out:



The first part has the numbers (or at least I thought):

323233331112132
333231322123312
111331132312233
333212123213113
311333313331111
211333323232211
232313331121231
33231312

Noticing this had 113 digits (which is a prime number) I went off on a wild goose chase around primes, and then around the interpretation of this number in hexadecimal as a string in ASCII, Unicode or binary... waste of time.

Then I started thinking about ternary again and wrote down the largest ternary numbers that can be expressed with 1, 2, 3, ... digits:

2₃ = 2₁₀
22₃ = 8₁₀
222₃ = 26₁₀
2222₃ = 80₁₀

One of those stood out: with three digits the maximum number is 26 and there are 26 letters in the alphabet! Then the only question was was how to map the three digits used in the code (1, 2, 3) to the three ternary digits (0, 1, 2).

To simplify things I wrote a small Perl program that tries out all the possible mappings and outputs the ternary interpreted as a string (with 001 = A, etc.):

use strict;
use warnings;

my $top = $ARGV[0];

$top =~ tr/321/abc/;

my @chunks;

while ( $top =~ s/^([abc]{3})// ) {
  push @chunks, $1;
}

my @digits = ( '0', '1', '2' );

foreach my $d0 (@digits) {
  foreach my $d1 (grep {!/$d0/} @digits) {
    foreach my $d2 (grep {!/[$d0$d1]/} @digits) {
      print "($d0$d1$d2) ";
      foreach my $c (@chunks) {
 my $v = 0;
 my $m = 1;
 foreach my $d (reverse split( //, $c )) {
   $d =~ s/a/$d0/;
   $d =~ s/b/$d1/;
   $d =~ s/c/$d2/;
   $v += $d * $m;
   $m *= 3;
 }
 print chr( 64 + $v );
      }
      print "\n";
    }
  }
}

With my initial interpretation of the top part of the coded message I got the following output:

(012) [email protected]@[email protected]@CJQJFBWKAF
(021) [email protected]@[email protected]@FTVTCAPSBC
(102) JDNXUMEISOZNUODMFSGYQMPNZHMJCHCPNTELP
(120) [email protected]@RMPWRWJLFUNJ
(201) THYLOZGRKUMYOUHZCKENVZWYMDZTFDFWYJGXW
(210) [email protected]@IZWPIPTXCOYT

A ha! The 021 block (which corresponds to the mapping 3 -> 0, 2 -> 2, 1 -> 1) seems to have a partial message: [email protected]@WOULD and then it's garbage. Going back to the original message I realized that 113 is not divisible by three and that I'd either missed a symbol, or had two too many.

After much fiddling around I discovered that the correct interpretation of the top block is that two of the threes are wrapped from one line to another (there appears to me some indentation in the message that indicates this, take a look at the original, but this could be just random).

323 233 331 112 132
333 231 322 123 312
111 331 132 312 233
333 212 123 213 113
311 333 313 331 113
113 333 232 322 133
231 333 112 123 133
231 312

Rerunning my Perl program output the full message:

(012) [email protected]@[email protected]@[email protected]
(021) [email protected]@[email protected]@[email protected]
(102) JDNXUMEISOZNUODMFSGYQMPNYYMCIVEMXSVEO
(120) [email protected]
(201) THYLOZGRKUMYOUHZCKENVZWYNNZFRQGZLKQGU
(210) [email protected]

So much for the first part. The second part took me off into Z-80, 6502 and 6809 machine code wondering if it was a program and then nowhere. I still don't understand what this part is trying to say.

The third part looked initially like binary but on closer examination I decided that the 2s (||) were actually separators and the message should be interpreted as number separated by 2s by counting the 1s (|). That yields:

31211112111312
32213123123331
12213111332312
23333333233123
12313123332311
33223232312312
112

(Once again there was a wrapping 'problem' in the message where a run of 8 |s was actually 3 |s then 1 || and 3 more |s.) Using the little Perl program reveals:

(012) [email protected]@[email protected]
(021) [email protected]@[email protected]
(102) OZTYSBOOMXGZLODMLNEEOMEVACOOX
(120) [email protected]@NKVMNLUUKMUDYWKKB
(201) UMJNKAUUZLEMXUHZXYGGUZGQBFUUL
(210) [email protected]@YSQZYXOOSZOHNPSSA

So, the same mapping between digits is used.

That leaves some final questions:

1. Who is Frank Shoemaker?
2. Why is base spelt incorrectly?
3. Is the extra S in BASSE a reference to the middle section where three symbols start with S.
4. If #3 is correct, then those three symbols could be intepreted as FC₁₆ which is 252. Could this be the employee number of the author?
5. Why is the letter A missing from the middle section when all the other hexadecimal digits are there?

Which countries have the most beautiful women? (My deeply flawed analysis)

So, I happened upon the Wikipedia page about the Miss World pageant and noticed that it had a list of winners by country. For example, India has won Miss World 5 times. But, of course, India has a very large population so you'd expect it to be able to churn out a few beauties. So, to get a better idea here is a population adjusted list of countries that have won Miss World:

Country	Wins	Pop.	Wins/Pop.	Normalized
Bermuda	1	66163	0.0000151141876879827	100.00%
Iceland	3	316252	0.00000948610601672084	62.76%
Grenada	1	110000	0.00000909090909090909	60.15%
Guam	1	173456	0.00000576515081634536	38.14%
Jamaica	3	2651000	0.000001131648434553	7.49%
Trinidad and Tobago	1	1305000	0.000000766283524904215	5.07%
Sweden	3	9182927	0.000000326693221017656	2.16%
Puerto Rico	1	3994259	0.000000250359328225836	1.66%
Austria	2	8316487	0.000000240486157195941	1.59%
Ireland	1	4339000	0.000000230467849734962	1.52%
Finland	1	5308208	0.000000188387493481793	1.25%
Venezuela	5	28199822	0.000000177306083705067	1.17%
Israel	1	7282000	0.000000137324910738808	0.91%
Netherlands	2	16408557	0.000000121887622415548	0.81%
Dominican Republic	1	9760000	0.000000102459016393443	0.68%
Czech Republic	1	10381130	0.0000000963286270377117	0.64%
Australia	2	21290000	0.0000000939408172851104	0.62%
Greece	1	11216708	0.0000000891527175353054	0.59%
Peru	2	28674757	0.0000000697477575834383	0.46%
UK	4	60487300	0.0000000661295842267716	0.44%
Argentina	2	40301927	0.0000000496254186555397	0.33%
South Africa	2	43700000	0.000000045766590389016	0.30%
Poland	1	38518241	0.0000000259617255107781	0.17%
France	1	64473140	0.0000000155103350015216	0.10%
Turkey	1	70586256	0.0000000141670639111387	0.09%
Egypt	1	80335036	0.0000000124478689472424	0.08%
Germany	1	82210000	0.0000000121639703199124	0.08%
Russia	1	142008838	0.00000000704181524251329	0.05%
Nigeria	1	148000000	0.00000000675675675675676	0.04%
US	2	304072000	0.00000000657738956562919	0.04%
Brazil	1	186757608	0.00000000535453420457174	0.04%
India	5	1132446000	0.00000000441522156464856	0.03%
China	1	1321851888	0.000000000756514409124179	0.01%

So, far and away, the top three are Bermuda, Iceland and Grenada. Given that Bermuda is the winner, and a tax-haven, and has a sub-tropical climate... Hamilton here I come!

The Spammers' Compendium finds a new home

Shortly after I announced that I was getting out of anti-spam the folks at Virus Bulletin contacted me about taking over The Spammers' Compendium. I was delighted.

Today the transfer is complete and the new home is here. It will be maintained and updated by Virus Bulletin. Please send submissions to them.