Tuesday, November 22, 2016

Sessions I moderated at the Cloudflare Internet Summit

In September of this year Cloudflare held its annual Internet Summit. I was asked to moderate three discussions. Videos of all the sessions have now been put online so folks who weren't able to attend in person (remember, it's free and open to the public), or watch the live stream aren't left out.

My three sessions were...

What Can We Expect from the Internet in 2020?

I sat down with Ilya Grigorik to talk about predictions for the Internet in 2020. Ilya is Web Performance Engineer at Google, author of High Performance Browser Networking and an expert on Internet performance.

Who Will Win the Cloud Wars?

It was a privilege to sit down with James Watters, SVP of Products at Pivotal, and John Engates, CTO at Rackspace to dig into what's happen with public and private clouds in the current 'cloud war'.

The Fastest and Most Secure Internet is Closer Than You Think

And, finally, Eric Rescorla from Mozilla and Jana Iyengar from Google (where he works on QUIC) sat down with me to talk about security and performance on the Internet.

Saturday, September 10, 2016

The CloudFlare Internet Summit

This coming Thursday (September 15), CloudFlare is holding its second Internet Summit in San Francisco. This year, as I did last year, I will be on stage helping to moderate discussions. The summit is organized as a sequence of 'fireside chats' where one or two guests are on stage talking with someone from CloudFlare.

Guests at the summit range from policy makers, to CEOs, to technologists. The goal is to talk about major issues affecting the growth, security and performance of the Internet worldwide. But without the talking being done by pundits, journalists or investors. The people invited to the summit are leaders, thinkers and doers, not professional talkers.

Last Year

Last year, for example, the President of Estonia talked with CloudFlare CEO Matthew Prince to talk about Estonia's push to be a digital nation.

We also had Andrew Ng from Baidu and Carmen Chang from NEA sit down and talk about the relationship between CloudFlare and Silicon Valley.

I spoke with Adam Langley from Google and Richard Barnes from Mozilla to talk about security on the Internet.

And I also spent time with Andy McAfee, co-director of the MIT Initiative on the Digital Economy, and Larry Smarr, former head of the San Diego Supercomputing Center to talk about where the Internet will be in 5 years time and beyond.

This Year

Guests for 2016 include Jen Easterly, Special Assistant to the President & Senior Director for Counterterrorism, National Security Council, John Mulligan, Deputy Director, National Counterterrorism Center, Brendan Eich, President & CEO, Brave Software, John Engates, CTO, Rackspace, Peter Eckersley, Chief Computer Scientist, EFF, Brewster Kahle, Founder & Digital Librarian, Internet Archive, Ilya Grigorik, Web Performance Engineer, Google and Jana Iyengar, Software Engineer, Google QUIC.

This year I'll be discussing the future of Internet privacy, security and performance with key people from Google, Mozilla and elsewhere.

The Internet Summit is an unusual event because it brings together policy makers, business leaders, hardcore technologists, privacy advocates and more.

Oh, and it's free and open to the public.

Sign up here if you want to attend.

Saturday, July 23, 2016

Lazy GNU make variables

GNU make has two 'flavors' of variable: simple and recursive. Everyone is familiar the the recursive style (which uses = for definition):

    BAR = banana
    FOO = one $(BAR), two $(BAR)s
    BAR = guava

The value of FOO is determined each time FOO is used. In a large Makefile that can mean that FOO is expanded over and over again. If expanding FOO is an expensive operation then determining its value each time can slow down the Makefile. For example,

    TODAY = $(shell date +%d/%m/%y)

would be very expensive is it would shell out to run date every time TODAY is used (and might have the strange effect of returning different dates if you run your make near midnight).

(I've written about that before and in my book on GNU make)

The other flavor of make variable is simple: its value is fixed the moment it's defined:

    BAR = banana
    FOO := one $(BAR), two $(BAR)s
    BAR = guava

There FOO is set to one banana, two bananas and not one guava, two guavas (as was the case above) because the right hand side of the := is expanded when FOO is being defined and BAR has the value banana at that point.

But suppose you wanted a mixture of the two: a variable that is only ever expanded once and only expanded if it is used. This might be useful if the variable was used rarely as part of the build and was expensive to expand.

You can concoct a recipe for that like this. Suppose you want to define a variable FOO to have value expensive-to-calculate then you can use the following pattern:

    FOO = $(eval FOO := expensive-to-evaluate)$(FOO)

It's initially a recursive variable and so the right-hand side is not expanded until it is used. But the first time it is used FOO is redefined (by the $(eval)) and becomes a simple variable with the value of expensive-to-evaluate.

Having become a simple variable it's possible to return its value (note that if FOO was still recursive it would not be allowed to reference itself like that).

You can see that in action with the following Makefile. To make it clear when NOW is actually expanded I've added a call to $(info). And calls to $(flavor) to show when NOW changes from recursive to simple.

    $(info Start Parse)

    NOW = $(eval NOW := $(info Expand NOW)$(shell date -u))$(NOW)

    .PHONY: all foo
    all: foo ; @echo "$@ /    $(flavor NOW) / $(NOW) / $(flavor NOW)"
    foo: ; @echo "$@ / $(flavor NOW) / $(NOW) / $(flavor NOW)" && sleep 1

    $(info End Parse)

Which will output

    Start Parse
    End Parse
    Expand NOW
    foo / recursive / Sat Jul 23 15:01:15 UTC 2016 / simple
    all /    simple / Sat Jul 23 15:01:15 UTC 2016 / simple

The one gotcha with this is that you need to be sure that when NOW is expanded any variable it depends on have the values you expect.

It's possible to wrap that up in a function like this:

    $(info Start Parse)

    make-lazy = $(eval $1 = $​$(eval $1 := $(value $(1)))$​$($1))

    NOW = $(info Expand NOW)$(shell date -u)
    $(call make-lazy,NOW)

    .PHONY: all foo
    all: foo ; @echo "$@ /    $(flavor NOW) / $(NOW) / $(flavor NOW)"

    foo: ; @echo "$@ / $(flavor NOW) / $(NOW) / $(flavor NOW)" && sleep 1

    $(info End Parse)

Just $(call) make-lazy on any recursive variable to make it lazy.

Wednesday, July 06, 2016

Le Sixième Élément : le code source dans le cinéma

Depuis les années soixante, les ordinateurs jouent un rôle important dans le cinéma. Parfois, ils jouent le rôle de méchants, parfois celui d’outils. Parfois ce sont les enemies jurés de l’humanité, parfois leurs amis. Mais quand un informaticien voit l'écran d'un ordinateur sur le grand (ou le petit) écran, qu'est-ce qu'il voit réellement? Est-ce que c'est du vrai code source ou n'importe quoi ?
 Dans Terminator, Schwarzenegger voit un Apple II. Dans le Martien, on montre vraiment du code de la NASA. En revanche, dans Ironman, c'est du code pour un ordinateur LEGO.
Du coup, le code est parfois hilarant, fascinant, stupide ou révélateur. Il y a même des 'easter egg'.

Friday, June 03, 2016

Solving the SETI Decrypting Challenge

I spotted a challenge to decode a message received from a star 50 light years away. Well, at least a message that pretends to be from a star 50 light years away. The idea is if we were to receive a message like this would we be able to understand it. 

The message is 1,902,341 binary digits. Clearly that needs chopping up in some way so I factored it and its prime factorization is 7 × 359 × 757. I suspected that the message was either an image or multiple images. After a bit of fiddling around (initially I thought it might be a single image made of 7 colours overlaid) it turned out to be 7 images of size 359 × 757.

The code I used to generate the images below is here.

The first image confirmed that we were going to see 7 images of size 359 × 757 as it has a horizontal white bar at the top spanning the width and a vertical column on the right giving the height. This first image is a sort of 'frame' indicating that correct way to view the images.

The second image has the number 0 to 756 encoded as binary (white is a 1, black is a 0) down the left hand side. Perhaps the alien is telling us that it will use binary encoding for all numbers.

The third image has another sequence of binary numbers. In this case these are the first 757 prime numbers. I'm not clear why the alien wanted us to know that other than telling us that they know about primes.

Each of the subsequent images has two binary numbers encoded in the first two rows (zoom into this image to see them).

The third fourth image shows a wave. The transmission was received on a frequency of 452.12919 MHz from a star/planet 50 light years away. Working on the assumption that this wave is meant to represent the transmission itself I guessed that the two binary numbers at the start represent a distance and a time period as a reference point.

The two numbers are

00000000000000000000000000000000000000000000000000000000b = 16368191637088910834159098202685440


000000000000000000000000000000000000000000000000000000000b = 11677159761321922952849403009790256654968431476473856

I guessed that the first number represents (in some units that the alien uses) the wavelength of the transmission (66.31cm)  and that the second number was the time it would have taken the transmission to reach Earth (50 years). With those as a basis for distance and time measurements it's then able to calculate the other information requested.

The next image shows the alien itself with two binary numbers:

= 60479561273104168652304174731493376


= 42037775140758923161949049149211273119409177427443712

Assuming the first is the height of the alien then that would be

66.31 cm × 60479561273104168652304174731493376 / 16368191637088910834159098202685440

which is 245.01 cm. So the alien stands about 2.45m tall.

Assuming the second number is the lifespan of the alien was get

50 years × 42037775140758923161949049149211273119409177427443712 / 11677159761321922952849403009790256654968431476473856

That's 180 years. So perhaps this particular alien is 180 years old, or they generally live that long.

The next image shows what appears to be a large area covered in radio telescopes. I guess this is the source of the transmission. I'm pretty sure that this is actually a picture of the proposed Square Kilometre Array in Australia taken from Wikipedia.

Anyhow the two numbers are 

= 2468553521351190513386359178720371015680


0000000000000000000000000b = 2335431952264384665006648365913340213606881670994067456

Assuming the first is the size of the array then it's

66.31 cm × 2468553521351190513386359178720371015680 / 16368191637088910834159098202685440

which is 100km. So perhaps this is a square array 10km x 10km.

The second number might be how long they've been trying to talk to us.

50 years × 2335431952264384665006648365913340213606881670994067456 / 11677159761321922952849403009790256654968431476473856

which is 10,000 years. Sorry it took us so long to answer the phone.

Finally we get some information about where the alien lives. I think this depicts a very large planet with three moons (or perhaps it's a sun with three planets). The two numbers at the top are:

= 948909505053876287754040784031183263414353920


00000000000000000000000000000000000000000000b = 1401259171358630776572575392119740616658474880694279974420480

Assuming the first number is a size of the solar system (or planet, perhaps) we get

66.31 cm × 948909505053876287754040784031183263414353920 / 16368191637088910834159098202685440

which is 3.8442×10^7 km (roughly 55 times larger than our sun). So I'm going to guess that might be the size of the sun that they are orbiting.

The other number might be how long their solar system has existed:

50 years × 1401259171358630776572575392119740616658474880694279974420480 

So, 6 billion years. Longer than our solar system has been around but not an unreasonable time frame.

Thursday, May 05, 2016

Setting up CloudFlare's Universal SSL and Origin CA on Plan 28

Although I work for CloudFlare there are some products that I haven't experienced as a customer and it's always fun and useful to actually behave as a customer and try them out. I like to do this to make sure the experience is good and try to spot bugs.

CloudFlare recently released a feature called Origin CA that generates a certificate you can drop onto your web server to ensure that the connection between CloudFlare and the server is secure. CloudFlare also offers a feature called Universal SSL that offers free SSL connections for the connection between a web browser and CloudFlare. Put the two together and you've got SSL from browser to CloudFlare and CloudFlare to the origin web server. Neat.

One of my domains, plan28.org, had a web site that was served over HTTP and I decided to SSL it using CloudFlare. I'd seen it demoed but there's nothing like trying it out for yourself. It was really quick to get set up. plan28.org is on CloudFlare's free tier.

Here are the steps I took starting from logging into my account and verifying that I didn't have any SSL set up for this web site:

Then I clicked the Crypto button to get to the settings for Universal SSL and Origin CA.

And I enabled SSL for the site by clicking the SSL button from Off to Full (Strict). That makes CloudFlare issue a certificate for plan28.org and start serving it publicly and at the same time ensure that it will secure the connection from CloudFlare to my web server by using SSL and checking the validity of the certificate that my web server presents.

Boom. plan28.org was now available over http:// and https:// (although the latter wouldn't work until I had a valid certificate on the server). Next stop was a quick scroll down to find the Origin Certificates settings on the same page.

Clicking Create Certificate gave me a pop up where I could select the certificate type (RSA or ECDSA), validity period and add SAN names as needed.

With those selected hitting Create resulted in the very fast creation of a certificate and the corresponding private key.

I copy and pasted them over to the server, configured NGINX for SSL, restarted NGINX and... it worked!

Total time: 10 minutes (most of which was messing around getting NGINX configured correctly). Now you can visit plan28.org securely.

Tuesday, May 03, 2016

What the "Silicon Valley" Easter Egg code does and how

In the TV series Silicon Valley there was apparently a snippet of code presented that is part of a compression algorithm. The code (or at least part of it) can be executed and the the program will output:


The code itself has been published first as a screen shot, then as text and got some press. But I hadn't seen a good explanation of how it works. It looks pretty complicated but most of the code (written in C) is unused and the part that is actually executed is pretty simple (if you've spent years coding in C-like languages). Here it as presented:

#include <stdio.h>
#include <stdlib.h>

typedef unsigned long u64;

/* Start here */
typedef void enc_cfg_t;
typedef int enc_cfg2_t;
typedef __int128_t dcf_t;

enc_cfg_t _ctx_iface(dcf_t s, enc_cfg2_t i){
	int c = (((s & ((dcf_t)0x1FULL << i * 5)) >> i * 5) + 65);
	printf("%c", c); }
	enc_cfg2_t main() {
	for (int i=0; i<17; i++){
		_ctx_iface(0x79481E6BBCC01223 + ((dcf_t)0x1222DC << 64), i);
/* End here */

The writers have tried to make this look more complicated by using a bunch of typedefs and deliberately messing with indentation so that the main() function is kind of hidden.  After a bit of clean up the code looks like this:

#include <stdio.h>
#include <stdlib.h>

void _ctx_iface(__int28_t s, int i){
	int c = (((s & ((__int128_t))0x1FULL << i * 5)) >> i * 5) + 65);
	printf("%c", c); 
void main() {
	for (int i=0; i<17; i++){
		_ctx_iface(0x79481E6BBCC01223 + ((__int128_t)0x1222DC << 64), i);

So, main() calls _ctx_iface() 17 times with i set to 0, 1, ... 16 and passed in a weird looking number 0x79481E6BBCC01223 + ((__int128_t)0x1222DC << 64) which seems to have been obscured a bit with a left shift of 64 bits and an addition. The parameter is actually just 0x1222DC79481E6BBCC01223. Still looks odd but it's pretty simple: that's DREAM_ON_ASSHOLES encoded where each 5 bits indicating one upper case character or symbol (A = 0, B = 1, C= 2, ...).

If you express that number in binary you can separate it out into five bit chunks like this: 10010 00100 01011 01110 00111 10010 10010 00000 11110 01101 01110 11110 01100 00000 00100 10001 00011. You can even manually discover that this represents S E L O H S S A _ N O _ M A E R D if you take a guess that 11110 (30 = _).

The _ctx_iface() function extracts each character of DREAM_IN_ASSHOLES (17 characters) one by one and prints them. If does that with the code (((s & ((dcf_t)0x1FULL << i * 5)) >> i * 5) + 65) which uses i to indicate which 5 bits to extract and then shifts 0x1f (which is 11111 in binary) to the right position in the string, does a binary AND to extract just those five bits and then shifts the five bits back so that they become a number between 0 and 31.

The +65 turns a number between 0 and 31 into the characters ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_ because 65 is the ASCII character for A. Note that this encoding also explains why the program outputs DREAM_ON_ASSHOLES and not DREAM ON ASSHOLES. There's no way for the encoding to result in a space (which is ASCII 32).

All a bit of a disappointment really.

The rest of the code is more fun. There's a function called HammingCtr which computes the Hamming Weight of a binary string and even references and uses bit twiddling tricks from seander to make the code run quickly.

The other function is ConvolutedMagic which I have yet to decipher. Readers?

Two factor paper passwords

I guess it makes me boring but I try to get people to use stronger passwords.

People love to use the same password over and over again, or they invent some amazing scheme like the same single word followed by their birth year, or replacing a's with 4's. And no matter how many password database get hacked the idea that password security matters doesn't seem to really sink in.

When I do get someone to listen I tell them to use diceware generated passwords and them write them down in a little book and guard the book jealously (actually, I tell them to use a password manager but most people seem to balk at using software I think for fear of losing their passwords).

So I advise them to buy something like this and then keep a record of their passwords generated using diceware. Usually people seem happy to have something that creates them passwords like this:

But then they often ask the sensible question: "What if someone steals that book?" And so I suggest a 'two factor' solution. Two factor authentication is often characterized as a combination of something you know (e.g. a secret password) and something you have (for example, a smartphone with an app on it that generates a unique number valid for a few seconds).

In my two factor paper passwords scheme these are reversed: the something you have is the password (since it's written down and not something you can remember) and the something you know is how to transform the password into what you actually type in.

For example, suppose the password written down is anger lunar greek worry brown hole. The second factor might be that the second letter of each word will be capitalized. So the user would type in aNger lUnar gReek wOrry bRown hOle.

Or maybe it's that the first vowel in each word is omitted: the user types in nger lnar grek wrry brwn hle.

Or that the first and last letters of each word are swapped: rngea runal kreeg yorrw nrowb eolh.

Or the letters at the end and start of each word are swapped: angel runag rreew korrb wrowh nole.

Or that the password is followed by digits giving the number of consonants in each word: anger lunar greek worry brown hole333442.

Or that the number of consonants in each word is multiplied together to create a number added at the end: anger lunar greek worry brown hole864.

There are all manner of schemes someone can use to make possession of the book useless and it's a good way to get a user who isn't interested in password security engaged in the topic. The goal should be something easy to remember that's reasonably easy to compute in your head.

Of course, this doesn't protect against an attack where a single password has leaked and someone has access to the book because they may be able to figure out the scheme used, but this is a huge advance on the poor habits of most users.

PS If you are planning to do this... please consider using a password manager first.

PPS This will not protect the user against disclosure of an individual password. Two factor systems like Google Authenticator do help there. Always enable two factor authentication if it is available.

PPPS If you read this blog post and only take one action... go make sure your email is secured.

Sunday, April 17, 2016

Apple's amusingly round reuse figures

If you take a look at Apple's Environment page you might well be impressed by the amount of reuse Apple is getting by taking back devices. The numbers look great:

What's weird about these numbers is that they are round numbers. In fact they are very simple, round numbers. You just have to be using the right units.

Just look at that... 2,204 lbs of gold.

But wait! Isn't 1kg roughly 2.204 lbs? (Why, yes. It's 2.20462 lbs). So, sounds like Apple recovered 1,000 kg of gold. Now take a look at tin... 4,408 lbs... 2,000 kg. And silver... 6,612 lbs... 3,000 kg. And lead... 44,080 lbs... 20,000 kg.

Here are the figures above converted to kg using 1 kg = 2.204 lbs.

Steel    28,101,000 lbs = 12,750,000 kg
Plastics 13,422,360 lbs = 6,090,000 kg
Glass    11,945,680 lbs = 5,420,000 kg
Aluminum  4,518,200 lbs = 2,050,000 kg
Copper    2,953,360 lbs = 1,340,000 kg
Cobalt      189,544 lbs = 86,000 kg
Zinc        130,036 lbs = 59,000 kg
Lead         44,080 lbs = 20,000 kg
Nickel       39,672 lbs = 18,000 kg
Silver        6,612 lbs = 3,000 kg
Tin           4,408 lbs = 2,000 kg
Gold          2,204 lbs = 1,000 kg

Every single number given by Apple is an exact number of metrics tons (tonnes).

It's doubtful that Apple obtained precisely 1,000 kg of gold or 86,000 kg of cobalt. My guess is someone in Apple came up with rounded figures and they got converted to lbs for public consumption.

In the process they ended up looking precise.

One mystery remains. The total recovered is 27,839,000 kg which is 61,357,156 lbs (using 1kg == 2.204 lbs). But Apple reports an extra 644 lbs (total 61,357,800 lbs). I'll send a copy of my book, The Geek Atlas, to the first person to send me a convincing argument where those 644 lbs came from.

PS This comment points out that the UK Apple site gives the figures in tonnes.

PPS Another comment points out that the Australian Apple site appears to have converted back from lbs to kg.

PPPS Khalil Kacem (and below) has a convincing answer to the 644 lbs question. He gets the book.

Friday, March 25, 2016

Making the BBC micro:bit display the BBC Micro owl logo

Back in the 1980s I had a BBC Micro Model B. I still have that machine and repaired its power supply not long ago. The logo of the BBC Micro was a dot matrix owl:

Well, I was lucky enough to be lent (for one evening!) the most recent incarnation of the BBC's attempt to teach Britain's children to code: the BBC micro:bit. It's a lovely little machine and I hope many children have fun programming it. Unfortunately, I suffer from the Demon Machine problem: show me a computer and I have to make it do something.

Programming the device is really simple. There are multiple options and an online editor for each language. When you are ready to program the micro:bit it's trivial because the editor downloads a .hex file that you drag and drop onto the device (it appears mounted as a volume).

I chose to use MicroPython and write a small program that displays the (old) BBC Micro logo (the owl above) on the micro:bit's 5x5 LEDs. Since the owl is much larger than 5x5 pixels I've used the accelerometer to allow the user to scroll around the owl by tilting the micro:bit.

The code is here on Github. But to get a sense of how easy it is, here's the main loop that does all the work.

while True:
    lastx = ox
    lasty = oy
    xa = accelerometer.get_x()
    if abs(xa) > sensitivity:
        if xa > 0 and ox > 0:
            ox -= 1
        elif xa < 0 and ox < w - 5:
            ox += 1

    ya = accelerometer.get_y()
    if abs(ya) > sensitivity:
        if ya > 0 and oy > 0:
            oy -= 1
        elif ya < 0 and oy < h - 5:
            oy += 1

    if lastx != ox or lasty != oy:

The paint() function draws a 5x5 section of the owl (which is stored in an array) starting from position (ox, oy).  The height and width of the owl are stored in h and w.

def paint():
    for x in range(0, 5):
        for y in range(0, 5):
            display.set_pixel(x, y, 9 * owl[y+oy][x+ox])

And here's what that looks like on the micro:bit itself:

Hmm. I wonder if the micro:bit could be used as a game controller for Elite?

Thursday, March 10, 2016

Finding free images for commercial use

From time to time (especially when writing for the CloudFlare blog) I've needed to find an image that will help illustrate a piece of web writing. As I want to only use images to which I have a license and don't want to spend money the answer is... Creative Commons. And my first port of call is flickr which has nice options for searching for content by license.

Typically when using an image from flickr I will add a caption to the image with the license type, a direct link to the image page on flickr and a direct link to the person who took it.

For example, here are some Creative Commons cupcakes from flickr with an appropriate line of information about the image giving the license, image link and photographer.