Thursday, October 29, 2009

Der Geek Atlas

The Geek Atlas ist jetzt auch in Deutsch.


Kaufen Sie es hier.

Die lebendige Geschichte der Wissenschaften ist überall um uns herum, man muss nur wissen, wo man hinschauen muss. Mit diesem einzigartigen Reiseführer kann man 128 Orte auf der Welt kennen lernen, die für bedeutsame Ereignisse in Wissenschaft und Technik stehen. Erlebe das Foucaultsches Pendel, das in Paris schwingt; erfahre Interessantes über das größste Wissenschaftsmuseum der Welt, das "Deutsche Museum" in München; besuche einen Ableger des Newtons Apfelbaums am Trinity College in Cambridge und vieles, vieles mehr...

Jeder Ort in Der Geek-Atlas stellt eine außerordentliche Entdeckung oder Erfindung in den Mittelpunkt und befasst sich darüber hinaus auch mit den Menschen und Geschichten, die hinter diesen Erfindungen stehen. Alle Orte werden mit interessanten Fotos vorgestellt und die Themen mit zahlreichen Zeichnungen illustriert. Das Buch ist nach Ländern aufgeteilt, für alle interessanten Orte werden auch -- neben nützlichen Tourismusinformationen -- die genauen GPS-Daten aufgeführt.

Eine kleine Auswahl der interessanten Orte: * Bletchley Park in Großbritannien, wo der Enigma-Code geknackt wurde * die Alan-Turing-Gedenkstätte in Manchester * die Hornantenne in New Jersey, wo die Big-Bang-Theorie bestätigt wurde * das National Cryptologic Museum in Fort Meade in Maryland (USA) * die Trinity Test Site in New Mexico, wo die erste Atombombe gezündet wurde * das National Museum of Scotland in Edinburgh, wo das Schaf Dolly ausgestopft ausgestellt wird Jeder Ort, der im Der Geek-Atlas vorgestellt wird, hat einen besonderen mathematischen, technischen oder wissenschaftlichen Hintergrund. Orte, die das Geek-Herz schneller schlagen lassen.

Thursday, October 22, 2009

Some real data about JavaScript tagging on web pages

Since March of this year I've been running a private web spider looking at the number of web tags on web pages belonging to the Fortune 1000 and the top 1,000 web sites by traffic. Using the spider I've been able to see which products are deployed where, and how those products are growing or shrinking.

The web tags being tracked are those used for ad serving, web analytics, A/B testing, audience measurement and similar.

The spider captures everything about the page, including screen shots, and I'm able to drill in to see the state of a page and all its includes at the time of spidering. Here's shot of Apple with all the detail that the spider keeps.



The first interesting thing is to look at the top 1,000 web sites by traffic and see how many different tags are deployed per page. The average is 2.21, but if you exclude those that have no tags at all then the average is 3.10. Here's the distribution of number of tags against percentage of sites.


And of course, it's possible to see the market share of various different products. Here are the top 10 that I am tracking. Google Analytics has an impressive 43% of the top 1,000 web sites by traffic.


Since I've been tracking over time it's also possible to watch the growth (and decline). Here's the growth in the average number of tags on a web page (excluding pages that have no tags) since March 2009.

Since I also keep all the JavaScript and HTML for a page it's a breeze to calculate page weights. Here's a chart showing the size of HTML and JavaScript for the top 1,000 web pages by traffic. The x-axis shows the size of the page (excluding images) in kilo- or megabytes. The y-axis is the percentage of sites in that band.


I was shocked when I saw that list and suspected a bug. How could there be web sites with megabytes of non-image content? It turned out that it wasn't a bug. For example, at the time of downloading the HTML and JavaScript for Gawker was over 1Mb.

In a previous post I showed in detail the tagging on a site and that 29% of the non-graphic content was JavaScript used for web tagging. Here's another chart showing what percentage of web page markup is included JavaScript (this can include stuff like jQuery and web tagging products).


The really surprising thing there is how much JavaScript there is on pages. For many pages it's the majority of non-graphic content. Take for example Subscene where the home page HTML is about 18k but then masses of JavaScript are included (including over 200k from Facebook, a similar amount from UPS and various other bits of code).

If you delve into the tags actually used by various products you'll see that the sizes of JavaScript used for them varies a lot. comScore's Beacon is tiny (just 866 bytes)!



Finally, you might be asking yourself which site had 16 different tags on it. The winner is the celebrity gossip site TMZ.

Nerd is the new normal

When I was writing The Geek Atlas there was a big debate about the title. My original title included the word geek, but O'Reilly quickly overruled it. By the time the final title was agreed, we went with a title that O'Reilly themselves suggested: The Geek Atlas.

And then, just the other day, a US TV station did a report called "Nerd is the new normal".

Friday, October 16, 2009

Dyson's overpriced fan and the Coandă Effect

The news the other day was filled with the news of James Dyson's new bladeless fan which costs an arm and a leg (seriously, £199.99 for a desk fan!?!). There are a number of interesting things about this fan.



First, it appears to have no blades. It's just an annulus (or a ring) with a big space in the middle. Using Dyson Magic (tm) air magically flows through the ring.

Well, actually this thing does have blades which are hidden in the base and air is sucked in there and blown upwards into the ring. Dyson calls this part an impeller because instead of being a fan it's, well, an impeller: basically a pump enclosed in the base.

Impellers themselves are rather neat because if they are well made they are efficient and they can exploit the Venturi effect to get lots of flow.

But the really interesting thing about the Dyson Air Multiplier is how it gets so much air in your face. The air coming up from the base is directed into the ring and exits through a slot that runs around the back of the ring and faces forward.

This causes air to be blown towards you touching the ring itself. The ring is shaped like an airfoil and the blown air will 'stick' to the airfoil shape. Then the Dyson Magic (tm) happens: the Coandă Effect causes additional air to be 'sucked' into the flow (this is called entrainment).

This is where Dyson gets his 'multiplier' word from. The Coandă flow of air generated by his impeller coming out of the slot entrains the surrounding air causing it to flow.

The Coandă Effect is very well known, and Dyson hasn't invented something revolutionary here (although I assume the engineering is very good). The effect has been used on aircraft and experimental flying saucers. Here's a lovely YouTube video of a Coandă Effect flying saucer. The thing to realize when watching this is that the air flow causing this to float is going over the surface of the outside of the craft.



I'd probably pay £199.99 for one of those, but not for a fan.

PS If you want you can read Dyson's patent that makes all this clear.

Wednesday, October 14, 2009

What is jsHub?

Some time ago I blogged about a new open-source project I'm involved in called jsHub. Since then there's been a little bit of confusion about what jsHub is all about.

Hopefully, I can clear this up in this blog post with an example.



The Problem

The home page of World Wrestling Entertainment has a total of 11 pieces of JavaScript for tracking and ad-serving. If you have Ghostery installed in your browser it will tell you that that page contains the following:



Using our own internal tool we see that page contains DoubleClick, Google Analytics (which they include three times), LeadBack, Microsoft Atlas, Omniture, OpenX, Quantcast, Quigo AdSonar, Revenue Science, Tacoda and comScore Beacon.

The problems with having so many different pieces of tracking JavaScript are many:

1. They add to the page weight. In the case of WWE the HTML of the page is 54687 bytes (the total non-graphic content downloaded is 433211 bytes).

The JavaScript for tracking and ad-serving is a total of 125454. i.e. 29% of the non-graphic content of that page is JavaScript code used to track usage and serve ads.

2. They create a risk of data integrity problems.

A typical problem occurs when one piece of JavaScript works and sends tracking information back and another doesn't. This creates a discrepancy between products that is a problem when trying to reconcile page counts between say an analytics product and an advertising system.

This is not a theoretical problem. It's easy to have it occur because a page may be viewed and the user may hit stop while the page is loading. A piece of JavaScript near the top of the page may have executed, while a piece near the bottom has not.

Indeed, the page code for the WWE site contains the comment: <!-- Add Google anlytics after omniture --> indicating how important placement of JavaScript code is.

3. They add unnecessary processing time.

Just take a look at this shot of downloading all the JavaScript for the WWE page. This was taken using Firebug and shows how wasteful all that extra code is in terms of download time, and execution time.



4. They are next to impossible to check for security problems.

The only option the web master of WWE.com has is to run all the JavaScript he receives from third-parties through something like Google Caja to ensure that it's safe, or insist that they are ADsafe.

Here, for example, is a section of code from Ominture's tracker used on the WWE page:



But the web master actually doesn't have that luxury because typically the JavaScript is being loaded remotely from the analytics vendor's or ad-server's web site and the end web master has no control at all over what's being loaded. Just look at what happened to the New York Times when a malicious ad turned into malware.

In the case of WWE there are 24 includes of JavaScript code from web sites that WWE do not control. And because of the browser security model all these pieces of code are getting equal access to the page.

5. End-users have no way to understand what they are doing.

Although programs like Ghostery are excellent they can't tell you what's actually happening inside that JavaScript. For example, there's no easy way for an end-user to determine what information is being gathered, or where it's being sent.

There is a tool called WASP but it's aimed at people debugging web site tagging problems, not at the privacy-aware consumer. Here's what WASP says about Tacoda on the WWE web site:



6. They represent duplicated effort as vendors are forced to write and maintain their own JavaScript code.

For example, all those tags have to find a way to send data back to their respective servers meaning there's duplicated code that has to be tested on a wide range of browsers to ensure that it all works.

7. Their inner working are often obscure.

See above!

Enter jsHub

jsHub is designed to eliminate these problems. It's a single piece of JavaScript (a "tag") that can handle reading different sorts of page information and then send them to many different vendors' products. One piece of code to send to Google Analytics, Omniture SiteCatalyst, WebTrends and Mixpanel.

Instead of one piece of JavaScript per vendor, jsHub has a single piece of code (the "hub") and plugins that know how to translate into the required wire protocol for each vendor. Vendors only maintain the plugin for their product.

With one piece of code the page weight is less, there's no danger of one product getting a page view and another not and processing time is reduced.

Then to make the entire thing debuggable and easy for an end-user to understand, there's the tag inspector. It's a user interface that talks to the jsHub tag and interrogates its operation. That way a user can see what's being gathered on a page, and who is receiving it.



Since the entire project is open-source it's possible to inspect the code to ensure that it is well written and secure. And it's licensed under a BSD-license so that it's open and includeable everywhere.

To further ensure that the code is of high-quality (and can handle all the different types of browsers that it might be executed in), there's a complete test suite and cross-browser testing system.

To make exactly what data is being gathered clear we are also proposing (public domain) standards for marking up page metadata using microformats. Our proposed standard is called hPage.

We're just getting started with jsHub. It's running on a small number of sites and we're working to build vendor interest. We strongly believe that a shared, open-source tag is the best solution for the entire web world.

If you want to get involved, contact the team.

The Times' new science magazine: Eureka

Hannah Devlin at The Times prodded my via email to tell me about the new science magazine they have launched called Eureka. What a wonderful idea. I'm so glad to see a major newspaper take science so seriously. So, I ran out and grabbed a copy.

Oddly, the next day I received the following mail from Carrie Mash at Mindshare:

Hello!

I've just been having a look through your blog and I wanted to get in touch as I think you'd be a fantastic audience for a new science magazine from the Times.

I am contacting you on behalf of News International who publish The Times and today they have launched a new Science Magazine called Eureka.

In order to make the magazine as informative as possible we are reaching out to the UK science community to help steer its editorial content.

As you are a passionate blogger, who discusses science/environmental issues, we invite you to join our online focus group and we would appreciate your feedback and comments about Eureka.

We are currently sending out free copies of Eureka to all members of the focus group. If you would like to participate in the program all you need to do is join the dedicated Facebook Fan Page and send us a postal address of where you would like the magazine to be delivered (UK Only).

The Fan Page will be monitored by the Eureka editorial team and over time we may even ask you to attend a focus group based at the News International offices in London. Please do treat the Fan page as a genuine feedback point; we will endeavour to respond to your comments as quickly as possible.

I am happy to answer any direct questions that you may have; in the meantime please feel free to share the news of the magazine with your blog readers.

I look forward to hearing from you.

Best Regards,
Carrie

Well, I don't have time to take part in a focus group, and I don't need a free copy (especially since it was given away free with The Times), and I love the bit where she gives me permission to blog about it ("feel free...").

But the magazine is worth reading, and having a science supplement to a major newspaper would be marvellous (although it appears that they plan to sell it). My only gripes are... the section highlighting people spent a lot of space of photographs of them and whitespace and not enough on the science they are doing. For example, in explaining quantum computing they get into quantum entanglement and Schrodinger's Cat yet fail to explain why quantum entanglement makes "a quantum computer lighting quick compared with a bog-standard PC".

The other gripe is about a fun little piece on page 5 called "The Quantum of Cool" where an attempt is made to make a funny scale going from 1 bogart (a thing that is very cool) to 1 bono (something that is very uncool). But the explanation of the scale is odd:

Here's how it works: The scale starts at neutral 0, equivalent to 0 millihanks (Tom Hanks, everyman), 1,000 millihanks equal 1 bradpitt, 1,000 millipitts equal 1 clooney, 1,000 millicloonies equal 1 bogart. Negative cool: 1,000 gumps equal 1 becks, 1,000 millibecks equal 1 cheney, 1,000 millichenies equal 1 bono.

(I assume that becks is referring to David Beckham.) Given that the prefix milli- indicates one thousandth this can be rewritten:

Here's how it works: The scale starts at neutral 0, equivalent to 0 millihanks (Tom Hanks, everyman), 1 hank equals 1 bradpitt, 1 bradpitt equals 1 clooney, 1 cloony equals 1 bogart. Negative cool: 1,000 gumps equal 1 becks, 1 becks equal 1 cheney, 1 cheney equals 1 bono.

So, that scale is totally broken.

But overall the magazine is great, and a great idea. I look forward to reading it each month.

Back to Carrie Mash. It was nice of you to write to me, but do you think you could reply to my mail when I reply to you with a question?

Tuesday, October 13, 2009

Damn the torpedoes

The other day a comment of mine on Hacker News received the following response:

Your work, however, suggests that you are not actually human. Seeing some of the things you have produced makes me think that you are inhumanly competent, and hence Joel's remarks do not apply to you.

I would have ignored the flattery if it hadn't been upvoted 10 times on a story that only got 8 upvotes. No one, except me, commented in disagreement.

So suppose that some group of Hacker News people does agree with that comment, I have to ask myself why, and what I've done to deserve that praise. I can really think of only one thing: I've often found myself thinking: "Damn the torpedoes". And if you are going to look up to me then I suggest that you do the same.

But damning the torpedoes comes in many different forms.

"You can't"

When I was around 9 years old I told a school science teacher that I intended to go to Oxford and get a doctorate. Since my parents had never pushed me in any particular direction this was a bit of a surprise. It also came shortly after the teacher had criticized me in an exercise that we had been doing about estimation of the number of earthworms in the school playing field. Damn her!

But getting to Oxford was one of the first times an external torpedo came my way. I told my school (when I was around 16) that I intended to apply to Oxford and I was immediately told that Oxford would never accept me because I did not know Latin. Now, it's true that at one time lacking a classical education would have precluded me entering Oxford or Cambridge.

But I left that staff room infuriated and determined to find out the truth. And the truth was I "simply" needed to sit the Oxford mathematics entrance exam and then pass a sequence of what would end up being around 10 interviews.

I did that, studied mathematics and computation and stayed to get a doctorate. Damn the teacher who tried to torpedo my desire to go to Oxford with her ridiculous outdated notions.

[As an amusing aside when I was being interviewed for entrance to study for my doctorate I was interviewed by Tony Hoare who asked me where else I had applied. I stated flatly and truthfully that hadn't applied anywhere else. To this day I don't know if he thought I was determined or simply crazy.]

I can't

Some torpedoes you create for yourself through doubts. Although doubts can be useful because if you examine them you can understand your own limitations, it's also important to recognize those doubts that have no foundation and spit in their face.

When I was at school I was very bad at foreign languages. I took French and German and was pretty bad at both. I barely spoke a word when I left school and my exam results were nothing to write home about.

Today, I speak French fluently.

I had a great personal motivation to learn French as an adult because of a relationship. But I doubted I could ever do it. So, I damned the torpedoes, dropped out of the useless evening course with a bunch of losers who were just doing it to be able to order bread in a boulangerie and took private lessons. Once a week I sat down with a Swiss woman who forced me to read and speak French. Little by little my doubts about not being a 'language person' dropped away. Now I'm ashamed I only speak two languages.

A similar thing happened when I worked on the copy/move forgery detection code. I knew nothing about computer graphics at all and considered my ability to learn about it pitiful. But I really wanted to implement that algorithm so I read and reread the paper and over a period of months I worked on the code. I had very little spare time so I would grab an hour one day and work on just the JPEG handling, or just the matrix code, and then drop the entire projects for weeks or even months.

But it works. Little by little I learnt about graphics and built on the knowledge necessary to finish the project.

They won't

And finally, the Alan Turing petition. For that I simply ignored the possibility that people wouldn't be interested in the man or the issue. Damn them all!

If you have an idea, or a cause, then you are going to face people who disagree with you. It's still disappoints me that my live interview with the legendary interviewer Jeremy Paxman was cancelled.

I was very scared of doing it, but damn Jeremy Paxman!

If you are going to do something, go all the way. I can't promise that you'll succeed, but you'll be better off for recognizing the obstacles you create for yourself through doubts, the obstacles others create for you for no good reason, and for sticking to what you believe.

I don't consider myself deserving of the Hacker News comment above, but if you think I do then try to damn the torpedoes in your own life.

PS And while you are being single-minded and purposeful, don't forget to be joyful. The happiest moment of my entire day yesterday came when I encountered a fox wandering down my street and I followed it to see what it was up to.

Monday, October 12, 2009

Leaving gaps on the iPhone helps a lot

One of the navigation nightmares of the iPhone is that by default the apps you download are all clustered together on page after page of identical seeming pages. Each page contains 16 apps.



I kept having problem where I'd start the wrong app because muscle memory would take my finger to the right place and I'd have clicked before I'd realized the icon wasn't the one I was looking for.



But I solved this problem in a simple way. I rearranged my apps over three pages (after deleting a lot of stuff I wasn't using) with gaps on them. By doing that I end up knowing which page I am on just based on the black space.



This is the iPhone equivalent of the dog-eared page in my notebook, or the coffee stain on the project plan. These simple queues make sub-conscious navigation easy. But I'm lucky because I've managed to arrange unique black-space on each page. Since the iPhone forces all the apps together other people might not be able to do this if they want to group applications together logically. Would be nice if Apple allowed arbitrary positioning.

Dear PR People... Send me stuff

The other day some enterprising folks sent me a DisplayLink adapter in response to a blog post. That was a good idea, but I'm not a professional reviewer, so it's only worth sending me stuff I actually need.

Here's a list of problems I'd like solved, or gadgets I'd like to try:

1. In-ear headphones with microphone that work with the iPhone
2. Aircraft noise cancelling loud speakers for my bedroom
3. A MagLite style torch that isn't cylindrical and hence doesn't roll
4. I'd also like a torch with a part specifically made so you can hold it in your mouth
5. A device that can clone RFID cards so that I only carry one
6. A transparent, red, plastic sheath for the iPhone for nighttime use
7. A decent application that aggregates email, Twitter, RSS, ... i.e. a personal dashboard
8. A messenger-style bag that contains a white or illuminated interior so I can find stuff
9. Slip on, water-proof covers for my dress shoes
10. A cheap Ethernet interface for Arduino

Send me stuff and I'll review it.

Monopoly .com Edition

I love Monopoly and have a small collection of Monopoly games from around the world. The oddest one is Monopoly .com Edition which was released in 2000.



In it the streets are replaced with '30 of today's hottest web sites'. These are: Sportsline.com and FoxSports.com, Yahoo! Geocities, Oxygen and iVillage, shockwave.com, games.com and E! Online, Priceline, Expedia, and eBay, weather.com, about.com and cnet.com, ETrade, monster.com and MarketWatch.com, Ask Jeeves, AltaVista and Lycos, and Excite@Home and Yahoo! (Yes, there are only 22!)



The railway stations are replaced with telecom companies: MCI WorldCom, Nokia, Sprint and AT&T. The playing pieces are made of pewter and depict Mr Monopoly sitting at computer, the Internet Explorer Hand, a surfboard, a computer screen, a web browser, a PC, an email, a mouse and a microchip. The Mr Monopoly piece is a special token that can take any web site 'offline' making it unavailable for purchase.



The buildings are houses and office blocks (instead of hotels), all the money is in millions of $ and Community Chest and Chance are replaced with Email and Download.



The back of the box (sorry about the poor quality of these images, had to use my iPhone camera):

Friday, October 09, 2009

The DisplayLink USB video adapter just works

The other day I wrote a post about the hell of Apple video adapters. Out of the blue I received an email telling me about DisplayLink. The company offered to send me a review unit to see if it would help with my problem.

I accepted and received one a few days later. Here it is.



There are three components: a standard USB to mini-USB cable. The DisplayLink box (top right) itself which has two connectors: mini-USB on one side and a DVI connector on the other. It also comes with a DVI to VGA converter (top left).

The adapter works by simply plugging it into the USB port of a computer. After installing the appropriate driver (Windows or Mac OS X) you can then connect a display or projector to the adapter and it appears as if it were directly connected to your machine.

For example, I connected the adapter to a MacBook in our conference room which is normally connected directly to our Epson projector. Instead I attached the connector to the DisplayLink. The Epson appeared in the list of displays detected by Mac OS X (the only poor part of the experience was downloading the correct driver which was OS version specific).

I then tried it on a MacPro and on a MacBook Air. Worked like a charm. Then I switched to a laptop with Windows Vista installed. Once again, it worked perfectly. The video was crisp and smooth.

I'm not a gamer so I didn't try it with fast moving graphics (although I did watch both YouTube and iPlayer video streaming video).

The DisplayLink adapter is available through a variety of OEMs, you can't get it directly from the manufacturer so I can't tell you the cost of the model I was using. But, as an example, the Diamond BVU195 HD USB Display Adapter costs about $80.

The only problems I see with it are: it got very hot with extended operation, and it only partially solves my problem. Now I have to ensure that all the machines that I might connect to the projector have the right software on them. But, at least, I don't have to get into trouble when someone steals the one Apple video adapter I need.

Overall, for our needs (office applications) the adapter is a nice solution. We'll probably be sticking with it.

Monday, October 05, 2009

A reply from Buckingham Palace

About a month ago I wrote a letter to Her Majesty The Queen asking her to consider knighting Alan Turing.

Here's the reply from the palace:

Friday, October 02, 2009

The hell of Apple video connectors

At my day job I made the fateful decision to go with an all Apple shop. Everyone has an Apple MacPro maxed out with dual monitors (we don't buy Apple monitors or RAM because of the excessive price). (Some of the team have switched their machines to run Windows XP or Windows Vista and one team member uses Ubuntu; the only person on PC hardware is the CEO).

For mobile users we have a range of laptops including the MacBook Air and different generations of MacBook Pro and one MacBook. We even have an Xserve as our main server.

But there's a price to pay... when connecting laptops to the projector in the meeting room we are in adapter hell.

To solve these problem I have one of each adapter with little stickers on them. One end of each of these is a standard 'VGA' style connector for the projector.

The red is for MacBook Air, because it has a micro DVI connector.

The green is for the MacBook, because it has a mini DVI connector.

The yellow is for the recent MacBook Pros, because they have a mini DisplayPort connector.

The blue is for the old style MacBook Pros, because they have a Apple DVI connector.

Oddly all these machines manage to have the same MagSafe power connector, but I'm in video connector hell. When is someone going to do a wireless video standard?