Monday, March 30, 2009

Measuring download speed by embedding a GIF inside a JavaScript comment

As part of my day job I've been spidering a large number of web sites. While debugging the spider to make sure that its results were consistent I came across a curious web page that contains binary data inside a comment inside <script> tags.

The spider was simply requesting http://www.usaa.com/ which then goes through a sequence of complicated redirects. Here's the output from the excellent HTTP debugging tool HTTPFox:

GET 301 Redirect: https://www.usaa.com//
GET 301 Redirect: https://www.usaa.com/inet/ent_logon/ent_login_redirect.jsp
GET 301 Redirect: https://www.usaa.com/inet/ent_logon/Logon?redirectjsp=true
GET 200 https://www.usaa.com/inet/ent_logon/Logon?redirectjsp=true
POST 200 https://www.usaa.com/inet/ent_logon/Logon?redirectjsp=true

The critical page to watch is https://www.usaa.com/inet/ent_logon/Logon?redirectjsp=true. When loaded with no cookies it is served containing JavaScript code that captures information about the browser and posts it back to the server:

var originalUrl = "https://www.usaa.com/inet/ent_logon/Logon";
var queryString = "?redirectjsp%3Dtrue";
function submitForm() {
var obj = document.frmBrowserProfile;
obj.action = originalUrl ;
obj.action = obj.action + decodeURIComponent(queryString);
obj.bpsw.value = getScreenWidth();
obj.bpsh.value = getScreenHeight();
obj.bpsd.value = getColorDepth();
obj.bpbw.value = getClientWidth();
obj.bpbh.value = getClientHeight();
obj.bpap.value = getAcrobatVersion();
obj.bpfp.value = getFlashVersion();
obj.bpcs.value = getConnectionSpeed();
obj.submit();
}

The interesting part is the function getConnectionSpeed() which is written as follows:

function getConnectionSpeed()
{
var connectionSpeed = "999999999";
try
{
var datasize=10240;
var diffTimeMilliseconds = endTime - startTime;
if(diffTimeMilliseconds > 0)
{
var diffTimeSeconds = diffTimeMilliseconds/1000;
var bits = (datasize*8);
var kbits = bits/1024;
var connectionSpeed = kbits/(diffTimeSeconds);
}
}
catch (e) {} ;
return connectionSpeed;
}

And that relies on knowing a startTime and endTime. They is done by embedding a 10k GIF inside the web page, not by loading it, but by embedding it inside a JavaScript comment. The time is measured at the start and end of the GIF being loaded into the browser and the bandwidth determined from that.

Here's a shortened (and wrapped) version of the code:

<script language = "Javascript">
date = new Date();
startTime=date.getTime();
</script>
<script language = "Javascript">
/*GIF89a?*÷ÿÿÿÿ)))111999BBBJJJRRRZZZccckkksss
{{{????????????¥¥¥µµµ½½½ÆÆÆÎÎÎ{ssRJJ9111))Æ­¥?
{sJc!ïÖÆR!?Jν­½­??Z)µk!µc?Jc½s!?R½kk9Jµk?R?Z!?
c?R­k{JsB?k9?c1cB?c?c?RR9B){Z!kJ?kZ9??R?sBcJ?k
?c¥s¥?9¥{!R9?{Rsc9­?RkZ1cR)ZJ!?s!ƽ¥­?k??BcR!µ?
?kkRcJ{Z?c?{B{k9sc1¥?9­?Z¥?R??1ZJ??k??c¥?s?{R?
?B?{1{c??c??Z{sJ­?B??9JB?{!ZJ­¥s??RcZ!RJ¥?)?{{k
B9?s¥?Z?{9sk1cZ­¥cµ­c??R­¥R??B¥?J??B{s!kcRJZR??)
??!??!{s­­¥µµ­Æƽ???{{sssk½½­Îνµµ¥RRJJJBÆÆ­¥¥????
ÆÆ¥ccR??{½½?µµ?­­?RRBÎÎ¥½½?µµ?JJ9¥¥{{{Z!!­­{??c¥¥
k??R­­k¥¥c??RZZ1??Bss9BBZZ!??)kk!RR??)ss!??cc))
11??RZc!µ½???{ckJÆÎƵ½µ????¥?ksk­­µ?????¥BBJ{{?
99Bkk?{{?ZZsss?ss?))9kk?cc?BBckk¥JJsZZ?cc¥kkµ
cc­RR?cc½JJ?11cRR­BB?99{!!JBB?))cJJµJJ½99?RRÖBB
µ))sBBÆ))?99½))?11Æ!!?))­))Æ!!­!!µ!!Æ!!Î!!Ö÷ÞÎ!
ùÿ,?*ÿÿýÛ?!@?U2$4¸

...

Æ;[email protected]?SHX༯?Aq ?âA@³-yRº`?ÔW85¾?ò*/
</script>
<script language = "Javascript">
date = new Date();
endTime=date.getTime();
</script>

Now the GIF data looked valid (you can see the global color table at the start) I decided to have a go at downloading it and viewing the GIF. Immediately I ran into trouble because the GIF was invalid. Looking at the code in a hex editor I immediately noticed that it was UTF-8 encoded. A quick run through UTF-8 decoding and the GIF was valid, but truncated. Unfortunately it's not very interesting when viewed in a web browser:



I wrote to the webmaster of the USAA web site, but have heard nothing back. Does anyone know if anyone else uses this technique?

PS This system measured my connection speed as 20Mbps. Which is about 5x too large.

Wednesday, March 25, 2009

Thursday, March 12, 2009

A punctuation failure

I guess the reviewer meant that Billy Elliot was "funny, touching and shamelessly enjoyable", but without the comma apparently the musical about a boy who wants to be a dancer involves "funny touching".

Oops.



Or perhaps I'm just an old punctuation curmudgeon.

Friday, March 06, 2009

3M: An old company working at the speed of Twitter

A few hours ago I tweeted a random thought I had about Post-It notes and my iPhone:

3M should make a Post-It that fits exactly on the front of an iPhone. The one's in my office are slightly too big.

Within two hours 3M had @replied to me with a suggestion from their product catalog that would fit on my iPhone:

Hi John - you might be interested to try this size. let us know how you get on http://tinyurl.com/a67xqh

Nice.

Tuesday, March 03, 2009

"Smell the Glove" playing on the iPhone

Dig that cover art:



"It's like, how much more black could this be? and the answer is none. None more black."

Monday, March 02, 2009

Taking your iPhone to the Sahara

On my last holiday I took my iPhone into southern Tunisia for a trip across the desert. To keep it juiced up and protected I used two cool pieces of technology: a powermonkey eXplorer and The Ultimate iPhone Sand/Sea Case.

powermonkey eXplorer

The powermonkey eXplorer is a universal charger and battery backup that can charge from a USB connection, 110V/220V in most countries in the world and via a solar panel. Here's mine recharging in the Tunisian sunshine.



Once recharged the little battery can be used to charge any device through the collection of tips that are provided. Or you can skip the battery if USB or mains power are available and charge your device directly.

The Ultimate iPhone Sand/Sea Case

To protect my iPhone I used The Ultimate iPhone Sand/Sea Case AKA a ziploc bag. The ziploc bag protected the phone from sand damage and kept it dry when I took it in the bathroom with me.

And you can use the phone without removing it from the bag. Here it is in action:



PS Yes there was cell phone service in the desert.

How to get technical support

The following steps have worked for me:

1. Contact technical support at the vendor in question (in this case HP)

2. Fail to get the support needed

3. Write blog post criticizing the company's technical support

4. Have blog post appear on front page of Google search results when looking for procurve support



5. Receive response from head of technical support giving his personal email address.