Monday, November 30, 2015

The secret message hidden in every HTTP/2 connection

If you spy on an HTTP/2 connection starting up you'll notice that it sends an almost-but-not-quite valid HTTP request at the very start of the connection. Like this:


Written a little more clearly that's:

    PRI * HTTP2.0

    SM

The HTTP verb is PRI and the body contains just SM. Put them together and you get... PRISM. This occurs right at the start of the connection to ensure that the server really supports HTTP/2.0. It is detailed in Section 3.5 of RFC7540 as follows:

   In HTTP/2, each endpoint is required to send a connection preface as
   a final confirmation of the protocol in use and to establish the
   initial settings for the HTTP/2 connection.  The client and server
   each send a different connection preface.

   The client connection preface starts with a sequence of 24 octets,
   which in hex notation is:

     0x505249202a20485454502f322e300d0a0d0a534d0d0a0d0a

   That is, the connection preface starts with the string "PRI *
   HTTP/2.0\r\n\r\nSM\r\n\r\n").

I tried to find an explanation of the specific letters used and why they spell PRISM. After a bit of spelunking the following comes to light.

May 29, 2013
IETF draft-ietf-httpbis-http2-03 describes this connection mechanism and indicates that the string to send is FOO * HTTP/2.0\r\n\r\nBA\r\n\r\n.

July 8, 2013
IETF draft-ietf-httpbis-http2-04 changes the string to PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n.

Strange. 

I wonder what happened between May 29, 2013 and July 8, 2013? Could it be "U.S., British intelligence mining data from nine U.S. Internet companies in broad secret program"?

Thanks to this comment on Hacker News here's the actual commit that introduced this change. On June 14, 2013 the string was changed with the comment "Exercising editorial discretion regarding magic."

3 comments:

Chester said...

Interesting. It seems to have been noticed before in https://html5experts.jp/jovi0608/1622/ - you can Google Translate and search for "PRI SM" that they notice the pun to be related to the (then) recent Snowden disclosure, emphasizing its provisory aspect (it may be anything that is *not* valid non-HTTP/2 server) and even dismisses further discussion on that as "bikeshedding". Since provisory names tend to stick, I will not be surprised if this indeed becomes the norm...

mnot said...

We needed two pseudo-HTTP requests for the "magic" to assure it wasn't being interpreted as HTTP/1; I'd done runs across something like 2% of the HTTP servers on the Internet and found that that form failed most reliably.

In the SF meeting at Twitter HQ, we were trying to nail this down; see Originally, we were looking at "STA" and "RT .

However, PRISM had just broken, and it was all that was being discussed in the hallway. People were pissed. It didn't get into the minutes, but it came up as an idea to replace START since it had five letters, and people were unlikely to every want a "PRI" or "SM" method.

Since this was effectively a bikeshed painting exercise, we let Martin determine the result using editorial discretion; although it doesn't get into the minutes, it was pretty clear that people were OK with this (since it was split into two, it was pretty harmless anyway).

Later, over dinner at the Zurich F2F, a number of people thought it would be good to replace "GOAWAY" with "GTFO" (after WAY too much fondue). That application of editorial discretion go so well in the end.

This is all from my quite fallible memory, YMMV.

Matty K said...

Everyone knows Martin is Illuminati, why else would he be allowed on the internet with that accent?

But more seriously, I had a bit of a whinge on my blog around that time wherein I compared it to a crude phallic graffito scrawled on the side of a bike shed. I ended with:

But I can't complain, because that would be "bike shedding." Surely there must come a point where the colour of the shed is important. Trivial details in a spec are still details in the spec, and this one is part of a MUST-level interop requirement, so absolutely everyone who implements or interacts with HTTP/2 is going to have to reproduce (or at least look at) that c**k-and-balls every time they delve into the spec.

So, a question: do bike-shedding details ever become important? And if so, when?

No one ever answered, so I still don't know.