Skip to main content

Trusted Email Connection Signing (rev 0.2)

IMPORTANT: This blog post deprecates my previous posting on this subject. The blog post Proposal for connection signing reputation system for email is deprecated.

Sign the medium, not the message

The motivation behind TECS (Trusted Email Connection Signing) is that what managers of MX servers on the public Internet really care about is the ability to distinguish a good connection (coming from a legitimate sender and which will be used to send wanted email) from a bad connection (coming from a spammer). If you can identify a bad connection (today, you do that using an RBL or other reputation service based on the IP address of the sender) you can tarpit or drop it, or subject the mails sent on the connection to extra scrutiny. If you can identify a good connection it can bypass spam checks and help reduce the overall false positive rate.

If you are a legitimate bulk mailer (an email marketer, for example) then you care deeply that you reputation being recognizable and that mail sent from you be delivered. Currently, you have to carefully tend your IP addresses to make sure that they don't appear on blacklists, and you have to ensure that new IP addresses are clean.

If you are running a large email service (e.g. Yahoo! Mail) then you are currently trying to build white and blacklists of IP addresses, when what you really want are white and black lists of entities.

Currently, the options used to identify a bad connection are rather limited (RBLs, paid reputation services and grey listing), and good connections are hard to manage (whitelists on a per-recipient basis, or pay-per-mail services). What's needed is a different approach.

The idea is to identify and determine the reputation of the entity connecting to a mail server in real-time without resorting to a blacklist or whitelist. This is done by signing the connection itself. With the signature on a per-connection basis a mail server is able to determine who is responsible for the connection, and then look up that entity's reputation in a database.

Current reputation databases are based on IP addresses. This is a very inflexible system: IP addresses must be added to blacklists very fast as spammers churn through zombie machines, and any legitimate emailer needs to make sure their mail servers are whitelisting with multiple email providers (e.g. Yahoo!, Gmail, Brightmail, ...) to ensure delivery. And if a legitimate mailer wants to bring on line new servers, with new IP addresses they have to run through the entire whitelisting process again.

This is inefficient. The mapping between IP address and entities (e.g. knowing that Google's Gmail services uses a specific set of IP addresses) is unwieldy to manage and the wrong level of granularity. Google should be free to add and remove email servers at will, while carrying their good reputation with them.

That's what TECS gives you.

Connection Signing

TECS is an extension to the existing SMTP AUTH mechanism (see RFC 2554) and implements an authentication mechanism that I'll refer to as TECS-1 (the 1 here acts as a version number on the protocol). TECS-1 would need to be registered as a SASL (see RFC 2222) authentication mechanism.

When a mail sender connects to an SMTP server wishing to sign its connection it issues the EHLO command and if that SMTP server is capable of handling AUTH the mail sender then signs the connection using the AUTH command, with the TECS-1 mechanism followed by an initial response (which contains the TECS signature) as defined in RFC 2554.

Here's an example session:

S: 220 ESMTP server ready
S: 235 Authentication successful.

The 'initial response' section of the of the AUTH command is a base-64 encoded string containing the following structure (this is deliberately similar to the DKIM fields):

a=rsa-sha256; q=dns;; b=oU0Nnbmh1YWVlMDljNDBhZjJiO==

a= is the cryptographic method used (default would be RSA/SHA-256 with suitable padding as described in PKCS#1 version 1.5 RFC 3447).

d= is the name of the domain signing the connection. In the example above I am showing a connection that is being signed (and hence claimed by)

q= is a query type with the default being the use of a DNS TXT record. This query method is used to obtain the public key associated with the signing domain. The public key would be obtained by looking up and getting the associated TXT record.

b= is the binary signature for the connection generated using the method in a= by the d= domain.

The connecting server signs the tuple consisting of ( destination IP/port, source IP/port and epoch ); that way they sign the current connection and verify that they are responsible for the mail sent across it.

Each entity has an RSA key public/private key pair. When signing a connection the entity generates a SHA-256 hash of the tuple. The destination IP/port pair is the IP address and port on the mail server that the mail sender is currently connected to; similarly the source IP/port pair is the IP address and port of the connection being used by the
mail sender. The epoch is the standard Unix epoch rounded to the nearest 30 seconds.

The entity making the connection then encrypts the hash with their private key.

Update (February 8, 2007): A number of people have suggested getting the public key from the _domainkey (i.e. DKIM) label of the d= domain. This seems like a good idea since there's no need to reinvent the wheel.

Update (February 9, 2007). A few people pointed me in the direction of the MARID CSV proposal (see CSV). I've addressed this below.

I don't want to sign my outbound SMTP connections
Well don't then. If TECS were implemented you'd probably find you'd have trouble getting your mail delivered as non-signing (and hence not-taking-responsibility) would be looked upon very poorly.

I'm big ISP and I don't want to sign as myself, can I sign as a customer?
Sure, for example,'s mail is actually handled by When that server was sending mail for it could sign as as long as it has access to's private key. It would simply specify in the TECS data.

Why don't you just use STARTTLS with certificates?
Because that's a very heavyweight system, designed for something else. A SASL extension using SMTP AUTH is simple and clean.

Why do you think connection signing is useful?
Because SMTP server resources are precious. Being able to make a decision about a connection before any mail is delivered is very useful. An SMTP server owner could use reputation data from a public or private source to decide whether to accept or reject a connection, slow down a connection, apply little or extra scrunity to a connection, etc. Being able to do this before receiving a ton of mail and tying up a server is very valuable.

Won't spammers just sign their connections?
No doubt, but that's hardly a worry, being able to identify the good senders fast is the most important goal.

Why don't you just signed the destination IP/port pair? That's known before the connection is made and avoids problems with NAT
TECS could just sign the tuple ( destination IP, port, epoch ) but I think it's a bit weaker than my proposal. Since the destination IP and port are fixed for a given MTA the signature is really a signature on the time. An eavesdropper could reply the signature within 30 seconds (or other timeone on the epoch value) and get an authenticated connection from any source IP address.

What about the MARID CSV proposal?
The CSV proposal is a lightweight (DNS-based), non-cryptographic method of estabilishing whether a host claiming a certain domain name in HELO/EHLO is authorized to be an SMTP client. Clearly, CSV aims to provide a simple method of determining whether a connecting SMTP client is authorized to be an SMTP client with the claimed name. This seems like a useful extension, but is very different from TECS. TECS operates at the level of a specific connection, and with an entity that is distinct from the domain of the SMTP client. This is valuable for two reasons: it allows the identity to be moved from SMTP service provider to provider, and it means that shared SMTP servers can operate claiming different 'responsible parties' for each connection. This latter point is important for ISPs that provide SMTP services to email marketers where the same SMTP server may be shared across many clients. This can result in a clean emailer being blacklisted because the IP of the shared server was blacklisted because of some other unrelated misbehaviour.

Whilst CSV is a useful extension which would help with the zombie problem, it does not address the needs at the connection level where I believe the problem needs to be addressed.

CSV also provides specific services for checking domain names against accreditation services. That is outside the scope of TECS, although the assumption is that such services would exist for TECS signed connections against the domain name claiming responsbility. The bottom line is that TECS deals with the party responsible for a connection, CSV the party responsible for the server.

What about mailing lists that forward mail?
By signing their connections they take responsibility for the mails they are sending. So mailing lists would need to have appropriate email policies in place for unsubscriptions, and deal themselves with spam to the list. Since the connection is signed any concern about munging of From: addresses for VERP handling, or adding headers/footers to email are irrelevant.

Is this compatible with SPF, Sender-ID, DomainKeys?
They are orthogonal. There's no direct interaction. Although, it might be sensible to use the _domainkey record from DKIM to obtain a public key thus sharing the same key between DKIM and TECS.

Will this reduce spam?
I'm not going to make any predictions. The goal would be to build a database that makes it easier to recognize someone who is legitimate, and scrutinize those who abuse the system or who choose not to sign.

What about anonymity?
Anoymous remailers are unaffected. They could sign their outbound connections with the system but that would not affect any changes they make to anonymize messages since its the conneciton, not the message content that's signed.

What if I change the mail servers or IP addresses I am using?
There's no effect. Keep signing the connections and you can take responsibility for any IP address you want to.

I think you are wrong, right, stupid, a genius.
Please comment here, or write to me directly.

Many thanks to all members of the REDACTED discussion forum, and to Toby DiPasquale.


Justin Mason said…
I dunno John -- I would say the massive deployment of SMTP+TLS is still a major plus in its favour, over a new scheme. getting changes made to SMTP infrastructure is *extremely* slow.

Also, the fact that the connection winds up encrypted with SMTP+TLS is, if anything, a bonus!
I don't agree about the encryption part. That's a major minus from my perspective because of the cost of encrypted connections.

Unknown said…
Sorry I didn't get to reply to your questions about my original comment.
I was initially thinking you could have the string added here so that it could be ignored or used without any new states/commands in the SMTP conversation:

220 ESMTP 18si4862540nzo
EHLO TECS:*TECS-string-here* at your service, [w.x.y.z]
250-SIZE 20971520
221 2.0.0 closing connection 18si4862540nzo

But using existing standards seems better probably.

The things I'd like to see are:

* Use the exact same key as DK/DKIM that way you have the bond between the two sigs; connection level and individual email level and you maybe can do just one dns lookup. In order to easily use the DK dns records, maybe have the selector string be a part of TECS AUTH.
Just use instead of the field.
That way you have a huge amount of flexibility in specifying the signing key and room for the standard to easily evolve.

* For the signed fields, have them all be things that the client knows prior to making the connection (so that the TECS can be built by an async process just prior to initating the connection), maybe: your server IP, the time, (not sure what else). The question here is do the fields that are signed beyond those two really matter? What value is added by signing certain fields vs others? The client doesn't know the ports beforehand, but those can be/are tracked by the server. You (the server) have all those fields src IP/port, dest IP/port, something signed by the client, the time of the connection - what spoofing is prevented or record keeping is enhanced with the additional fields encrypted? By signing only the dest IP and time, it would allow for TECS strings to be cached in a SMTP connection cache, since it could be reused for 30 or whatever seconds.

About the outgoing IP address - People will argue a "perfect world" scenario in regards to this - but do mail servers really always know what their external outgoing IP is? I say no with all the load balancing, NATing, proxying that goes on, but...

Other than those two things, I think its a great idea.

Why is the discussion forum a secret?
Unknown said…
What exactly is the cost? 5% CPU? Are you 100% maxed out on CPU already?
Unknown said…
"The idea is to identify and determine the reputation of the entity.... This is done by signing the connection itself."

Signing the connection does not give you reputation. It gives you an entity whose reputation can be measured.

These seem to be highly conflicting statements:
-[problem]...determine the reputation of the entity connecting to a mail server in real-time without resorting to a blacklist or whitelist.
-[solution]...determine who is responsible for the connection, and then look up that entity's reputation in a database.

White and Blacklists *are* reputations stored in a database. They are generally not very granular though.

It seems to me your root problem is that you want better ways to measure reputation than the blackbox algorithms RBLs/accreditation entities use. But, this protocol just changes the entity that gets measured.

Assume everyone in the world changes their MTAs to do the AUTH TECS-1 command today. Why won't RBLs begin to generate domain blacklists using the same mechanisms to determine the list as today?

I do think that domain is a better entity to measure than IP. But, that's exactly why the industry has been working on sender authentication mechanisms for several years. With DKIM in IESG last call, why not just use that domain to do your reputation lookup?
Unknown said…
A novel approach, and I like the simplicity of it. Have you tried to quantify the impact from the added step of a DNS lookup request? You're asking the MTA to rely on another process for a timely response (one that's more than likely on a completely different computer, seperate network segment, etc).

On a similar note, if you want to use the DKIM format for the TXT records then it's important to note some flavours of DNS don't support the leading underscore the DKIM record requires. BIND seems okay with it, but the MS implementation seems to silently ignore it (which makes it devillishy difficult to diagnose if you don't know what to look for).
Unknown said…
"but the MS implementation seems to silently ignore it"
which makes this pretty ironic:
dig TXT
;; ANSWER SECTION: 3600 IN TXT "v=spf1 ip4: ip4: ip4: ip4: ~all"
Unknown said…
Well, the MS server that replied to my query for the SPF record was fingerprinted as:
fingerprint (, Microsoft Windows 2003
Unknown said…
How *does* this address the problems for MTAs behind a NAT? The MTA has no knowledge of what the public IP will be.

Popular posts from this blog

How to write a successful blog post

First, a quick clarification of 'successful'. In this instance, I mean a blog post that receives a large number of page views. For my, little blog the most successful post ever got almost 57,000 page views. Not a lot by some other standards, but I was pretty happy about it. Looking at the top 10 blog posts (by page views) on my site, I've tried to distill some wisdom about what made them successful. Your blog posting mileage may vary. 1. Avoid using the passive voice The Microsoft Word grammar checker has probably been telling you this for years, but the passive voice excludes the people involved in your blog post. And that includes you, the author, and the reader. By using personal pronouns like I, you and we, you will include the reader in your blog post. When I first started this blog I avoid using "I" because I thought I was being narcissistic. But we all like to read about other people, people help anchor a story in reality. Without people your bl

Your last name contains invalid characters

My last name is "Graham-Cumming". But here's a typical form response when I enter it: Does the web site have any idea how rude it is to claim that my last name contains invalid characters? Clearly not. What they actually meant is: our web site will not accept that hyphen in your last name. But do they say that? No, of course not. They decide to shove in my face the claim that there's something wrong with my name. There's nothing wrong with my name, just as there's nothing wrong with someone whose first name is Jean-Marie, or someone whose last name is O'Reilly. What is wrong is that way this is being handled. If the system can't cope with non-letters and spaces it needs to say that. How about the following error message: Our system is unable to process last names that contain non-letters, please replace them with spaces. Don't blame me for having a last name that your system doesn't like, whose fault is that? Saying "Your

The Elevator Button Problem

User interface design is hard. It's hard because people perceive apparently simple things very differently. For example, take a look at this interface to an elevator: From flickr Now imagine the following situation. You are on the third floor of this building and you wish to go to the tenth. The elevator is on the fifth floor and there's an indicator that tells you where it is. Which button do you press? Most people probably say: "press up" since they want to go up. Not long ago I watched someone do the opposite and questioned them about their behavior. They said: "well the elevator is on the fifth floor and I am on the third, so I want it to come down to me". Much can be learnt about the design of user interfaces by considering this, apparently, simple interface. If you think about the elevator button problem you'll find that something so simple has hidden depths. How do people learn about elevator calling? What's the right amount of