Thursday, June 11, 2015

AWS SES, VERP and hMailServer


A .NET MVC (c#) application I'm responsible for has occasion to send emails to clients of the applications users. The email address associated with a client comes from two sources:

  • The application users entering it on behalf of their clients 
  • From third party systems that users can elect to integrate our system with

Of course, we do some basic 'well formed' validation of email addresses, but the acid test comes when you attempt to use that address.

Being fully paid up AWS hosted citizens, we use AWS SES as our SMTP server. But we also self host hMailServer for our domain; for a variety of reasons, it makes sense to partition the 'work horse' email sending to SES, and the basic domain email handling to hMailServer (given that SES does not 'do' POP).

So how do handle bounced emails from SES? We need to let our users know if any of their clients did not receive an email. But how? As usual with SMTP, there is no trivial answer, but thankfully there is a relatively easy one.

I looked at the usual suspects (message id, custom headers and so on), but VERP seemed to provide the best route (and SES supports it).

In passing, I should note it is possible to do some of this with SNS - but really, for our application, it offered no real advantage - 'real time' delivery of notifications was not important, just best efforts, and there was only one sink for the bounce message.

So, in summary, this is what I ended up doing, and it is working well enough in production now.

Dispatching an email to a client: I use the basic SmtpClient of .NET to dispatch the email to the (in our case) single recipient. When a MailMessage is created, the From address is set to an email address verified with SES and belonging to our domain. But I also set the sender address to be a modified form of that address, using a standard VERP style separator, example:

From address:   mailer@corporate.server.com
Sender:             mailer+8922@corporate.server.com

The point of this is to ensure that the context I need (8922 in the Sender address above) to interpret a bounce message is honoured by SES; this is one of the few ways to do it.

SES interpretation: SES dispatches the email to the recipient, and on detecting some form of failure, sends an email to the Sender address, not the From address; that is, the VERP marked up address, not the plain verified address.

hMailServer: hMailServer is a well featured mail server for Microsoft platforms. But it hides its VERP support well, calling it instead 'Plus addressing'.

In short, if we don't activate this part of hMailServer, our bounce messages from SES will bounce as well.

If you need to activate VERP style addresses for hMailServer, you need to:

  • Open the hMailServer application
  • Click on the domain you are interested in enabling VERP support for
  • Go to the advanced tab
  • Click 'Enabled' for 'Plus addressing'
  • Select a separation character; in my case of course, I used '+'
Final actions: All the above being done, we had a reasonable feedback loop for bounce handling. As the platform we host already has a cooperating set of agents that perform scheduling, it was easy to slot in another component (discovered by MEF) to:
  • Periodically query the mailer@corporate.server.com mailbox (using OpenPOP), and read the content
  • For every message that has an SES bounce signature and a To address that matches our VERP address, attempt to decode the to address, extract the context, and the act upon it
  • When done with a message that could be handled, audit it and delete it
It's a simple approach I grant you, but one that works without issue to date.

No comments: