Sender Rewriting Scheme

For a mail transfer agent (MTA), the Sender Rewriting Scheme (SRS) is a scheme for rewriting the envelope sender address of an email message, in view of remailing it. In this context, remailing is a kind of email forwarding. SRS was devised in order to forward email without breaking the Sender Policy Framework (SPF), in 2003.^[1]

Background

In a number of cases, including change of email address and mailing lists, an MTA accepts an email message that is not destined to a local mailbox but needs to be forwarded. In such cases, the question arises of who deserves to receive any related bounce message. In general, that is either the author, or a person or other entity who administers the forwarding itself.^[2] Sending bounces to the author is administratively simpler and used to be accomplished by just keeping the original envelope sender. However, if the author address is subject to a strict SPF policy (-all) and the target MTA happens to enforce it, the forwarding transaction can be rejected. As a workaround, it is possible to synthesize on the fly a temporary bounce address that will direct any bounce back to the current MTA. The scheme provides for recovering the original envelope address, so that if a bounce does arrive, it can be forwarded along the reverse path—with an empty envelope sender this time.

While there are other workarounds, SRS is a fairly general one. It is interesting to note how its notion of reversing the path resembles the original routing dispositions for email, see below.

The Rewriting Scheme

SRS is a form of variable envelope return path (VERP) inasmuch as it encodes the original envelope sender in the local part of the rewritten address.^[1] Consider example.com forwarding a message originally destined to bob@example.com to his new address bob@example.net:

   ORIGINAL
   envelope sender:     alice@example.org
   envelope recipient:  bob@example.com

   REWRITTEN
   envelope sender:     SRS0=HHH=TT=example.org=alice@example.com
   envelope recipient:  bob@example.net

The example above is adapted from Shevek.^[3] With respect to VERP, the local part (alice) is moved after her domain name (example.org), further adding a prefix (SRS0), a hash (HHH), and a timestamp (TT). That reflects an operational difference: Eventual bounces back to a VERP address are handled within the rewriting domain, and forged messages can at most unsubscribe some users, a kind of abuse that hasn't seen significant exploits in the last decades. Instead, SRS aims at remailing a possible bounce back to Alice, so that forged bounces can become an alluring technique for injecting spam apparently originating from the rewriting sender.

• The local part, in this case alice, is moved because it may contain equal signs (=), so putting it at an extremity of the rewritten local part makes the latter parsable.
• The timestamp (TT) has a one-day resolution in order to make the address invalid after a few days. Computed as  ^{unix time}⁄_(60×60×24) mod 2¹⁰, it can be stored as a two base32 characters, with a recycling period of about 3.5 years.
• The hash-based message authentication code (HHH) is computed against a local secret, but only a part of it is used; for example, storing the first 4 characters of a base64 representation provides 24 bits of security. The hash is checked by the domain who generated it, in case a bounce arrives.
• The prefix, SRS0, is meant to disambiguate regular addresses from rewritten ones; it is up to example.com to ensure that none of its users has an email address starting with SRS0=.

SRS provides for another prefix, SRS1, to be used for rewriting an already rewritten address, in a multi-hop scenario. If example.net has to forward the message in turn, it can spare adding another timestamp and repeating the original local part (alice). That is, each new forwarder adds just its own hash (HHH) and the domain name of the preceding forwarder:

   FURTHER REWRITTEN
   envelope sender:     SRS1=HHH=example.com==HHH=TT=example.org=alice@example.net
   envelope recipient:  bob@further.example

Database alternative

Using a database can definitely control the growth of rewritten addresses, since it is sufficient to put just a unique key in the rewritten local part. It also allows for a certain amount of anonymity in the resending process, if this is desired. However, a database requires centralization and is a single point of failure.^[4]

Header field alternative

Another possibility is to store the long rewritten address somewhere in the message header. The i= tag of a DKIM-Signature may be a good place, as such choice considerably improves the security. This technique has been just observed.^[5] Unless there is a backup mechanism, it can only work if the bounce message is in a standard format.^[6]

Historical background

Historically all mail transfer agents (MTAs) added their host name to the reverse path. In the Simple Mail Transfer Protocol (SMTP) this reverse path is also known as MAIL FROM, but paths were also used before and outside of SMTP, e.g. as bang paths in UUCP and Usenet (NetNews). All news articles still contain a Path header, example:

Path: news.server.example!other.example!not-for-mail

The same information in an RFC 5321 e-mail envelope - that is the SMTP info like MAIL FROM - would be:

1. MAIL FROM:<not-for-mail@other.example>
2. MAIL FROM:<@news.server.example:not-for-mail@other.example>

The 1st step reflects the sender, the 2nd step the next MTA, etc. In this example let's assume that the 2nd MTA forwards the mail to a 3rd MTA, where it is finally delivered. The final MTA is also known as Mail delivery agent (MDA), putting the mail into the mailbox of the recipient. The MDA transforms the reverse path into the known Return-Path header field:

Return-Path:<@news.server.example:not-for-mail@other.example>

SMTP uses MX records for its forward routing. Explicit source routes as in…

RCPT TO:<@news.server.example:user@destination.example>

…to route mail from other.example via MTA news.server.example to MDA destination.example were cumbersome. To make things worse sometimes the new (1982) style of addresses was mixed with old UUCP bang paths in constructs like…

destination.example!user@news.server.example
other.example!not-for-mail@news.server.example

…and various other kludges. SMTP and MX records made all this essentially useless. Therefore, source routing was deprecated 1989 in RFC 1123.

One special case in RFC 1123 are gateways from or to other networks like UUCP and NetNews, where the first sending MTA cannot reach the final receiver directly with TCP. It is solved by MX records and if necessary rewriting foreign addresses at the gateway. MX is an acronym for Mail eXchanger.

Another special case are mailing lists, where the list server rewrites all reverse paths to its own error handling address for bounces (error messages) by recipients. The list server could automatically unsubscribe bouncing recipients. This type of address rewriting is known since RFC 821 and still used today (RFC 5321, as well as RFC 2821, updated the SMTP chapter in RFC 1123).

Last but not least forwarding to another address always worked by rewriting the address in the forward path also known as RCPT TO, if and only if the forwarding MTA accepted the responsibility for both forwarding the mail and returning potential bounce messages to the sender. RFC 821 and all later SMTP specifications offer two result codes for this situation:

• 251 user not local (attempted forward)
• 551 user not local (mail rejected)

For privacy reasons these result codes are today rarely used; they include the forwarded to (251) or not forwarded to (551) address. But the meaning and the effect of forwarding to third parties is identical for result code 250 and error code 550 respectively.

As noted RFC 1123 deprecated source routing, that implicitly also deprecated the reverse routing of bounces. It was a relatively small Internet back in 1989, mail admins (postmasters) often knew each other and fixed problems on the fly. Routing bounce messages back via any forwarders was a waste of time and bandwidth if the MTA noting a problem (e.g. a rejection with a 5xx error code) could send the error message directly back to the MX of sender.

Since RFC 1123 forwarders to third parties still rewrote the RCPT TO address, but kept the MAIL FROM as is. As a side effect MTAs wishing to accept mail from forwarders generally accept any MAIL FROM address.

More than a decade later spammers started to abuse this flaw in post-1123 SMTP, today most mails are spam and most reverse paths are forged. Note that spammers typically forge working reverse paths, many MTAs reject mail if callback verification or other plausibility checks fail for the reverse path.

RFC 5321, as well as RFC 2821, states that non-delivery reports (bounces) must be sent to the originator as indicated in the reverse path after an MTA accepted the responsibility for delivery. However, the bounce message may be suppressed when the original content is hostile (cf. spam or virus mail) or the message is forged (RFC 5321, Section 6). Note that all current forgery detection methods require the mailbox owner to supply information for them to work. Failing to supply the criteria should not make any bounce message classifiable as backscatter, although some people mistakenly think it should.

Open relays and forwarders are in an unlucky position with regards to this issue, generally they can't guarantee that the MAIL FROM address indicates the originator, and they also can't guarantee that final delivery will succeed.

This SMTP problem caused as side effect of RFC 1123 is addressed by SPF, and the executive summary is SPF breaks forwarding - actually that's not the case, SPF FAIL only asks receivers to check SPF at their border MTA, not later.

Receivers can arrange their forwarding in a way that works with SPF with in essence three strategies:

1. not checking SPF behind their border, e.g. white list forwarders
2. just reject SPF FAIL, resulting in a bounce (SMTP error 550)
3. rewrite the MAIL FROM at the forwarder (as done by mailing lists)

Sender Rewriting Scheme (SRS) is one way for the third strategy.

See also

• Sender Policy Framework (SPF)
• Bounce message (SMTP non-delivery report)
• Bounce Address Tag Validation (BATV)
• Simple Mail Transfer Protocol (SMTP)

References

External links

• libsrs2 home page
• Paper on SRS (PDF)
• Historical SRS draft by Meng Weng Wong (2003)
• qmail SRS patch