Internet-Wide Study: State Of SPF, DKIM, And DMARC — RedHunt Labs

9 min readFeb 10, 2022

At RedHunt Labs, (under Project Resonance), we frequently conduct internet-wide research in different shapes and formats to understand the state of security across the internet. In this iteration, we conducted a study about the current state of DNS configurations (SPF, DKIM and DMARC) from a security perspective. We scanned more than 2.2 billion domains but before we discuss the insights, let’s quickly go over the basics.

What if a malicious actor could send an email on behalf of your domain? Spoiler alert; they can.

In this blog, we will cover results from our Internet-scale study of DNS Records and their misconfigurations that can lead to such attacks, their preventions and some take aways.

Before we dive into what did we study and talk about our observations, let’s get to some basics.

What is DNS?

DNS stands for Domain Name System, a set of protocols that translates addresses of various devices connected to the internet. It’s like a phonebook for the internet, it maps IP addresses to domain names.

As simple as it sounds on the surface, DNS has evolved quite a bit after it was created. As of now, there are tons of mechanisms running on top of DNS supported by DNS records. That leads us to,

What is a DNS Record?

DNS records are instructions stored in a DNS server for a given domain name. Below are some common DNS records and their uses.

A/AAA: Contains IP addresses associated with said domain
CNAME: Used to set given domain as an alias of another domain
MX: Specifies mail server for the said domain
TXT: Contains arbitrary text, usually meant to be read by third-parties

Only the TXT record falls under the scope of this study. A TXT record can contain arbitrary text and hence has various uses such as proving ownership of a domain to a third party and supporting mechanisms such as SRV, SPF, DMARC etc.
Feel free to read more about DNS records.

Domain Spoofing

Domain spoofing is an attack where an attacker sends an email impersonating someone else’s domain. This is frequently used in spam & phishing attacks.

This attack is a byproduct of how SMTP (Simple Mail Transfer Protocol) works; a protocol used for email communication. In SMTP, every part of an email can be forged and there is no default system in place to verify the contents.

Over time, implementations such as SPF (Sender Policy Framework) have been put in place to prevent such abuse. However, they need to be set up properly by users for mitigating the risk of email spoofing. Below is an example of modern SMTP implementation:

Let’s talk about the methods used to verify the authenticity of emails and their common misconfigurations.

SPF (Sender Policy Framework)

SPF is an email authentication method designed to validate sender addresses during the delivery of the email. SPF is used to authorize certain mail servers to send emails on the behalf of the given domain. SPF alone, though, is limited to detecting a forged sender claim in the envelope of the email. Here’s how a typical SPF record looks:

v=spf1 include:spf.gmail.com ipv4:8.8.8.8 ~all

The configuration above enforces that only the IP address 8.8.8.8 is allowed to send emails from this domain. Other conditions mentioned in the SPF record of spf.example.com should be considered by the recipient server. The ‘~all’ flag asks the recipient to mark the email as spam if these conditions are not met.

The SPF record helps prevent the spoofing of “mail-from” or “reply-to” email header. You probably have received such spoofed emails from marketing, customer support, newsletters etc.

Below are the most commonly used SPF flag:

Let’s take a look at how SPF misconfigurations can render it ineffective.

Common SPF misconfigurations

The “all” flag tells the receiver what to do in case an email fails verification. It has 4 possible configurations:

+all: accept the verification failed email
-all: reject the email
~all: accept the email but mark it as a spam
?all: disables the SPF policy as if it isn’t there

If an SPF record is configured with ‘+all’ or ‘?all’ flags, it provides no protection against domain spoofing.

If the SPF record isn’t properly formatted then according to the RFC standard, it is up to the recipient server how to interpret it. They might mark it as spam or ignore the SPF record altogether.

While parsing a SPF record, the recipient server makes DNS queries to fetch SPF records of the domains mentioned via the “include” flag as well as for the flags such as ‘a’ and ‘mx’. If the number of DNS queries during parsing exceeds 10, the SPF record becomes invalid and gets ignored.

It is important to note that the “include” flags of the queried domains are also processed and contribute towards the score.

While the recipient server is making DNS queries to the domains in the SPF record, if more than 2 query attempts return an NXDOMAIN error, the SPF record is considered invalid. This threshold is configurable but 2 is the recommended value.

In case there is more than one SPF record for a domain, the SPF record is considered invalid.

The ‘ptr’ flag in SPF records has been deprecated and discouraged because of its speed, performance tax on ARPA servers, and lack of reliability. Using this flag may not be a vulnerability in itself but certain mail providers may mark your emails as spam.

DKIM (DomainKeys Identified Mail)

DKIM is a security standard to prevent the alteration of emails between a sender and the recipient. This is how a typical DKIM record looks:

v=DKIM1; p=MIGfMA9GCSqGSIb3DQSBAQUAA4TNADCBiQKBgQC1TaNgLlSyQMNWVLRLvyY/neDgaL2oqQE8T5illJqCgDtFHc6eHVAU+nlcaGmrKmDMw9dbgiGk1ocgZ56NR4ycfUHwQhvQPMUZw0cveel/8EAGoi/UyPmqfcPibytH81NFtTMAxUeM4Op8A6iHkvAMj5qLf4YRNsTkKAV;

Common DKIM misconfigurations

If the public key mentioned in the DKIM record is too short, it becomes cryptographically insecure and can be exploited. A minimum length of 1024 bits of key size with a strong hashing function is recommended.

DMARC (Domain-based Message Authentication, Reporting & Conformance)

DKIM is a security standard to prevent the alteration of emails between a sender and the recipient. It achieves this by affixing a digital signature, linked to a domain name, to each outgoing email message. This is how a typical DKIM record looks:

v=DMARC1; p=reject; rua=mailto:dmarc@example.com

A well-configured DMARC record prevents the spoofing of the “from” email header.

Here’s a list of common flags in the DMARC standard:

Common DMARC misconfigurations

The pct flag defines the percentage of emails to be filtered.

This value should be set to 100 or it negates the protection provided by DMARC. For example, pct=30 means only 30% of emails sent from a domain will be filtered by the recipient.

As mentioned in the table above, the three valid values for the p (policy) flag are:

none: do nothing when an email fails authentication checks i.e. accept it
quarantine: accept the email but mark it as spam
reject: reject such emails

If the “none” policy is used, the protection becomes ineffective.

Similar to ‘p’ (policy) flag, the ‘sp’ (subdomain policy) flag is used for defining a policy for the subdomains. If it’s value is set to ‘none’, it excludes the subdomains from DMARC protection.

DMARC relies on either DKIM or SPF records and if both of them don’t exist or are invalid, DMARC becomes useless as well.

We started by curating more than 2.2 billion domain names using our hundreds of bots across the internet. Then, as per Project Resonance’s policy, we found an optimal rate of DNS queries to make sure our research does not disrupt anything. While we scanned about 2.2 billion domains, only 1.5 billion responded with a TXT record.

Finding security risks was rather simple, as our Attack Surface platform NVADR already has a component that scans for DNS-related issues. We simply used it on the TXT records we got from those 2.2 DNS queries. It gave us the statistics used to derive the following insights.

Results and Observations

Out of 1.5 billion domains that returned an answer to our DNS query, we found that ~32% of domains had an SPF record. Statistics for DKIM, DMARC and domains having more than two layers of protection are illustrated through the following Venn diagram:

State of Misconfigurations in SPF Records

SPF is the first line of defence against domain spoofing. However, we found out that more than 68% of the domains did not have an SPF record. That means there’s no straightforward way to determine if an email received from that domain was sent by the legitimate sender.

For the domains that have SPF implemented, let’s see how many got it right.

7.7% of SPF records were improperly formatted (invalid syntax). Including but not limited to — invalid flags, invalid delimiters, and improper use of flags.
Despite PTR being a deprecated standard, we found that about 1% i.e. 45k domains out of those having SPF records are still using it.
If SPF policy is not set to FAIL or SOFTFAIL, using SPF is pointless. Based on our study, 3.3% of these domains have an SPF policy that allows all emails to be delivered regardless of the verification.

State of Misconfigurations in DKIM Records

DKIM records are responsible for making sure that an email’s contents are not tampered with. It has been 15 years since the DKIM standard was introduced but we found that only ~0.04% of 1.5 billion domains have implemented it.

From the domains having a DKIM record, 7% of domains had improper syntax, and surprisingly all domains had a proper public key.

These numbers are concerning and it seems a lot more work and awareness is needed to increase the usage of such useful security best practices.

State of Misconfigurations in DMARC Records

Now let’s talk about DMARC, the DNS record responsible for authentication checks and reporting. DMARC’s first draft was published 9 years ago and just like the DKIM record, only ~0.036% of hosts had one.

We found that 24% of domains had a weak DMARC policy. Further, 1.1% of domains had a weak subdomain policy and 1% used a very low PCT value.

How to do it right?

For Domains without Email Services

You might own more than one domain name but some of them might not be used for sending emails.
Thus, the very first step to configuring email-related DNS records is to completely disable emails on unwanted domains. This can be done with the following SPF record:

v=SPF1 -all

This will ask the recipient mail server to reject all the emails sent from the domain.

Now, for the domains that are meant to send emails, we must properly configure SPF, DMARC, and DKIM records. Let’s start with SPF.

Properly configuring SPF

Before setting up the SPF record, you will need to know 2 things — IP addresses that you want to send emails from and if you have any third-party email managers e.g. Gmail.

Once you have that knowledge, create a TXT record like the following:

v=SPF1 ipv4:127.0.0.1 include:_spf.gmail.com ~all

Properly configuring DKIM

Generate a domain key and add the public key to your DNS records as follows:

v=DKIM1; p=your-key-here;

Once the DKIM record is in place, turn on DKIM signing on your email server/platform. You can also add “t=y” to your DKIM record to tell recipients that you are currently testing the DKIM implementation.

Properly configuring DMARC

Make sure your SPF and DKIM records are well configured before setting up the DMARC record. Once that’s done, add the following record to your DNS records:

v=DMARC1; p=reject; rua=mailto:dmarc-reports@example.com;

Where “dmarc-reports@example.com” is the email address where you want to get DMARC related reports. As per your need, you can “p=quarantine” instead of “p=reject” which will mark the unverified emails as spam instead of asking the recipient to reject them altogether.

If you want to enable DMARC for your subdomains as well, add “sp=reject” or “sp=quarantine” to the record.

Continuous / Recurring Monitoring

It is of paramount importance that these configurations are monitored more than once, ideally every day, but if that's too much, at least on a weekly basis.

In Conclusion

While talking about cyber security, we should keep in mind that it is more than a technological problem. Studies done by sources such as Verizon suggest that a major chunk of attacks are directed at people such as guessing credentials, phishing, and other social engineering-based attacks, etc. While there has been a significant growth in corporations educating their employees, attacks that negate common best practices such as “checking the sender of an email” are still a problem.

If a spoofed email doesn’t seem suspicious of the content itself, the only way to tell if it’s legitimate is quite technical. One would need to query the A records of the concerned domain and match them to the IP address from the email’s metadata.

As you can tell, doing this for every email is not practical. Instead, properly implement SPF, DKIM, and DMARC records to protect your employees and customers.

Originally published at https://redhuntlabs.com on February 10, 2022.