Lately, many of our clients have been reaching out to us saying that their emails have been landing in the spam folder or being blocked for many recipients. This seems to happen even when no changes were made to their email settings or servers. This problem is a mystery for many people. We have researched this problem and want to offer an explanation and easy solution for you.

First, I would like to describe this problem as comprehensible as possible, free from technical terms so it is easy to understand.

Where Does the Spam Email Problem Come From?

This problem has occurred only recently, and is due to the changes in email technologies over recent years. Despite this, you still have to deal with it today in 2024.

The cause of this technical change is part of the overall spam email problem in this world. To tackle the spam problem, three new email technologies have evolved and they are called SPF, DKIM and DMARC.

What is SPF, DKIM and DMARC?

For many, these are three completely bizarre acronyms that do not help describe or explain the issue. Many professional system administrators and even Linux professionals feel the same way: they have never even heard of SPF, DKIM or DMARC.

No surprise, because these technologies are relatively new. DMARC was launched in 2015, SPF in 2014 and DKIM in 2011. All three technologies are also now the standard in 2024 for almost all Internet servers worldwide.

Anyone who has not configured any of these standards for their email servers will most likely see their emails ending up in the recipient’s spam folder or are being blocked.

What About SMTP (Simple Mail Transfer Protocol)?

Without going into a deep technical answer, here’s a relatively simple explanation that should be pretty easy to understand:

Today’s email technology dates back to 1982 and was introduced to the public as SMTP (Simple Mail Transfer Protocol). It has stayed like this for almost 40 years without change as the standard email protocol. However, it comes with a lot of security issues, which are a rising concern in 2024. It is a relic of times when email spam was still unknown, and the Internet was not available to the general public.

Because of this old technology, basically anyone can falsify an email that looks like it has been sent by any sender. For example, if you want to send an email from president@whitehouse.gov, you can easily do so: just change the so-called “header” of an email before sending it by changing the line where it starts with “from:”.

For example, you could simply change “from:john@test.com” to “from:president@whitehouse.gov”, and your email would be sent as the president of the United States — that’s how it would arrive on the recipient’s email address.

This weakness is used by scammers daily to send spam from trusted email addresses you are likely to click and open. You surely must have received an email from your bank or a friend by now, which did not originate from that source — you know what I’m talking about.

Benefits of SPF, DKIM and DMARC

For the first time with these new technologies; SPF, DKIM and DMARC, servers are now able to check whether the sender is actually the original, genuine sender. SPF is the most common method and stands for Sender Policy Framework — it’s a new standard for email, which can guarantee the authenticity of the email’s sender.

Meanwhile, many email servers have gone so far that they automatically push all emails back that have no SPF configuration. This includes the largest email servers such as Google, Yahoo, Microsoft — but also tens of thousands other servers started to act that way.

Configuring SPF On Your Email Server

Anyone who has not configured SPF on their email server will end up with emails going into the spam folder or being blocked.

The first step is to get the system administrator to configure a correct SPF entry in the email server. This is what every system administrator will understand:

• An SPF entry (v = spf1 a) is to be stored in the DNS records TXT.

If the system administrator does not know what an SPF entry is, this can be easily researched on the internet.

Configuring DKIM and DMARC On Your Email Server

I will leave out other security measures around DKIM and DMARC to keep this article simple. DKIM and DMARC are mandatory for only few email servers as a security measure. For those who are also having troubles with DKIM and DMARC not being accepted should talk to their email server administrators to accept these emails. I would estimate that more than 90% have not setup DKIM and DMARC, yet. However, they are detected by almost all servers when applied as of March, 2024.

The above SPF can be set up relatively quickly. And almost all web servers and hosting providers now have instructions on how to setup an SPF entry.

Continued Troubleshooting After Setting Up SPF

SPF configured? Wonderful! Problem solved? Not quite. Despite setting up SPF, my emails end up in spam or being blocked anyways. Bummer!

Let’s have a look: So, you have created the SPF entry, and as a result, many of the emails do not end up in the spam folder of or blocked by others. Great. For example, Gmail accepts the SPF record. Yahoo, however, continues to report the sender as spam (this may change in the future).

For many system administrators this is a cause of frustration and leads to new questions. They usually ponder around what the cause might be and start to see the main issue in the DKIM and DMARC setup — only to find out that this was not the problem.

I’ve Finally Configured This Weird SPF, But Why Are Emails Still Landing In Spam or Are Being Blocked?

The source of this problem can be the receiving server’s configuration. No matter where emails are being sent from or hosted, if the IP of the email sender is being classified as spam, your email will be blocked. This wasn’t always like this .

Here’s an example:

If you are hosting your email through HostEurope, GoDaddy, BlueHost, 1 & 1, Strato or many other mass providers, your emails are most likely to be blocked . And that will happen despite a successfully configured SPF.

Problems With Shared Email Hosting & Spam

The problem is that with many of these providers, email services do not cost extra, but are included in the overall hosting plan, whether it’s an e-commerce website, WordPress hosting, or similar.

HostEurope as an example offers free and unlimited email services when you simply book a hosting plan for your domain or website. It’s definitely a good deal to have unlimited emails for free. However, these are so-called “shared hosts”, i.e. all these emails are being sent by the same or similar servers with the same IP.

To offer such a cheap solution at such low rates, these providers are forced to offer all emails as “shared services”. That is, emails are sent through a single (or a few) IPs. You are literally sharing the server with millions of other customers. And this is precisely the reason why these emails end up being blocked: These massively used IPs have been abused by tens of thousands of users, so that their IPs are now in the blacklist of many other servers.

Will An Email Spam Check of My Server’s IP help?

It also doesn’t help anymore to perform a spam check of the server’s IP, such as through services offered at mxtoolbox.com/blacklists.aspx. You will most likely see that your email’s IP and server is not listed as a spam server. That’s because spam lists are no longer maintained on public lists — servers like Gmail and Yahoo started to have their own non-public spam lists nobody can see: not being listed on a Public Spam List means nothing, anymore.

It is how it is: If you send an email through an email provider such as HostEurope to a Yahoo email, it will automatically end up in the spam folder of others or simply blocked. For example, in the case of HostEurope, Yahoo displays the following in the HostEurope email header:

• X-YahooFilteredBulk: 80.237.132.61

The number 80.237.132.61 is the IP address of HostEurope. And “X-YahooFilteredBulk” stands for the internal spam filter of Yahoo, which has classified this IP as spam. This means that Yahoo has now classified all emails coming from HostEurope as spam. With or without SPF.

Incidentally, you also can’t go to Yahoo and try to add the IP 80.237.132.61 from HostEurope to their white list. Because HostEurope is a giant hosting provider with millions of customers, and that’s why neither Yahoo nor any other server is ready to open their gates for HostEurope. You just have to accept that the IP 80.237.132.61 is being rejected no matter what.

The truth: The times where you were able to use shared email together with domain hosting are over since about 2017.

The Solution to Emails Going to Spam or Are Blocked — What Can You Do?

The solution is to stop hosting emails with mass or badly configured email servers. As a professional company, you should no longer rely on mass email providers. An external and professional email server is necessary for peak performance and customization.

The most well-known email servers are Zoho, GSuite and Microsoft. For example, Microsoft costs $15 per email and GSuite costs $4. Zoho is the cheapest with only $1 per email and also one of the most advanced email providers in the world, if not even better than its two competitors Google and Microsoft.

In the case of moving to another dedicated email server, you have to change the old MXP entries to the newly selected dedicated email server. Within a few hours, the email service then switches to the new email server.

The spam/block problem is then usually solved in most cases. SPF, DKIM and DMARC entries are often not even necessary because these servers assign a dedicated and unique IP to your email, which is not marked by any other server as spam. Even so, Zoho allows DKIM entries. So, if you want to be on the safe side, you can turn on all the standards of SPF, DKIM and DMARC.

Another solution is to host a server yourself (for example via AWS), and then configure an email service completely under its own dedicated IP. However, this is a relatively tedious and expensive configuration.