Thank you for Signing Up |
Lately, many of our clients have been reaching out to us saying that their emails have been landing in the spam folder or being blocked for many recipients. This seems to happen even when no changes were made to their email settings or servers. This problem is a mystery for many people. We have researched this problem and want to offer an explanation and easy solution for you.
First, I would like to describe this problem as comprehensible as possible, free from technical terms so it is easy to understand.
Where Does the Spam Email Problem Come From?
This problem has occurred only recently, and is due to the changes in email technologies over recent years. Despite this, you still have to deal with it today in 2024.
The cause of this technical change is part of the overall spam email problem in this world. To tackle the spam problem, three new email technologies have evolved and they are called SPF, DKIM and DMARC.
What is SPF, DKIM and DMARC?
For many, these are three completely bizarre acronyms that do not help describe or explain the issue. Many professional system administrators and even Linux professionals feel the same way: they have never even heard of SPF, DKIM or DMARC.
No surprise, because these technologies are relatively new. DMARC was launched in 2015, SPF in 2014 and DKIM in 2011. All three technologies are also now the standard in 2024 for almost all Internet servers worldwide.
Anyone who has not configured any of these standards for their email servers will most likely see their emails ending up in the recipient’s spam folder or are being blocked.
What About SMTP (Simple Mail Transfer Protocol)?
Without going into a deep technical answer, here’s a relatively simple explanation that should be pretty easy to understand:
Today’s email technology dates back to 1982 and was introduced to the public as SMTP (Simple Mail Transfer Protocol). It has stayed like this for almost 40 years without change as the standard email protocol. However, it comes with a lot of security issues, which are a rising concern in 2024. It is a relic of times when email spam was still unknown, and the Internet was not available to the general public.
Because of this old technology, basically anyone can falsify an email that looks like it has been sent by any sender. For example, if you want to send an email from president@whitehouse.gov, you can easily do so: just change the so-called “header” of an email before sending it by changing the line where it starts with “from:”.
For example, you could simply change “from:john@test.com” to “from:president@whitehouse.gov”, and your email would be sent as the president of the United States — that’s how it would arrive on the recipient’s email address.
This weakness is used by scammers daily to send spam from trusted email addresses you are likely to click and open. You surely must have received an email from your bank or a friend by now, which did not originate from that source — you know what I’m talking about.
Benefits of SPF, DKIM and DMARC
For the first time with these new technologies; SPF, DKIM and DMARC, servers are now able to check whether the sender is actually the original, genuine sender. SPF is the most common method and stands for Sender Policy Framework — it’s a new standard for email, which can guarantee the authenticity of the email’s sender.
Meanwhile, many email servers have gone so far that they automatically push all emails back that have no SPF configuration. This includes the largest email servers such as Google, Yahoo, Microsoft — but also tens of thousands other servers started to act that way.
Configuring SPF On Your Email Server
Anyone who has not configured SPF on their email server will end up with emails going into the spam folder or being blocked.
The first step is to get the system administrator to configure a correct SPF entry in the email server. This is what every system administrator will understand:
- An SPF entry (v = spf1 a) is to be stored in the DNS records TXT.
If the system administrator does not know what an SPF entry is, this can be easily researched on the internet.
Configuring DKIM and DMARC On Your Email Server
I will leave out other security measures around DKIM and DMARC to keep this article simple. DKIM and DMARC are mandatory for only few email servers as a security measure. For those who are also having troubles with DKIM and DMARC not being accepted should talk to their email server administrators to accept these emails. I would estimate that more than 90% have not setup DKIM and DMARC, yet. However, they are detected by almost all servers when applied as of October, 2024.
The above SPF can be set up relatively quickly. And almost all web servers and hosting providers now have instructions on how to setup an SPF entry.
Continued Troubleshooting After Setting Up SPF
SPF configured? Wonderful! Problem solved? Not quite. Despite setting up SPF, my emails end up in spam or being blocked anyways. Bummer!
Let’s have a look: So, you have created the SPF entry, and as a result, many of the emails do not end up in the spam folder of or blocked by others. Great. For example, Gmail accepts the SPF record. Yahoo, however, continues to report the sender as spam (this may change in the future).
For many system administrators this is a cause of frustration and leads to new questions. They usually ponder around what the cause might be and start to see the main issue in the DKIM and DMARC setup — only to find out that this was not the problem.
I’ve Finally Configured This Weird SPF, But Why Are Emails Still Landing In Spam or Are Being Blocked?
The source of this problem can be the receiving server’s configuration. No matter where emails are being sent from or hosted, if the IP of the email sender is being classified as spam, your email will be blocked. This wasn’t always like this .
Here’s an example:
If you are hosting your email through HostEurope, GoDaddy, BlueHost, 1 & 1, Strato or many other mass providers, your emails are most likely to be blocked . And that will happen despite a successfully configured SPF.
Problems With Shared Email Hosting & Spam
The problem is that with many of these providers, email services do not cost extra, but are included in the overall hosting plan, whether it’s an e-commerce website, WordPress hosting, or similar.
HostEurope as an example offers free and unlimited email services when you simply book a hosting plan for your domain or website. It’s definitely a good deal to have unlimited emails for free. However, these are so-called “shared hosts”, i.e. all these emails are being sent by the same or similar servers with the same IP.
To offer such a cheap solution at such low rates, these providers are forced to offer all emails as “shared services”. That is, emails are sent through a single (or a few) IPs. You are literally sharing the server with millions of other customers. And this is precisely the reason why these emails end up being blocked: These massively used IPs have been abused by tens of thousands of users, so that their IPs are now in the blacklist of many other servers.
Will An Email Spam Check of My Server’s IP help?
It also doesn’t help anymore to perform a spam check of the server’s IP, such as through services offered at mxtoolbox.com/blacklists.aspx. You will most likely see that your email’s IP and server is not listed as a spam server. That’s because spam lists are no longer maintained on public lists — servers like Gmail and Yahoo started to have their own non-public spam lists nobody can see: not being listed on a Public Spam List means nothing, anymore.
It is how it is: If you send an email through an email provider such as HostEurope to a Yahoo email, it will automatically end up in the spam folder of others or simply blocked. For example, in the case of HostEurope, Yahoo displays the following in the HostEurope email header:
- X-YahooFilteredBulk: 80.237.132.61
The number 80.237.132.61 is the IP address of HostEurope. And “X-YahooFilteredBulk” stands for the internal spam filter of Yahoo, which has classified this IP as spam. This means that Yahoo has now classified all emails coming from HostEurope as spam. With or without SPF.
Incidentally, you also can’t go to Yahoo and try to add the IP 80.237.132.61 from HostEurope to their white list. Because HostEurope is a giant hosting provider with millions of customers, and that’s why neither Yahoo nor any other server is ready to open their gates for HostEurope. You just have to accept that the IP 80.237.132.61 is being rejected no matter what.
The truth: The times where you were able to use shared email together with domain hosting are over since about 2017.
The Solution to Emails Going to Spam or Are Blocked — What Can You Do?
The solution is to stop hosting emails with mass or badly configured email servers. As a professional company, you should no longer rely on mass email providers. An external and professional email server is necessary for peak performance and customization.
The most well-known email servers are Zoho, GSuite and Microsoft. For example, Microsoft costs $15 per email and GSuite costs $4. Zoho is the cheapest with only $1 per email and also one of the most advanced email providers in the world, if not even better than its two competitors Google and Microsoft.
In the case of moving to another dedicated email server, you have to change the old MXP entries to the newly selected dedicated email server. Within a few hours, the email service then switches to the new email server.
The spam/block problem is then usually solved in most cases. SPF, DKIM and DMARC entries are often not even necessary because these servers assign a dedicated and unique IP to your email, which is not marked by any other server as spam. Even so, Zoho allows DKIM entries. So, if you want to be on the safe side, you can turn on all the standards of SPF, DKIM and DMARC.
Another solution is to host a server yourself (for example via AWS), and then configure an email service completely under its own dedicated IP. However, this is a relatively tedious and expensive configuration.
Thank you for Signing Up |
I'm just using my personal email- for personal things- it's a gmail account..... and people are telling me my emails are going to their spam folder. For anything and everything.
I started a kitchen remodel project- the contractor, the designer, the flooring guy- I've emailed them individually and none of them get it. I messaged two campgrounds to get reservation info- both of them said my emails went to their spam.
I fear that my email has been added to some sort of blacklist? Help! It's just a gmail account that I use through gmail.
I am in the US. This email is not the email for my business. I am using GoDaddy as my host company and email company through Microsoft. Though I think at this time my email is direct through Microsoft. My business emails are going into clients' spam, which could cause me a lot of business. Any suggestions on how to find out what to do to ensure my emails are not sent as spam?
Hello Zai,
That might be the reason why your email goes to spam, because you are using the free email service. My advice would be to switch to a professional email provider, e.g. by changing your MX Records. Good and reliable email hosts I know of are Zoho G Suite, and Microsoft.
I hope this helps.
Have a wonderful day,
William
Hello Dr. William Sen,
This is a very clear explanation of the problem why my emails are going into spam.
I am using the free email service from my small business provided by my ISP. I don't send bulk emails through this service but recently am getting complaints tat my emails end up in spam folders. I get notified from people I t4rust wh I deal with regularly but it may be also happening to others whi I email once in a while as most people do.
I really appreciate the advice and will look to change my email server to another suitable service.
Thanks!
Zai
Wow....a clear answer to a tech question. I'm astonished. I think my problem is the shared hosting issue... Zoho looks like the answer. Thank you.
Awesome post William! Thanks for sharing such informative stuff. Many times my emails also go in spam and I become confused why it is going in spam now have found the exact reason.
Sorry, but Microsoft 365 Personal is USD$59.99 / year (microsoft.com/en-ww/microsoft-365/outlook/outlook-personal-email-plans?market=af). Nice article, thanks.
Many thanks for this article - even though I already know about SPF, DKIM and DMRC, it has provided me with some very useful info and better a understanding of the problem.