Is spam so bad that we no longer care about legit e-mail?

Recently a couple of servers I know were blacklisted, one by AT&T and another by Securence's ShieldWall (these are two different servers in two different cases). In both cases these companies treated the legit mail server like a criminal without any way for us to contact them and get something other than a form letter back. In the case of Securence it was even worse, we were not allowed to contact them at all unless we were a customer. Whaaa? Looking at the mail logs nothing was out of order, the server was not compromised, spam was not being sent but still they were blacklisted. The really scary thing here is that I'm starting to refer to servers as people now, so this does not bode well for me.

Since the servers in question are legit, not sending spam and yet were blacklisted we started trying to figure out what happened. Neither AT&T nor Securence is willing to let you know why you're blacklisted, how to get off the blacklist or any ways to prevent it from happening again. The logs didn't show anything abnormal and the companies were not willing to work with us so we had to hound on them to get the data we needed. That pretty much failed too. This means that the legitimate mail that was flowing through these servers was being blocked by AT&T and Securence which is also known as a false positive in the spam blocking world.

Blocking spam is actually very easy. Blocking only spam and having no false positives, now that's hard yet very important. The latest stats I've seen state that 95% of all e-mail is spam which means that clearly we have a huge problem here. It seems that larger ISPs like AT&T and smaller spam filtration services like Securence are only worried about blocking spam at all costs without any care for false positives. If more and more companies start to take this approach, the value of e-mail will start to decrease.

Before you think you're not vulnerable to this ask yourself if you're 100% sure that you're getting all your messages. Systems like AT&T and Securence simply block the message and send absolutely no notification to the recipient that the message has been blocked. The only way to know if something has been blocked is that if the *senders* e-mail server is configured to send a deliver bounceback, which many servers are not set up to do. This means that there would be no notification of a delivery failure and no notification of a blocked message sent to anyone at any time. How much business have you lost, how many parties did you miss, how many messages from loved ones got blocked simply because your ISP or mail provider didn't provide the proper tools to alert you of a possible false positive?

I'm not saying that these companies have to block only spam and never have a false positive. That's unreasonable. What irks me is the idea that both AT&T as well as Securence seem to think that they are better at this than everyone else (I wonder what would happen if Securence blocked a message from AT&T). Neither company gave us the tools to determine what was wrong and fix any potential issues to allow them to pass our legitimate mail. Instead they simply blocked the message, told us to go away and we can't contact them about it. Or in other words their customers getting these messages was less important than blocking a message about Viagra.

The solutions are simple yet incredibly complex. With the amount of spam that we have to deal with it only seems logical to rip out the SMTP/POP/IMAP infrastructure that we have today and replace it with something new. We need to be able to authenticate the senders and have a system in place to verify every message. Looking to the future I see little way around this, but maybe a clever hack will be created. No too long ago we thought we were going to run out of V4 IP addresses, then a clever little hack called NAT came along. Hopefully something similar will help us in e-mail. The second solution is to have companies stop using any spam filtering service that thinks even a single false positive is OK. The best way to do that is to purchase a spam firewall such as a MailFoundry or IronMail (I would personally stay away from Barracuda as their support is simply awful). By running your own spam firewall you're able to better protect yourself, have a log of messages in case something gets improperly held and the senders who are getting blocked get real data as to what happened, how to fix it and who to contact all within the mail header (unlike AT&T and Securence).

In the end it turns out that the server blocked by Securence happened because a single user (of 20,000 users) sent a company-wide message to 2,500 people asking for help, but had 70 old e-mail addresses that were no longer legit. Securence put us into a blocking pattern while the mail server tried re-sending every 2 hours since the Securence server never sent a proper 'mailbox not found, reject' message to the sending server. Thinking that we had a delivery failure our server did what it was supposed to do and made the situation worse and worse until finally our side gave up and send a bounceback to the original user. We still have no idea what happened on the AT&T side but after filling out form after form after form we were finally able to get off their blacklist. I have no idea if it will happen to that server again, or to a different server. In the end these companies, in my opinion, are just as bad as the spammers themselves in destroying the e-mail infrastructure. It's time to make companies like this obsolete and replace e-mail with something that works. I fear that won't happen soon enough. It will take a global e-mail disaster of massive proportion to force change here as most users sit by thinking everything is just fine. 95% spam, higher and higher false positive rate and taking an anti-customer approach to e-mail. Everything is not just fine. Viva La Revolucion!

On my mail server, I send a daily report showing any messages blocked and the reason.

You'll notice that the DNS checks block a huge amount of spam to the address the spammers have got hold of, but it occasionally blocks a legitimate message from a misconfigured site. The report lets the user know this has happened and gives a brief description they can pass on to me or the administrator of the site who tried to send the message.

We had a similar issue due to a blacklist service called Outblaze, which decided to blacklist our server within about a month. Our email list is double opt-in,

Yusuf Goolamabbas of Outblaze describes the data as coming from spam trap message body analysis: "We have an extensive set of spamtraps. The emails to these accounts are analysed and URIs are extracted. For any domain found in these emails, we check for 'new' and if so, its blocked." In this case, 'new' is defined as the domain being registered within the last 90 days. This newness should help prevent legitimate domains from being listed since spammers routinely register new domains, use them for a only a few days or weeks, then move on to a fresh domain for their spam advertised sites.

In other words, we were blacklisted basically because a) we're new, b) we send HTML email with images and c) we're talking about things like money and the economy. That's messed up.

Scott, it is messed up. That's completely the wrong attitude that spam filtration services should be taking. Something closer to what Mokel is doing seems like a good fit, allow the end user to see everything just in case, and whitelist users if they are getting caught incorrectly. Spam services are not smarter than the human brain.

I'm not sure if the monetary solution can be implemented and I fear that the Government will get involved and start taxing e-mail if we go down that road. Worth discussing as an option though.

Spam has gotten so bad that our spam filters no longer allow legitimate email to come through. All I get is the new spam that my filters have not figured out yet.

Teresa, if you run your own mail servers or have influence over IT at your company, check out MailFoundry.com. If you don't control your mail server but do control your own domain name, try their hosted service: http://www.mailfoundry.com/anti-spam/hosted_anti-spam/ or now that GMail supports IMAP, Google for Business might have everything you're looking for as well.

We are having the same issue where we are being blocked when sending emails to our congregation that use sbcglobal/ att accounts. Can you point me in the right direction to get the "forms" that you had to fill out to get whitelisted. Would be greatly appreciated.

For AT&T you'll want to send an e-mail to abuse_rbl@att.net and ask to be removed. From that point forward it's all form letters and you'll never get a real person. To date I've gotten nothing back but form letters and have no idea who to actually call or how to get someone who can answer one simple question: why was I blocked?

Comments

1. Posted by: Shannon on October 23, 2007 7:05 PM:

I heard a idea long ago that i would love to see implemented. If there would be a .02 postage for every email it would take care of the vast majority of our email problems. I am not sure how this would be implemented or who would get the money. Could each person pick where their postage money goes? Mine could go toward Habitat for Humanity someone else could pick another approved charity. For a legitimate user the cost would be insignifigent, but it would ruin the spammers business plan. I know this plan is not perfect but something does need to be done or the whole medium will be lost.

Name:

Email Address:

URL:
Remember personal info?

Comments:

HTML Tags you can use in your posts:
<b>Bold</b> = Bold
<i>Italicized</i> = Italicized
<a href="http://www.othersite.com">Link to Other Site</a> = Link to Other Site

Please keep comments on-topic. Contact authors or other commenters
directly for off-topic conversations.

Notify me of future comments via e-mail

Technology Evangelist Digest - Free Newsletter

Sign up for the free Technology Evangelist Digest to receive daily updates, editorials, and practical advice on emerging technology trends in hardware, software, webware, marketing and beyond.

Technology Evangelist Digest will keep you up to date on the technology trends that will help make you more productive and efficient both in business and your personal life.

Let's face it: If you made it to this line, you must have found something valuable on this page, right? Think about how cool it would be to have something free and interesting to read every day from Technology Evangelist by signing up today.

1. Fill in your email below,
2. Then click on the confirmation email you receive.
3. That's it. Your first Technology Evangelist Digest will arrive within 24 hours.