So, after finally getting Postini deployed and enabled for all accounts, the Postini stats report that just over 50% of incoming mail is being tossed out as “blatant spam,” with over 25% of the remaining mail being quarantined as “potential spam.”  This is with Postini on the default lowest aggressiveness setting.  So, assuming the worst case (“all incoming mail is spam”, which is almost true), our users are already receiving less than 37.5% as much spam as they used to.  Once we’re satisfied that things are working fine, I’ll bump up the aggressiveness.

Anyways, I noticed that my FirstClass inbox was still being bombarded by spam messages (a few per hour), most of them obviously spam.  Why weren’t these being blocked by Postini?

Checking the headers on these spam messages, I noticed that they weren’t being routed through the Postini servers.  This means that the spammers are ignoring the MX records for our domain and delivering mail directly to the FirstClass server.  They must have cached the old MX record and kept using it after we switched it to point to a Postini server, because what spammer wants to send mail through Postini if they have the choice?  Anyway, yuck.

Turns out that this is not an unknown problem.  I find it pretty interesting, though.  I didn’t know that spammers did this since I had never bumped into this situation before; just another pretty smart thing that the spammers are doing to get around your efforts to stop them.

Anyway, the solution in this case is to set the mail server (the FirstClass server in this case) or a firewall in between your mail server and the Internet to only accept connections from where it should be coming from.  Seems simple enough?  We are actually routing the mail through Google Apps, so the answer was to only allow connections from addresses that Google’s SPF record says that mail should be coming from.

209.85.128.0/17
216.239.32.0/19
64.233.160.0/19
66.249.80.0/20
72.14.192.0/18
66.102.0.0/20
74.125.0.0/16
64.18.0.0/20
207.126.144.0/20
173.194.0.0/16

Anyway, I set these filters in our firewall and, presto, no spam messages all weekend.

First of all, I’ve never liked to install the Microsoft Office plug-in or Outlook e-mail scanner plug-in that comes with AVG. In my experience, this has led to it wanting me to quit Outlook whenever it wants to install an update (or reboot the machine after the update is complete), which is a bother, and I don’t think it’s a very big risk to leave those components out. So, on my system with only the “basic” e-mail scanner installed, it is only able to scan mail sent through SMTP.

I recently switched to Thunderbird and I have one of my accounts set up to send through SMTP, which means AVG is able to intercept the connection and scan the mail. Even though I do not have AVG set to “certify mail,” it still makes some modifications to the message. Perhaps the most obvious is a message that it adds to the headers of the message…

Received: from 127.0.0.1 (AVG SMTP 8.0.169 [270.6.15/1649]); Wed, 03 Sep 2008 15:30:57 -0500

Of course, it makes sense for it to put this there, because it is actually intercepting the TCP connection used to send the mail and then relaying the mail to the server you specified in your e-mail client. Adding these relay messages to an e-mail will not make it fail validation as they are not covered by the digital signature.

However, oddly, AVG makes some other modifications to the message, as we see in this diff between the mail sitting in my Thunderbird “sent” folder and the mail that showed up in the recipient’s inbox:

I know that AVG caused this because I duplicated the behavior several times, and it stopped right after I disabled AVG’s mail scanner.

There are a number of other small changes like this. These modifications are to the body of the e-mail message, which is covered by the digital signature. The result is, the recipient cannot verify the authenticity of the message, as it appears to have been modified since it was sent (duly so).

Why were these modifications made to the message? Are they just strange errors, or is AVG actually trying to make the style information in the message more consistent for some reason? Whatever the case, your anti-virus software should not be modifying mail that you send, especially without informing you that it is doing so, whatever the modifications may be.

So, here we have a plus for digital signatures and a minus for AVG. I don’t seem to be able to communicate this to AVG support because I am using the free version. If any of you have a paid AVG anti-virus product, maybe you could pass the word along?

Anyway, I now am running AVG without any mail scanning at all. (I hope that common sense in dealing with attachments can keep me safe. )

Google Apps for your domain is great. For less than $10/year, you can get yourself a domain name. With this service from Google, you can set up things like Google Docs, Google Talk, and Google Calendar to run on your domain name. Perhaps the coolest one is Gmail. Google will host the e-mail on your domain for free and give you the nifty Gmail web interface on top of that.

But I’ve never been one for web mail. I’d rather use a desktop e-mail client, like Outlook or Thunderbird, mostly so that I can check all of my different e-mail accounts through one interface, utilize S/MIME digital signatures and encryption, and file all of my incoming e-mail without having to worry about running out of online storage space. (Granted, Gmail’s online storage space is very generous and makes this a non-issue… And it also pushed Yahoo! and [Microsoft] Hotmail into offering large online storage for mail as well.)

Fortunately, Google offers POP3 and IMAP access to your Gmail mailbox, so that you can use virtually any mail client with their service. This is also free. Great, I can have them host my e-mail, that means I don’t have to worry about setting up a server and making sure that it is up all the time, or pay someone else to run a mail server for me. And, I can create whatever account names I want, they will all be at my domain, and I don’t have to worry about name collisions with existing Gmail users.

Should I use POP3 or IMAP to access my mail?

IMAP may seem to be the obvious choice. You get access to the full Gmail “folder structure,” including the spam folder, trash, sent items, and all of your archived messages.

I’d rather use POP3, myself. My connection is a little slow, and in either Outlook or Thunderbird, when I try to view one of my Gmail folders via IMAP (the Inbox or whatever), the application becomes non-responsive while it connects to the IMAP server and refreshes. If I use POP3, mail is downloaded to a local store on my machine, automatically deleted on the Gmail server (preferred behavior for me), and the local mail client runs smoothly.

Enter Gmail’s spam filter. Google looks at all incoming mail and delivers suspicious e-mail to your “spam” folder instead of your inbox. There’s nothing wrong with this… In fact, I’d expect any respectable mail provider to do some spam filtering like this, as spam is a huge problem nowadays. Anyway, I have a problem with how this is implemented against your client mail application, whether you are using POP3 or IMAP.

When using POP3, only messages in your inbox are downloaded to your mail client. Messages in the spam folder are not downloaded. This means I have to remember to go to Google’s web interface and check to see if any messages have been misclassified as spam from time to time. (Don’t give me any “Google’s spam filter is great, you don’t need to worry about it!” talk. I had a misclassification that started me on this whole investigation, so that I don’t have to worry about it anymore.)

When using IMAP, at least you have access to the spam folder, so you can easily see if any mail has been misclassified. However, I use my mail client to check four different e-mail accounts, and only one of them is provided by Gmail/Google. If I want to take advantage of the mail client’s spam filter, this throwing the spam out of my inbox is annoying — I may have to check multiple folders for junk and I’d like all of the suspected junk from all of my accounts to go to one place.

Ideally, I’d just set it to POP3, disable the server-side spam filter, and do all of the filtering on the client. Google doesn’t give us this choice, though, there is no option to turn off the spam filter or even to change its behavior in any way. So, I’m stuck using the laggy IMAP account, because I want to keep an eye on the spam folder.

I’ve been searching the Internet for solutions to this problem. Lots of people have posed the question “How do I download the contents of my spam folder via POP3?” Maybe someone has answered that question, but I didn’t find an answer after checking 20 or so pages. Most of the responses were, “You can’t,” or, “Use IMAP instead.”

But today, I believe I figured it out! Here we go.

• Log into Gmail, go to Settings, and then filters.
• Create a new filter.
• In the from field, type: *
  (That’s right, a single asterisk.)
  Leave the other fields blank.
• If you do a “Test search”, it should match all of the mail in your account (except mail that is already in the spam folder).
• Click on “Next step,” and then check the box “Never send it to Spam.”
• Save the filter. Now no incoming mail should end up in the spam folder, which means your POP3 client is free to download it.

Thanks Google for making this straight-forward. Hope my little rant helps someone.

I mostly like the default Outlook message layout, but I don’t really have any use for the “Categories” or “Flag Status” things that it wants to display next to every message by default. You can remove these columns, sure, and doing this manually on each folder in a small set of folders is not too much trouble, but it starts getting annoying to make changes to each folder once you have a lot of them. And then there’s the thought of, if I ever migrate this to another machine, will I have to do this all again?

So, I investigated how to modify the default Outlook view. It’s not very hard, but it’s not very obvious either.

First, start Outlook with the /cleanviews switch. That is, go to Start -> Run and type “outlook /cleanviews” (without the quotes). This will reset view changes you have made to any folders.

Now, go to your Inbox, and go to “View -> Current View -> Define Views.” You’ll be presented with a list of views. Click on one (like “Messages”) and select “Modify,” and then change it however you want. When you’re done, click on Apply View. Now, any folder that uses that view should have been changed. “Messages” is the default view for most e-mail folders (and all of the folders should be using the default view after you started with the /cleanviews switch).

This seems to mostly work, but I’ve seen a number of behavioral oddities. For example, when I first open a folder, the “Categories” box is usually still next to all of the messages, even though I turned it off in the Messages view. However, if I click out of the folder and back in again, they are gone.

Anyway, after messing with it for a while and visiting all of my folders, everything seems to be set up like I like it.

So, there you have it. Good luck changing your Outlook views.