Hi Matt,
At SFU, we've run three different solutions over the past decade:
- home grown (Spam Assassin+McAfee AV and our own Milter to tie it into Sendmail) - about 4 years
- Barracuda, both appliances and VMs - 4 years
- Proofpoint (VMs) - entering our second year
The SA solution was obviously cheap, but I was the only one who could manage it. And I didn't really have the time (or inclination) to devote a few hours a week to tweaking rules (I know other sites running SA who do this), so a fair amount of spam got through and a LOT of phishing did
The initial Barracuda install was a single hardware appliance, with a second one added a year later. Their claim of 10,000,000 msgs per day for their large appliance was BS - it was falling over dead at 2m (hence the second appliance). After 3 years, we switched to their VMs as it was cheaper to buy a bunch of VM licenses than renew maintenance on the hardware. Their VMs are licensed by size - the more CPU and memory you want, the more you pay. We went for relatively small ones (2 CPUs) and they were only $1500/yr each. We did 4 VMs. By and large this handled the volume, but whenever there was a spike (e.g. Alumni mailout, which actually came from off-campus from a bulk-mailer), the load was spike way up and mail would get delayed (sometimes hours). Often one or more of the VMs wouldn't recover properly without a reboot. This happened on a fairly regular basis (every month or two). You couldn't just throw more resources at a VM because they were license-restricted. Their spam blocking rate was fairly good, but their Barracuda BL was pretty aggressive, so we'd often get legitimate sites blacklisted
Before going with Barracuda, we did get quotes from Sophos and IronPort. The IronPort one stood out for its gold-plated price. It was nearly a million dollars for appliances + 3 years licenses. This was quite a few years ago now, but obviously the university balked at that. At the time, we were running open-source everything for email, so there was no appetite to pay for it.
We eval'd Proofpoint just over a year ago. We looked at their VM solution as it was easier than messing with appliances. They're licensed based on number of users and you can throw as many VMs (as big as you want) at the problem as you want. But it's not a strict license where if you have more than 'x' accounts on the cluster it won't work - the number of "official" users is all just part of the negotiation.
We ended up licensing just the basic anti-spam/virus functionality - we haven't licensed DLP, spear phishing. encryption, or any of those other add-ons. It is substantially more per year than the Barracudas but a LOT less than the IronPort quote we got. Their initial quote was way high, but there was a lot of room to negotiate. We just renewed for 3 years and got a further price break for doing a 3-year renewal.
For the first year, after initial setup, I didn't touch the Proofpoints at all. One of my colleagues used the web UI occasionally when we got reports of false-positives and had to release a message from quarantine, but that's about it. I will probably have to do some performance tuning though, and adding more (or resizing) VMs, because there's been a definite uptick in spam this fall and it's impacting load somewhat. Right now, we're not using many of the features of the product, but I do intend to turn on more things when I get time (e.g. load the box with our list of users so it can resolve users' multiple aliases to the same person, then turn on daily digest generation; and use the box for outbound filtering to catch compromised machines). I've heard there's a Zimlet in the works to integrate Proofpoint at the user level so that users can manage their own whitelists from within Zimbra. I haven't investigated this yet. At purchase time last year, the only "zimlet" that existed was an admin script that could take the spam and ham corpus from inside Zimbra and fire it at Proofpoint for training (at HQ, not at your own box though, so not that helpful)
I do have one major complaint about all of the commercial solutions though - none of them tell you what their rules do. So if you do get a false-positive, you can see what rules matched, but they're usually cryptic and don't include the weighting, so you can't determine why a particular message was marked as spam. The companies protect this info so even as a paying customer you can't pry it out of them. They just say "submit the message to us".
One note about Proofpoint's blacklisting based on IP: While they do support Spamhaus et al, their internal system, while IP based, still allows the message through (but throws it into quarantine). This is good if a site has been wrongly blacklisted as a user can still get their mail (by releasing from quarantine), but bad in that it substantially increases the volume of mail you handle (still not much in comparison to other campus traffic - our peak usage for inbound mail over the last 7 days was about 1.5GB/hour)
Hope that helps - feel free to contact me if you want more info.
Interesting that ProofPoint is even more expensive. Yes....that is the other consideration.....should we even still be doing email ourselves. For some people that is a hard sell. Some see it as a security issue, some see it as losing full control, but I know a lot of schools are doing that.
Matt
From: "Tom Golson" <tgolson@tamu.edu>
To: "zimbra-hied-admins" <zimbra-hied-admins@sfu.ca>
Sent: Wednesday, October 2, 2013 6:47:19 PM
Subject: Re: Reviewing Spam Appliance Options
Hi Matt,
Texas A&M also uses IronPort, but we are currently looking at ProofPoint. However, it is a more expensive option, not a less expensive option. If you really want to not spend on providing email, send the service to Office365 or Google Apps.
--Tom
--
Steve Hillman IT Architect
hillman@sfu.ca Institutional, Collaborative, & Academic Technologies (ICAT)
778-782-3960 Simon Fraser University