Re: Let's get the ball rolling: Hardware requirements for Zimbra

[Steve Hillman <hillman@sfu.ca>]

Forwarding in a bounced posting:

----- Forwarded Message -----
From: tgolson@neo.tamu.edu
To: hillman@sfu.ca
Sent: Wednesday, March 5, 2008 5:40:01 AM (GMT-0800) America/Los_Angeles
Subject: Re: Let's get the ball rolling: Hardware requirements for Zimbra

Last month we went live with ~62,000 mailboxes and nine servers.  We're running four, four-socket, dual-core IBM LS41 Opteron blades with 16 GB RAM each, for the mailstores, and then we have three, daul-socket, quad-core HS21 Xeon blades with 4 GB RAM for LDAP and two more for MTA's.  The entire complex is running the 64-bit version of Suse Linux Enterprise Server 10.

We run the proxy service on the mailstores and do most of our spam/virus filtering further out, but using a similar postfix/amavis environment.  The mailstores are backed by an IBM N5500 (NetApp 3050, I believe) with clustered controllers that we connect to over iSCSI.  And all of that is living behind a pair of F5 load balancers.

We'd initially configured our hardware for an upgrade to Sendmail's Mailcenter product, but that wound up falling apart at literally the last minute.  So, Zimbra was our emergency, fallback solution, that we were able to migrate to and go live on in less than 90 days.  The biggest downside for us is that we really had to shoe-horn the migration into what we had.  About the only change to hardware we were able to make before going live, was to bump the mailstore servers from 8 GB RAM to 16.  That was a life-saver! 

Zimbra doesn't seem to be computationally intensive (8 cores is kind of overkill), but it loves memory.  Even with more, smaller servers, I'm not sure how small the connecting population would have to be, per server, to get away with less than 8 GB of RAM on a mailstore.  We'll probably be adding two more LS41's to the mix, in order to give folks a warmer fuzzy about capacity, but we're really not doing too badly, right now.  We just don't have a lot of extra capacity for unexpected demand peaks.  But, we are delivering around 300,000 messages a day to the complex, and supporting peak usage accross the complex of about 6,000 concurrent connections.

A couple of us are in Sunnyvale, this week, so maybe we can dig up some of those sizing and/or tuning gems to pass along.

--
Tom Golson
Senior Lead Systems Engineer
Open Systems Group
Computing and Information Services
Texas A&M University
(979) 458-4373

----- Original Message -----
From: "Damion Alexander" <daalexander@vassar.edu>
To: "Steve Hillman" <hillman@sfu.ca>, zimbra-hied-admins@sfu.ca
Sent: Tuesday, March 4, 2008 5:40:21 PM GMT -06:00 US/Canada Central
Subject: Re: Let's get the ball rolling: Hardware requirements for Zimbra


Steve Hillman wrote:

> But I need to plan for the next step beyond pilot, because our data centre hardware lease comes due this summer. Assuming we go full production with Zimbra, we'll have about 50,000 accounts. I'd like to hear what other sites are running on, especially those of you who have already gone production - were your hardware choices the right ones, or are you having to scale up?
> 
> Thanks!
> 

We're running on 6 IBM HS21 Dual QuadCore Xeon Blades with 10G of 
Memory. RHEL4-64 is installed on internal harddrives with the mail 
stores, backup file and redolog each stored on their own LUNs on the 
NetApp. 2x LDAP Servers, 2x Mailboxes (with Proxy service) and 2x MTAs 
(AV/AS handled by IronPort appliances).

We have 6k accounts and the only sizing issue we have run into really is 
storage related.

During the initial changeover we used imapsync and some custom scripts 
to pull information from our old system and put it into zimbra.  The 
problem arose where the redolog was tracking all of the migrated mail. 
So we kinda panicked and gave the redolog it's own LUN as large as the 
mailstore (600G for each mailbox server). Turns out that that was really 
only needed for the migration as the redolog only needs about 100M or so 
  (for our volume :) ).

Our other issue is backing up said data.  Full zimbra backups for about 
3k accounts (with the zip option) takes about 2 days to complete. 
Without the zip option our NetVault tape system starts to choke on the 
number of files its processing  nightly.

If I could do it all over again, barring any insight zimbra (or anyone 
else) could give to better size storage, I would make many smaller 
mailbox servers instead of a few big ones.


Damion

-- 
Damion Alexander
Systems Administrator
Vassar College
124 Raymond Ave
Box 13 - Computer Center 205
Poughkeepsie, NY 12604
(845) 437-7759




-- 
Steve Hillman                                IT Architect
hillman@sfu.ca                               IT Infrastructure
778-782-3960                                 Simon Fraser University
Sent from Zimbra