Re: Do you do HSM?

[Rich Graves <rgraves@carleton.edu>]

> 1. How many user accounts per mailbox server

3691 on one server.

> 2. How much mailbox quota per account

7GB default. Increased on demand, after some instruction on how to find/delete large attachments.

> 3. Storage architecture: do you use different storage for zimbra root
> /opt/zibmra, and zimbra backup /opt/zimbra?

/opt/zimbra/data/amavisd/tmp is a RAM disk

/opt/zimbra/backup is a NexSan SATABeast in a secondary data center, accessed via 4G FC (primary) and 1G iSCSI (failover). We do --noZip, auto-grouped backups spread over 10 days. To ensure that backups stop before 8am, I run basically /opt/zimbra/bin/zmbackupquery|head -1|awk '{print $2}'|xargs -r /opt/zimbra/bin/zmbackupabort -lb. On weekends, backups are allowed to run to completion, sometimes past noon. Effectively, a given account gets a full backup every 11-12 days.

These are all on a Compellent SAN with thin provisioned, automatic SATA/10K/15K tiered storage, multipathed over 4G FC (primary) and 1G iSCSI (failover, for when someone accidentally yanks multiple FC switches' power cords). Sizes are disk space in use, not total.

700G /opt/zimbra - original filesystem with binaries, redo, db, and one store
390G /opt/zimbra/store2 - easier to add than grow the existing store. I will probably add a store3 right before students arrive.
750G /opt/zimbra/hsm1 - email older than January 2009
440G /opt/zimbra/hsm2 - email older than January 2010

I only run HSM jobs a couple times a year, to migrate items more than 180 days old.

/opt/zimbra and /opt/zimbra/store2, but not the hsm's, are asynchronously replicated to a secondary data center. So if the primary SAN is lost we can bring up a VM with all email < 6 months old in minutes.

Local snapshots are the primary means of recovery from software fault, and the replicated copy is the primary means of DR. However, zmrestore is accessed frequently to deal with Zimbra errors and user errors, including the error of responding to phishing emails. Within the last few months, the Nigerians recently started deleting and purging stolen Inboxes, not just Sent folders.

> 4. Do you have dedicated network connection to the HSM storage?

Sharing the 4G FC is plenty.

> Will it be a performance problem if both backup and the secondary HSM
> volume share the same network connection?

As long as the link is full-duplex, I wouldn't think it would be too bad. You'll be reading from one and writing to the other.

> 5. Impact with backups

I actually don't think there would be much difference. The bottleneck is more on the writing side than the reading side.

> 6. Impact to server
> 
> Obviously moving data from primary disk to secondary disk will have
> impact to server performance. Do you let your HSM process run
> continuously?

No. The impact is not quite as bad as backups, but noticeable.

> 7. What HSM Age do you use? If you adjusted it from the default 30
> days, why?

180+. Our SAN already tiers data at the block level after 12 days. So for us, the real purpose of HSM is to prioritize more recent email for DR purposes.

> 8. When you initially turn on HSM, how long did it take for a complete
> run? How long does it take after the intial run?

I don't recall. Several days but less than a week, I think.

Does HSM still conflict with (one stage of) backups?