Re: Do you do HSM?

[Xueshan Feng <sfeng@stanford.edu>]

This is very helpful. Please keep your feedback coming. I will post summary sometime next week.  

Thanks everyone! 

Xueshan

----- Original Message -----
> > 1. How many user accounts per mailbox server
> 
> ~800 on one server.
> 
> > 2. How much mailbox quota per account
> 
> Unlimited. Largest mailbox is 58G. Our top 25 are all 10G+.
> 
> > 3. Storage architecture: do you use different storage for zimbra
> > root
> > /opt/zibmra, and zimbra backup /opt/zimbra?
> 
> Size Volume
> 20G /opt/zimbra
> 50G /opt/zimbra/db
> 100G /opt/zimbra/index
> 20G /opt/zimbra/log
> 100G /opt/zimbra/redolog
> 450G /opt/zimbra/store
> 1T /opt/zimbra/store2
> 4T /opt/zimbra/backup
> 
> All of the above volumes, except for store2 and backup, are on a
> Fujitsu Eternus 4000 model 500 connected via multi-pathed fibre
> channel. They are made up of (4) dedicated 6 disk RAID10 arrays, fed
> into LVM. The volumes are created as 4 way stripes across these 4
> RAID10 volumes.
> 
> /opt/zimbra/store2 is made up of shared RAID5 arrays from the same
> Fujitsu. They are fed into LVM, and simply concatenated. We'll be
> creating a store3 soon, but all future HSM volumes will be 500G or
> less. With ext3, we expose ourselves to too much potential downtime
> if an fsck is needed by having such a large volume (1T). We also run
> the attached script (e2croncheck) on each volume each month to help
> ensure we don't have untimely fsck checks, and to identify potential
> filesystem issues early on. I've also taken the liberty of attaching
> the script we use to do our LVM snapshots prior to upgrades
> (snapshot).
> 
> /opt/zimbra/backup is a 1G ethernet connected NFS share off a Sun 7410
> clustered pair with read and write cache. This FS is optimized on the
> 7410s to cache metadata only. Using a 7 day auto-group backup scheme,
> a nightly backup typically takes ~2hours. This volume used to be on a
> NEXSAN SATABeast with concatenated RAID6 volumes, connected via
> multi-pathed fibre channel. Backups used to range between 4 and 10
> hours. Since moving to the 7410s with cache, backup time has not only
> dropped, but has become more consistent.
> 
> We have a cron job that runs HSM once a week at 9:pm on Saturday, and
> aborts it at 11:59pm, if it's still running, so the backups aren't
> competing with HSM for I/O. If it's not still running, then the abort
> is harmless.
> 
> 
> > 4. Do you have dedicated network connection to the HSM storage?
> 
> HSM storage is FC connected. Not dedicated, but switched. Bandwidth
> is fine.
> 
> 
> > Will it be a performance problem if both backup and the secondary
> > HSM
> > volume share the same network connection?
> 
> We try to not run HSM jobs and backup jobs at the same time to avoid
> contention.
> 
> 
> > 5. Impact with backups
> 
> We try to not run HSM jobs and backup jobs at the same time to avoid
> contention.
> 
> 
> > 6. Impact to server
> >
> > Obviously moving data from primary disk to secondary disk will have
> > impact to server performance. Do you let your HSM process run
> > continuously?
> 
> No. The impact is not quite as bad as backups, but noticeable.
> 
> > 7. What HSM Age do you use? If you adjusted it from the default 30
> > days, why?
> 
> 60days. We try to stay around 50% used on our tier1 store. That's
> about the right number for us right now. But truthfully, the
> frequency with which things older than even 30 days are accessed in
> our environment is very low. So because of less user traffic to our
> tier2 store, having it on slower disk is essentially unnoticable by
> users.
> 
> 
> > 8. When you initially turn on HSM, how long did it take for a
> > complete
> > run? How long does it take after the intial run?
> 
> You don't have to let it run out. You can abort it at any time, and
> then restart it again later. I'd recommend running it via cron during
> low usage hours. We just do it once a week, but if you have a lot to
> initially migrate, you could run it a few hours a day off hours until
> you get your data migrated. Here are the commands we run out of cron
> (/etc/cron.d/zimbra-hsm -- locally created cron file):
> 
> 00 21 * * 6 zimbra /opt/zimbra/bin/zmhsm --start >/dev/null
> 59 23 * * 6 zimbra /opt/zimbra/bin/zmhsm --abort >/dev/null
> 
> 
> 
> --
> Brian Elliott Finley
> Manager, Application Platforms - Infrastructure and Operations
> Computing and Information Systems, Argonne National Laboratory
> Office: +1 630.252.4742 Mobile: +1 630.447.9108

-- 

Xueshan Feng <sfeng@stanford.edu>
Technical Lead, IT Services, Stanford University

-- 

Xueshan Feng <sfeng@stanford.edu>
Technical Lead, IT Services, Stanford University