Expected performance of importing mail?

[Steve Hillman <hillman@sfu.ca>]

Hi folks,
  We have written some scripts to import users' mail from our legacy Webmail system and from homedir-based IMAP folders into Zimbra using their SOAP interface. I'm not getting stellar performance out of them, and I'm wondering what other sites have experienced in terms of throughput when importing mail.

The SOAP import mechanism is, as far as I know, the same mechanism that the Outlook migration wizard uses to get mail in, so I would expect to see similar performance numbers to it. On our servers, we're getting about 10-12 messages per second throughput, depending somewhat on the size of the mail (on large mailboxes, it all averages out anyway). The servers don't appear to be maxed out during the import - CPU is 10-20% and iowait is 10-30%. The /opt/zimbra volume, which is iSCSI-mounted from a NetApp, is seeing about 1-2MB/s in disk writes.

Are these numbers consistent with what other sites have seen? Have you tried to optimize anything on the server to get better throughput than this? I'm not seeing a particular bottleneck on the server, so I'm wondering why it can't go faster (if I modify my client-side script to write to /dev/null instead of the http socket, I get about 60 messages per second, so most of the latency is definitely coming from the Zimbra side)

I'm a Solaris guy and am used to a much bigger bag of tricks when it comes to doing performance analysis. Unfortunately, a year of banging on Zimbra folks to support Solaris hasn't yielded the desired result, so we're stuck running on Linux, leaving me at a bit of a loss to dig deep into the server-side 

-- 
Steve Hillman                                IT Architect
hillman@sfu.ca                               IT Infrastructure
778-782-3960                                 Simon Fraser University
Sent from Zimbra