Netmail Blog

Storage Fun: What to Expect for Storage Consumption During and After Migrating to Microsoft Exchange 2010/2013 from Novell GroupWise

Posted by Brian Hatchell

Find me on:

Feb 12, 2014 11:59:00 AM

One of the common messaging system migrations is from Novell GroupWise to Microsoft Exchange 2010 and 2013. There are many differences between GroupWise and recent versions of Exchange and one of the more challenging issues involved in planning this migration path is deciding how much storage to allocate to Exchange.

Single Instance Storage (SIS) is a system’s ability to keep one copy of content that multiple users share. It is a means to eliminate data duplication and increase efficiency. In the case of email systems that support SIS, this would mean that a single copy of a message is held within its database whilst individual mailboxes or accounts access the content through a reference pointer. While disk space reduction is a prime goal of SIS, it also greatly enhances delivery efficiency of messages sent to large distribution lists.

Exchange 2010 marked a change in how storage is consumed in Exchange. Microsoft decided to eliminate SIS in favor of storage throughput optimization, allowing Exchange data to be stored on slower storage such as 7200 RPM SATA HDD systems and facilitating the Database Availability Group feature. While this significantly reduced the storage speed requirements to run Exchange, it now means that each messaging item will get stored for each user instead of just one per mailbox database as in previous versions of Exchange. The net result of this is that while storage speed requirements have diminished, storage volume will increase overall for Exchange 2010/2013 implementations as compared to GroupWise or Exchange 2007 and earlier versions.

Novell GroupWise conducts SIS within the post office database, attachment data is replicated only once per post office so if you had a 5 MB attachment sent to 100 users in one post office the message would consume 5 MB of space as the item would be written once and other users would be linked to the existing attachment. If the same 5 MB message is sent to user across post offices only one entry would be made per post office, the 5 MB attachment sent to 100 users in three different post offices would consume a total of 15 MB storage, one 5MB attachment per post office database. (There are other factors involved here, such as block size of allocated storage, but for simplicity’s sake we are ignoring that.)

The same attachment in Microsoft Exchange 2010/2013 will be written once per user. The same 5 MB attachment sent to 100 users would now consume 500 MB of storage. It does not matter if the users all reside in separate mailbox databases or separate mailbox databases. Each user will get their own copy of the attachment. It can be easily seen that organizations that send attachments frequently to large numbers of their users will consume storage at a greater rate than was consumed under Exchange 2007 or GroupWise. This growth is accelerated by large attachments being sent and since these large attachments are often kept by users in their mailbox, it further increases the ongoing storage costs of an organization.

To make matters worse, what makes Exchange such a robust platform with the advent of Database Availability Groups and automatic failover and redundancy further increases the storage required to save multiple attachments. With the average system having a minimum of 3 copies of each database, that 5MB attachment now consumes 1.5GB of total disk space.

An example: A small tech firm before migrating from GroupWise to Exchange 2013 consumed 18GB in GroupWise 8, but after migration to Exchange 2013 was consuming 114 GB of space. Accounting for the growth of Exchange transactions logs during the migration Exchange consumed 3.16 as much data as GroupWise did with the same messaging data. These are typical numbers that are seen during migrations. There are a lot of variables, including general messaging profiles of an organization and number of GroupWise post offices but a growth of 300 to 400 percent is typical.

The Higher Education (due to increase of online education and corresponding email communications) and Health Care (due to regulatory requirements on retention of records, including email) verticals can be especially challenging. In 2010 at one large university there were 15,835,664 new messages consuming 154 TB of space (Total cumulative storage 412TB). By 2011 their message volume grew to 28,693,343 items consuming 280 TB of storage (Total cumulative storage 693TB). There were a total of 100,104,927 items included in this analysis over 12 years. The average message size was 10MB.  This is a clear example of message storage bloat.

This ‘growth’ of email data post migration, combined with increasing volume of email messages overall results in large cost increases for storage. Combining storage with other costs (management, time lost to users self-managing email, fulfilling discovery requests) this results in frustrated users, aggravated email and storage administrators, and CIOs wondering if they ever need to stop feeding the email storage beast.

While Microsoft does present an archive solution (requires an Enterprise CAL), this solution is basically a separate mailbox for the user that items can be moved to manually by the user or by automated archive policies. While this allows the user to tidy up their mailbox, it does nothing to reduce total storage costs as these archived items are still maintained in the same multi-instance format.

Using the Netmail platform (which includes Secure, Store, Detach and Archive) we here at Netmail cannot only migrate your GroupWise data to Exchange, we can prevent much of the GroupWise data from even being injected into Exchange, rather retaining the data on our inexpensive SIS solution, Netmail Store. This data is accessible to users through the Outlook Client or Outlook Web App. We can also prevent ongoing messaging bloat, enforce retention, provide secure email, manage attachments and provide easy ediscovery. Call or email us today for more info.

Topics: Email storage, Novell GroupWise, email migrations, Microsoft Exchange 2013