File and Object Storage

File and Object Storage

Software-defined storage for building a global AI, HPC and analytics data platform 

 View Only

Sharing data to minimize email attachments

By Tony Pearson posted Mon March 10, 2008 09:58 AM

  

Originally posted by: TonyPearson


Last week, I covered backup issues in [Deduplicationversus Best Practice for Backups]. This week, I thought I would cover issues with email.

At IBM, our standard is to have a limit of 200MB per user mailbox. A few of us get exceptions and have up to500MB limit because of the work we do. By comparison, my personal Gmail account is now up to 6500MB. Whenthis limit is exceeded, you are unable to send out any mail until it is brought down below the limit, and a request to be "re-enabled for send" is approved, a situation we call "mail jail".

The biggest culprit are attachments. Only 10 percent of emails have attachments, but those that do take up 90percent of the total space! People attach a 15MB presentation or document, and copy the world ondistribution list. Everyone saves their notes with these attachments, and soon, the limits are blown. Not surprisingly, deduplication has been cited as a "killer app" to address email storage, exactly for this reason.If all the users have their mailboxes all stored on the same deduplication storage device, it might find theseduplicate blocks, and manage to reduce the space consumed.

A better practice would be to avoid this in the first place. Here are the techniques I use instead:

Point to the document in a database

We are heavy users of Lotus Notes databases. These can be encrypted and controlled with Access Control Lists (ACL)that determine who can create or read documents in each database. Annually, all the database ACLs are validatedso that people can confirm that they continue to have a need-to-know for the documents in each database. Sendinga confidential document as a "document link" to a database entry takes only a few bytes, and all the recipientsthat are already on the ACL have access to that document.

Point to the document on a web page

If the document is available on an internal or external website, just send the URL instead of attaching the file.Again, this takes only a few bytes. We have websites accessible only to all internal employees, websites thatcan be accessed only by a subset of employees with special permissions and credentials based on their job role, and websites that are accessible to our IBM Business Partners.

In my case, if I happen to have a blog posting that answers a question or helps illustrate an idea, I will sendthe "permalink" URL of that blog post in my email.

Point to the document on shared NAS file system

Internally, IBM uses a "Global Storage Architecture" (GSA) based on IBM's Scale-Out File Services [SoFS] with everyone getting initially 10GB of disk space to store files, with the option to request more if needed. The system has policy-based support for placing and migrating older data to tape to reduce actual disk usage, and combines a clustered file system with a global name space.

My SoFS space is now up to 25GB, and I store a lot of presentationsand whitepapers that are useful to others. A URL with "ftp://" or "http://" is all you need to point to a filein this manner, and greatly reduces the need for attachments. I can map my space as "Drive X:" on my Windows system,or as a NFS mount point on my Linux system, which allows me to easily drag files back and forth.

Departments that don't need to offer "worldwide access" use NAS boxes instead, such as the IBM System Storage N series.



Pointing to files in a shared space, rather than as attachments in email, may take some getting used to. I've hada few recipients send me requests such as "can you send that as an attachment (not a URL)" because they plan toread it on the airplane or train, where they won't have online connectivity.

This all relates to new ways for employees to collaborate. Shawn from Anecdote writes in the post[Fostering a Collaboration Culture]:

"Have you invested in the latest and greatest in collaboration technology but still feel people are still not collaborating? How many Microsoft Sharepoint servers and IBM Quickplaces remain relatively untouched or only used by the organization's technorati? I think it's a big problem because this narrow view of collaboration starts to get the concept a bad name: "yeah, we did collaboration but no one used it." And then there the issue of the vast amount of money wasted and opportunities lost. We can't afford to loose faith in collaboration because the external environment is moving in a direction that mandates we collaborate. The problems we face now and into the future will only increase in complexity and it will require teams of people within and across organizations to solve them."

Well, sending pointers instead of attachments works for me, and has kept me out of "mail jail" for quite some timenow.

technorati tags: , , , , , , , , , , , , , , ,

3 comments
6 views

Permalink

https://community.ibm.com/community/user/blogs/tony-pearson1/2008/03/10/sharing-data-to-minimize-email-attachments

Comments

Archive User
Tue March 11, 2008 04:44 PM

Yeah, I know first-hand how difficult it can be to get things done in a big company like IBM - I worked there too!
It's sorta like trying to get gourmet cuts of meat down at the local SuperMarket for a dinner party.
Sometimes the specialty butcher shop is the best answer, like when you need elk tenderloins or a few pheasants for that special recipe!
If you get tired of waiting and you want a market-proven solution, you might want to check out EMC EmailXtender for IBM Lotus Notes/Domino - should do the trick:
http://www.emc.com/products/detail/software/emailxtender-for-ibm-lotus-notes-domino.htm
And look, No Waiting!

Archive User
Tue March 11, 2008 12:52 AM

BarryB,Thanks for the offer. I've been notified that our internal IT is working on deploying "IBM CommonStore for Lotus Domino" which is the IBM product that archives and migrates out emails and their attachments in compliance with regulations. However, IBM is a bigger company, and can sometimes take longer to implement these major changes.-- Tony

Archive User
Mon March 10, 2008 10:05 PM

Notes is so antiquated - I remember struggling with these limitations back when I was an IBM employee. You are to be commended for working around its limitations.
At EMC, everyone has an "infinite" mailbox, thanks to the integration of Microsoft Exchange, EMC eMail eXtender, EMC tiered storage, and Centera archives.
For the first "n" days of an email's life, it resides on the Exchange server of the recipient(s), on either Symmetrix or CLARiiON storage (dependent upon your locale). Exchange will single-instance attachments, but only within a single server instance; still, there should be one (and only one) copy of each attachment in all the servers of the recipients.
After "n" days, which is dependent upon retention policies and the "real" size of your mailbox, attachments and messages are extracted from the Exchange servers by our eMail eXtender product, and stored on Centera. Centera will single-instance all attachments, so that there is only a single copy of attachments servicing everyone. Users see an "archived attachment" icon, but the attachments are retrieved in only a few more seconds than it would have taken if they still were in the Exchange store. (A neat side benefit is that your Exchange footprint is minimized, greatly improving Exchange performance).
Since IT converted over to this automated, policy based tiering of email, there's basically no more "mail jail."
More importantly, attachments are preserved along with the emails, as is required by many compliance mandates and directives - something it seems you must be sacrificing with your separate-and-unmanaged approach to mailing pointers to attachments. Somehow I doubt your model would work for regulated agencies (financials, government, etc.).
On top of that, both the Exchange server stores and the Centera archive can be remotely replicated to meet the BC/DR requirements of compliance in many industries.
If you'd like, I can have someone contact your IT department and help you migrate to this more robust and easy-to-use way of staying out of (email) jail!
Stop by my blog and drop me an email...