Where to Store those Large Files? A Tale of a BLOB

Few months ago I noticed that the WSS 3.0 introduced the notion of a external file storage provider to SharePoint world. Since then I kept myself updated about it, and I have mixed feelings for it.

The Rationale

If your SharePoint is mainly a glorified File Share, then this may be interesting to you. In essence, SharePoint saves the binary data for your files in SQL Server as BLOBs(Binary Large OBjects). So, it's easy for the SQL Server database files to grow rapidly in size as you add more and more large binary files.

The Hope

If only we could say to SharePoint not to save that large files into the database but to a file server or a distributed file system.....then we could do as Documentum does, saving the metadata into a database and the binaries to the file system.

The Good News

Well, it's possible. SharePoint allows us to create our own external binary large object (BLOB) store provider, or EBS Provider. We must inherit from ISPExternalBinaryProvider class and implement our own Save and Load procedures.

This provider operates deeply in SharePoint infrastructure stack, low enough to be completely transparent to the content and the SharePoint object model.

The Bad News

Unfortunately, too many of them:

• Currently it can only be configured at SPFarm level, which means that we use it for ALL the lists and libraries or NONE of them. A major screw-up issue that prevents it to be really useful in production.
• Any updates will create new BLOBs instead of updating the old ones. Want to garbage-collect the old ones? Do it yourself!
• The code runs outside .NET environment, but it a dreaded COM world. HRESULTs, hex errors, mysterious hangs.....the old friends are welcome again!
• Microsoft says that SharePoint 2009/2010 might have another implementation of EBS Provider, possibly related to SQL Server 2008 transactional FILESTREAM storage system. No compatibility is assured nor suggested.
• The performance will almost surely be crippled, compared to SQL Server database storage.

Still Want to Try?

You can find more information on MSDN SharePoint SDK: http://msdn.microsoft.com/en-us/library/bb802976.aspx