Data destruction -- RIP

20.12.2006
After much prodding from my wife (plus the fact that I could no long stand watching the Patriots being shut out by the Dolphins), I finally cleaned out my desk last weekend. I found lots of items that at one time I thought were worth keeping: memorabilia from past storage conferences, receipts for old PC software (anyone remember Stacker?), old product manuals and engineering documents, and an assortment of computer parts and disk drives ranging from 20MB to 20GB.

Sorting through this hodgepodge typifies, albeit on a minute scale, the problem organizations experience facing that final stage of data life-cycle management -- data destruction. I've written recently about long-term data retention and archiving, but one of the most difficult aspects of the process is making the decision to finally pull the trigger to purge data.

To a large extent, data deletion does not happen regularly (and in some environments, it has never happened -- at least intentionally). Thus, there can be considerable costs associated with an effort to identify and clean out unneeded data. Although the growing use of WORM storage, such as CAS-type arrays, can be useful for housing inactive data, it also provides an easy way to prolong the data-purge decision, making it only more challenging in the future. In many situations, the term WORM -- write once, read many -- is actually a misnomer, since much of the data written to these devices is rarely retrieved. More appropriate terms might be "WORST" (suggested by a colleague) -- write once, read sometimes -- or even "WORN" (suggested by a client) -- write once, read never!

If data can be stored cheaply enough, should we even care about purging it? Certainly the primary driver is not storage cost. The biggest reason is potential liability. An analysis in one organization using extremely conservative probability assumptions estimated the liability exposure to be well into the millions of dollars -- easily justifying the cleanup cost.

After investing in a one-time cleanup, it's vital to institutionalize and proceduralize the regular purging of data. Right now, my desk looks OK, but I know that if I don't change my behavior, it'll be a mess again in a year. Comedian George Carlin remarked that "all you need in life is a little place for your stuff," but it's not good to become buried by it.

Jim Damoulakis is chief technology officer of GlassHouse Technologies Inc., a leading provider of independent storage services. He can be reached at jimd@glasshouse.com.