How to escape the financial trap of storage scaling

Enterprise content management (ECM) and enterprise information management (EIM) are some of the most important business processes at many companies, although not always by choice. Because ensuring nearly all the enterprise’s transaction data is collected and stored in a compliant solution that adheres to legal requirements for data retention rarely ranks among the favorite responsibilities of CEOs and CIOs. Policymakers, on the other hand, are very active in this space, contributing to the astounding flood of data companies are exposed to with federal and industry-specific regulations on data retention. The second cause is the advance of digitization within the enterprise itself. Because once something exists in digital form, it almost always needs to be archived as well.

Compliance involves a degree of complexity that has given rise to technologies and certified IT infrastructures that vendors can only provide by investing heavily in hardware, expertise, and operations. It’s a segment dominated by conventional technologies and a handful of monopolists with highly certified solutions calling the shots. This gets to be particularly painful when it becomes necessary to scale a solution up — due to conventional system architectures, the resulting costs for systems, licenses, and project services can be bracing.

Yet in today’s world of skyrocketing data volumes, it’s a common requirement for solutions to scale up.

In response to this pressure, a changing of the guard has been underway for some time now in the ECM and EIM space. Conventional ECM systems have already fallen behind the pack, innovative big data systems are starting to advance, using NoSQL architectures to raise the enterprise to a revolutionary new level with the untapped opportunities that archival data presents. Scaling in the system core has also become far more cost-efficient and flexible as a result. And with big data processes, many new, value-added applications are becoming possible for what appeared to be a “dead money” investment. But that’s not our topic today.

Because the shift on the platform side makes it all the more apparent where the true source of costs are in the event that scaling becomes necessary: it’s the storage infrastructure.

Traditionally, WORM (write once read many) storage systems have been necessary to ensure compliance. Compliance is established using closed systems and proprietary standards. Users become extremely dependent on a number of manufacturers (NetApp, EMC Centera, Hitachi). Scaling becomes expensive, migrations consume considerable resources. The key issue, however, is that the price per terabyte of WORM storage is particularly high and rises further as soon as redundancy or high availability is required.

The transition to cloud storage systems would reduce storage costs by as much as 80%. The range of vendors available in this segment is large. It’s a true buyer’s market, resulting in extremely low costs. You can change storage providers quickly if needed. Scaling is always possible and costs are linear. All of that, and high-availability and always-on infrastructure is included.

How can WORM storage systems remain on the market? Because until now, there haven’t been any viable compliant solutions for the cloud. And because cheap storage in ECM archival solutions has not been an available option.

But that’s the thing about Gordian knots — someone always manages to unravel them eventually. This time, it was us. By using blockchain technology to achieve full compliance for cloud storage. How did we do it? By setting up a private permissioned blockchain that uses hashing and time stamping to secure data. Concurrently, the data and documents were protected against deletion via distribution across the blockchain. Compliance functionality and storage locations remain completely separate.

That’s the basic principle. Taking it from a model to a practical, complete business solution required equipping the system with a comprehensive feature set able to handle any scenario that arises in enterprise ECM and EIM applications. We call the result of these developments Deepshore Blockchain Storage.

At the core of the system is the implementation of an autonomous certification layer with complete administration of the storage layer. It is also able to manage different storage classes and geo locations. Building on this foundation, individual rules are implemented using policies (such as domestic storage only, limiting copies in specific geographic regions, saving copies to multiple storage devices). And last but not least, thanks to the complete abstraction of the physical storage layer, migrations become simpler than they have ever been in our blockchain storage — all you have to do is re-assign the database and replicate it.

Eliminating proprietary technologies in ECM and EIM results in cost savings of up to 80%, an end to vendor lock-in, and a considerable increase in availability and scalability. It’s getting very difficult to justify basing future plans on WORM storage.