Posted by on
Categories: Enterprise Storage SSD

A recent piece in the Atlantic details an ambitious project to preserve the world’s digital knowledge in event of global catastrophe. This is a serious issue, as most of today’s digital storage needs to be copied every few years – disks wear out, flash in #SSD s loses charge – and there are multiple efforts underway to protect our digital civilization from massive data loss

Headed by Martin Kunze, the project hopes to snapshot all of humanity’s knowledge, commit it to durable ceramic plates, and hide it in an Austrian salt mine. Ceramics are a time-tested storage medium: the world’s oldest written records are recorded on clay, and are over 5,000 years old.

Kunze’s idea is to

. . . laser-print a microscopic font onto 1-mm-thick ceramic sheets, encased in wafer-thin layers of glass. One 20 cm piece of this microfilm can store 5 million characters; whole libraries of information- readable with a 10x-magnifying lens-could be slotted next to each other and hardly take up any space.
Magnetically-encoded data – disk or tape – is remarkably vulnerable to large-scale events, such an electro-magnetic pulse from an atomic weapon, or a powerful solar coronal blast, as well as a host of lesser ills. While data capacity has been getting cheaper for decades, making it economical to preserve old data, that trend can’t last forever – so it won’t – and we’ll be faced with deciding what data to keep and what to delete.

Most current preservation projects, such as LOCKSS (Lots Of Copies Keeps Stuff Safe) or DuraSpace, address preservation, but primarily with magnetic storage. My favorite personal archival medium is the 1,000 year DVD, which stores almost 1,000 times the data of one of Kunze’s ceramic plates, but isn’t human readable.

Any human readable storage medium, even one that uses microscopes, will have relatively low storage density. Five gigabytes of ceramic plates would require over two meters – about 10 feet – of linear shelf space. Five terabytes would require 2km of shelving, while five petabytes would need 2,000km of shelving. And the world is producing thousands of petabytes of data every year.

Thus the project requires curating the permanently preserved data. Who chooses what is preserved and what isn’t? How would the information be updated, if new discoveries render old data moot?

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.