Let’s say I have some data that I absolutely want to keep for twenty years or more. I’m not worried about file format or interface compatibility, just physical data integrity. What’s the best media in terms of long-term reliability? This would be in the conditions you’d find inside an ordinary home - I can probably keep the climate relatively well-controlled, but there will still be temperature, humidity and cleanliness fluctuations.
Don’t say tape. I have something like a phobia for storing anything valuable on tape. I just don’t trust it. Burning CDs and DVDs are out of the question: they’re way too unreliable.
How about hard drive, or flash memory? I have a data sheet for a flash ROM chip in front of me now, and it promises data retention of “up to 20 years”. That’s not a strong enough assertion for me, and the manufacturers of actual flash drives do not even bother to make that promise.
I remember hearing a lot about magneto-optical drives a while ago, and a number of companies still manufacture disks that are guaranteed for 50 years. However, the only company still making drives is Fujitsu, so I can’t believe that market is especially vibrant.
What do banks and hospitals use for long-term data storage?
When I worked in a bank, we used tape.
Each month the tape would be copied to a new tape.
Tape is not reliable as you say, but with the volume of data it was cheap and copying to a new tape was just as cheap.
Everything is unreliable for such a long time. The best solution is to copy the data on 3-4 DVDs of different brands using a medium recording speed. Then copy those DVDs into new DVDs every couple of years.
I think that if you just want to store it for 20 years without accessing it in the interval, a hard drive would probably be the best solution. Some light googling didn’t turn up any estimates on data longevity for hard drives, but as far as I know, most hard drives fail due to a mechanical or electrical problem, not loss of information from the media itself. Whether the information would be totally stable on a drive that was never powered up I dont know, but I’m guessing it would be more reliable than DVD-R or CD-R. Hard disk space is cheap, though, so having two (or more) copies is reasonably inexpensive insurance.
The secret to long-term storage is maintenance; as others have said, keep extra backups and copy them to new media at an interval small enough to insure you don’t lose anything.
But if you absolutely must keep something, untouched, for decades, go with the archival medium of choice: paper. Print out your data and stick it away. Even today, OCR is pretty good, in twenty years you can probably assume it will be effectively flawless, so getting the information back into digital form won’t be a problem. Of course, if your data is several gigabytes, the initial printout issue might be daunting, and if it’s not text, you’ll have to convert it to something visual or textual first. But many agencies still print out copies of their stuff on paper and file it the old fashioned way. And you’ve hit on why.
If I was going to settle for an 80% chance of retrieval, I’d go with either a hard drive or burned CD’s, placed in a metal, lightproof box and kept somewhere dark and out of the way. I’d be very surprised if a flash drive or any flexible magnetic media held it’s data for that long; certainly I don’t have any 20-year-old floppy disks that I can still read.
If you’re going to start eliminating the best choices, then maybe the best remaining choice is punch cards.
Seriously, tape is the storage medium of choice for archivists and those concerned with long term storage. But the key is not to blindly trust the stated life span of a medium. At the National Archives, for example, they check and re-copy their tapes at regular intervals. No reason you couldn’t use that strategy with CDs effectively, too.
Floppy drives are no longer standard, and while 5.25 disks where the storage medium of choice 20 years ago, you’d be hard pressed to find a drive that would read it. Although the OP specifically dismissed that concern, that’s the major issue for any storage medium when we’re talking about storing something for retrieval 20 years later.
Paper’s not a bad idea, although it’s susceptible to damage – from preventable elements like fire and water, but also less evident elements like temperature, light, and humidity.
I’d recommend storing the data on several formats; DVD, hard drive, flash memory, etc. and testing each after a year or two and then transferring it to any new formats that arise over the decades. And for really long-term storage, look at theRosetta Disk being developed, which is a two-inch nickel disc with text micro-etched on it.
‘Vintage computing’ on eBay is probably all that you’d need. Windows XP supports 5.25" drives. On the other hand, you’re right about degredation, and Dog80 is right to suggest regularly copying the data onto new discs, or whatever replaces them.
Convert the data to a decimal stream. Get a metal rod, then make a single notch on the rod exactly 1/x from one end, where x equals the data stream.
For example, if your data is 8675309, make the notch 1/8675309 from one end of the rod.
Don’t forget to mark the end you are measuring from
For any significantly sized data storage, you need a very accurate measuring tape to calculate where to put the notch. You may even need one accurate to Planck space, if you have enough data.
An accepted alternative to paper, if you are willing to convert your data into an analog medium and have scan and redigitize it back in again is microfilm. Many companies maintain microfilm archives as an alternative to paper documents. It’s less bulky, and properly stored microfilm is supposed to last 500 years. SOX compliance is convincing many companies that it is unwise to keep permanent records solely in electronic form, and many are opting for microfilm.
The metal rod is made up of atoms, so I think the highest resolution you can get is 1 atom. At this resolution, a 1-meter rod can store less than 20 bytes of data, if I did my math correctly. You’re much better off carving symbols on a stone tablet.
I thought the degredation problem with floppies is that there are “soft” magnetic media (unlike the “hard” magnetic media of hard drives) and those just naturally start to degrade after a decade or so. I have the drives to read both 5.25" and 3.5" disks but I’m having problems reading these old floppies these days (some of which are nearly 20 years old) and I assume it’s because of this degredation.
Someone at my local library said something very amusing and true: “When it comes to reliable data storage, mankind has yet to improve upon the stone tablet.”
In addition to all of those, put them on a HD inside your computer. The advantage of this is that you wont get silent failure. You’ll know when the hard drive dies and can take active measures to restore it. The biggest risk you take is coming back to something in 5 years and realising that all of your backups have failed at once.
Okay, a few things: first, I’m looking to store several gigabytes of digital information, that cannot be reasonably turned into analog form. So digital storage is the best.
Second, it’s obvious that I can achieve essentially any reliability I want by periodically buying new media and copying my backups onto it. I’m wondering which media will last the longest without such measures (or, which media will allow me to perform the copying as rarely as possible).
I do not think that tape is the best solution here. I will accept that tape is optimal for storing enormous volumes of data - I’m talking about just a few gigs.
I’m wondering if anyone has anything to say about magneto-optical, or its successor, UDO (ultra-density optical).
It seems I can buy a magneto-optical drive for $299, and the 2.3 GB disks it uses for $20 apiece. Quite a deal, if I can really trust the disks and data to last for 50 years. Apparently it and UDO are widely used for storing medical and financial records. UDO can store up to 30GB on a disk, but I don’t need that, and it’s fairly expensive.
