LinuxTECH.NET
advertisement
  Home    |     News    |     Reviews    |     Features    |     Tips+Tricks    |     Downloads    |     Contact Us  
 

Best Reliable Long-term Data Storage Media

how to preserve your digital photos, family videos, music files and documents for your children and grandchildren

In recent years all consumer audio/video recording devices have gone through a radical change from traditional 'analogue' media to digital. Photo cameras have moved from film to memory cards, video cameras have moved from video tapes to digital tapes and mini DVDs and then on to hard disks and memory cards, audio cassettes have long disappeared in favour of mp3 players and recorders, and the home VCR has been replaced by DVD-recorders and more recently hard disk recorders.

All these devices now have one thing in common: they store the recorded data in files, which are easy to duplicate and manipulate with the help of a PC, which in turn has ensured that most people transfer their pictures or recordings onto their PC and store them on the internal hard disk of their PC. But computer hard disks are fragile and prone to defects and have a relatively short life-span (not to mention accidental deletion of files due to user error or viruses), therefore they are not well suited for long-term archiving of your precious memories.

To make sure that none of your precious files ever gets lost you need to have at least two archived copies (in addition to the copy you keep on your PC), that need to be stored on at least two different kinds of physical media in at least two different and distant locations.
For example, say you have 4 GBytes worth of photos of the first two years of life of your child that you need to keep safe, copying the files onto a DVD-R and a 4GB CF memory card and placing the DVD-R in your bank safe deposit box and the memory card in a cupboard at your home should ensure that you won't lose your photos no matter what happens. To be even safer you could place a third copy at your parent's or your brother's/sister's house.

But the aim of this article is primarily to help you chose the right kind of media to store your files on, so lets get back to that. Archiving media needs to be robust, reliable and durable. Currently there are three main types of media that can be used for data archiving, external hard disks, memory cards and optical discs. All three have advantages and disadvantages which we list briefly here:

USB hard disk
WD Passport USB hard disk

Spindles of DVD-Rs and CD-Rs
Spindles of DVD-Rs and CD-Rs

Various types of memory cards
Various types of memory cards
External USB hard disks:
  • + large storage capacity
  • + no need for a separate reader device
  • + very low cost per GB
  • - magnetic storage, prone to gradual decay and bit flips
  • - relatively bulky, heavy, fragile
  • - complex, therefore more parts that can fail

Optical discs:
  • + very cheap (CD-R and DVD-R)
  • + insensitive to magnetic fields and cosmic radiation
  • - relatively low storage capacity
  • - data layer prone to physical decay
  • - quality among brands is very variable

Memory cards:
  • + small, light, robust
  • + cost per GB is getting lower all the time
  • + good capacity and increasing all the time
  • - possibility of gradual decay and bit flips
  • - quality among brands is very variable
  • - many different incompatible types available

Currently, good quality optical discs (DVD-R) and good quality Compact Flash (CF) memory cards are the best choice for reliable long term archival of digital documents and files. We recommend the use of external USB hard disks only for large files (video recordings) as using DVD-R or CF cards for video files would be impractical and/or too expensive due to their more limited storage capacity.

Compact Flash (CF) memory cards
Compact Flash memory cards are the best type of memory card for long term data storage due to their robust physical casing, physical size (not too small like some other card types), and even more importantly, the fact that they are based on a stable standard. CF cards haven been around since 1994 and are based on the same ATA interface as common hard disks. This has ensured that the standard hasn't changed over the years (unlike many other card formats that have come and gone and had incompatible changes of specifications) and is a guarantee that there will still be card readers able to read CF cards for many years to come.
As quality of flash chips varies a lot and because there are lots of fake branded CF cards available, it is crucial to buy only good quality branded CF cards from reputable dealers (never on online auction web sites, as those are full of fakes!). Specifically, good brands that make good quality CF cards are: SanDisk, Kingston, Transcend and Lexar. Out of these four brands Kingston and Transcend generally offer best value for money without compromising on quality.

We have listed our recommended CF cards (which offer best value for money) with some links to trusted reputable online retailers here for your convenience:
Transcend TS32GCF133 CF card
Transcend 32GB CF card

Kingston CF/32GB-S2 CF card
Kingston 32GB CF card

Transcend 133x CF Card - check price at:

8 GB: Amazon (US) | Newegg (US) | Amazon (UK) | Amazon (DE) | Conrad (DE)


16 GB: Amazon (US) | Amazon (UK) | Amazon (DE) | Conrad (DE)

32 GB: Amazon (US) | Amazon (UK) | Amazon (DE) | Conrad (DE)

Kingston Elite Pro 133x CF Card - check price at:

8 GB: Amazon (US) | Amazon (UK) | Amazon (DE)


16 GB: Amazon (US) | Amazon (UK) | Amazon (DE)

32 GB: Amazon (US) | Amazon (UK) | Amazon (DE)

Use only FAT32 or possibly ext2 (on Linux) as filesystem for the CF cards, to ensure maximum compatibility even in future. While encryption could be desireable for some content, you need to make sure that you won't forget the encrytion key/password even in 10-20 years and that the software you use to encrypt the card will be still available in 10-20 years and that your relatives which you want to be able to access the data know the key too.

Optical discs
When it comes to optical discs, there is currently really only one type on the market that can be trusted for long-term storage, the Verbatim Gold Archival DVD-R. This particular DVD-R has been made specifically to ensure long-term data stability and has been rated as the most reliable DVD-R in a thorough long-term stress test by the well regarded German c't magazine (c't 16/2008, pages 116-123). According to that test, the Verbatim Gold Archival DVD-R has a minimum durability of 18 years and an average durability of 32 to 127 years (at 25C, 50% humidity). No other disc came anywhere close to these values, the second best DVD-R had a minimum durability of only 5 years.
Verbatim Gold Archival DVD-R Spindle

Verbatim Gold Archival DVD-R, spindle of 25 or 50 (US/AU) - Check price at:
Amazon (US) | Amazon (UK) | Amazon (DE)


One very important factor when burning DVDs that often gets overlooked, is that burn speed plays a very important role with regards to recording accuracy and reliability. For archival purposes we strongly recommend that you burn the DVDs at no more than 4x speed.

External USB hard disks
As mentioned previously, external USB hard disks should only be chosen for large files (video recordings) where using memory cards or optical media would be impractical, as hard disks are not as reliable and durable as good quality memory cards or DVD-Rs. That said it is actually much easier to chose an external hard disk for archival purposes as the actual disks inside the casings are generally all of similar quality. Avoid 7200rpm drives, as due to the higher speed they get hotter and wear out quicker, chose a 5400rpm or 4200rpm drive instead.
Here is an example of a suitable external USB hard disk:

Seagate FreeAgent Go, 2.5", 5400rpm, 250/320/500/640 GB, (comes with 5 years warranty)
Check price at: Amazon (US) | Amazon (UK) | Amazon (DE)

Before using an external USB hard disk for the first time, it is highly recommended to do a thorough surface check of the disk first, to avoid storing your data on a flawed disk. On Linux this can be achieved by running the following command as root user in a terminal:

badblocks -svw /dev/sdX1

Use the correct device letter instead of 'X', this can be found for example by looking at the output of:

fdisk -l

Make sure you get the device letter right, as using the wrong device letter could overwrite all data on one of your other disks! Also make sure the external USB hard disk isn't mounted when running the 'badblocks' command. The 'badblocks' test will take several hours, depending on the size of the disk, it can even take 24 hours. Once the 'badblocks' test has completed, you will need to recreate the filesystem on the USB hard disk, as this has been erased by the test.

Final considerations
Currently Verbatim Gold Archival DVD-Rs and good quality Kingston or Transcend CF cards are the best choice for safe long-term private data storage. USB hard disks are ok for large files, providing they have passed the surface test and providing they are handled with care.

We have excluded USB memory sticks as they are generally made with very low quality flash chips and therefore cannot be relied upon for long-term storage. We also don't regard online storage offers as suitable for long-term private data storage, as there is no guarantee that the companies offering these services will still be around in 5 years (even less so in 10-20 years!). Also they are not cost effective, if you consider the service fee over many years.

Whatever media you use, make sure you label them clearly and check the integrity of the data every 5-10 years or so, a md5 checksum of the whole DVD-R or memory card can be a good method for this.


You might also be interested in the following article:
[27/12/2009 - art-best_safe_long-term_data_storage.html]