Haystack Software Blog

Do you use an online backup product/service? Ever wonder where your data are actually being stored? Ever wonder how safe and reliable that storage is?

It comes down to 1 question:

 

How much redundancy do you have?

 

Let’s look at the types of redundancy. But first a word about tape:

Disk vs. Tape Backup

In the past most backup systems used tape for storage. Tape was slow but it had much higher capacity than disk drives. Another killer feature was redundancy. Best practices for tape-based backup include keeping multiple historical tapes containing backups of your files at various points in history. Perhaps you needed to keep historical data for compliance reasons, but you also kept multiple tapes for redundancy.

This redundancy also helps protect you from data loss. If your most recent backup tape isn’t readable, you can always use the prior backup tape. You will lose the most recent items but that’s better than complete data loss.

RAID Is Not Backup

Most online backup offerings don’t use tape. They use disk. It’s cheaper now (and getting cheaper all the time), faster, and easier for the provider to use. Also, it’s “random access” — you don’t have to wind through the tape to get the file you want. But unlike tape there’s no extra disk with last week’s data.

Many providers use RAID arrays to protect against failure of an individual disk drive. This RAID can be effective in mitigating that risk, but it can fail.

How does your provider mitigate against disk failure within their data center?

Multi-Site Redundancy

In addition to risk of disk failure, there’s the risk that a data center experiences some catastrophe. Does your provider replicate your data across multiple data centers? They may store your files in an underground former bank vault with armed guards, but what if the vault takes on water or suffers a lightning strike? Can they withstand the loss of one data center, or even more than one, without losing your data?

Ongoing Integrity Monitoring

Unlike paper or film which degrade gracefully (yellowing and fading but still readable), magnetic media (disks and tapes) often fail catastrophically — one minute they’re readable and the next they’re not. Corruption happens. If you’re going to keep your data on disk, you should periodically verify the data’s integrity. Does your provider verify your backups on your behalf?

Provider’s Recovery Strategy

If an online backup provider loses a customer’s data, the only option is to start uploading the current files from the customer’s computer and hope the upload finishes before the customer suffers a disk failure or other form of data loss (e.g. customer inadvertently deleting an important file). Historical data are gone forever; the history of changes to your files can’t be recreated.

You Get What You Pay For

Most consumer-oriented online backup offerings are focused on price. Consumers would rather pay $5/month for “unlimited” backup. (Many providers limit things in one way or another by excluding certain file types or deleting old backups of external drives, but that’s another blog post). Customers get some sort of data protection, but it often comes with one or more of the risks described above.

Amazon S3 (“Simple Storage Service”) takes a different approach. It focuses on durability. S3 is:

• Designed to provide 99.999999999% durability and 99.99% availability of objects over a given year.
• Designed to sustain the concurrent loss of data in two facilities.

S3 is just a cloud storage system. It doesn’t come with software. That’s why I wrote Arq. Because it uses your S3 account for storage it’s a very reliable online backup solution.

Questions For Your Provider

Ask your online backup provider the following questions:

• Where are my data stored?
• How many data centers are my data redundantly stored at?
• If you lose my data in one of your data centers, can you repair by retrieving it from another data center?
• How many data centers can simultaneously lose some of my data without you permanently losing my data?
• Do you regularly verify the integrity of my data and repair corruption using your redundant copies of my data?
• What’s your durability design goal?

Then decide what price vs. redundancy trade-off is right for you.

 

This entry was posted on Tuesday, June 21st, 2011 at 2:29 pm and is filed under backup. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.