Archive for the ‘glacier’ Category

Amazon Glacier Pricing Explained


May 29th, 2014

Arq is our Mac backup app that backs up your files to your own Amazon Glacier account.

Backing up to Glacier is very popular because the storage cost is only $.01/GB per month! If you have, say, 100GB of files, it would only cost $1/month to store them in Glacier.

But people sometimes find Glacier pricing confusing, especially when it comes to restoring (downloading) your files. In this essay I hope to make the restore costs much clearer.

Glacier Retrieval Fees

For this example we’ll use the “US East” region glacier pricing (other regions cost slightly more).

Restoring a file from Glacier is a 3-step process. First you issue a Glacier restore request for an object. Then you wait approximately 4 hours for the object to become available for download. Then you download it.

Requesting that an object be made available for download costs $.05/1000 requests, and data transfer out of Glacier is $.12/GB with the first 1GB free each month.

You can restore up to 5% of your Glacier data for free each month, prorated daily. For example, if you’ve backed up 100GB of files to Glacier, you can restore 5GB for free each month — 160MB each day.

Data Restore Fee

If you exceed the 5% in a month, Amazon charges a data restore fee. This is where it gets confusing. Amazon’s description of the data restore fee is complex. Put more simply, the data restore fee is equal to the total size (in GB) of the object(s) requested, multiplied by $7.20, divided by 4 hours, minus the prorated 5% free tier.

There’s one other quirk about this data restore fee: It’s only incurred once for the entire billing month. If you request objects from Glacier and incur a data restore fee of, say $5, you could continue requesting objects from Glacier for the rest of the month at that rate (or slower), and the charge on your bill for that month will be $5. But if, during some other hour in that month, you request objects at a rate that equals a data restore fee of, say, $6, the charge at the end of the month will be $6.

Cost vs Speed

If you requested all 100GB of your Glacier data all at once, the data restore fee would be substantial! But in practice that isn’t realistic. For one thing, you probably can’t download 100GB of data very quickly. A 10 megabit/second ISP connection would allow downloading of about 3.6GB/hour; 100GB would take 27 hours to download, so there’s no need to ask Glacier to make all 100GB available in 4 hours.

The best way to download from Glacier is to request only what you can download in the next 4 hours, and repeat every 4 hours as necessary while you download objects as they become available for download.

Arq is an Amazon Glacier client that manages the Glacier retrieval cost for you. When you restore your files from Arq Glacier backups, you first select a transfer rate, and Arq’s Amazon Glacier calculator calculates the data restore fee (labeled “peak hourly request fee”) for you:

amazon glacier pricing example

If you change the download rate, Arq updates the cost estimates. Here we’ve changed from 686 KB/sec to 330KB/sec, and the data restore fee is cut in half:

glacier retrieval cost


When you click “Restore”, Arq begins requesting objects, and continues until it has requested the amount of data that would take 4 hours to download at the rate you’ve chosen. After 4 hours have elapsed, it begins requesting another 4 hours’  worth of objects, and simultaneously begins downloading objects that are becoming available. It continues this pattern until all the files have been downloaded.

You’re In Control

The great thing about Amazon Glacier is that you’re in control of your data. Your data are in your own Amazon account.

If/when it comes time to restore, you can choose the rate at which you want to restore your files, choosing a balance between speed and cost.



Arq 3.2 is out!


June 12th, 2013

Arq 3.2 is a free update for all Arq 3 users.

The big new feature in 3.2 is restoring onto an existing folder. While it looks like no big deal on the surface (it just asks whether you’d like to overwrite the existing folder or not), under the covers it was a big change in the restore process. Arq now asks you for permission to launch a helper program as “root”, the super user. The helper program needs to be “root” so that it can write files into folders that your regular user account may not have permission for. Arq compares the existing file contents to the backup record and only downloads files that are different or missing. Then it applies all the metadata correctly as before.

Full list of new features:

  • Restore into an existing folder of files, only downloading the files that are different or missing. Restore runs using administrator privileges to avoid permission issues.
  • Added a “pause on battery power” feature — set it in Arq’s preferences.
  • Added a “setthrottle” command-line option to change the transfer rate setting. This is useful for those who wish to change Arq’s throttle setting via a script.

Fixed Bugs:

  • Fixed an issue during very large Glacier restores where Arq would request too many items, which would then expire from AWS before Arq got a chance to download them.
  • Improved memory usage and performance during Glacier restore and S3 restore.
  • Fixed 10.6-related crashes.
  • Fixed broken throttling issue.
  • Removed AddressBook features (pre-populating name and email in crash report forms and in-app purchase) so that users aren’t asked to give Arq permission to access contacts.
  • Added explanation of command-line options to Help documents.

To get Arq 3.2, pick “Check for Updates” from Arq’s menu.