Archive for the ‘sftp’ Category

Arq 4 Beta

By

February 4th, 2014

UPDATE March 3: Arq 4 is officially released.

 

I’m really excited about this release! It’s got features that many people have been asking for, and it opens Arq up to a whole new range of options for storing backup data.

PLEASE NOTE: Arq 4 should officially ship sometime before the end of February. Every Arq 3 license purchased on or after December 1, 2013 is eligible for free upgrade to Arq 4.

ALSO: Arq 4 is for OS X 10.7 and later.

New Storage Options

For the first time, Arq can back up to not just Amazon Web Services. You can choose GreenQloud, DreamObjects, Google Cloud Storage, or any other S3-compatible target:

 Target types

You can even choose to back up to an SFTP server! If you have a NAS in your home or office that allows SSH/SFTP access, you can back up to that and pay $0 in monthly storage charges. Or back up to a VPS (virtual private server) like Dreamhost for cheap offsite backup.

More Control

 Several new features are aimed at providing you with more control over your backups:

  • Backup only on selected wireless networks — prevent upload when tethered to your phone, for instance
    Networks
  • Email notifications — great for monitoring headless/remote Macs and customer Macs 
    Email prefs
  • A unified budget across S3 and S3/Glacier backups (see “A New Approach to Glacier” below)
  • Optionally specify a “window” of time during the day/night when Arq pauses — useful for networks that are underutilized at night, and for ISPs that charge less at certain times of day
    Backup window

Other Features

Arq 4 includes several other features and improvements, including a “Stop Backup” function, the display of the last backup date in the agent’s menu, and less prompting for administrator privileges when restoring. Also, the process for setting up and restoring to a new computer is more straightforward.

A New Approach to Glacier

When Amazon announced their Glacier offering in the fall of 2012, we built Arq 3 to take advantage of it. Some time after Arq 3 was released, Amazon announced an S3 Glacier Lifecycle feature through which Amazon would automatically store certain S3 objects in Glacier. Arq 4 uses this new S3 Glacier Lifecycle feature for Glacier backups (existing Glacier backups made with Arq 3 will continue to use the old Glacier API as Amazon offers no way to move Glacier objects to S3). There are several benefits to using the S3 Glacier Lifecycle feature:

  • S3 objects with Glacier storage class have all the benefits of regular S3 objects — a known location/name, S3 object query abilities
  • No more creating Glacier vaults, which are hard to use and even harder to delete.
  • Restored objects are at known/expected locations, unlike restored Glacier objects which receive a random name.
  • Restored objects can be persisted for much longer, unlike restored Glacier objects which have a fixed 24-hour expiration.
  • Restarting a restore with Arq means only requesting restore of objects which haven’t been requested yet because previously-requested objects are at known locations.
  • Restoring is less complex and much faster — no more creating SQS queues and SNS topics; no more taking the time to read all messages from the SQS queue before beginning to download.
  • Budgeting is possible using the same logic as the S3 budget feature.

Changing to the New S3/Glacier Format

Unfortunately Amazon doesn’t provide a mechanism for creating “pointers” in S3 to existing Glacier objects, so if you want budgeting and easier restoring with Glacier, you’ll have to re-upload your files. I know this is far from ideal, but in the long term I believe it pays dividends in cost and ease of use. 

Arq no longer restricts you from adding the same folder twice, or adding a folder contained by a folder that Arq is already backing up. So, if you’ve been backing up a folder to Glacier with Arq 3, just add the folder to Arq again, and choose “Glacier storage class”:

Adding folder

Arq will back up the folder to S3 in a subdirectory of your S3 bucket called “glacier” which Amazon will automatically archive to Glacier storage class (Arq creates the Glacier lifecycle policy automatically). When it’s done uploading, delete the old folder (the old-style Glacier backups created with Arq 3).

Beta Testing

The Arq 4 beta is available here: http://www.haystacksoftware.com/arq/Arq4beta.zip

Your Arq 3 license will work with it just fine.

The beta will expire March 17, but Arq 4 should be shipping by then. If not, a new beta version will be available.

Please submit your feedback, questions, and bug reports via Twitter @arqbackup or via email to support@haystacksoftware.com.

I look forward to getting your feedback!

- Stefan