Warning: What follows is technical nitty-gritty. If you just want to know why Mac metadata are important in the real world, read The Importance of Metadata on the Mac.

UPDATE December 13, 2010: CrashPlan have released a new version of their app which passes all but 1 of the Backup Bouncer tests. The version is called “3.0″. Within the app the version is listed as “3.0 (1291687724486)”. This blog post analyzed the “3.8.2010″ version which was the latest as of February 2010.

Detailed Technical Analysis

Here’s what CrashPlan says it backs up and restores on the Mac:

• Data fork
• Resource fork
• Owner information for regular files and directories
• Symlink owner information
• POSIX permissions
• Finder Flags/Info (including type/creator)
• Locked flag (this is part of Finder Flags)
• Modification date
• Creation date
• Finder comments (.DS_Store file)
• ACLs

CrashPlan states that the following are unsupported:

• Other named forks
• BSD Flags, which can be set via chflags
• HFS+ extended attributes (i.e. xattr)
• Aliases

That list a bit redundant: “other named forks” is the same as “HFS+ extended attributes”, and an “alias” is a special type of file containing some data as well as 2 extended attributes (com.apple.FinderInfo and com.apple.ResourceFork) and some Finder Flags (‘Alias’ and ‘Custom Icon’).

Also, CrashPlan doesn’t mention that they do not support hardlinks, FIFOs, or device files.

Test Results

I tested CrashPlan 3.8.2010 on a MacBook Pro running OS X 10.6.3. I followed Backup Bouncer‘s test procedure:

1. Download Backup Bouncer 0.2.0
2. ‘sudo bbouncer create-vol Src’ to create the source volume for testing
3. ‘sudo bbouncer create /Volumes/Src’ to create the files to be backed up
4. ‘sudo bbouncer create-vol Dst’ to create a volume for the restore
5. Back up all the folders under /Volumes/Src using CrashPlan (to CrashPlan Central)
6. Restore all the folders from CrashPlan to /Volumes/Dst
7. ‘sudo bbouncer verify -d /Volumes/Src /Volumes/Dst’

Here’s the output of that ‘bbouncer verify’ command:

backup-bouncer-0.2.0 $ sudo ./bbouncer verify /Volumes/Src /Volumes/Dst
Verifying:    basic-permissions ... ok (Critical)
Verifying:           timestamps ... ok (Critical)
Verifying:             symlinks ... ok (Critical)
Verifying:    symlink-ownership ... FAIL
Verifying:            hardlinks ... FAIL (Important)
Verifying:       resource-forks ...
   Sub-test:             on files ... ok (Critical)
   Sub-test:  on hardlinked files ... FAIL (Important)
Verifying:         finder-flags ... FAIL (Critical)
Verifying:         finder-locks ... ok
Verifying:        creation-date ... ok
Verifying:            bsd-flags ... FAIL
Verifying:       extended-attrs ...
   Sub-test:             on files ... FAIL (Important)
   Sub-test:       on directories ... FAIL (Important)
   Sub-test:          on symlinks ... FAIL
Verifying: access-control-lists ...
   Sub-test:             on files ... ok (Important)
   Sub-test:              on dirs ... ok (Important)
Verifying:                 fifo ... FAIL
Verifying:              devices ... FAIL
Verifying:          combo-tests ...
   Sub-test:  xattrs + rsrc forks ... FAIL
   Sub-test:     lots of metadata ... FAIL

Detailed Failure Analysis

symlink-ownership test failure

The source directory looks like this:

$ ls -l Src/15-symlink-ownership/
total 32
-rw-r--r--  1 root  staff  14 Jun  1 07:07 some-file
lrwxr-xr-x  1 _www  _www   11 Jun  1 07:07 symlink1@ -> ./some-file
lrwxr-xr-x  1 root  wheel  11 Jun  1 07:07 symlink2@ -> ./some-file
lrwxr-xr-x  1 root  staff  10 Jun  1 07:07 symlink3@ -> ./symlink1

The files restored by CrashPlan look like this:

$ ls -l Dst/15-symlink-ownership/
total 32
-rw-r--r--  1 root  staff  14 Jun  1 07:07 some-file
lrwxr-xr-x  1 root  staff  11 Jun  1 07:11 symlink1@ -> ./some-file
lrwxr-xr-x  1 root  staff  11 Jun  1 07:11 symlink2@ -> ./some-file
lrwxr-xr-x  1 root  staff  10 Jun  1 07:11 symlink3@ -> ./symlink1

CrashPlan doesn’t restore symlink owner information, even though they claim to on their web site.

hardlinks test failure

The source directory looks like this:

$ stat -f 'inode:%i file:%N' Src/20-hardlinks/*
inode:48 file:Src/20-hardlinks/link1
inode:48 file:Src/20-hardlinks/link2
inode:48 file:Src/20-hardlinks/link3
inode:48 file:Src/20-hardlinks/some-file

Note the ‘inode’ number is the same for each file. That’s what a “hardlink” is. Wikipedia has more information on hardlinks.

The files restored by CrashPlan look like this:

$ stat -f 'inode:%i file:%N' Dst/20-hardlinks/*
inode:57 file:Dst/20-hardlinks/link1
inode:59 file:Dst/20-hardlinks/link2
inode:58 file:Dst/20-hardlinks/link3
inode:60 file:Dst/20-hardlinks/some-file

The ‘inode’ numbers are all different in the restored files.

finder-flags test failure

CrashPlan correctly restores almost all the Finder Flags, but Backup Bouncer marks this test as “failed” because CrashPlan fails to restore the Extended Finder Flag “busy”. The source files look like this:

$ GetFileInfo -P -a Src/40-finder-flags/mucho-flags-file
AVBSTClINMEDZ

$ GetFileInfo -P -a Src/40-finder-flags/mucho-flags-dir
aVbstClInmEDZ

The files restored by CrashPlan look like this:

$ GetFileInfo -P -a Dst/40-finder-flags/mucho-flags-file
AVBSTClINMEDz

$ GetFileInfo -P -a Dst/40-finder-flags/mucho-flags-dir
aVbstClInmEDz

The ‘z’ flag is “busy” (kExtendedFlagObjectIsBusy) in the Extended Finder Flags.

bsd-flags test failure

CrashPlan restores 1 of the 4 BSD flags, UF_NODUMP (1), UF_IMMUTABLE (2), UF_APPEND (4), and UF_OPAQUE (8). The source files look like this:

$ stat -f 'flags:%f name:%N' Src/60-bsd-flags/*
flags:15 name:Src/60-bsd-flags/dir-with-flags
flags:15 name:Src/60-bsd-flags/file-with-flags

The files restored by CrashPlan look like this:

$ stat -f 'flags:%f name:%N' Dst/60-bsd-flags/*
flags:2 name:Dst/60-bsd-flags/dir-with-flags
flags:2 name:Dst/60-bsd-flags/file-with-flags

In the output above, the “flags:15″ is all 4 flags (1+2+4+8=15) and “flags:2″ is UF_IMMUTABLE. CrashPlan actually restores the UF_IMMUTABLE flag correctly because it’s the same as the “Locked” Finder Flag in OS X.

extended-attrs test failure

Backup Bouncer creates some extended attributes with silly names on a file, a symlink, and a directory. The names of the extended attributes of each file are listed here (‘xattr-test’ has 2 extended attributes):

$ xattr-util l Src/70-extended-attrs/dir-with-xattrs
mamma.mia
$ xattr-util l Src/70-extended-attrs/symlink-with-xattrs
good.grief
$ xattr-util l Src/70-extended-attrs/xattr-test
message
this.that

The files restored by CrashPlan look like this:

$ xattr-util l Dst/70-extended-attrs/dir-with-xattrs
$ xattr-util l Dst/70-extended-attrs/symlink-with-xattrs
$ xattr-util l Dst/70-extended-attrs/xattr-test

Those 3 ‘xattr-util’ commands show no output because no extended attributes exist on the restored files.

fifo test failure

The source file is a “fifo” (a first-in-first-out file, or “named pipe”):

$ file Src/90-fifo/some-fifo
Src/90-fifo/some-fifo: fifo (named pipe)

CrashPlan doesn’t restore anything at all in this case:

$ file Dst/90-fifo/some-fifo
Dst/90-fifo/some-fifo: cannot open `Dst/90-fifo/some-fifo' (No such file or directory)

Backing up and restoring fifos is quite esoteric and probably of no interest to most people.

devices test failure

The source files are “device” files:

$ file Src/95-devices/devvn0
Src/95-devices/devvn0: block special
$ file Src/95-devices/devzero
Src/95-devices/devzero: character special

CrashPlan doesn’t restore anything at all in this case:

$ file Dst/95-devices/devvn0
Dst/95-devices/devvn0: cannot open `Dst/95-devices/devvn0' (No such file or directory)
$ file Dst/95-devices/devzero
Dst/95-devices/devzero: cannot open `Dst/95-devices/devzero' (No such file or directory)

Like fifos, device files are quite esoteric and probably of no interest to most people.

Conclusions

As I said at the beginning of this post, I wrote Arq, which competes with CrashPlan, so I’m biased.

If you’ve analyzed your programs and your files and determined you don’t need the metadata that CrashPlan skips, then you should be OK, at least in terms of metadata.

If you want to be sure all your data (and metadata) are backed up and restored correctly, then Arq and Jungle Disk are the only 2 options I know of. Arq has a Mac-native UI, a one-time software license fee (instead of an endless software subscription fee), and has gotten excellent reviews (see MacUpdate and MacStories).

Files on the Mac can have a bewildering array of important metadata attached to them, and it’s critical to restore all the metadata, not just the file’s main data (contents). Fortunately there’s an awesome open-source tool for testing whether your backup solution is capable of restoring correctly: Backup Bouncer.

Arq passes every Backup Bouncer test. It’s safe to use for backing up your files. What are you using for backup? Does it pass all the Backup Bouncer tests?

Here are the Backup Bouncer results:

Verifying: basic-permissions ... ok (Critical)
Verifying: timestamps ... ok (Critical)
Verifying: symlinks ... ok (Critical)
Verifying: symlink-ownership ... ok
Verifying: hardlinks ... ok (Important)
Verifying: resource-forks ...
Sub-test: on files ... ok (Critical)
Sub-test: on hardlinked files ... ok (Important)
Verifying: finder-flags ... ok (Critical)
Verifying: finder-locks ... ok
Verifying: creation-date ... ok
Verifying: bsd-flags ... ok
Verifying: extended-attrs ...
Sub-test: on files ... ok (Important)
Sub-test: on directories ... ok (Important)
Sub-test: on symlinks ... ok
Verifying: access-control-lists ...
Sub-test: on files ... ok (Important)
Sub-test: on dirs ... ok (Important)
Verifying: fifo ... ok
Verifying: devices ... ok
Verifying: combo-tests ...
Sub-test: xattrs + rsrc forks ... ok
Sub-test: lots of metadata ... ok