Backup to S3 using duplicity
June 14th, 2009 by Mark
I recently put a nice big hard drive in my file server. I got a 1TB drive, which is kind of silly because I don’t have anywhere near that much data. But, I figured that it’s better to be prepared. I wanted to backup the data on the drive and Amazon’s S3 seemed like a good choice. However, I wanted the ability to only send changed files across the wire, and a simple way to encrypt all of the data as well. I chose to use Duplicity to manage the backups as it was able to only send changes as well as encrypt everything using GPG.
I decided to only backup a subset of my data since it didn’t seem necessary to backup Linux ISOs or the local copy of Wikipedia that I have on this drive. So, I ended up backing up 31.2 GB of data. Here’s the stats that duplicity produced:
--------------[ Backup Statistics ]--------------
StartTime 1244688149.16 (Wed Jun 10 22:42:29 2009)
EndTime 1244842521.31 (Fri Jun 12 17:35:21 2009)
ElapsedTime 154372.15 (42 hours 52 minutes 52.15 seconds)
SourceFiles 0
SourceFileSize 33486993762 (31.2 GB)
NewFiles 0
NewFileSize 0 (0 bytes)
DeletedFiles 0
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 0
RawDeltaSize 33484666346 (31.2 GB)
TotalDestinationSizeChange 32831751689 (30.6 GB)
Errors 0
-------------------------------------------------
Yeah, just in case you didn’t notice. The elapsed time was nearly 43 hours. That’s just under two full days. The 15/2 service that I’m getting from Cablevision was clearly the limiting factor. But, I don’t plan on doing full backups very often, and the incrementals will be considerably smaller.
Amazon charges for S3 by the amount of storage you actually use. They charge $.15/GB/month. So, for 31.2 GB I’ll be paying $4.68 per month. Not bad for a remote, redundant backup. They also charge for bandwidth which is usually $.10/GB, but it’s currently on sale for $.03/GB. So, the upload only cost $.93.








