Playing with rsync on Ubuntu

Update: it was announced on May 13, 2008 that a vulnerability was discovered in a Debian patch to a library used to generate SSH (and SSL) keys. If you followed these instructions before May 15, 2008 to generate a passphraseless key, you’ll need to update your system. All you need to do is goto System > Administration > Update Manager and click “Install Updates” from the GUI (or sudo apt-get update && sudo apt-get dist-upgrade at the command line). If you don’t know what I’m talking about, you’re probably not affected.

Right after I wrote How to regularly backup Windows XP to Ubuntu, using rsync for my T42 ThinkPad, I ended up installing Ubuntu on it. In the end I was mostly interested in backing it up for the purpose of safely wiping Windows. So much for regularly.

Then something unexpected happened: I started to use my 5 year old X23 exclusively. Partly because I’d already gotten comfortable with Ubuntu on it, and partly because I was using it at the time to edit photos on the bus ride home from work. When I got home, it didn’t make sense to switch to another laptop to finish whatever I was working on.

It also may have something to do with a feeling that I think Mark Pilgrim put well:

There’s something magical about breathing new life into old hardware, hardware that you had given up on. I don’t know quite how to describe it. It’s very… tender.

So I’ve been meaning to get back to rsync to start regularly backing up Ubuntu to Ubuntu. Well my little Mini-ITX server with external hard drive is already up and running. So that half of the equation was in place. What I was a little rusty on was how to set the whole thing up to run without needing a password.

That turns out to be really easy. If you’ve got ssh, then you’ve probably got ssh-keygen, which exists for the sole purpose of generating public and private keys, which when created without a passphrase can be used for password-free logins. So I ran ssh-keygen to generate a 2048 bit RSA key without a passphrase (aka a passphraseless key). I could also have generated a 1024 bit DSA key. I’m not sure I understand the difference. I’m not sure it matters.

ssh-keygen -b 2048

Inside ~/.ssh, ssh-keygen created two standard files, id_rsa and id_rsa.pub, the private and public keys respectively. The next and final step is to copy and “install” the public key on my backup server (192.168.0.100).

ssh-copy-id -i ~/.ssh/id_rsa.pub jwatt@192.168.0.100

ssh-copy-id uses ssh to copy the public key to the remote server and appends it to the ~/.ssh/authorized_keys file. Of course I didn’t know about ssh-copy-id when I started, so I just scp-ed the file over and pasted the public key into the authorized_keys file.

At which point I could use ssh to login without a password! ssh knows to automatically check for the existence of the id_rsa private key and try logging in with that.

ssh 192.168.0.100

Hot damn! That alone makes me want to start distributing my public key around to every server I access regularly. Of course the other benefit (and the whole point of this post!) is that now I’ll also be able to cron an rsync backup without requiring a password.

My ideal backup is a relatively current mirror of my home directory. I’m not looking for modified file snapshots or entire bootable filesystem images, I just want to know that if my hard drive crashes, most of my data (especially the photos) is recoverable. To that end, my rsync needs are relatively simple, though it took some tweaking to get to this point:

rsync -aze ssh --delete --exclude=".*/" /home/jwatt/ jwatt@192.168.0.100:/home/jwatt/backup/x23/

The -a option means archive files—it’s really an alias for a lot of other options having to do with maintaining permissions and timestamps, etc. The -z option uses compression when transferring files. The -e ssh option tunnels the file transfer over an encrypted ssh connection. The --delete option deletes any destination files that have been deleted from the source. The --exclude=".*/" option skips hidden files and directories. Finally the last two parts are the source (in this case everything under my home directory) and the destination I’ve already set up on my backup server.

And that’s it. I added it to my cron to run daily at 10pm. Set and forget it.

Feel free to if you found this useful.

4 Comments

Eric

Cool man. I do the cron rsync thing for a variety of my machines. I run it every hour now, it only takes a few seconds to check for updates, so I’m always up to date. Muy bueno.

Japh

Brilliant, thanks for this :)
I was just wondering if you could explain how you keep track of whether a backup was successful or not with this method?
Cheers,
Japh

Dragonslayr

Install mailutils, then pipe it..

/bin/backup.vim | mail -s Homeserv Backup dragonslayr@example.com

you may find this useful:
http://www.rcrnet.net/downloads/backup.vim

Posted an update to the top of this post about the recently discovered weakness in SSH and SSL keys generated by Debian and Debian-based Linux distributions, such as Ubuntu.

Care to Comment?

Or if you'd prefer to get in touch privately, please send me an email.

Name

Email (optional)

Blog (optional)