How to regularly backup Windows XP to Ubuntu, using rsync
Back in September I revived my Mini-ITX box to serve as a backup server. I set up BackupPC, ran it once, it seemed to work, and then ignored it for weeks. When I checked back, it hadn’t run successfully since. Ugh, I want backups to just work!
A few nights ago I decided to try again, this time dropping the constraint of not installing software on my laptop. Turns out I already had the building block I needed: rsync, installed in the form of Cygwin.
I began with these Rsync for Windows instructions, and everything went smoothly until the very end—rsync on Windows wasn’t connecting to rsync on Ubuntu. My gut told me rsync’s port 873 isn’t open on Ubuntu, but I had no idea how to open ports anymore. Luckily I found How to start rsync daemon at boot in the Ubuntu forums which told me exactly what I needed to know.
Partly as an aide to my memory, and partly to help anyone out there who might be struggling with the same project (unless everyone except me is already backing up their computers) I decided to summarize the process without all the false starts and dead ends it took me to figure it all out. For more information, the rsync man page is useful, as well as the results in Google for windows rsync.
Set up rsync server on Ubuntu
- Run
sudo apt-get install rsync
(it’s probably already installed) - Create a file named rsyncd.conf in /etc
-
sudo nano /etc/rsyncd.conf
- Add the following to rsyncd.conf, replacing all instances of
username
with your Ubuntu username:[usernamebackup] path = /home/username/backup comment = Backup uid = username gid = username read only = false auth users = username secrets file = /etc/rsyncd.secrets
-
sudo chmod 644 /etc/rsyncd.conf
-
- Create a file named rsyncd.secrets in /etc
-
sudo nano /etc/rsyncd.secrets
- Add the following to rsyncd.secrets, replacing
username
with your username andpassword
with a password of your choosing:username:password
-
sudo chmod 600 /etc/rsyncd.secrets
-
- Open rsync port by editing /etc/default/rsync and setting
RSYNC_ENABLE=true
- Restart rsync
sudo /etc/init.d/rsync restart
Set up rsync client on Windows
- Install Cygwin, making sure Editors > nano and Net > rsync are selected
- Add
C:\cygwin\bin;
to the Windows PATH statement- Right-click on My Computer and select Properties
- Switch to the Advanced tab and click the Environment Variables button at the bottom
- Find the “Path” or “PATH” variable in the System variables list at the bottom and click Edit
- Add
C:\cygwin\bin;
to the beginning of the list
- Create secret file to store password in Cygwin
- Start Cygwin Bash Shell
- Create secret file in the filesystem root and enter only the password in rsyncd.secrets above, with no spaces or line breaks
nano /secret
-
chmod 600 /secret
-
chown Administrator:SYSTEM /secret
- Create bat file to run rsync
- Open Notepad and enter the following command, replacing User Name with your Windows User Name directory, username with your Ubuntu username, and ipaddress with the IP address of your Ubuntu server (e.g. 192.168.0.100):
C:\cygwin\bin\rsync.exe -qrtz --password-file=c:\cygwin\secret --delete "/cygdrive/c/Documents and Settings/User Name" username@ipaddress::usernamebackup
As you may have guessed, the
"/cygdrive/c/Documents and Settings/User Name"
command line option designates where to start backing up from. As currently configured, this will backup your Windows home directory (Desktop, My Documents, etc). If you want to backup your whole hard drive, change that option to"/cygdrive/c"
. - Save the file as C:\rsync.bat
- Open Notepad and enter the following command, replacing User Name with your Windows User Name directory, username with your Ubuntu username, and ipaddress with the IP address of your Ubuntu server (e.g. 192.168.0.100):
Create scheduled task to run C:\rsync.bat once a day
- Create scheduled task
- Goto Start > Programs > Accessories > System Tools > Scheduled Tasks
- From the File menu, select New > Scheduled Task
- Name this task “rsync backup”
- Right-click on the task and select properties
- Enter
C:\rsync.bat
in the Run field - Switch to the Schedule tab and select the time you want the backup to run every day and click Ok
- Test the scheduled task
- Create a folder called C:\data and put a few photo files in it
- Edit C:\rsync.bat and change
"/cygdrive/c/Documents and Settings/User Name"
to"/cygdrive/c/data"
- Add the command
pause
on a new line at the bottom of C:\rsync.bat and save the file - Right-click on the “rsync backup” scheduled task and select “Run”—A command window should popup and with either errors or the list of files being transfered. If there are errors, troubleshoot them.
- Once the scheduled task and C:\rsync.bat appear to be working correctly, change
"/cygdrive/c/data"
back to"/cygdrive/c/Documents and Settings/User Name"
and remove thepause
command - Finally, edit the scheduled task properties and change “Run as:” to
NT AUTHORITY\SYSTEM
—this will ensure that the process runs in the background, without popping up a command prompt window
Run your first backup
Run C:\rsync.bat from the command line before going to bed. Backing up 35GB over a wireless-g connection took me over 8 hours. Subsequent backups take less than a minute. Behold the beauty of rsync.
Update: for information on how to backup Ubuntu to Ubuntu (or Linux to Linux really) using rsync with passphraseless keys, check out Playing with rsync on Ubuntu. For information on backing up Ubuntu to an external hard drive, check out How to rsync your Ubuntu home directory to an external hard drive.
BackupPC just run ok for me but if you don’t like it maybe you’d check out Bacula.
[…] blog covers one of the more exotic solutions – backup Windows XP to Ubuntu, using rsyncread more | digg […]
I’d tunnel rsync over ssh (you get compression and encryption that way) but that’s just me..
Glate, I started looking into that, but it made the process much longer—I wanted this howto to be something pretty much anyone (within reason) could follow. Plus I didn’t realize at the time that I could generate a key without a passphrase. I’ll probably add that to this post (or another) later.
Good post. Don’t forget you can configure cron on cygwin and use that as the scheduler. Then use a shell script instead.
I like this layout. Anyway, good article.
I use a similar method (cwrsync – a minimal cygwin+rsync install package) to backup my Windows data to my offsite rsync.net account. Check out http://itefix.no/cwrsync/ as well as http://www.rsync.net which I can’t praise highly enough.
me, I didn’t realize you could cron things cygwin. I’ll have to try that.
My friend Corey recommends Unison, which sounds like SVN+rsync for managing 2+ filesystems.
Arone, I tried cwrsync but as I recall, it seemed a little too complicated. I also tried DeltaCopy, but it kept erroring out. Wouldn’t it be nice if Windows just shipped with rsync (and a hundred other useful linux command line utilities)?
Look into rdiff backup. Both the benefits of a mirror and an archive with not much increase in disk space because only changes are stored in compressed files. Always, can drag the latest file to get the most current version if you are in a hurry. I use it and really love it.
mark, thanks for the heads up, I hadn’t heard of that, will have to check out. I was mostly just looking for a mirror of my content (in the event of a hard drive crash) and not worried about recovering files deleted in the distant past.
Windows XP already has a backup program built in which can backup to anywhere. (Start > Programs > Accessories > System Tools > Backup) I would just create a backup share on the Ubuntu box via Samba and then map the shares to my WinXP box and then run the Windows backup program. No need for cygwin or anything extra on the Windows side.
Keep it simple!
[…] My How to regularly backup Windows XP to Ubuntu, using rsync made the top of Digg this […]
John, I’ve found Samba somewhat slowish and confusing to configure. I have tried using Windows Backup (though not across a network) and I didn’t like it because it used a proprietary archive format. With a mirrored backup on another filesystem, I can just navigate to the directory where the files live and access them directly.
Plus eventually I can tunnel rsync over ssh.
well done
Great how to! I’ve used rsync for my server backup needs, and just started to toy with S3Sync and having Amazon backup my data. On my desktop I still prefer a remote solution like BackupRight or iBackup.
you should run rsync over ssh as it’s a “clear text” service.
Steven, S3 is expensive. I assume you’ve read this cost comparison with Dreamhost?
yo, I’m only running this across my local network (granted it’s wireless), but I plan to set up ssh with public keys now that I know you can use them without passphrases.
I use robocopy to mirror my XP files to a second hard drive. Can I do the same with rsync XP=>XP box?
ale, presumably if you had an rsync server set up on the backup Windows box (Rsync for Windows has instructions on how to do this) you could do this, but using a native windows app might be a better way to go. Several people have recommended SyncToy from Microsoft, which is available to download for free.
Have you considered “Windows Services for UNIX“
Have you tried BackupPC on Ubuntu in combination with rsync? It is very similar to this setup but gives you a web interface on Ubuntu, incremental backups , retention policies, compression, pooling, etc. Very, very cool. I have a full month of incremental backups of my Windows machine using rsync and BackupPC (http://backuppc.sourceforge.net). It also backs up my Macs and FreeBSD server. There is a BackupPC write-up on my site if you’re interested.
Hi, I seem to be having problems with rsync-ing binary files. Some are ended up damaged.
Have you encoutnered this problem as well?
Thanks.
Dad, nope I didn’t even know it existed (added a link to it in your comment). I skimmed the Introduction to Microsoft Windows Services for UNIX 3.5 article, which says it includes “More than 350 commonly used UNIX commands and utilities” in the form of Interix. Sounds just like Cygwin. Apparently it comes packaged with Windows Vista. Not readily apparent if that includes rsync.
Casey, in the second sentence of the post I said I first tried BackupPC, and because of the plethora of config options, its dependence on Samba, and the fact that it didn’t work after I first ran it manually, I decided against wrestling with it further. Plus I didn’t need or want incremental backups or a (clunky) web interface.
sufehmi, nope, no known problems. I’m not the best resource, I’d probably talk to the rsync folks.
[…] backup Windows XP to Ubuntu, using rsync Filed under: Uncategorized — recar @ 11:02 am How to regularly backup Windows XP to Ubuntu, using rsync This blog covers one of the more exotic solutions – backup Windows XP to Ubuntu, using […]
Great article! Once this process is in place, what’s the best way to restore? Are there similar options to conduct a restore via rsync? Thanks!
Rsync is definitely one of the most useful tools, however sometimes I find it has problems with copying the same file over and over again. I definitely agree with the comment that you should simply run rsync over ssh instead of setting up a daemon just for rsync.
… aaammm, great. You have two working systems when backing up. Now assume, you need to restore windows … eeerrr .. how? You either have a broken windows or you cant’ restore ’cause windows doesn’t let you do it while it is running.
The net is full of backup ideas but VERY few ventures into restoring windows. At this point it becomes a nightmare.
Using linux to write a file to ext3 is easy. Using Linux to write to a NT partition … is still in alpha. (gpart)
Yes, I am struggling with the same problem. Ghost is not an answer either, I used it. It did the backup without any problem. On restore it reported ‘damaged archive’. Good luck to anybody relying on Ghost(10). You’ll need it.
Restoring windows is not something I’m particularly interested in. Why restored a crudded up OS when you could just start fresh. I’m more interesting in not losing my data.
Thank you for this excellent how-to! In case something goes wrong, be sure to have a look at var/log/rsyncd.log on the server. This helped me to spot a nasty typo in my rsyncd.conf. In Windows it’s easier (though less secure) to set the password with the RSYNC_PASSWORD user variable instead of using a password-file. Also, cygwin has problems with filenames containing high-bit or wide characters. These can be avoided by using this
alternative build of cygwin1.dll.
Sorry for another plug for BackupPC. I setup it up on a spare Ubuntu PC at my parents place and installed rsyncd on their WinXP box to back up their “Docs and Settings” folder. Going on 8 months, it runs fine and I haven’t changed a thing.
My backup always ends with an error saying that a path name is too long. Is there any way for me to find all the files on my Windows box with path names that are too long, so I can rename them?
Is it possible to do it the other way round? I want to synchronise linux to windows.
Hey, this looks like a great how-to. I was planning to setup a similar backup solution, and this will be a good resource. I have a question about steps 2a, b, and c. You talk about both rsyncd.conf and rsynd.conf. I assume it’s just a typo, but I thought I’d check. Thanks for the good info!
Can you please provide a user guide for Fedora?? Thank you
anon, sorry I don’t use Fedora, but I imagine it would be fairly easy to translate, primarily replacing
sudo apt-get
with the Fedora-appropriate package manager,yum
I believe it is called.Fantastic Guide. Thanks, it helped me out alot. I read other solutions that suggested mountain a windows file share and running rsync locally on the ubuntu server. However i did not like that solution because of its obvious vulnerbilities. Thanks Again
***This is to Bill
Have you considered “Acronis” it builds a perfect image of your partition (including windows-even while windows has its gui up and is logged in.) Should your system fail you use a premade bootup disc (this not custom to every system so i can be used to bootup any system in-house) follow the prompts, insert the dvd you backed up to (or network location) and restore. I “ghosted” about 50 pc’s this way at an office location a while back. Flawless (mostly)
-Acronis is not automated as far as i know though…This is why i’m considering an Ubuntu solution- But, Acronis may be the answer to your prob.
This is for Glen, re the long file names problem. The same problem happens when I back up my Vista box, although the Win-XP-Home box backs up just fine. The solution which worked for me, on Vista, was to exclude file names which include “Application Data”. That appears to be a Vista synonym (possibly the same as AppData), built for the user environment while logged in, and it loops back to inself so Vista complains that file names are too long.
In any case, adding the following option in the rsync command line fixed the trouble with backup from Vista:
— exclude=”Application?Data”
The question mark matches the single space between Application and Data.
The option goes somewhere after the rsync command and before the “cygdrive…” specification.
Hope that helps. And thank you all for an excellent guide and discussion. Very pleased with the results. I do backups manually, and called the script “bu2ub2.bat” as a handy mnemonic. It is great for my purposes which is to save the files from Windows boxes which are personal – eg photos – and do not want to lose when Windows eventually goes wonky. If Windows or its box fails, I would just replace with a re-install or repaired box or upgrade to new box/version of Windows, and not try to restore Windows as earlier. So I’m not needing a backup-restore, more a file-save for preservation. And the method described was just perfect.
For simplicity, I omitted the –delete option for rsync. Some overhead, end up retaining photos etc that were deleted or renamed, but perfer that risk to the possibility that an rsync runaway might delete stuff that was previously saved.
Merry Christmas to all!
Justin: stunning how-to, many thanks
Vista doesn’t recognize “Administrator” as a user when I run: chown Administrator:SYSTEM /secret. How can I get around this? Thanks.
is the ipadress the only way to specify the rsync host? Would it be possible to use the fqdn instead?
Erich, yes, you should be able to use a fully qualified domain name (FQDN), I just referred to a specific IP address because I was on a local area network behind a NAT router.
Excellent tutorial. I tired following guides on a few other sites and was unsuccessful before finding yours. Thanks for the help.
Thanks for this post.
When “run” from Scheduled Tasks get status “could not run”. When I run rsync.bat from command prompt, I get @ERROR:chroot failed. Removing password file reference and entering password manually gets same error.
Seems I set up the Windows XP with no password. When I added (the correct) one, the whole thing worked! It is running the first backup as I write this. It lists only those files that it cannot back up, because they are busy or whatever — stuff like NTUSER.DAT and ntuser.dat.LOG. It does not list every file as it runs.
Also, when I tried to change “run as” in the Scheduler setup to NT AUTHORITY\SYSTEM, I get an error message that I don’t have authority to make this change. What’s up with that?
For those talking about restoring Windows, I have a suggestion. (Even though this topic is over a year old now! Late to the party’d)
If you install Windows on its own partition, and store all of your files on a separate partition, you can use Justin’s method to backup the file partition and then simply use a free imaging program like Maxtor Maxblast to create an image of the Windows partition that you can easily restore if it gets corrupted.
followed the whole thing and it failed. changed the path in the /etc/rsyncd.conf from path = /home/username/backup to /home/username and its worked fine. So how do I backup to a disctory other than my home dir ?
DeltaCopy is a “Windows Friendly” wrapper around the Rsync so you don’t need to install cygwin. It can function in client or server mode.
http://www.aboutmyip.com/AboutMyXApp/DeltaCopy.jsp
Per the web page: If you have an “rsync” daemon running on Linux/Unix/BSD or any other platform, DeltaCopy client will successfully connect to them.”
To re-enforce a previous post: If you are running deltacopy or cygwin-rsync on Vista, you have exclude certain hidden links that vista uses to provide backwards support for XP applications (My Documents links to Documents, e.g.). see this link: http://www.svrops.com/svrops/articles/jpoints.htm
Additionally, wrt to DeltaCopy client on Vista, you have to install the client in something other that Program Files, since it doesn’t have a manifest and writes it’s configuration files to the install directory.
hey chuck, i am trying to figure thing deltacopy out? CAn you help me i have installed it on windows, and windows to windows works just fine. Only now i am tryint to setup rsync but thats hell, i dont nkow where to start, could you give me a step by step tutorial pleasse? The folder is named c:/backup and now i need to backup this one to my linux server(im using centos).
THx
good post. i’m with newbie, in that my path directive is being ignored, instead DeltaCopy is using the ssh user’s home directory as a starting point.
newbie, I’m pretty sure backup needs to be written as Backup since the directory structure is case sensitive
There is a slight flaw in this tutorial. Step 3, sub-step C should read “sudo chmod 600 /etc/rsyncd.secrets”. If you set 644 with modern copies of rsync it will refuse to authenticate against it as this file should not be readable by average users.
Steve, thanks, I didn’t know that, but I think you misread the instructions which have people correctly chmod the secrets file to 600, while chmodding the rsyncd.conf file to 644.
Oops. I did misread that. Disregard that comment. ;-)
Great tutorial! This was exactly what I was wanting to do with my home network. I have an old gateway that’s running xubuntu and I have three windoze machines that I’m backing up data from.
I had trouble with the “chroot failed” error and checked all my paths. They were set correctly. It wasn’t until I changed the password in rsyncd.secret & /secret to my xubuntu account passwd that everything worked.
I also discovered that I could specify different [usernamebackup]s in rsyncd.conf and call them from their respective PC in the bat file. Just copy and paste all the lines, change the [usernamebackup] to [nameofpc1backup], [nameofpc2backup], etc. and change all the paths to where you want each pc backed to, then call them from the bat file at username@ipaddress::nameofpc1backup.
Thanks again, Justin!!!
Great guide, thanks for taking the time to write this up and share it.
I read the guide and like the windows idiot I am, insisted on putting a colon (:) after the drive letter in the bat file when you clearly showed in the example to do as follows: “/cygdrive/c/Documents and Settings/User Name”, needless to say, I spent some time trying to figure out why I kept getting error’s about “no such file or directory”.
Anyhow, once I discovered that error it worked like a charm.
Thanks
Chris
Justin….you’re a legend, nice one!
Got it working no problem, other than the documented chroot error (ubuntu username and pass solved that one). Works great over my ovenvpn without pratting about with ssh port forwarding.Thanks dude ;)
Step 3a trying to open Cygwin bash shell. It will open then immediately close. did I miss something?
I’m getting this error, any idea?
rsync: failed to connect to 192.168.1.7: Connection refused (111)
rsync error: error in socket IO (code 10) at /home/lapo/packaging/rsync-3.0.4-1/
src/rsync-3.0.4/clientserver.c(122) [sender=3.0.4]
Ah, thank you so much. Now if only my ISP didn’t cap our bandwith…
Excellent straightforward process. Got me up and running in no time, so thanks very much. I am using DeltaCopy under Windows and it works great.
Thanks for posting this tutorial.
JG
GREAT post. This worked flawlessly on Vista -> Ubuntu (except instead of
chown Administrator:SYSTEM /secret
, I usedchown <my_user_name> /secret
(replacing <my_user_name> with my actual username of course). I’m also running rsync.bat as administrator to eliminate any Permission Denied errors. I don’t know how that’s going to work with Task Scheduler.Anyway, great simple guide. Thanks for taking the time.
I ran into the Vista Application Data “file name too long” problem and solved it as follows —
Andrew, thanks for the comments (I fixed up the formatting issues you had). Glad you found this post helpful.
Is the full Cygwin needed to act as a client for backups? I started to download Cygwin and gave up after 20mins it had a 400Mbyte download and only 1/2 finished. Is there a smaller version just fo backing up to ubuntu server?
roger, take a look at http://www.aboutmyip.com/AboutMyXApp/DeltaCopy.jsp for a windows rsync client … definitely better than installing cygwin :)
Hi, Very nice article. I have some problem to run rsync as daemon on Red Hat Linux server. Also, the link “How to start rsync daemon at boot” seems not working anymore. I like to read the article but not successful. Any suggestion will be appreciated.
Installed cwRsync package on Windows XP and then used my Fedora 6 as the server. These instructions worked perfect. Transferred a test file just fine. This may be just the beginning of a large project for me, but it is a good start.
Hello I did all the procedures, more to run rsync manually or scheduling, I am not having success, what could this happening? the need to review this, I did everything step by step. Thanks!
I’m using DeltaCopy (a Windows GUI for cygwin + rsync), however I believe the same problem happens when using cygwin + rsync without the GUI.
Problem is, the file names are too long. I receive the error: “failed: File name too long (91)” for pretty much all of my files.
Does anybody else also receive this error? I see Andrew did. However, I receieve this error for pretty much every file it tries to rsync – not just files in my Application Data folder.
Is anybody aware of a fix to this problem? I don’t understand how nobody else has come across this downfall! But it’d be really cool if I could get this whole backup system working.
Btw, thanks for this excellent article! It’s really easy to follow, and explained things well along the way. Thanks.
I tried this method and unfortunately some of my filenames were too long for the Windows version of rsync to handle. However, the version on my Ubuntu server handles long filenames perfectly fine.
Armed with that knowledge, I thought about instead treating the Windows PC like a server, and just use the rsync tool in Ubuntu (not the rsync server/daemon).
I now have a setup in which the Ubuntu server sends a WOL packet to the Windows PC, and then it waits 2 minutes for it to boot. It then mounts the C$ administrative share via the Samba client. Then I use the rsync tool to backup the remote PC as if it was part of the local filesystem. This means that long filenames can be handled! :-)
After rsync finishes, I simply unmount the C$ share and shutdown the Windows PC remotely using “net rpc shutdown -I -U administrator%password”.
Job done! Essentially, I’ve created a fully automated script that can boot the Windows PC, back it up, then shut it down again. All without any software installed on the client! It also means I don’t have to run the rsync daemon because it’s not required.
Thanks for this great tutorial. Easy to follow and works like a charm. Much appreciated!
Darryl, glad you found it useful.
i followed all the instructions. but i get an error when i run rsync.bat
@ERROR: auth failed on module rodkbackup
rsync error: error starting client-server protocol
at /home/lapo/packaging/rsync-3.0.6-1/src/rsync-3.0.6/main.c [sender=3.0.6]
i wrote in my /etc/rsyncd.conf
[rodkbackup]
path = /home/rodk/backup
comment = backup
uid = rodk
gid = rodk
read only = false
auth users = rodk
secrets files = /etc/rsyncd.secrets
and in my rsync.bat
C:\cygwin\bin\rsync.exe -qrtz --password-file=c:\cygwin\secret --delete "/cygdrive/c/Documents and Settings/Rod" rodk@192.168.7.12::rodkbackup
please help me
no its fine now, but when i set auth users = rod, then it asks password, it’s not processing but the password i typed is correct. I want to have a secured sync
another question, can you have multiple rsyncs in your rsync.bat
for example: C:\cygwin\bin\rsync.exe -qrtz –password-file=c:\cygwin\secret –delete “/cygdrive/e/Shared Documents/” rodk@192.168.7.12::rodkbackup
C:\cygwin\bin\rsync.exe -qrtz –password-file=c:\cygwin\secret –delete “/cygdrive/e/Armie/” rodk@192.168.7.12::armiebackup
C:\cygwin\bin\rsync.exe -qrtz –password-file=c:\cygwin\secret –delete “/cygdrive/e/Rod/” rodk@192.168.7.12::rodbackup
Hi.. a little help here pls.. i stuck in the windows procedures. i need to know how to create secret file to store password in cygwin.
and also how do i start cygwin bash shell
thank you so much..
Great stuff! I have 250gb so I’m trying to see if I can only back up files that have been modified, so it doesn’t back up all 250gb every night. Think lsyncd is the way forward for this… not sure about going Windows Server – Linux though!
Thanks mate.
Hi, I followed the steps as described here, however I get this error.
I somehow have cleared that error above. But now I am getting this alert.
I have set the file permissions to 600. But still I get this error. Any ideas on this will help.
I think –password-file=c:\cygwin\secret should be unix style so –password-file=/cygdrive/c/cygwin/secret that did it for me, connecting works fine now but I have permission problems instead.
rsync: recv_generator: mkdir “/Contacts” (in userbackup) failed: Permission denied (13)
Even though I have set 777 permissions on the sync root folder, any thoughts?
nice article
Using sync on windows. Pretty amazing, I didn’t knew about it earlier… Well the software which I am using is also very effective. I am using GS Richcopy 360 which helped me a lot in solving many issues and yes its GUI based. Fun to use!