Idea For A Good Backup Solution

[daniel3000]

Hello,

still searching for the "perfect" backup solution for my C3000. Requirements:

* backup the COMPLETE Zaurus setup with one shell command or better automatically each night with cron job or similar, i.e. without booting to single user mode and without going through any service or diagnose menu

* Easy restore in case of a total hardware failure, either on another device (cloning the setup) or on the same device

* No 4GB+ media must be needed (for Sharp's built-in microdrive backup this is needed, I was told)

-----

Now I have an idea. Please tell me if this could fulfill my needs or if I missed something important:


Use a large SD or CF card, probably 1GB is enough (excluding video and mp3 files on hdd3 from the backup saves a lot of space and time!).

Do the folllowing from the shell script:

Code: [Select]

#!/bin/bash

# backup entire home partition:
dd if=/dev/mtdblock3 | bzip2 > /mnt/card/mtdblock3.img.bz2

# backup entire user program area:
dd if=/dev/hda2 | bzip2 > /mnt/card/hda2.img.bz2

# backup hdd3 contents, but without multimedia stuff
# (file dontbackup contains names of Multimedia directories to exclude)
gtar cvzf /mnt/card/hdd3.tgz -X /mnt/card/dontbackup

# suspend Zaurus after backup:
sleep 5 && qcop QPE/System "execute(QString)" suspend
hdd1 is readonly so probably does not have to be backed up I think. Isn't it fully restored by a Cacko ROM reflash?

Restore procedure would be:

0. (if necessary): Restore factory hdd strutrure via service menu
1. Re-flash Cacko ROM per instructions
2. bunzip2 /mnt/card/hdd2.img.bz2 | dd of=/dev/hda2
3. bunzip2 /mnt/card/mtdblock3.img.bz2 | dd of=/dev/mtdblock3
4. install gtar
5. cd /hdd3
6. gtar xzvf /mnt/card/hdd3.tgz
7. reboot

Would this work?

Could there be problems

* restoring hda2 because we execute something from hdd2 (bunzip2 is on hda2 in Cacko)
* backing up the entire partitions while they are in use (non-consistent file systems)?


Can these potential problems be solved by using single user mode? (I would like to avoid this, because I want to be able to make a backup daily / at night, and I really do not want to reboot the Zaurus that often!).

Can the backed up partitions be unmounted cleanly during operation of the Zaurus in order to avoid non-consistent file systems in the backup?

In order to unmount them cleanly, how can the Zaurus still access the necessary programs (bzip2 and gtar on hdd2) - maybe copy them onto a Ramdisk temporarily. But how to do that?

Any thoughts about this?

Thank you
daniel

[Meanie]

Hello,

still searching for the "perfect" backup solution for my C3000. Requirements:

* backup the COMPLETE Zaurus setup with one shell command or better automatically each night with cron job or similar, i.e. without booting to single user mode and without going through any service or diagnose menu

* Easy restore in case of a total hardware failure, either on another device (cloning the setup) or on the same device

* No 4GB+ media must be needed (for Sharp's built-in microdrive backup this is needed, I was told)

-----

Now I have an idea. Please tell me if this could fulfill my needs or if I missed something important:


Use a large SD or CF card, probably 1GB is enough (excluding video and mp3 files on hdd3 from the backup saves a lot of space and time!).

Do the folllowing from the shell script:

Code: [Select]

#!/bin/bash

# backup entire home partition:
dd if=/dev/mtdblock3 | bzip2 > /mnt/card/mtdblock3.img.bz2

# backup entire user program area:
dd if=/dev/hda2 | bzip2 > /mnt/card/hda2.img.bz2

# backup hdd3 contents, but without multimedia stuff
# (file dontbackup contains names of Multimedia directories to exclude)
gtar cvzf /mnt/card/hdd3.tgz -X /mnt/card/dontbackup

# suspend Zaurus after backup:
sleep 5 && qcop QPE/System "execute(QString)" suspend
hdd1 is readonly so probably does not have to be backed up I think. Isn't it fully restored by a Cacko ROM reflash?

Restore procedure would be:

0. (if necessary): Restore factory hdd strutrure via service menu
1. Re-flash Cacko ROM per instructions
2. bunzip2 /mnt/card/hdd2.img.bz2 | dd of=/dev/hda2
3. bunzip2 /mnt/card/mtdblock3.img.bz2 | dd of=/dev/mtdblock3
4. install gtar
5. cd /hdd3
6. gtar xzvf /mnt/card/hdd3.tgz
7. reboot

Would this work?

Could there be problems

* restoring hda2 because we execute something from hdd2 (bunzip2 is on hda2 in Cacko)
* backing up the entire partitions while they are in use (non-consistent file systems)?


Can these potential problems be solved by using single user mode? (I would like to avoid this, because I want to be able to make a backup daily / at night, and I really do not want to reboot the Zaurus that often!).

Can the backed up partitions be unmounted cleanly during operation of the Zaurus in order to avoid non-consistent file systems in the backup?

In order to unmount them cleanly, how can the Zaurus still access the necessary programs (bzip2 and gtar on hdd2) - maybe copy them onto a Ramdisk temporarily. But how to do that?

Any thoughts about this?

Thank you
daniel
[div align=\"right\"][a href=\"index.php?act=findpost&pid=97184\"][{POST_SNAPBACK}][/a][/div]

you probably can copy some essential utilities such as gtar and bzip2  to /hdd1 since there is quite a bit of space left there. once you have all your essential binaries on /hdd1 you probably want to make a master backup of it but you don't need to back it up regularly since it would not change much (being read only most the time)
so /hdd2 /hdd3 and /home would need to be backed up.  probably will be able to unmount hdd2 and hdd3 if you stop all the non essential applications and services (ie samba, httpd, ftpd, sshd, musicplayer, texteditor, etc.).

btw, the Backup/Restore utility that comes with the Sharp ROM does backup everything except /hdd3. wish i knew how to invoke it from the command line to schedule it from a script

[bam]

or use your linux box to grep mtab for the mounted samba share and backup documents directory after of course a std backup. Although that may be slow.


but i like your idea, i might use it, actually i probably will

[daniel3000]

you probably can copy some essential utilities such as gtar and bzip2  to /hdd1 since there is quite a bit of space left there. once you have all your essential binaries on /hdd1 you probably want to make a master backup of it but you don't need to back it up regularly since it would not change much (being read only most the time)
so /hdd2 /hdd3 and /home would need to be backed up.  probably will be able to unmount hdd2 and hdd3 if you stop all the non essential applications and services (ie samba, httpd, ftpd, sshd, musicplayer, texteditor, etc.).

btw, the Backup/Restore utility that comes with the Sharp ROM does backup everything except /hdd3. wish i knew how to invoke it from the command line to schedule it from a script
[div align=\"right\"][a href=\"index.php?act=findpost&pid=97186\"][{POST_SNAPBACK}][/a][/div]

So if the internal backup/restore program does this, it probably also unmounts and stops services etc.? Or does it risk bad files by backing up a running fs?

Maybe the backup/restore application is only a wrapper for command line utilities.
Is there a way to trace what utilities it calls (with watch ps or something like that) and with what options? We could reengineer it in a shell sctipt then. :-)


Re. hdd1: I would like to avoid copying the utilities to hdd1, because my idea for the resotre procedure would not restore hdd1, so the utilities are missing on a restored machine. One more thing to remember when restoring: to copy the utilities back onto hdd1.
Thus my idea with a temporary RAM disk simply created from within the backup script and deleted afterwards.
But I do not know how to do that.

Thanks for your hints!

daniel

[T-RuX]

It's a good idea.
Thanks.! I'll adapted for OpenZaurus...

[bluedevils]

If using a *nix box is possible, then why not nfs mount and do your dd to the nfs mount?

[daniel3000]

If using a *nix box is possible, then why not nfs mount and do your dd to the nfs mount?
[div align=\"right\"][a href=\"index.php?act=findpost&pid=97217\"][{POST_SNAPBACK}][/a][/div]

Sure, but I want to be independent. :-) And for a good data insurance (=flash card) I am willing to pay qhite some money.

But you are right, as long as I am at home, and WLAN and the server in my basement are running, I can also let the backup script write to a drive on the server in addition to the SD card. Would mean an additional insurance against theft of the card.

My plan is this: Before I fall alseep in bed, I usually read some emails or ebooks. Then I prepare KO/PI to wake me up in the morning and put the Zaurus aside.
In addition I now want to start the backup routine, which then may take as long as it needs to write everything onto SD and/or server drive NFS mount and then suspends.


Thanks
daniel

[Meanie]

If using a *nix box is possible, then why not nfs mount and do your dd to the nfs mount?
[div align=\"right\"][a href=\"index.php?act=findpost&pid=97217\"][{POST_SNAPBACK}][/a][/div]

Sure, but I want to be independent. :-) And for a good data insurance (=flash card) I am willing to pay qhite some money.

But you are right, as long as I am at home, and WLAN and the server in my basement are running, I can also let the backup script write to a drive on the server in addition to the SD card. Would mean an additional insurance against theft of the card.

My plan is this: Before I fall alseep in bed, I usually read some emails or ebooks. Then I prepare KO/PI to wake me up in the morning and put the Zaurus aside.
In addition I now want to start the backup routine, which then may take as long as it needs to write everything onto SD and/or server drive NFS mount and then suspends.


Thanks
daniel
[div align=\"right\"][a href=\"index.php?act=findpost&pid=97239\"][{POST_SNAPBACK}][/a][/div]

I backup my Z to my USB disk so even when I travel, I can make regular backups and keep multiple copies. However, I kick them off manually and only do them once a week. I should had done them daily! Lost a whole week's work when my C3000 died

[bam]

learned that lesson myself...

[jfs.w]

about the backup/restore utility that comes with the Sharp ROM - it seems that it does sort of boot the system into a single-user mode!! (my guess) - cos after activating it, u'll see that screen that u see when u first boot it up...

So the sequence goes - boot screen, backup starts, boot screen again, configure timezone screen.

oh, and btw - i discovered something interesting in /hdd1/.sys - there's this file called 'hdimage2.tgz' which is apparently a backup of the "standard" stuff in hdd3?! (and a few folders in /hdd2 - namely 'lost+found', and '.sys') Have a look at it...

[Meanie]

about the backup/restore utility that comes with the Sharp ROM - it seems that it does sort of boot the system into a single-user mode!! (my guess) - cos after activating it, u'll see that screen that u see when u first boot it up...

So the sequence goes - boot screen, backup starts, boot screen again, configure timezone screen.

oh, and btw - i discovered something interesting in /hdd1/.sys - there's this file called 'hdimage2.tgz' which is apparently a backup of the "standard" stuff in hdd3?! (and a few folders in /hdd2 - namely 'lost+found', and '.sys') Have a look at it...
[div align=\"right\"][a href=\"index.php?act=findpost&pid=97322\"][{POST_SNAPBACK}][/a][/div]

That's used to restore the directory structure on hdd3.
operational wise, you could wipe hdd3 and the Z would still function alas it will occasionally complain that it could not find some directories/files unless you restore the directory structure it likes, but except getting some annoying popup asking you to specify and alternate directory, ie your sd card, it would work. but for most of us, hdd3 holds our data which we do not like to loose and hence want to backup as well.