OESF Portables Forum
Everything Else => Zaurus Distro Support and Discussion => Distros, Development, and Model Specific Forums => Archived Forums => Debian => Topic started by: ArchiMark on April 01, 2008, 11:20:15 pm
-
Decided to take the plunge and got a new 32GB A-Data CF Card today....
Want to do this swap the right way and hopefully easy way....
I assume that I can 'clone' my current Debian EABI setup on my existing 6GB microdrive over to the new 32GB CF card, right?
Would this be the right way to do it?
1) Format new 32GB card and make 2 partitions; ext3 (/dev/hdc1 /mnt/cf) and a small swap (512MB on /dev/hdc2 /mnt/cf2).
2) Copy over current microdrive debian contents on hda2 over to new ext3 partition on 32GB CF card by doing:
#dd if=/dev/hda2 of=/dev/cf
* Note: hda1 on my existing microdrive is my swap partition.
So, is this correct way???
Thanks!
Mark
-
Decided to take the plunge and got a new 32GB A-Data CF Card today....
Want to do this swap the right way and hopefully easy way....
I assume that I can 'clone' my current Debian EABI setup on my existing 6GB microdrive over to the new 32GB CF card, right?
Would this be the right way to do it?
1) Format new 32GB card and make 2 partitions; ext3 (/dev/hdc1 /mnt/cf) and a small swap (512MB on /dev/hdc2 /mnt/cf2).
2) Copy over current microdrive debian contents on hda2 over to new ext3 partition on 32GB CF card by doing:
#dd if=/dev/hda2 of=/dev/cf
* Note: hda1 on my existing microdrive is my swap partition.
So, is this correct way???
Thanks!
Mark
Oooooo Nooooo. How fast is it? Was it priced reasonably?
I've never moved stuff with dd like that--I'd have tried tar, but only on fa,iliarity, not on the belief that tar would be a better way to do it. You are probably right.
-
@Archimark: I think that your cloning progedure needs some input from someone who has tried it. Possibly ZDevil or some of the others who have tarred up filesystems might be able to give some good advice here about using their methods.
@adf: Archimark is talking about using and A-Data card. I remember you posting about major problems getting Debian to work on your transplanted A-data card. Did you solve those problems?
Cheers
-
Oooooo Nooooo. How fast is it? Was it priced reasonably?
I've never moved stuff with dd like that--I'd have tried tar, but only on fa,iliarity, not on the belief that tar would be a better way to do it. You are probably right.
I'm not sure how fast it is....but it was priced very reasonably I though from SuperMediaStore.com....
$149.99 for the 32GB CF and $69.99 for a 16GB SDHC Card....
@Archimark: I think that your cloning progedure needs some input from someone who has tried it. Possibly ZDevil or some of the others who have tarred up filesystems might be able to give some good advice here about using their methods.
Sounds like a good idea, tux.....
ZDevil, where are you???.... or anyone else....
Cheers
-
Decided to take the plunge and got a new 32GB A-Data CF Card today....
no, you definitely don't want to use "dd" to copy a file system between different types of media unless you guarantee the number of blocks in the partition is the same or larger on the target.
anyway, "cat" is faster - it uses more buffering!
the best way to copy a filesys is like this:
cd /mnt/partition1
tar cf - | (cd /mnt/partition2 ; tar xf - )
tar will copy all files and devices, preserve the datestamps and protections etc.
if you were l33t, you could probably also do it with cpio but I'm too lazy to look it up.
so, just partition the CF appropriately. oh, do NOT use ext3, the journalling will kill the card faster.
-
Decided to take the plunge and got a new 32GB A-Data CF Card today....
no, you definitely don't want to use "dd" to copy a file system between different types of media unless you guarantee the number of blocks in the partition is the same or larger on the target.
anyway, "cat" is faster - it uses more buffering!
the best way to copy a filesys is like this:
cd /mnt/partition1
tar cf - | (cd /mnt/partition2 ; tar xf - )
tar will copy all files and devices, preserve the datestamps and protections etc.
if you were l33t, you could probably also do it with cpio but I'm too lazy to look it up.
so, just partition the CF appropriately. oh, do NOT use ext3, the journalling will kill the card faster.
Thanks for your help with this, speculatrix!!
Will report back after I try this out...
-
[snipped]
oh, do NOT use ext3, the journalling will kill the card faster.
This may not be true in practice, as argued in this article, which in a way also applies to CF disks: http://wiki.eeeuser.com/ssd_write_limit (http://wiki.eeeuser.com/ssd_write_limit)
And given that my Z with 16GB vs. my Eee PC with 4GB ...
-
[snipped]
oh, do NOT use ext3, the journalling will kill the card faster.
This may not be true in practice, as argued in this article, which in a way also applies to CF disks: http://wiki.eeeuser.com/ssd_write_limit (http://wiki.eeeuser.com/ssd_write_limit)
And given that my Z with 16GB vs. my Eee PC with 4GB ...
Thanks for that info/link....sounds reassuring as I have a feeling I won't be using my Z in 25 years....
-
@adf: Archimark is talking about using and A-Data card. I remember you posting about major problems getting Debian to work on your transplanted A-data card. Did you solve those problems?
No, I went to pdaxii13. I am hoping the sharpbootloader method will work, though.
-
anyway, "cat" is faster - it uses more buffering!
I just realized that I don't see 'cat' below in your instructions???
Also, I thought that 'cat' is for just viewing the content of a file?
the best way to copy a filesys is like this:
cd /mnt/partition1
tar cf - | (cd /mnt/partition2; tar xf - )
tar will copy all files and devices, preserve the datestamps and protections etc.
So, just want to make sure I understand this correctly for my situation of copying my hda2 partition with Debian EABI (hda1 is my swap partition) on it over to new partition on 32GB CF card....
So, here's the existing and new setups:
Existing 6GB Microdrive
hda1 is Swap Partition
hda2 is Debian EABI Partition
New 32GB CF Card
cf1 will be new Debian EABI Partition
cf2 will be new Swap Partition
cd /mnt/cf1
tar /mnt/hda2 - | (cd /mnt/cf1; tar xf - )
Does this look right?
Thanks!
-
rsync also works and is faster if you need to redo.
-
anyway, "cat" is faster - it uses more buffering!
I just realized that I don't see 'cat' below in your instructions???
Also, I thought that 'cat' is for just viewing the content of a file?
no, you could do, for example, "cat /dev/hda > /dev/hdb" to clone an entire hard drive
So, here's the existing and new setups:
Existing 6GB Microdrive
hda1 is Swap Partition
hda2 is Debian EABI Partition
New 32GB CF Card
cf1 will be new Debian EABI Partition
cf2 will be new Swap Partition
cd /mnt/cf1
tar /mnt/hda2 - | (cd /mnt/cf1; tar xf - )
Does this look right?
Thanks!
you're missing a dot
cd /mnt/hda2
tar cf - . | (cd /mnt/cf1 ; tar xf - )
you cd into the source directory so that when you do the tar the current directory is the "root" of all the files; the "-" in the tar command outputs the stream to stdout; the stream is piped into a separate shell (the brackets) which starts in the destionation directory and unpacks the incoming stream from stdin.
the same trick works over an ssh session.
-
anyway, "cat" is faster - it uses more buffering!
I just realized that I don't see 'cat' below in your instructions???
Also, I thought that 'cat' is for just viewing the content of a file?
no, you could do, for example, "cat /dev/hda > /dev/hdb" to clone an entire hard drive
So, here's the existing and new setups:
Existing 6GB Microdrive
hda1 is Swap Partition
hda2 is Debian EABI Partition
New 32GB CF Card
cf1 will be new Debian EABI Partition
cf2 will be new Swap Partition
cd /mnt/cf1
tar /mnt/hda2 - | (cd /mnt/cf1; tar xf - )
Does this look right?
Thanks!
you're missing a dot
cd /mnt/hda2
tar cf - . | (cd /mnt/cf1 ; tar xf - )
you cd into the source directory so that when you do the tar the current directory is the "root" of all the files; the "-" in the tar command outputs the stream to stdout; the stream is piped into a separate shell (the brackets) which starts in the destionation directory and unpacks the incoming stream from stdin.
the same trick works over an ssh session.
THANKS for the explanation/clarification, speculatrix!!
Think I got it now....
-
rsync also works and is faster if you need to redo.
Thanks for your suggestion too, iwantprogress, didn't know about rsync....
Just googled it and found the rsync website....sounds interesting....
-
OK, tried the 'cat' approach and here's what I get:
#cat /dev/hda2 > /dev/hdc1
#df
Filesystem 1K-blocks Used Available Used% Mounted on
/dev/hda2 5589680 1317484 3988248 25% /
tmpfs 31180 0 31180 0% /lib/init/rw
udev 10240 80 10160 1% /dev
tmpfs 31180 4 31176 1% /dev/shm
/dev/hdc1 5184312 -25616124 30516488 - /mnt/cf1
#cd /mnt/cf1
EXT3-fs error (device hdc1): ext_readdir: bad entry in directory #2: directory
entry across blocks - offset=0, inode=4294967295, rec_len=65536, name_len=255
So, should I just try the other way or did I do something wrong???
Thanks!
-
Cat gives same results as dd - won't work unless devices are the same or close enough
Use the tar method!
-
given that the cards are different sizes and therefore your partitioning needs to be different, you're going to run into a world of pain copying the card raw block style; just use "fdisk" to partition the target, "mkfs.ext2 /dev/X" to format it, then mount and copy files. really, it's actually not hard :-)
-
you're missing a dot
cd /mnt/hda2
tar cf - . | (cd /mnt/cf1 ; tar xf - )
you cd into the source directory so that when you do the tar the current directory is the "root" of all the files; the "-" in the tar command outputs the stream to stdout; the stream is piped into a separate shell (the brackets) which starts in the destionation directory and unpacks the incoming stream from stdin.
the same trick works over an ssh session.
Just to clarify, in looking at the above, shouldn't the first 'tar' above be for the 'hda2' partition, not 'cf' as shown??
In other words, shouldn't the above look like this?
#cd /
#tar / - . | (cd /mnt/cf1; tar xf - )
Thanks!
-
given that the cards are different sizes and therefore your partitioning needs to be different, you're going to run into a world of pain copying the card raw block style; just use "fdisk" to partition the target, "mkfs.ext2 /dev/X" to format it, then mount and copy files. really, it's actually not hard :-)
I did format partition after I created it, speculatrix....so that's already done...
As for copying files, I can do that, just assumed that this might not work as when you try to do that with a Windoze install...it won't make for a bootable drive, you have to use a cloning app in order to get an exact duplicate of drive that boots....
So, if Linux is different than Windoze in this regard, then no problem, I'll just copy the directories over to the CF card....
Hope this makes sense....
-
"tar cf" is the command, "c" means create, "f" means the next argument is the output file (- means use stdout). "tar xf" means extract with source being stdin.
if you're copying the root partition, you will need to exclude the target mount point, eg.
cd /
tar cf - . --exclude=/mnt/newdir | ( cd /mnt/newdir ; tar xf - )
-
p.s. if you're copying with tar, the target disk/partition needs to be formatted and mounted in advance.
if you're cloning with cat or dd, the partition should NOT be mounted at the time.
-
"tar cf" is the command, "c" means create, "f" means the next argument is the output file (- means use stdout). "tar xf" means extract with source being stdin.
if you're copying the root partition, you will need to exclude the target mount point, eg.
cd /
tar cf - . --exclude=/mnt/newdir | ( cd /mnt/newdir ; tar xf - )
I get it now, speculatrix!!!....I thought that the 'cf' there was referencing 'cf' as in 'compact flash'......
p.s. if you're copying with tar, the target disk/partition needs to be formatted and mounted in advance.
if you're cloning with cat or dd, the partition should NOT be mounted at the time.
Thanks for clarifying that....so, I did the clone with 'cat' wrong as I had mounted the partition I think....
Meanwhile, I'm just copying over directories to CF card now....
-
Copying over directories...most copying over OK, but have had a few error messages:
#cp -r /proc /mnt/cf1
Warning: /proc/ide/hd?/settings interface is obsolete, and will be removed soon!
cp: cannot open `/proc/sys/net/ipv4/route/flush' for reading: Permission denied
cp: cannot open `/proc/sys/net/ipv6/route/flush' for reading: Permission denied
process `cp' is using deprecated sysctl (syscall) net.ipv6.neigh.default.retrans_time:
Use net.ipv6.neigh.retrans_time_ms instead.
cp: reading `/proc/sysrg-trigger': Input/output error
No more HD green light activity showing and command prompt has not returned after 10 minutes...
????
Thanks!
-
The stuff in /proc you needn't worry about, it's auto generated by the kernel
-
ooops, yes, exclude /proc and /sys too. probably not a bad idea to clean out /tmp too.
-
Thanks for the advice, rogalian & speculatrix!
-
let us know when you're back up and running.
-
let us know when you're back up and running.
Thanks speculatrix.....
Meanwhile, copied over directories, cleaned out 'tmp'.....
Just did 'df' and here's what I see:
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/hda2 5589680 1317404 3988328 25% /
tmpfs 31180 0 31180 0% /lib/init/rw
udev 10240 80 10160 1% /dev
tmpfs 31180 4 31176 1% /dev/shm
/dev/hdc1 5589680 2419708 2886024 46% /mnt/cf1
So......why does my existing microdrive with Debian EABI only use 25% of the drive and the new 32GB CF card use 46% of drive???
Maybe I'm just being overly cautious with all of this...but I'm trying to avoid situation where I yank out microdrive, take apart my Z and install CF card, put Z back together, and voila! it doesn't boot up or something like that....
Thanks!
-
I had to do something similar last night, when I managed to make my 16gb CF unbootable, fortunately i had my 4gb I was using until about 1 month ago :-
1) Attached 4gb microdrive to laptop using USB adapter, checked the filesystem (just to be safe).
2) Mount 4gb microdrive (/media/disk in this case)
3) cd /media/disk
4) copy contents to laptop HD (I wanted a) a backup a fast way to redo the process if I messed up)
mkdir /home/andy/z/cf_recovery
find . | cpio -pdmv /home/andy/z/cf_recovery
5) wait .......
6) once the copy was complete umount /media/disk
7) Disassemble Z (c3100 in my case)
9) Retrieve CF card from Z, attach to lappy via USB.
10) It automounted on /media/disk, so I unmounted it.
11) Create an ext2 filesystem (yes I want ext3 journaling, but i am going to dump a lot of data in a stable environment so I don't need the journal - it would only slow things down).
mke2fs /dev/hdb1
tune2fs -i 0 -c 0 /dev/hda1
(turns off fsck after a number of mounts).
12) mount the cf card (mount /dev/hdb1 /media/disk)
13) cd /home/andy/z/cf_recovery
13) find . | cpio -pdmv /media/disk
14) wait .....
15) umount cf card, insert in Z, rebuild Z - hey presto - working fs
BTW. If you need to repeat step 13) for any reason either remake the filesystem or add the 'u' flag (unconditional overwrite) to the cpio.
Hope this helps
ajnk
(Then I did an apt-get update ; apt-get upgrade )
-
Good to know that things are working now.
ajnk: so do you mean the latest yonggun kernel can boot off an ext2 partition? I remember the previous one only takes ext3 ...
-
Good to know that things are working now.
ajnk: so do you mean the latest yonggun kernel can boot off an ext2 partition? I remember the previous one only takes ext3 ...
I don't know the answer to your question as I enabled the journal (tune2fs -j /dev/hda1) before inserting the CF into the Z. I left journaling off while copying about 2.5Gb of data to it (the recovery from the laptop) - it sped things up A LOT and stopped my laptop from crashing. I almost looked like the CF was thrashing - but it's memory not a disk ?!?!
Also edited /etc/fstab on the Z to add the noatime option to the filesystem mount - to turn off recording off access times - again should help performance.
-
So......why does my existing microdrive with Debian EABI only use 25% of the drive and the new 32GB CF card use 46% of drive???
hmm, odd.
try doing an "ls -laR" within each, redirecting output to file and diff'ing them? might need to "grep -v ^d" to omit directory names.
-
So......why does my existing microdrive with Debian EABI only use 25% of the drive and the new 32GB CF card use 46% of drive???
hmm, odd.
try doing an "ls -laR" within each, redirecting output to file and diff'ing them? might need to "grep -v ^d" to omit directory names.
Thanks for the suggestion, speculatrix....
Meanwhile, decided to just bite the bullet and deleted everything on CF card and copied over stuff again....
Now, when I did 'df' I get:
#df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/hda2 5589680 1317392 3988340 25% /
tmpfs 31180 0 31180 0% /lib/init/rw
udev 10240 80 10160 1% /dev
tmpfs 31180 0 31180 1% /dev/shm
/dev/hdc1 5589680 1386936 3918796 27% /mnt/cf1
So, hda2 and cf1 are close now, but still slightly different.
Is this OK?? Or ???
Also, since I've:
1) 'flipped' location of Debian from second partition (hda2) on microdrive to first partition (cf1) on 32GB CF card and
2) just copied over directories as-is from microdrive to CF card
Is there going to be any issue when I yank out microdrive and put in CF card in it's place?
In other words is there any other things I need to do in order to make sure that CF will be recognized and then boot up Debian after CF card is installed?
Thanks!
-
you will find that a file copy using tar, rsync or cpio will be different.. say, for example, a directory had loads of files in it and the directory occupied lots of blocks, but then most of the files were deleted, the directory's usage wouldn't diminish. back to the original size.
when you mkfs, you can specify how much of the disk should be reserved - the default IIRC is 10% but on large disks that's often more than necesssary. On a 100GB+ disk I'll drop it to 5%, on a 750GB+ I'll use just 3. DONT set to 0 because it'll cause fragmentation as space runs out. You can change the % with tune2fs.
--edit--
if you're moving debian from one part'n to another you may need to flash a different kernel, as the location of the root part'n is wired in at build time. See the multi-boot thread on debian for details of the different kernels to be used for different root locations.
-
you will find that a file copy using tar, rsync or cpio will be different.. say, for example, a directory had loads of files in it and the directory occupied lots of blocks, but then most of the files were deleted, the directory's usage wouldn't diminish. back to the original size.
when you mkfs, you can specify how much of the disk should be reserved - the default IIRC is 10% but on large disks that's often more than necesssary. On a 100GB+ disk I'll drop it to 5%, on a 750GB+ I'll use just 3. DONT set to 0 because it'll cause fragmentation as space runs out. You can change the % with tune2fs.
--edit--
if you're moving debian from one part'n to another you may need to flash a different kernel, as the location of the root part'n is wired in at build time. See the multi-boot thread on debian for details of the different kernels to be used for different root locations.
Thanks for the explanations, speculatrix!
So, I guess that I can
1) swap out internal drive, then
2) reflash kernel from SD card, then
3) let 3200 boot up to new CF card....
and hope this Z boots up properly....
-
Does it make sense replacing the internal microdrive with a bigger compact-flashcard? So, if you use a part of your cf as swap-image in Debian it will probably have a lot of writing cycles on it. I was told, a cf-card has not as many overwriting cycles (about 200.000 or so) as a microdrive . Anyway, writing a cf-card is much slower than reading it - and might be slower than reading a microdrive.
Pro: Bigger, less power consumption
Contra: Might be slower, shorter life, when used as swap
agree?
-
Does it make sense replacing the internal microdrive with a bigger compact-flashcard? So, if you use a part of your cf as swap-image in Debian it will probably have a lot of writing cycles on it. I was told, a cf-card has not as many overwriting cycles (about 200.000 or so) as a microdrive . Anyway, writing a cf-card is much slower than reading it - and might be slower than reading a microdrive.
Pro: Bigger, less power consumption
Contra: Might be slower, shorter life, when used as swap
agree?
Good points, maemorandum....
However, please see earlier posts in this thread, like ZDevil's post #7 that has link to webpage on this subject, it appears that it might take 25 years for the effects that you refer to show up...
-
so, archimark, have you done it yet? we eager readers are desperate to know!
-
so, archimark, have you done it yet? we eager readers are desperate to know!
Not yet...
Per my other thread on XFCE4, stylus stopped working, no response to screen taps, want to fix that first then, do swap out.....so that I know I have fully working setup when I swap...
So, if I can fix this today, then I'll swap tonight or if not, tomorrow....will report back after I do it...
Thanks.