Help - Search - Members - Calendar
Full Version: Making Fat Sd Cards More Robust
OESF Portables Forum > General Forums > General Discussion
daniel3000
Hi guys,

I have observed, that when you remove a FAT-formatted SD card without unmounting it cleanly, the entire FAT contents may be erased.
Sometimes only the files which have been added to the FS are gone, but sometimtes the card will appear EMPTY after such faulty handling.

Two questions:

Is there a way to make a FAT-formatted SD card more robust, so something like this won't happen? (e.g. switching off some FAT write cache, adding more FAT copies etc. etc.)?

Is there a way to rebuild the FAT (automatically, not manually) after such an incident?

Windows' scandisk won't recover the FAT, and fsck.vfat won't, too, even when using the first OR the second FAT copy.

Thank you for your ideas!

Daniel
InSearchOf
well you could change the mount option to async... that may help... instead of sync which is the default

Late
desertrat
QUOTE(InSearchOf @ Mar 26 2007, 10:20 PM)
well you could change the mount option to async... that may help... instead of sync which is the default

But isn't sync supposed to keep the filesystem (hence FAT) updated immediately (at the expense of using extra flash write cycles) whereas async will cache changes to the filesystem until the kernel(?) decides it's time to write the changes or you do sync or umount, whichever is sooner.
Meanie
QUOTE(desertrat @ Mar 27 2007, 02:57 PM)
QUOTE(InSearchOf @ Mar 26 2007, 10:20 PM)
well you could change the mount option to async... that may help... instead of sync which is the default

But isn't sync supposed to keep the filesystem (hence FAT) updated immediately (at the expense of using extra flash write cycles) whereas async will cache changes to the filesystem until the kernel(?) decides it's time to write the changes or you do sync or umount, whichever is sooner.
*



yeah, desertrat got it right.

the only way is to properly unmount your partition, and adding an extra sleep in the eject script so it actually waits an additional second or two for the filesystem to sync up completely if it is a larger SD card.

note that the two FAT tables are only applicable if you are using FAT12 or FAT16. FAT32 uses both tables since it needs the extra space to store long file names and support larger filesystems...
daniel3000
QUOTE(Meanie @ Mar 27 2007, 12:39 PM)
yeah, desertrat got it right.

the only way is to properly unmount your partition, and adding an extra sleep in the eject script so it actually waits an additional second or two for the filesystem to sync up completely if it is a larger SD card.

note that the two FAT tables are only applicable if you are using FAT12 or FAT16. FAT32 uses both tables since it needs the extra space to store long file names and support larger filesystems...
*


aha, so for a card up to 2GB a FAT16 fs would be better (i.e. safer) than a FAT32, right?

Is the only disadvantage of FAT16 the large cluster size of 32kB and the fixed root directory size?

daniel
daniel3000
QUOTE(Meanie @ Mar 27 2007, 12:39 PM)
note that the two FAT tables are only applicable if you are using FAT12 or FAT16. FAT32 uses both tables since it needs the extra space to store long file names and support larger filesystems...
*


On a German website (don#t remember where, though) I got other information.
There it was written "FAT32 uses, as opposed to FAT16, the second FAT really for error correction".
which seems to imply that the second FAT in FAT16 was simply redundant and unused...

Which information is correct?
Do you have a good source for correct information?
There is not much information about that on the internet.

thanks

daniel
InSearchOf
QUOTE(desertrat @ Mar 27 2007, 12:57 AM)
QUOTE(InSearchOf @ Mar 26 2007, 10:20 PM)
well you could change the mount option to async... that may help... instead of sync which is the default

But isn't sync supposed to keep the filesystem (hence FAT) updated immediately (at the expense of using extra flash write cycles) whereas async will cache changes to the filesystem until the kernel(?) decides it's time to write the changes or you do sync or umount, whichever is sooner.
*



But if Daniel was stating that removing a sd card on the fly instead of umount is where the issue is caused... this is the exact reason why floppy drives are mounted sync because a typical person doesnt umount mount a floppy before removing it.

Sync will allow a user to remove there sd with out the hassle of umount'in it but yes you will suffer when it comes to write counts on a "write limited" device...

Late
daniel3000
QUOTE(InSearchOf @ Mar 27 2007, 04:05 PM)
But if Daniel was stating that removing a sd card on the fly instead of umount is where the issue is caused... this is the exact reason why floppy drives are mounted sync because a typical person doesnt umount mount a floppy before removing it.

Sync will allow a user to remove there sd with out the hassle of umount'in it but yes you will suffer when it comes to write counts on a "write limited" device...

Late
*


Just found this text on Microsoft TechNet (and they should know well about their FSs):

CODE
FAT32 is more robust than FAT16. FAT32 has the ability to relocate the root directory and use the backup copy of the FAT instead of the default copy. In addition, the boot record on FAT32 drives has been expanded to include a backup of critical data structures. This means that FAT32 volumes are less susceptible to a single point of failure than FAT16 volumes.


It's found here:
http://www.microsoft.com/technet/prodtechn...d.mspx?mfr=true

Well, this is the theory and probably true in Windows.
but is it also true in Linux with the vfat fs driver?
The driver may be old and thus support FAT16 better than FAT32.

Who has experiences or even knowledge?


Regarding the sync mount:

Is sync the default?
in /etc/sdcontrol the line "FATOPTS=..." is commented out, and it even doesn't contain sync or async, so there is no parameter "sync" for the SD card mount command.
When looking at the mout table, there is no mention of "sync".

Also, as InSearchOf says, the mout man page says:

CODE
All I/O to the file system should be done synchronously. In case of media with limited number of write cycles (e.g. some flash drives) "sync" may cause life-cycle shortening


so actually from this point of view, sync is not good for flash cards.
However, I would rather be failsafe now than in the future because I expect to exchange my SD card anyway after some time.
Some even have swap on SD and say it works for months without a problem.
So lifeycle is probably not a big issue anymore.

Is there any known value in how mucht "sync" will increase the number of write processes? Probably depends heavily on the kind of usage. But typical usage, i.e. open a file, work on it, save it, will not let "sync" increase write cycles much, would it?

Maybe with a swap file on SD it would. Or with other random access type files.

What's your opinion?



daniel
daniel3000
Here is a very interesting article, discussing several aspects about mounting sync:

https://wiki.ubuntu.com/SyncMount

I think I will try it. :-)

daniel
Meanie
CODE
FAT32 is more robust than FAT16. FAT32 has the ability to relocate the root directory and use the backup copy of the FAT instead of the default copy. In addition, the boot record on FAT32 drives has been expanded to include a backup of critical data structures. This means that FAT32 volumes are less susceptible to a single point of failure than FAT16 volumes.


This is true if you are using a FAT32 enhanced driver and FAT32 enhanced fsck which should be win98 scandisk equivalent. however, the vfat support in linux and the corresponding fsck.vfat are not that advanced. vfat support has improved in that it can handle FAT32 lfn extensions, but fsck.vfat is still approximately equivalent to DOS6.0 chkdsk/scandisk. If you ever ran a DOS6.0 scandisk on a FAT32 filesystem (not a DOS 6.2 version which has been enhanced), then you will know what I mean.

as for sync/async mode, in SharpROM, I found that using sync mode on larger SD cards, ie 4GB cards, can prevent it from loosing its partition table, especially if the card is relatively full. When the card is empty, there is not much difference...
On OpenZaurus, sync mode causes the SD card to be extremely slow and almost unusable.
In pdaXrom depending on the kernel version and sd module being used, the default setting which I think is async, works fine...
speculatrix
so, maybe you need to run bochs or dosbox and fsck your fat32 flash memory card using that!
smile.gif
daniel3000
QUOTE(Meanie @ Mar 28 2007, 07:01 PM)
This is true if you are using a FAT32 enhanced driver and FAT32 enhanced fsck which should be win98 scandisk equivalent. however, the vfat support in linux and the corresponding fsck.vfat are not that advanced. vfat support has improved in that it can handle FAT32 lfn extensions, but fsck.vfat is still approximately equivalent to DOS6.0 chkdsk/scandisk. If you ever ran a DOS6.0 scandisk on a FAT32 filesystem (not a DOS 6.2 version which has been enhanced), then you will know what I mean.
*


Aha, so you mean that fsck.vfat can much better recover FAT16 than FAT32 corruption?
Would you recommend FAT16 over FAT32 on pdaXrom for an SD card which mainly holds user files?


I just did some tests with sync mounting:

1GB SD card with FAT32 sync-mounted.

Then:
touch /mnt/card/test
ls /mnt/card --> test is shown. Wait a few seconds, then:
Removed the card without cleanly unmounting it.
inserted the card
ls /mnt/card --> test is NOT shown.

touch /mnt/card/test && sync. Don't wait, but:
Removed the card without cleanly unmounting it.
inserted the card
ls /mnt/card --> test IS shown.

So - does the sync option have no effect??
Only when waiting quite a long time after the touch command (maybe half a minute) before ejecting the card preserves the new file entry.

So maybe a cronjob which executes "sync" every second would be good biggrin.gif

But at least I get no "filesystem panic" messages from the kernel now.
When doing similar tests without -o sync, this happened often.

Maybe, FAT16 with -o sync is really the safest option.

daniel
InSearchOf
QUOTE(daniel3000 @ Mar 28 2007, 01:17 PM)
QUOTE(Meanie @ Mar 28 2007, 07:01 PM)
This is true if you are using a FAT32 enhanced driver and FAT32 enhanced fsck which should be win98 scandisk equivalent. however, the vfat support in linux and the corresponding fsck.vfat are not that advanced. vfat support has improved in that it can handle FAT32 lfn extensions, but fsck.vfat is still approximately equivalent to DOS6.0 chkdsk/scandisk. If you ever ran a DOS6.0 scandisk on a FAT32 filesystem (not a DOS 6.2 version which has been enhanced), then you will know what I mean.
*


Aha, so you mean that fsck.vfat can much better recover FAT16 than FAT32 corruption?
Would you recommend FAT16 over FAT32 on pdaXrom for an SD card which mainly holds user files?


I just did some tests with sync mounting:

1GB SD card with FAT32 sync-mounted.

Then:
touch /mnt/card/test
ls /mnt/card --> test is shown. Wait a few seconds, then:
Removed the card without cleanly unmounting it.
inserted the card
ls /mnt/card --> test is NOT shown.

touch /mnt/card/test && sync. Don't wait, but:
Removed the card without cleanly unmounting it.
inserted the card
ls /mnt/card --> test IS shown.

So - does the sync option have no effect??
Only when waiting quite a long time after the touch command (maybe half a minute) before ejecting the card preserves the new file entry.

So maybe a cronjob which executes "sync" every second would be good biggrin.gif

But at least I get no "filesystem panic" messages from the kernel now.
When doing similar tests without -o sync, this happened often.

Maybe, FAT16 with -o sync is really the safest option.

daniel
*




How about add the sync option to fstab

Late
daniel3000
QUOTE(InSearchOf @ Mar 28 2007, 07:21 PM)
How about add the sync option to fstab

*


Should this make a difference?
I have added it to the /etc/sdcontrol script, and the card IS mounted with the sync option ("mount" shows (rw,sync) as the mount options).

daniel
InSearchOf
yea your right... I forgot sdcontrol mounts it... unless you umount and remount from the console... that uses fstab data...

Late
Drake01
QUOTE(Meanie @ Mar 27 2007, 05:39 AM)
the only way is to properly unmount your partition, and adding an extra sleep in the eject script so it actually waits an additional second or two for the filesystem to sync up completely if it is a larger SD card.
*

On a similar topic, what if you never remove your SD card and don't really need to umount it? Mine (ext2 formatted) only gets removed on rare occasions. I'm still concerned about the possibility of Something Bad happening, so I don't want a bunch of unwritten cache sitting there in memory.

Instead of actually ejecting / unmounting the card, wouldn't syncing the card offer a similar level of protection?
speculatrix
QUOTE(Drake01 @ Mar 28 2007, 11:42 PM)
On a similar topic, what if you never remove your SD card and don't really need to umount it?  Mine (ext2 formatted) only gets removed on rare occasions.  I'm still concerned about the possibility of Something Bad happening, so I don't want a bunch of unwritten cache sitting there in memory.

Instead of actually ejecting / unmounting the card, wouldn't syncing the card offer a similar level of protection?
*


if you almost never take your sd card out, then probably the only risks are a kernel crash or lockup, or sudden power failure so it can't shutdown without warning - the latter is more likely. Even if the Z seems totally locked, there's a good chance the kernel itself is still ok.
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.
Invision Power Board © 2001-2018 Invision Power Services, Inc.