OESF Portables Forum
General Forums => General Discussion => Topic started by: wirawan0 on June 15, 2004, 12:42:52 pm
-
[span style=\'font-size:25pt;line-height:100%\']Important corrections for the poll!!![/span]
Sorry folks, I was wrong when writing in the poll question:
* ZUG polling allows each of us to vote only once. For your vote, give the answer for the card that you use the most often.
* The question should have read: If you use SD card(s) with ext2 partition, have you ever encountered any problem (filesystem corruption, card damage, ...)? If there\'s an ZUG admin that could change the poll question, I\'d appreciate it *very much*.
---- [ original message below ] -----
Hi,
I start this new thread for a single purpose: trying to centralize the experiences of Zaurus users who employ their SD cards with ext2 filesystem. Could you share your experiences, whether good or bad? This may help others too, to judge whether it is worth trying using SD + ext2.
My experience has been a bad one. (Only 1 experience, so it\'s unfair to conclude right away.) I used to own a Lexar SD 256MB 32x which I re-format to ext2 right after I received it. After 2 months it broke down, and reformatting it would result in a completely unusable card.
I have checked around and seems like people suggest to NOT mess up with filesystem on an SD card. See e.g.
http://docs.zaurus.com/index.php?id=managi...ing_sd_cf_howto (http://docs.zaurus.com/index.php?id=managing_sd_cf_howto)
Now, MMC card seems to be okay to use for ext2. But MMC is slow, and NOT necessarily cheaper than SD anymore (note that, folks!).
So, please contribute your experience, and we\'ll see if this law applies:
SD + ext2 = corruption
Thanks,
Wirawan
-
My experience doesn\'t fit neatly into your poll. I have been running ext2 on my 512 SD card for about a year. For the most part it works fine. Once in awhile, generally after doing a lot of file copying -- ie untaring a build, the file system gets corrupted and I have to run e2fsck to fix it.
I\'m wondering if another file system would work better.
-
Hi Wirawan,
My experience is:
PNY 256Mb SD Card (bought May 2003) gives almost faultless performance as a mixed ext2 and fat16 card. Infrequently I have needed to e2fsck it, because an application has crashed and then I\'ve powered it down to reset it. Note that /mnt/card is ext2, on /dev/mmcda1 and that /mnt/card2 is fat16, on /dev/mmcda2.
Sandisk 512Mb SD Card (bought May 2004) worked okay-ish, for about a week. Then I noticed problems accessing certain files. A reboot later, and e2fsck-ing gave lots of errors. I ended up fdisk-ing and mke2fs-ing the card. Many trials and tests later and I gave up on the Sandisk card... It was sold to a digital photographer friend of mine.
OEM-branded MMC 128Mb card - formatted as Ext2. Installed applications on it (/mnt/card). Worked fine. Has worked fine for 10 months.
By the way, the polling mechanism only let me cast one vote, despite the fact I have 3 cards (across 2 Zaurii).
Regards,
Edo.
-
Never had a problem of any kind. All but 3 apps are installed to SD 256 Lexar.
-
edo,
Ouch. I didn\'t realize the limitation on the polling mechanism. Anyway, thanks for your input. Do you guys use `noatime\' option to \"conserve\" the SD lifetime?
Note for others: if you have good and OK experiences, please jot down yours, too. I hope this would be a fair sampling on the experiences with SD/ext2 combination.
As for other filesystem, I would put my bet on jffs2. It\'s supported on (the stock) Sharp Kernal anyway. But the toolset to make the filesystem isn\'t there. :-(
-
My SD card is working in ext2 since I\'ve bouht it (2 month ago) without any major problem ... just done an e2fsc on it and repaired a few bad sectors btw .... so I\'m now crossing fingers and hope it wont fail in a few days/weeks
-
My SD card is working in ext2 since I've bouht it (2 month ago) without any major problem ... just done an e2fsc on it and repaired a few bad sectors btw .... so I'm now crossing fingers and hope it wont fail in a few days/weeks
[div align=\"right\"][a href=\"index.php?act=findpost&pid=31480\"][{POST_SNAPBACK}][/a][/div]
Bad sectors, fsck's etc - does not sound all that reliable...
needs some comparision with VFAT - is it really better or does it just not get detected as often because of less checks in that FS?
What choices for filesystem are there at all?
Filesystems optimized for Flash's?
BTW - how to check the remaining life-time of Flashs before error? just 'badblocks' ?
Operating temperature affecting lifetime?
-
What choices for filesystem are there at all?
Filesystems optimized for Flash's?
At the end of this article http://www-106.ibm.com/developerworks/library/l-embdev.html (http://www-106.ibm.com/developerworks/library/l-embdev.html)
is a little overview. They recommend not to use ext2 but jffs2 (I am not familiar with which roms support that out of the box though). To quote some of this:
The second Extended Filesystem (Ext2fs)
Ext2fs is the de facto standard filesystem for Linux, having ousted its predecessor, the Extended File System (or Extfs). Extfs supported a maximum file size of 2 gigabytes and a maximum file name size of 255 characters -- and it did not support inodes (including data modification timestamps). Ext2fs does much better; it has these advantages:
* Ext2fs supports up to 4 terabytes of memory.
* Ext2fs filenames can be up to 1012 characters in length.
* The administrator can choose the logical block size when creating the filesystem (typical sizes are 1024, 2048, and 4096 bytes).
* Ext2fs implements fast symbolic links: no data blocks need to be allocated for this purpose, and the target name is directly stored in the inode table. This leads to an increase in performance, especially in speed.
Because of its stability, reliability, and robustness, the Ext2 filesystem is used on almost all Linux-based systems including desktops, servers, and workstations -- and even some embedded devices. However, Ext2fs has some disadvantages when it comes to embedded devices:
* Ext2fs is designed for block devices like IDE devices, where the logical block size will be on the order of 512 bytes, 1 kilobyte, and so on. This is not well suited for flash devices where sector sizes vary.
* The Ext2 filesystem does not provide good management of sector-based erase/writes. To erase a single byte in a sector in Ext2fs, a whole sector has to be copied to RAM, erased, then rewritten. Considering that flash devices have a limited erase lifecycle (about 100,000 erases) after which they can't be used, this is not a particularly good practice.
* Ext2fs is not crash-proof in the case of a power failure.
* The Ext2 filesystem does not support wear levelling, thereby reducing sector/flash life. (Wear levelling ensures that different areas of the address range are used for writes and/or erases in rotation to extend flash life.)
* Ext2fs does not have particularly brilliant sector management, making the designing of a block driver extremely difficult.
For these reasons, an MTD/JFFS2 combination is generally preferred over Ext2fs for the embedded environment.
---------------------------------------------------
Journaling Flash File System, version 2 (JFFS2)
The original JFFS was developed by Axis Communications of Sweden, and was improved upon by David Woodhouse at Red Hat. The second version, JFFS2, is emerging as the de facto filesystem for raw flash chips for tiny embedded devices. The JFFS2 filesystem is log-structured, meaning that it is basically a long list of nodes. Each node contains some information about the file of which it is part -- possibly the name of the file, maybe some data. JFFS2 is being increasingly favored over Ext2fs for diskless embedded devices for these advantages:
* JFFS2 performs flash erase/write/read on the sector level better than the Ext2 filesystem.
* JFFS2 provides better crash/power-down-safe protection than Ext2fs. When a little amount of data needs to be changed, the Ext2 filesystem copies the whole sector to memory (DRAM), merges the new data in memory, and writes back the whole sector. This means that for changing a single word, the read/erase/write routine has to be done for the whole (64 KB) sector -- which is highly inefficient. On the off chance that there is a power failure or other catastrophe while data is being merged in DRAM, the entire collection of data is lost, since the flash sector is erased after the data has been read to DRAM. JFFS2 appends files rather than rewriting whole sectors, and features crash/power-done-safe protection.
* Perhaps most importantly, JFFS2 was specifically created for embedded devices like flash chips, so its overall design provides better flash management.
Having been written primarily for use with flash devices, the disadvantages of using JFFS2 in an embedded environment are few:
* JFFS2 can tend to slow down a great deal when the filesystem is full or nearly full. This is because of garbage collection issues (see Resources for more information).
-------------------------------------------------------------
tmpfs
Once an embedded device becomes a fully functional unit with Linux running on top of it, lots of daemons tend to run in the background generating lots of log messages. In addition, all of the kernel logging mechanisms like syslogd, dmesg, and klogd, generate a lot of messages under the /var and /tmp directories. Since a huge amount of data is produced by these processes, it is not advisable to allow all of these writes to happen to flash. As these messages need not be persistent across reboots, the solution to this problem is to use tmpfs.
Tmpfs is a memory-based filesystem, which is mainly used for the sole purpose of reducing unnecessary flash writes to the system. Since tmpfs resides in RAM, operations to write/read/erase happen in RAM rather than in flash. Therefore, log messages go to RAM rather than to flash, and they are not preserved across reboots. Tmpfs also makes use of the disk swap space for storage, and of the virtual memory (VM) subsystem when requesting pages for storing files.
Advantages of tmpfs include:
* Dynamic filesystem size -- The filesystem size can shrink or grow depending on the number of files or directories that are copied or created or deleted. This results in optimal usage of memory.
* Speed -- As tmpfs resides in RAM, the reads and writes are almost instantaneous. Even if files are stored in swap, the I/O operations are at very high speed.
One disadvantage of tmpfs is that all data is lost when the system reboots. Therefore, important data can't be stored on tmpfs.
-
ext2fs
jffs2
tmpfs...
[
Performance slowdowns at nearly full filesystem are certainly true for all filesystems, for journalling maybe little more than without, choose your poison ;-)
/tmp as tmpfs might be no choice either with so little RAM aboard.
The little microdrive would theoretically ease things down on read/write cycles, are there practical experiences/reliability available with that drives?
(I've seen normal IDE drives dying early in the office with all kind of manufacturers last years, smart-ide always finds the failure - after it already happened.)
One could still use a very cheap throw-away USB or CF - Flash as /tmp and swap...
BTW - when a file is written to disk or flash within a sync-period of the kflushd - will it actually be written twice or just once? One could set it relatively large value to safe some writes too.
Any plans/success with JFFS2 on Z?
beyond: https://www.oesf.org/forums/index.php?showt...6&hl=jffs&st=15 (https://www.oesf.org/forums/index.php?showtopic=3986&hl=jffs&st=15)
Sounds really slow... and if the internal ROM is JFFS too - wouldnt it have to be that slow as well? Totally confused now...
-
[span style=\'font-size:25pt;line-height:100%\']Important corrections for the poll!!![/span]
Sorry folks, I was wrong when writing in the poll question:
* ZUG polling allows each of us to vote only once. For your vote, give the answer for the card that you use the most often.
* The question should have read: If you use SD card(s) with ext2 partition, have you ever encountered any problem (filesystem corruption, card damage, ...)? If there's an ZUG admin that could change the poll question, I'd appreciate it *very much*.
---- [ original message below ] -----
Hi,
I start this new thread for a single purpose: trying to centralize the experiences of Zaurus users who employ their SD cards with ext2 filesystem. Could you share your experiences, whether good or bad? This may help others too, to judge whether it is worth trying using SD + ext2.
My experience has been a bad one. (Only 1 experience, so it's unfair to conclude right away.) I used to own a Lexar SD 256MB 32x which I re-format to ext2 right after I received it. After 2 months it broke down, and reformatting it would result in a completely unusable card.
I have checked around and seems like people suggest to NOT mess up with filesystem on an SD card. See e.g.
http://docs.zaurus.com/index.php?id=managi...ing_sd_cf_howto (http://docs.zaurus.com/index.php?id=managing_sd_cf_howto)
Now, MMC card seems to be okay to use for ext2. But MMC is slow, and NOT necessarily cheaper than SD anymore (note that, folks!).
So, please contribute your experience, and we'll see if this law applies:
SD + ext2 = corruption
Thanks,
Wirawan
[div align=\"right\"][a href=\"index.php?act=findpost&pid=31445\"][{POST_SNAPBACK}][/a][/div]
My experience with the Sandisk 512 SD is that it does not handle ext2 very well, it appears to format fine, it looks fine - it just doesn't work, even after numerous reformats. I put it back to FAT16 and it functions OK, I formatted my old LEXAR 128 SD to ext2 and it works like a charm. One thing I did notice was that ext2 handles application installs fine, but stalls out on simple file transfers - is this normal?
-
My experience with the Sandisk 512 SD is that it does not handle ext2 very well, it appears to format fine, it looks fine - it just doesn't work, even after numerous reformats. I put it back to FAT16 and it functions OK, I formatted my old LEXAR 128 SD to ext2 and it works like a charm. One thing I did notice was that ext2 handles application installs fine, but stalls out on simple file transfers - is this normal?
[div align=\"right\"][a href=\"index.php?act=findpost&pid=85623\"][{POST_SNAPBACK}][/a][/div]
Still wonder why that is - it should be independend what filesystem writes a block at the disk so far its contents are stored correctly.
Does the vfat simply not check as much as ext2 or what happens inside the chip?
Idea: can it be, that ext2 relies on ordered write inside the chip, whereas vfat doesn't and the chip doing some write buffering?
-
At the end of this article http://www-106.ibm.com/developerworks/library/l-embdev.html (http://www-106.ibm.com/developerworks/library/l-embdev.html)
is a little overview. They recommend not to use ext2 but jffs2 (I am not familiar with which roms support that out of the box though).
[div align=\"right\"][a href=\"index.php?act=findpost&pid=85577\"][{POST_SNAPBACK}][/a][/div]
Actually, JFFS2 is not appropriate for SD and CF cards. JFFS2 is designed for raw memory devices where the filesystem can control where data is physically stored in the memory chips. SD and CF cards do not give the host any control over where data is physically stored which defeats most of the wear leveling features in JFFS2.
-
I have 2 partitions on my card, one vfat & one ex2. I've used the ext2 partition to install programs, write scripts, house swapfiles, etc. To date I haven't had any problems.
And yes I use the noatime flag.
-
Every one of my SD cards has failed me... I consider SD on Zaurus as 100% unsable.
This is frustrating when I have to use the CF for wifi.
Scott
-
Every one of my SD cards has failed me... I consider SD on Zaurus as 100% unsable.
This is frustrating when I have to use the CF for wifi.
Scott
[div align=\"right\"][a href=\"index.php?act=findpost&pid=85675\"][{POST_SNAPBACK}][/a][/div]
Question:
Do those SD-cards work flawlessly in other devices?
Maybe your SD-controller is broken...
-
Some more thoughts on filesystems details:
What block size would be useful for certain type/size of Flashs?
Does it make sense to map them at the same size (as suggested in JFFS2) ?
If i read it correct, JFFS2 does automatically re-locate badblocks when detecting a write error, how do ext2/3 or VFAT perform? So far ein know, they only handle badblocks when specifically told so, which might account for a number of the observed problems here.
On the other side - SD cards do have wear-levelling, do they also have automatically relocation transparent to Z? I searched the net, but could not find any reliable note on this, just that "SD have ECC ... data are corrected before written back to host".
That does not sound like relocation, just ECC.
Reason is:
Especially for journalling a log wandering through a badblock might be very nasty.
Now JFFS2 (3) would be perfect to adress this problem, but when used for SD - does it still make sense?
For all that different things to consider - a selection matrix would be nice:
(other things to consider is interoperability of course)
filesystem/card | vfat | ext2 | ext3 | jffs2
CF
MMC
SD
microdrive
(int. Flash?)
-
On the other side - SD cards do have wear-levelling, do they also have automatically relocation transparent to Z? I searched the net, but could not find any reliable note on this, just that "SD have ECC ... data are corrected before written back to host".
That does not sound like relocation, just ECC.
[div align=\"right\"][{POST_SNAPBACK}][/a][/div] (http://index.php?act=findpost&pid=85754\")
See [a href=\"http://www.sandisk.com/pdf/oem/WPaperWearLevelv1.0.pdf]this whitepaper[/url]
SD and CF both have controllers that do a significant amount of abstraction between the logical operation requested by the host and the physical operations performed on the actual flash. On top of that, each vendor's controller behaves differently as they all make different trade-offs between cost, speed, and reliability.
-
On the other side - SD cards do have wear-levelling, do they also have automatically relocation transparent to Z? I searched the net, but could not find any reliable note on this, just that "SD have ECC ... data are corrected before written back to host".
That does not sound like relocation, just ECC.
[div align=\"right\"][{POST_SNAPBACK}][/a][/div] (http://index.php?act=findpost&pid=85754\")
See [a href=\"http://www.sandisk.com/pdf/oem/WPaperWearLevelv1.0.pdf]this whitepaper[/url]
SD and CF both have controllers that do a significant amount of abstraction between the logical operation requested by the host and the physical operations performed on the actual flash. On top of that, each vendor's controller behaves differently as they all make different trade-offs between cost, speed, and reliability.
[div align=\"right\"][a href=\"index.php?act=findpost&pid=85772\"][{POST_SNAPBACK}][/a][/div]
Thanks for that informations.
I read it that way, that the card uses some internal storage to remember the sector-mappings between logical an physical location.
So indeed JFFS2/3 is trying to do something not really required - additional wear-levelling (simply wasting cpu, though compression of files might speed things up a bit).
When the internal mapping-logic of the flash is flawed any filesystem get screwed up totally.
So what filesystem to use? vfat no links, loopbacked-ext2 on vfat a waste of cpu (and putting a dangerous additional level of potential failure in), native-ext2 at least sometimes failing, ext3 certainly too, confusing...
-
So what filesystem to use? vfat no links, loopbacked-ext2 on vfat a waste of cpu (and putting a dangerous additional level of potential failure in), native-ext2 at least sometimes failing, ext3 certainly too, confusing...
[div align=\"right\"][a href=\"index.php?act=findpost&pid=85808\"][{POST_SNAPBACK}][/a][/div]
In my experience, the choice of SD card brand (an even model) is far more important than filesystem. The problems most folks experience with SD cards have nothing to do with flash wear. It's a well established fact that compatibility between SD cards and the interfaces on mobile devices leaves a lot to be desired. Ask owners of older Dell PocketPCs about SD compatibility and you're likely to get an earful
I've found that SD cards that work reliably in the Zaurus do so regardless of which filesystem you choose. Cards that don't will eventually have problems regardless of the selected filesystem. Now, the more complex the filesystem, the sooner an unreliable SD will exhibit problems. That's the reason some folks get by seemingly ok with FAT but experience failures with ext2 (FAT FS corruption can go by unnoticed by the OS for quite some time).
Note that CF cards don't seem to suffer the same compatibility issues (probably because the parallel bus oriented CF interface is far easier to implement than the tight serial timing required by SD).
-
I have a sandisk 512 sd, formatted ext 2 and have used it for a home on Cacko, and now on tKc 1.0 (with the Cacko kernel upgrade). I have also used it as a place to to install apps on OPIE, Hybrid and tKc. It has never given me any problems other than some apps (very few) loading slowly. I never had the infamous "sandisk problem."