Help - Search - Members - Calendar
Full Version: help with SD read/write speed
OESF Portables Forum > General Forums > General Discussion
chyang
hi, Is there anyone having tested the read/write speed of SD? I tested with a 10MB/s high speed Panasonic SD, but very disappointed to see the result is just about 200KB/s or so. Is it normal?
Thanks.
ScottYelich
I'd be happy to get 200KB/sec ... and stable SD.

I was getting 34 to 75KB/sec... and my SD would *still* go offline.

Scott
Anonymous
What are you using to test the I/O? On what devices?
ScottYelich
copying a large file... to lexar 512 mb SD ... with async on and off...

Scott
ps: I can't write much to the sd before it goes offline... can't untar... can't copy mp3s.... nothing.
derekp
Mounting the SD card async has no noticable effect using the standard Sharp kernel, because apparently Sharp has put in additional code in the filesystem drivers that force sync i/o on even if you mount with -o async. When you re-compile your kernel, you can turn off the "CONFIG_FS_SYNC" option, and then mounting your SD or CF card async gives you at least a 10-1 improvement in write speed. You just need to make sure that you don't pull it out without unmounting it first.
I'll go ahead an upload my kernel with this option turned off (it's not in the standard Linux kernel, only in Sharp's version). Been running it for a month or two, without any problems.

Something else you can try in the meantime is to mount your SD card through the loopback driver, if you have a lot of data to copy to it... i.e.:
umount /dev/mmcda1
mount -oloop,noatime,async /dev/mmcda1 /mnt/card
Anonymous
I get 198.0KB/s copying to the internal storage from 256MB simpletech SD card, and 302.8KB/s copying from internal storage to simpletech SD card. I get 349KB/s copying from simpletech SD to sandisk 256MB cf, and 312KB/s copying from CF to SD. The numbers are from a localhost sftp 70MB transfer on an 860.
chyang
I haven't met ScottYelich's problem with SD, but I have met the un-stable mmc using. The MMC card's read/write is so slow and I got the directory or files all in mess when I resume my Zaurus from suspend. :-(
ScottYelich
QUOTE
I haven't met ScottYelich's problem with SD, but I have met the un-stable mmc using. The MMC card's read/write is so slow and I got the directory or files all in mess when I resume my Zaurus from suspend. :-(


I recently was going to fly for 5 hrs ... so I planned to load up my 512MB SD with mp3s. Needless to say, I had to "rsync" in the mp3s. It took 4 hours for 400+MB of MP3s.

ie: rsync, crash, eject, insert, umount, fsck, mount, repeat

To me, SD is simply not an option. It's unusable. I'm mostly using mine as a read only source for things like mp3, etc... it's just a PITA to get the data there. Ya, I have a card reader... for pc. I can hook it up to my work w2k box, but will w2k support ext2/ext3 filesystems?

Scott
derekp
ScottYelich, if you want a quicker way to copy stuff to the SD, try the trick I mentioned above using a loopback mount. This helps if you are writing a bunch of files, as it filters out multiple writes to the same location (i.e., the inode table). This will reduce the write time in your example from 4 hours to about 15 minutes. Slightly faster results will com from using a kernel compiled with the CONFIG_FS_SYNC flag turned off (see the other thread where I posted my custom kernel).
Anonymous
ScottYelich, there are drivers for w2k that can read and wirte to an ext2 filesystem. Download one and found out that it doesint suppert ext3 though. JP
Anonymous
You may want to try the driver hiding in "Downloads" section. It may help with some of the problems mentioned (such as SD read speed, lost card on resume). Please, note that it will only run on SL5500 (SL5000) with 2.4.18(19?) kernels.
maslovsky
QUOTE
I get 198.0KB/s copying to the internal storage from 256MB simpletech SD card, and 302.8KB/s copying from internal storage to simpletech SD card.  I get 349KB/s copying from simpletech SD to sandisk 256MB cf, and 312KB/s copying from CF to SD.   The numbers are from a localhost sftp 70MB transfer on an 860.


With my own kernel built with CONFIG_FS_SYNC off, I more than 1 Mb/s transfer rate to SD/Cf cards.

Internal storage is till between 200-300 Kb/s though. I don't know if that's because NAND flash memory is so slow or there are still some places in the kernel left that cause internal storage to be monted aith sync option.
ciro
sa1100 models & pxa modles do have pretty much different hardware. We should expect different performance. Also CONFIG_FS_SYNC is not the only thing which keeps filesystem cache from working. As for 5500 Sharp roms, there's a daemon called "shsync" which calls sync() every 5 seconds. In addition, filesystem caches get invalidated before suspend (did you see the "invalidate: busy buffer message"?). I don't know any good reason for that.
chyang
QUOTE
QUOTE
I get 198.0KB/s copying to the internal storage from 256MB simpletech SD card, and 302.8KB/s copying from internal storage to simpletech SD card.  I get 349KB/s copying from simpletech SD to sandisk 256MB cf, and 312KB/s copying from CF to SD.   The numbers are from a localhost sftp 70MB transfer on an 860.


With my own kernel built with CONFIG_FS_SYNC off, I more than 1 Mb/s transfer rate to SD/Cf cards.

Internal storage is till between 200-300 Kb/s though. I don't know if that's because NAND flash memory is so slow or there are still some places in the kernel left that cause internal storage to be monted aith sync option.

Is there something wrong? The internal storage should be based on SDRAM for SL5500.
derekp
The shsync daemon is fine, even with it running you will still filter out most of the redundant writes (i.e., when you are writing a bunch of files, the inode table gets written / rewritten multiple times. With async io, only the last update to a particular sector will actually be written out to the device). Also, since sync runs in the background, you are still getting buffered writes even when it is kicked off.
As for invalidating the buffer on suspend... The kernel normally will mark all cached data as no longer valid when you suspend, in case someone swaps out a storage device while the system is suspended. If it didn't invalidate the buffers, then you would get filesystem corruption if you suspended, swaped your CF /SD card, then resumed and wrote out something. The error message itself (busy buffer) just means that a program was reading from the device at the time the kernel tried to invalidate the buffers. A search on linux-kernel mailing list shows that this message is usually harmless.
maslovsky
QUOTE
Is there something wrong? The internal storage should be based on SDRAM for SL5500.


I'm taling about C750, which has internal storage in flash.
Anonymous
Yes... but... Invalidating the buffers in that way won't prevent the filesystem corruption, I guess. Kernel should know that the media is changed, and shouldn't write it. A single sync before the suspend may be enough. Swapping mounted media won't do anything good anyway. And the third thing, you get all caches reloaded on resume. You need more time to resume and loose some battery life for extra flash reads.
ciro
Oops, I wasn't logged in. Previous post is mine.
chyang
What is this module's usage?
Does it solve the problem a little?
Thanks.

http://www.zaurususergroup.com/modules.php...download&sid=52
MMC/SD driver
Description: MMC/SD driver for SL5500 replacement for the original driver - sharp_mmcsd_m.o

Filesize: 16.82 Kb
Added on: 29-Jan-2004
Downloads: 114
Category: patches / 5500
ScottYelich
thanks derekp ... I'll give that a try! (I did take note of it...)

Scott
chyang
Thanks.
I tried the module and it improved the right speed about 3 times. :-)
Original module in 3.13 ,copy a 4358462 bytes file from SDRAM to sd, the time is about 20.945s, so the write speed is about 208KB/s.
While with the module mentioned above, the time decreases to 6.393s, so the write speed is about 682KB/s. IT's great!!
:-)
klrussell
Sorry for what is probably a ridiculously basic question, but how do I install the new module on my 5500. Do I have to recompile? Thanks for any help.
mike_r
klrussell,

gunzip the file (so it looks like mmcsd.o). Copy it to /lib/modules/2.4.18-rmk7-pxa3-embedix/kernel/drivers/block. (The 2.4.18 part may be a bit different, but you get the idea.) Remove the link to sharp_mmcsd_m.o. Make a new link:

ln -s mmcsd.o sharp_mmcsd_m.o

Reboot. I'm sure there are better ways to do this, but it worked for me.

As a test, I copied a 23 MB backup file from CF to SD. Straight 3.13 ROM, I got 150 kB/sec. Flashing in the async kernel got me to 250 kB/sec. Installing this driver got me up to 400 kB/sec (2.5 min down to 1 min). Not bad.

Mike
Anonymous
QUOTE
Something else you can try in the meantime is to mount your SD card through the loopback driver, if you have a lot of data to copy to it... i.e.:
umount /dev/mmcda1
mount -oloop,noatime,async /dev/mmcda1 /mnt/card


I am using the Cacko ROM for 5500 with homes on SD. Is there a way to use loopback on /home and /tmp to acheive faster write speeds?
From reading posts here it appears that mounting a card in asynchronous IO mode is buggy and can cause errors? It would be a bad idea to try mounting my SD card in asynch mode, then?

Thanks in advance
Anonymous
Well... so it's bad, there's no improvement for 5600 users.
derekp
QUOTE
I am using the Cacko ROM for 5500 with homes on SD. Is there a way to use loopback on /home and /tmp to acheive faster write speeds?
From reading posts here it appears that mounting a card in asynchronous IO mode is buggy and can cause errors? It would be a bad idea to try mounting my SD card in asynch mode, then?

Thanks in advance


Actually, I haven't heard any complants yet about using the async kernel. If you saw one, let me know so I can investigate. On the other hand, I have had some issues with suspend lockups when using /home or root on SD through the loopback driver during my tests, the way I got around it initially is to make sure that the /var/lock directy was pointing to the ramdisk (see the latest prep-home-sd_test5 script I posted for the tkc-2.0-alpha3 rom). But, using the async kernel removes this need.
The async kernel should be a good fit for the Cacko rom with it's /home on SD. Otherwise, you'd have to rebuild your rom and change the mount line where it mounts /home to add the -o loop option to it. I haven't tried that particular rom yet, so you'd be adventering into unknown teritory.

The only issue that I can see poping up with the async kernel (or using the loopback mount for that matter) is if you pop out the SD card after writing to it without ejecting it, you could have data loss. But otherwise you should be fine.

Try it and see, and report back the results... what's the worse that could happen? (famous last words) :-)
rjd_z
I installed the new module, and the time to copy an 18 MB file from the CF to SD improved significantly. Throughput increased from 184 KB/sec to 408 KB/sec.

Nice job.
albertr
May I ask the kind soul who anonymously posted the new MMC/SD module for 5000d/5500 - what are the changes/improvement over the old one from Sharp ROM 3.x days? I see that version string didn't change (still says "sharp_mmcsd 0.21 27 Sep 2002"), but some functionality was definitely changed. Does it mean that Sharp Japan is still burning money working on improvements for old 5x00 series, or mmc/sd code was given out to third party? Please shed some light if you can... and thank you for releasing it!
Thanks,
-albertr
Mickeyl
I am also interested to know where that module originally comes from and how it has been created. As you may know (or not) the SD module is our #1 problem for delivering newer kernels to the Zaurus family.
Anonymous
I don't know if there any legal issues exist, so I've posted the module anonimously. If you tell me that everything's ok, I can provide the source.
New module is 90% Sharp's code which was disassembled and re-assembled having some functions replaced.
Now I have approx. 50% code of the dirver converted to C.
The actual changes are:
1) Higher SPI bus freq. (I'have no idea why original driver doesn't default to higher speed, although it does provide ioctl interface for changing it via /dev/sd_slotstat).
2) Different way to prefetch blocks. Original driver guarantees every eighth block to be wasted for ext2.
3) Better checksum code.
4) Original driver has hardcoded delay to wait for the card after reset. New driver attempts to detect the card multiple times increasing the delay.
albertr
ha, I knew it was not Sharp wink.gif Very nice job, indeed. Thank you for sharing this information. I'm not sure about legal implications, I doubt Sharp would care, but SD association well might. Although, the code probably doesn't contain any top secret encryption or anything, you never know what those fat lawyers have in mind.

I'd say it won't surprise me if the code would be leaked eventually somewhere outside of US. Regardless, please keep up good work, your changes are definitely in the right direction and much appreciated.
-albertr
abcdef
Sounds very good. Can I try it on 5600? If not, can you compile one for 5600?
chyang
Cool and Great!! Thank you very much for your cool module!! laugh.gif
QUOTE
I don't know if there any legal issues exist, so I've posted the module anonimously. If you tell me that everything's ok, I can provide the source.
New module is 90% Sharp's code which was disassembled and re-assembled having some functions replaced.
Now I have approx. 50% code of the dirver converted to C.
The actual changes are:
1) Higher SPI bus freq. (I'have no idea why original driver doesn't default to higher speed, although it does provide ioctl interface for changing it via /dev/sd_slotstat).
2) Different way to prefetch blocks. Original driver guarantees every eighth block to be wasted for ext2.
3) Better checksum code.
4) Original driver has hardcoded delay to wait for the card after reset. New driver attempts to detect the card multiple times increasing the delay.
agosine
Just a question on the installation of this patch. If I rename the original file to .old and rename this to match the sharp name (sharp_mmcsd_m.o) that should work fine?
chyang
yes, remember to reboot after you rename the module with the sharp_mmcsd_m.o
panyo
QUOTE
klrussell,

gunzip the file (so it looks like mmcsd.o). Copy it to /lib/modules/2.4.18-rmk7-pxa3-embedix/kernel/drivers/block. (The 2.4.18 part may be a bit different, but you get the idea.) Remove the link to sharp_mmcsd_m.o. Make a new link:

ln -s mmcsd.o sharp_mmcsd_m.o

Reboot. I'm sure there are better ways to do this, but it worked for me.

As a test, I copied a 23 MB backup file from CF to SD. Straight 3.13 ROM, I got 150 kB/sec. Flashing in the async kernel got me to 250 kB/sec. Installing this driver got me up to 400 kB/sec (2.5 min down to 1 min). Not bad.

Mike


I tried this, including changing the read premission on mmcsd.o with
CODE
chmod +r mmcsd.o


but when I reboot, my SD card does not appear.
I am running OZ 3.3.5
Also, is there any way to take advantage of derekp's kernel in OZ? I tried substituting it for zImage in the flashing process I used for OZ, but I just got a screen that said "SHARP" and nothing else.
tom_servo
QUOTE
Actually, I haven't heard any complants yet about using the async kernel.  If you saw one, let me know so I can investigate.  

I have had trouble with the async kernel. It doesn't seem to be the async part that is causing trouble though. Ever since loading that kernel, my zaurus must be rebooted between uses of the the USB network interface to the computer. If I put the Zaurus in the cradle, in a few seconds it will start to respond to pings from the computer. If I take it out of the cradle and put it back in, it does not respond to ping, even if it was suspended and resumed while out of the cradle. When re-inserted the computer ifconfig shows the usb0, but ifconfig on the zaurus only shows the “lo”. I have to reboot to be able to connect to the computer again.

I re-flashed, and tried again as soon as the initial boot was finished, and had the same problem. I don't know if it is a problem with the kernel, my SL-5500, or incomparability with the tkcrom 1.0. I tried flashing the original tkcrom kernel, and the problem did go away, but the SD became unbearably slow again.

I would rather use the async kernel (even with this problem) because it makes SD write more then 10x faster, but it would be perfect if it didn't have this little problem.


Alain
Mahoro
After reading 3 pages of msg...I have some question...

How can I Patch that MMC/SD driver for SL5500 replacement
I am an ULTRA n00b on Zaurus, just got my 5500 from my friend.......and my friend went for a trip....sad.gif

And that kernal is for 2.4.18, so it won't work on my 2.4.6 kernal is it?
Thanks
ciro
It won't work on 2.4.6. Sorry.
jlagarde
QUOTE(ciro @ Jul 16 2004, 07:21 AM)
It won't work on 2.4.6. Sorry.

Any way for someone to compile one for that kernel (the same kind soul who made the original mod maybe)? If someone sends me the source I can make an attempt but I can't guarantee the result, as I have little experience in that domain.

Or is the issue more involved than a simple recompile?

Cheers
albertr
So the source code for thos module was never released?
-albertr
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.
Invision Power Board © 2001-2019 Invision Power Services, Inc.