SD/CF speed tests using CONFIG_FS_SYNC (long post)

[zbones]

First off, many thanks to Derekp for discovering this setting, my tests reveal (as did Dereks) that this significantly improves sd/cf read/write speeds at no apparent cost to stability.

In fact my Sandisk card which normally screws up after a single suspend when formatted to ext2 now works fine with ext2.  

So removing this setting not only provides blinding speed improvements, but is more stable.

I would love to know the reasons for the existance of this setting, as removing it gives a truly massive improvement, and so far, no downside.

TEST 1
======

The test file was a 46,409,728 byte tar file ( a backup from my c760 ).
I chose large files to try to eliminate buffer cache form the copy tests.
The first test was copy the file from my 512mb kingston cf card to my 256mb sd card (both formatted as fat16).
The second test was to extract the copied tar file from the sd card, to the sd card.
tests done on 5500 and c760 models.

The roms were all stock install, other than specified differences.

Code: [Select]

               copy from       tar -x

5500            cf to sd        sd->sd.    

=======================================

OZ 3.5          3m27s           21m15s    

                218913 b/sec    

                213 kb/sec



tkcrom 1.0      3m27s           30m28s

standard        218913 b/sec  

kernel          213 kb/sec



tkcrom 1.0      3m11s           5m10s  

CONFIG_FS_SYNC  242982 b/sec

disabled        237 kb/sec

(derekp\'s kernel)





                copy from       tar -x    delete

c760            cf to sd        sd->sd    files    

================================================

cacko QT        1m37s           13m46s    8m 55s    

CONFIG_FS_SYNC  4784512 b/sec    

enabled         467 kb/sec  

Cakco QT Jan.



Cacko QT        1m21s           2m8s      9sec    

CONFIG_FS_SYNC  572959 b/sec

disabled        559 kb/sec

Cacko QT Mar 5



As above but    1m13s           1m59s     9sec

overclocked,    635749 b/sec

cpu 471mhz      620 kb/sec

bus 235mhz

Observations,:-
1: The c760 is much faster for sd read/write. due to a faster bus speed, (already a known fact from previous tests)
2: Overclocking the c760 resulted in a further but small speed increase, again due to faster bus speed.
3: The overall write speed is not really much faster. (expected result)
4: Extracting tar files is much quicker due to removing redundant writes. (expected result)
5: Deleting lots of files shows the best improvements (535 seconds down to 9seconds), as it is only accessing the fat table (again this was expected).
6: One reason why Oz, performed better in the extract was the tar file had loads of symbolic links, oz did not display any error messages for these, but could not write them out as it was fat16.
The other roms did display error messages and had to spend cpu time doing this.
on the c760 test, I used 2]/dev/null for both extracts so screen display of error messages was removed.
7: Sadly I didn\'t test the delete times on the 5500, but expect similar improvement.


TEST 2
======
As my backup had several large files on it 7-14mb I decided to test an archive of my news spool which contains lots of small text files.  I also tested the cf card in a similar way to the sd test.

test2 news.tar file contains 7063 small text files size is 25,461,760 bytes
Test done on c760 only

Code: [Select]

               copy from      tar -x    delete         tar -x   delete

c760            cf to sd       sd->sd    files          cf->c    files

==============================================================================

cacko QT        43sec          45m0s     8m45s          15m15s   7m45s

CONFIG_FS_SYNC  592133 b/sec      

enabled         578 kb/sec





Cacko QT        39sec          5m46s     32s            4m14s    19sec

CONFIG_FS_SYNC  652865 b/sec

disabled        637 kb/sec



As above but    32sec          4m52s     27s            3m38s    16sec

overclocked,    795680 b/sec

cpu 471mhz      777 kb/sec

bus 235mhz

Observations:-
1: CF is way faster than sd regardless of settings (already proven fact).
2: Speed improvement between sd and cf is not as much when disabling CONFIG_FS_SYNC.
3: read/write speed is dependant on the bus speed, buying 32x sd cards is a waste of money as the bus on the Zaurus is not up to the job and offers no speed improvement over a standard speed card (again proved by me in other speed tests).

[maslovsky]

I think it still depends on card manufacture and file system. My rough tests shown speed incrase from 200-300 Kbs up to 1 Mbs when writing to my ext2 formatted SD card.

[zbones]

What were you writing? and from where?

I\'d like to try a similar test.

Peter

[maslovsky]

As I remember I copied 25 Mb file between CF, SD andl internal flash. BTW, the speed of internal flash realy sucks...

[zbones]

Some of it may well have been cached if your coping the same file about, but this option only adds another level of cache, which is where the speed increase may come from.

My sd and cf cards where freshly formatted inbetween tests to try and lose some of the effect from buffer cache.

I have had over 2mb transfer on some smallish files when trying to utilise any caching availible, by re-running the same test without flushing the cache.  My card should be good for 10mb/sec as tested on my usb2 reader, but you wil not get this from the zaurus.

Either way, this patch rocks for extracting tar files, deleting, compiling apps or even offline news reading. Anything with frequent read/writes to the same area fly\'s now.  Many thanks for including it in the cacko QT rom
and to derekp for discovering it.

Internal flash is faster on the 5500, by a long way.  but with this patch the sd/cf speed can outperform internal flash on the cxx0 series.

Peter

[derekp]

One thing to note, is that the internal rom (when used as a jffs2 filesystem) isn\'t a block device.  This is why you don\'t see a speed increase, as that filesystem normally isn\'t buffered anyways.
As far as why Sharp included this in the kernel, I\'ve been trying to research it but could only come up with a japanese weblog that mentions this config option.  I couldn\'t draw much of a conclusion from the babelfish translation.  The only thing I can think of, is that since the Zaurus is intended to be used as a consumer device, then it is known that most people will just pull out their SD & CF cards after writing to them, without hitting the eject icon.  Therefore, Sharp wanted to make double-sure that there would be no filesystem corruption.  And, for most consumer uses, you don\'t do a lot of writing at one time to the cards, so most people wouldn\'t notice the speed problem unless they are loading up a bunch of mp3 files on a card.   Now, why they chose this route, instead of calling mount with the \"-o sync\" option in userland, is beyond me.  Kind of like flying around the world to go next door.  Unless if the kernel patch does a better job of writing in sync mode than the -o sync option does.

[ScottYelich]

it took me 4+ hours to copy in 400mb to my 512mb lexar sd.... had to use rsync.

I have this option on in the latest qt from from cacko on my 700, I assume...
I\'ll test my card.

I hope it is part of the new pdaXrom soon, as qt isn\'t where it\'s at.

Scott

[abcdef]

Somebody can  build a kernel for 5600 with this option enabled? I want to try it
on my 5600 machine. Thanks.

[zbones]

I have discovered a strange issue with the config_fs_sync kernels.

I have two 256mb SD cards. With the Standard kernels my sandisk card worked fine as fat, but became corrupt very quickly (minutes) when formatted as ext2.

That\'s when I bought the lexar card, this has been stable in both my zauri as ext2 for 8 months in my 5500 and since xmas in my cl760.

With the config_fs_sync kernels this behaviour is totally reversed  :shock:

The sandisk is stable using ext2, the lexar however will now no longer take an ext2 filesystem and survive a suspend without giving off io errors and going ro.  IT is still stable as fat fomat.

This may explain a few things.
1) The reason for this setting in the first place, to make some cards more stable, but sadly it makes others unstable (lack of testing?)

2) The increasing numbers of posters saying that the Lexar SD cards are unstable, when previously there were no complaints.

I discovered this 2 days ago, and have run many tests since just to make sure that it wasn\'t a one of issue with the lexar card.

And yes I flashed my 5500 back to a sharp kernel and retested that too :?

I am not bothered by this, as I only need one sd card to run as ext2 the other does need to be fat so it will go in my camera.  I am just glad I kept the sandisk card now.

[maslovsky]

I have a Lexar SD card formatted as ext2. On my 5500, it would get corrupted periodicaly, even without any async fs option. On my 750 the same card works like a charm, regardless of the option. So it also seems to depend on actual hardware???

[albertr]

I have a Lexar and two SanDisk cards. All formated ext2. Never had a corruption problem with either one on 5000D and 5500.
-albertr

[derekp]

There is also an alternate SD driver floating around here, you might want to give that one a try it in combination with the async kernel, and report back the results.  It appears to fix a bunch of timing issues and such.

[maslovsky]

There is also an alternate SD driver floating around here, you might want to give that one a try it in combination with the async kernel, and report back the results.  It appears to fix a bunch of timing issues and such.

yes, I have noth the new driver and the updated kernel installed on my 550. But now I use my only SD  card mostly in 750   As son as I get another one, I\'ll be able to test it...