16gb Cf For $159.99 After Rebate

[Capn_Fish]

[there are A-Data 16 Gb cards available for @160 usd at tigerdirect

I wonder how they will survive swapping?  I guess with 16 gigs the write-levelling will handle swapfiles pretty well? opinions?

any insight on the a-data cards?

Anyway, re the price--possibly the cost of manufacture is lower as a result of new tech?
[div align=\"right\"][a href=\"index.php?act=findpost&pid=162563\"][{POST_SNAPBACK}][/a][/div]

I personally wouldn't risk wearing out a $160+ card.

[Meanie]

[there are A-Data 16 Gb cards available for @160 usd at tigerdirect

I wonder how they will survive swapping?  I guess with 16 gigs the write-levelling will handle swapfiles pretty well? opinions?

any insight on the a-data cards?

Anyway, re the price--possibly the cost of manufacture is lower as a result of new tech?
[div align=\"right\"][a href=\"index.php?act=findpost&pid=162563\"][{POST_SNAPBACK}][/a][/div]

I personally wouldn't risk wearing out a $160+ card.
[div align=\"right\"][a href=\"index.php?act=findpost&pid=162565\"][{POST_SNAPBACK}][/a][/div]

if it wears out, just buy another one. replacable parts dont worry me... now the next chance i get to go to HK, and I will go crazy shopping for new hardware...

[ZDevil]

[there are A-Data 16 Gb cards available for @160 usd at tigerdirect

I wonder how they will survive swapping?  I guess with 16 gigs the write-levelling will handle swapfiles pretty well? opinions?

any insight on the a-data cards?

Anyway, re the price--possibly the cost of manufacture is lower as a result of new tech?
[div align=\"right\"][a href=\"index.php?act=findpost&pid=162563\"][{POST_SNAPBACK}][/a][/div]

I personally wouldn't risk wearing out a $160+ card.
[div align=\"right\"][a href=\"index.php?act=findpost&pid=162565\"][{POST_SNAPBACK}][/a][/div]

if it wears out, just buy another one. replacable parts dont worry me... now the next chance i get to go to HK, and I will go crazy shopping for new hardware...
[div align=\"right\"][a href=\"index.php?act=findpost&pid=162568\"][{POST_SNAPBACK}][/a][/div]

@adf: If you want more safety, use a swap partition instead (which is what I do).

@Capn_Fish: imho this just sounds more like a horror story than the reality. We seldom hear Z users here got their cards worn out fast. I expect my card will sustain for at least 1 or 2 years. And i believe these cards are not made of paper.

@Meanie: I will be in HK soon for weeks. Feel free to drop me a list if you like. I'm more than happy to do that, because I simply enjoy buying (cheap & cool) gadgets there, even if i don't buy things for myself (... shopaholics!?  )!

[adf]

I wasn't sure about how partitioning works on flash. I've experinced wayy more data corruption using a swap file on flash cards that using a partition on my microdrive...  I had thought maybe that partitioning might screw with the write-levelling in some way, say by limiting the physical address useable for the writes (which would explain the data corruption I'd seen using swapfiles--though a buggy driver might have been responsible there as well, it was a while ago.)
Anyway, I won't be doing any z surgery for a little bit--gotta take it travelling this summer.

[ZDevil]

Off topic a bit ...
In my case I seldom had data corruption in the swapfile when I was doing quite a bit native compilation under pdaXrom. Swap partition is to prevent file corruption and wearing from spreading all over the card.
My 16gb card has two partitions: one big root (1st partition, ~15GB) and one swap (2nd, the rest of the space). Yet I am not sure if this is a good setting or not.
Now using OpenBSD on 3200, I am a bit amazed that the swap space is often not used (checking with swapctl -l), unless when running big apps with huge overhead such as firefox. Perhaps it's because of the resource-friendliness of OBSD?
For microdrive or HDD, it is advised to put the swap partition as close to the beginning to the drive as possible to improve transfer speed. But flash memory doesn't involve any spinning, so where to put the swap partition may not be an issue. I am no hardware guru at all. Please correct me if i am wrong.  

[kopsis]

Off topic a bit ...
In my case I seldom had data corruption in the swapfile when I was doing quite a bit native compilation under pdaXrom. Swap partition is to prevent file corruption and wearing from spreading all over the card.

Partitioning may help, but the reason why is a bit more complicated. The real key to long CF life is don't ever get all the sectors on the card "used". Wear leveling needs unused sectors to work. With wear leveling, when you write to logical sector X, the card actually picks a physical sector marked as "unused", marks it "used" and writes your data there. Next time you write to logical sector X, the card picks a different physical sector, writes your new data there, then erases the previous one and marks it "unused". As a result, writes (actually its the erase before the write that wears out flash) are spread over the whole physical flash keeping any one sector from being erased and written over and over.

CF cards start with all logical sectors "unused". The first time you write to logical sector X it becomes "used". Even if you later delete the file that was using it, it is still "used" (there will forever more be a physical sector mapped to it). The CF card doesn't know anything about your filesystem. Everything is a logical sector -- and a previously written logical sector that is back on the filesystem's free list is still "in use" as far as the CF card is concerned.

Using partitions doesn't stop your writes from being spread over the physical card (nor would you want it to), but it does stop them from being spread over all the logical sectors. Without the partitioning, when the OS needs to allocate another sector, it just pulls one off the free list. That may be one that was previously "used" or it may be an "unused" one. If it's an "unused" one, you just reduced your wear leveling pool by one. Over time, unconstrained writes will cause the number of never written sectors to asymptotically approach 0. Partitioning keeps them constrained. Once every logical sector in the main partition has been written once, every subsequent allocation from the free list is forced to grab one of these previously used logical sectors and leave the logical sectors outside the partition untouched.

Good CF cards come with a decent percentage of "spare" sectors that are invisible and are always available to the card for wear leveling. Think of it as a "hidden" partition. But if you want to prolong the life of your card even more (or if you have doubts about how much protection the manufacturer has included), create a "reserved" partition with the number of sectors that you want to forever keep available for wear leveling. Do not create a filesystem in this "reserved" partition. Use the rest of the space however you see fit. Once you have some reserved space, it doesn't really matter if you use a swap file or a swap partition.

Note that you have to do this early in the life of the CF card. Once a card has been thrashed, it's not clear if there is a way to ever get the "used" sectors back to "unused". There is a low-level ATA "format sector" command that may do the trick, but it's impossible to tell if it really has any effect (and it's tough to find a disk formatting program these days that offers a "low-level format" option). That's also why buying a used CF card is such a gamble!

[ZDevil]

Thanks for the precise explanation. This forum is such a good read that I learn new things almost everyday.  

Now another question about using such a big card on Z: does it make a difference in read/write time when using one big partition versus several? What would be the variables here? Card speed? Flash vs. mechanical drive? Sheer partition numbers? Or something else?

[speculatrix]

Partitioning may help, but the reason why is a bit more complicated. The real key to long CF life is don't ever get all the sectors on the card "used". Wear leveling needs unused sectors to work. With wear leveling, when you ...

this seemed to make sense, but I think that the logic breaks down. Lets say you have a digital camera fitted with huge 16GB CF card, and only ever write 4GB to it before "emptying". It will only take four lots before you'll have marked every sector as used?

[kopsis]

this seemed to make sense, but I think that the logic breaks down. Lets say you have a digital camera fitted with huge 16GB CF card, and only ever write 4GB to it before "emptying". It will only take four lots before you'll have marked every sector as used?
[div align=\"right\"][a href=\"index.php?act=findpost&pid=162646\"][{POST_SNAPBACK}][/a][/div]

The confusing part is the subtle distinction between "used" from a CF perspective and "used" from a filesystem perspective. In this example, you write your 4GB (let's say it's just one file for the sake of simplicity) to a brand new card. That marks roughly 8M logical sectors as "used" in both the filesystem and the CF logic. Now you delete the file. Those sectors are still "used" as far as the CF card is concerned (it won't use them for wear-leveling) but they're "free" as far as the filesystem is concerned.

Now when you write another 4GB file, the filesystem has to pick another 8M of logical sectors to write to. It *may* pick the same ones it used before (it sees them as "free"). In that case what really happens is that a new batch of physical sectors get written and the old ones get erased, but there is no net change in the number of logical sectors that are marked "used". In a controlled example like this that's very likely to happen. In reality where there are somewhat random file operations, the filesystem probably picks some previously used sectors and some never used sectors (they all look "free" to the filesystem). The result now is a net drop in "unused" logical sectors.

Repeat the process and the next time around the filesystem may pick a different batch of 8M logical sectors. Eventually it is possible that all the logical sectors will have been marked "used" in the CF. And though they may be available for the filesystem to reuse, they're no longer available for wear leveling.

Another example may make this easier to understand. Let's say you fill up a 16 GB card. Every logical sector is marked used in the CF and the filesystem. Now you delete all the files. The filesystem sees the card as empty but the CF logic says they're all "used". Now you create a zero byte file. All that does is write some stuff to the sector where the root directory is stored. Let's say that's logical sector 42  The CF card could just erase the physical sector that is mapped to logical sector 42 and then write the new data, but if you do that a million times the physical sector could wear out.

So it grabs an unused physical sector. Now, since every logical sector is marked as used and mapped to a physical sector, the only unused physical sectors available are in the reserve pool. So it grabs a reserved sector, writes it, updates logical sector 42 to map to it, then erases the physical sector that was mapped to logical 42, and puts that erased sector in its reserved pool (all in a matter of microseconds!). Assuming that the reserved pool has 10 sectors and that sectors can handle a million writes, now we can do our zero byte file create 10 million times before we get into trouble.

Now, that example is pretty far from what someone would really do, but it illustrates the problem. On a CF card that has a small pool of unused sectors, writing the same logical sectors over and over (as happens with a swap file and a swap partition) will greatly accelerate wear on those sectors due to the reduced size of the wear-leveling pool. Keeping a large chunk of logical sectors forever unused (through partitioning) can dramatically increase the size of the wear-leveling pool and protect against the abuse of things like swapfiles.

[ShiroiKuma]

So concretely speaking, how does that work?

If  you'll have a new 16GB card, you'd create one extra primary partition? Then not mkfs it? Is that all? How big should this partition be for a card this size, to provide the additional level of security you speak about and yet not be unnecessarily big?

[Cresho]

I just grabed this infro from the wikipedia.

It talks about "wear leveling" and specifically "some solutions" to be exact.

http://en.wikipedia.org/wiki/Wear_leveling

in any case, I did'nt find an answer on wear leveling on 2 partitions on a single drive.

[adf]

and on another note entirely, I was thinking more of using something like swapd that places relatively small files in; I suppose; random locations. I'm not sure now if this would help...it seems like it might.
the other option, of course, woild be to swap to the cheaper and easier to replace (due to the external slot) SD card.  of course SD writes on the Z are sloooooow.
would something like swad be an improvement ata least as wear levellingi s concerned?

[kopsis]

If  you'll have a new 16GB card, you'd create one extra primary partition? Then not mkfs it? Is that all?

That's it.

How big should this partition be for a card this size, to provide the additional level of security you speak about and yet not be unnecessarily big?

That's where things get a lot more "speculative". It's going to depend on a number of things you may not know (expected usage patterns) and some things you definitely don't know (reserved sector pool size, wear-leveling algorithm specifics). But, a quick "back of the envelope" type calculation can probably get us in the ballpark ...

In my previous "pathological" zero byte file example, we saw that the number of cycles a sector can withstand times the number of reserved sectors gave a kind of worst case life expectancy. So let's guess at some parameters. First we need to pick a mean-time-to-failure (MTTF) -- I'm going to say 10000 hours is sufficient. That's enough for 3.5 years of using the card 8 hours a day, every day. Now let's guess that on average we get 10 sector writes per second over that period. That's a complete guess but it "feels" pretty reasonable. Last but not least, let's guess that a sector can survive 100K write cycles (that's a decent number for today's flash technology).  Note that card size doesn't really factor into the equation.

(10000 hrs * 36000 cycles/hr) / 100000 cycles/sector = 3600 sectors

A sector is normally 512 bytes so that equates to about 1.75 MB. I bet that's not as bad as you thought it would be  Now there are lots of places where my assumptions could be way off, so on a small CF card I'd multiply by a factor of 10 and round up (20 MB) and on a big card I'd multiply by 100 (175 MB).

There are still no guarantees. Sectors can fail after far less than 100K cycles, implemented wear-leveling may not work exactly like the ideal case I describe, etc. But the bottom line is that the "cost" of setting aside a few MB is pretty low, definitely can do no harm, and might help dramatically.

[wsuetholz]

Not to get off topic, but does switching from the microdrive CF-II form factor to a solid state CF-I form factor give enough room to fit a WI-FI and or Bluetooth hardware internally?  C3X00 Zaurus.  I'd still like to know how Sandisk did their combined flash/WI-FI card,  I'd like it even more to see it with 16Gig WI-FI and Bluetooth, with a detachable Antenna header so I could put it internal.

[ShiroiKuma]

I've created a slideshow of the swapping surgery. So far so good. 
[div align=\"right\"][a href=\"index.php?act=findpost&pid=162486\"][{POST_SNAPBACK}][/a][/div]

ZDevil, can you report on your experiences on using this 16gb card in the place of the internal microdrive. Any issues? Still going strong?

I swapped it too, and it got killed. So am wondering if it was just one bad specimen, or whether there's some general issue with this model of the CF card.