Hmm, could this be the problem with SD cards

[iamasmith]

I am seriously thinking of putting an ext3 partition onto my 512Mb SD card but being of the  cautious type I decided to do a little digging.

Firstly a comparison of the existing partition table between a desktop card reader and an SL-C860.

On SuSE 9.1 through Dazzle USB 2 card reader (fdisk -v reports 2.12)

The number of cylinders for this disk is set to 13895.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): p

Disk /dev/sda: 512 MB, 512229376 bytes
9 heads, 8 sectors/track, 13895 cylinders
Units = cylinders of 72 * 512 = 36864 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1               4       13896      500107+   6  FAT16

Command (m for help):

On Zaurus SL-C860 running Cacko 1.21b (fdisk -v reports v2.11g)

# fdisk /dev/mmcda

Command (m for help): p

Disk /dev/mmcda: 16 heads, 63 sectors, 992 cylinders
Units = cylinders of 1008 * 512 bytes

     Device Boot    Start       End    Blocks   Id  System
/dev/mmcda1             1       993    500107+   6  FAT16
Partition 1 does not end on cylinder boundary:
     phys=(992, 8, should be (992, 15, 63)

Command (m for help):

Observations 1

Notice that the geometry reported is quite different between the two outputs. I then decided to have a look at the changelog for fdisk from the util-linux package and found that the versioning goes from 2.11g through 2.11z before it gets to 2.12 with MANY changes to fdisk including improved geometry stuff.

Having built a new copy of fdisk this is the observation...

On Zaurus SL-C860 running Cacko 1.21b with fdisk built from util-linux-2.12a

# ./fdisk-2.12a /dev/mmcda

The number of cylinders for this disk is set to 13895.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): p

Disk /dev/mmcda: 512 MB, 512229376 bytes
9 heads, 8 sectors/track, 13895 cylinders
Units = cylinders of 72 * 512 = 36864 bytes

     Device Boot      Start         End      Blocks   Id  System
/dev/mmcda1               4       13896      500107+   6  FAT16
/dev/mmcda2               1           1           0    0  Empty
Partition 2 does not end on cylinder boundary.

Command (m for help):

Observations 2

This is the interesting thing, the new version of fdisk reports the start block for mmcda1 at block 4 and also reports a secondary partition of 1 block at block 1. Could this be storing some metadata for the card that's getting zapped if people repartition with the older fdisk

What are your thoughts people ?

By the way I'm still not completely happy with flattening it and going ext3 yet.

[iamasmith]

As a followup I tried a 128Mb SD card with the new version of fdisk.
That also had a single block at the beginning of the card as partition 2 that shows up only when going throught the mmcda driver.

And finally the output of the new version of fdisk 2.12a through my Dazzle USB2 reader (on the SuSE 9.1 desktop machine again with the 512MB SD Card back in)...

The number of cylinders for this disk is set to 13895.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): p

Disk /dev/sdc: 512 MB, 512229376 bytes
9 heads, 8 sectors/track, 13895 cylinders
Units = cylinders of 72 * 512 = 36864 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               4       13896      500107+   6  FAT16

Command (m for help): q

Interesting huh, the extra partition only shows up through the mmcda driver.

I'm inclined to think at this stage that the mmcda driver is a little lacking and shouldn't be allowing reads/writes to a reserved area of the SD card.

Could someone try this with an MMC card ?

All you need is the util-linux-2.12a from www.kernel.org and a cross-compiler platform to build.

My theory is that the MMC cards don't have that extra partition.

[iamasmith]

Hmm, actually not so sure about the new version of fdisk now, tried a CF card and it had the second partition definition too but the interesting thing is that all SD cards seem to start partition 1 at block 4 whilst the CF card started partition 1 at block 1 - partition 2 was also at block 1 with a size of 1 so I may need to take a look at the source a little more closely.

Anyone fancy trying this on an MMC card ? Please, I would like to find out what they report (particularly the starting cylinder of partition1) - note I mean with fdisk 2.12a not 2.11g as this probably reports 1 regardless.

Regards,

Andy

[iamasmith]

OK the CF stuff was a false alarm, just tried it again with the newly compiled version of fdisk and it works fine (one partition showing).

Status thus far then..

Current version of fdisk on the Cacko ROM is 2.11g and I presume that this was obtained from the fdisk package on killefiz. This is darned old.

The current version of fdisk is 2.12a and here's a summary of the changes to fdisk since 2.11g (from the util-llinux HISTORY file).

util-linux 2.12a

* fdisk: fix for kernels 2.4.15-2.4.17
* fdisk: fix when all partitions are in use

util-linux 2.12

* fdisk: sgi layout fix

util-linux 2.12pre

* fdisk: make various variables unsigned to lengthen the life of 32-bit vars
* fdisk: some sgi fixes (Phillip Kesling)
* fdisk: k=1000, K=1024
* fdisk: removed last occurrences of HDIO_REQ
* fdisk: use BLKGETSIZE64

util-linux 2.11y

* fdisk: decimal units

util-linux 2.11x

* fdisk: allow addition of a new partition when logicals all used

util-linux 2.11w

* cfdisk, fdisk: allow slightly larger disk sizes
* fdisk: Makefile: also for m68 (sun3) (Kaj-Michael Lang)
* fdisk: allow to use the last partial cylinder, change display format
* fdisk: do not ask partition number in case there is only one choice
* fdisk: new sunlabel fix

util-linux 2.11v

* fdisk: add -C, -H, -S command line options
* fdisk: allow changing type 0

util-linux 2.11u

* fdisk: fix for fdisk on empty disk (Michael D. Black)

util-linux 2.11t

* fdisk: fix device names with sundisklabels under devfs (Kaj-Michael Lang)
* fdisk: minor polishing

util-linux 2.11s

* fdisk: fixes for disks with OSF/1 label

util-linux 2.11r

* fdisk: avoid <linux/hdreg.h>
* fdisk: x86_64 patch (Mads Martin Jrgensen)

util-linux 2.11q

* fdisk: geometry improvement

util-linux 2.11o

* *fdisk: added Darwin types (Vincent Bourgonjen)

util-linux 2.11m

* fdisk: added netbsd type (Steven J. Hill)

util-linux 2.11l

* fdisk: show partitions with empty type but some other nonzero field

util-linux 2.11k

* fdisk: added fclose()

util-linux 2.11j

* fdisk: fix for AIX label (Olaf Hering)
* fdisk: fix for creating DOS label when BSD label is present

util-linux 2.11i

* fdisk: avoid superfluous warning about DOS partitions
util-linux 2.12a

* fdisk: fix for kernels 2.4.15-2.4.17
* fdisk: fix when all partitions are in use

util-linux 2.12

* fdisk: sgi layout fix

util-linux 2.12pre

* fdisk: make various variables unsigned to lengthen the life of 32-bit vars
* fdisk: some sgi fixes (Phillip Kesling)
* fdisk: k=1000, K=1024
* fdisk: removed last occurrences of HDIO_REQ
* fdisk: use BLKGETSIZE64

util-linux 2.11y

* fdisk: decimal units

util-linux 2.11x

* fdisk: allow addition of a new partition when logicals all used

util-linux 2.11w

* cfdisk, fdisk: allow slightly larger disk sizes
* fdisk: Makefile: also for m68 (sun3) (Kaj-Michael Lang)
* fdisk: allow to use the last partial cylinder, change display format
* fdisk: do not ask partition number in case there is only one choice
* fdisk: new sunlabel fix

util-linux 2.11v

* fdisk: add -C, -H, -S command line options
* fdisk: allow changing type 0

util-linux 2.11u

* fdisk: fix for fdisk on empty disk (Michael D. Black)

util-linux 2.11t

* fdisk: fix device names with sundisklabels under devfs (Kaj-Michael Lang)
* fdisk: minor polishing

util-linux 2.11s

* fdisk: fixes for disks with OSF/1 label

util-linux 2.11r

* fdisk: avoid <linux/hdreg.h>
* fdisk: x86_64 patch (Mads Martin Jrgensen)

util-linux 2.11q

* fdisk: geometry improvement

util-linux 2.11o

* *fdisk: added Darwin types (Vincent Bourgonjen)

util-linux 2.11m

* fdisk: added netbsd type (Steven J. Hill)

util-linux 2.11l

* fdisk: show partitions with empty type but some other nonzero field

util-linux 2.11k

* fdisk: added fclose()

util-linux 2.11j

* fdisk: fix for AIX label (Olaf Hering)
* fdisk: fix for creating DOS label when BSD label is present

util-linux 2.11i

* fdisk: avoid superfluous warning about DOS partitions


Quite a lot of changes really...

On tests the 2.12a version of fdisk reports the same geometry for SD cards on the Z as it does on a desktop machine through a reader which 2.11g DOESNT - it reports quite different results (see the first message in this thread).

SD Cards seem always to show extra partition information when read on the Z through the mmcda driver with this new version of fdisk. These partitions do NOT show when using SCSI emulation through a desktop card reader on USB2. I'm not sure if this at the moment is coding in fdisk, a cross compilation issue (dev image won't build it ) or some 'undocumented feature' of the mmcda driver revealing something on the SD card that it isn't supposed to..

I would like to rule out the mmcda driver as a possible cause but to do this I need some input from someone with an MMC card to see if the same artifacts appear on the fdisk 2.12a partition list (second mmcda partition as discussed in first item in this thread).

Any thoughts people ?

[iamasmith]

OK, managed to get my dev_img working again.

The fdisk quirk isn't a cross compile artifact..

I now have two arm binaries fdisk-2.12a-cross and fdisk-2.12a-native and BOTH report the extra partition on SD Cards.

Anyone tried with MMC cards yet

BTW: Anyone with an SD Card that they have killed with the standard Z fdisk drop me a personal message if they want to try these newer versions of fdisk.

When I have some positive proof that these versions improve the situation I will make an IPK for distribution (subject to the usual disclaimers of course).

[iamasmith]

OK, this is obviously a bug that appears when fdisk 2.12a is compiled for ARM.

If you reset the partition table using o and create a new primary partition you immediately get the empty second partition too.

Hmm, I'll take a look when I get more time.....

[iamasmith]

OK,

Running gdb on my Z now, strangest of things is that I can dump the block of memory that is the second partition and it's all zero, however, when I let the code run this function....

static int is_cleared_partition(struct partition *p) {
   return !(!p || p->boot_ind || p->head || p->sector || p->cyl ||
       p->sys_ind || p->end_head || p->end_sector || p->end_cyl ||
       get_start_sect(p) || get_nr_sects(p));
}

(btw the get_start_sect and get_nr_sects are simple macros that calculate an offset in the partition entry and they return 0 too)

It returns true

There's definitely something wrong here, tried modifying the source and using #pragma pack(1) around the structures to see if it was alignment or basic compiler behaviour... alas no... there's more going on here...

Anyone point me at an arm assembler reference pdf ?

Regards,

Andy

[iamasmith]

OK, it seems that enabling -mstructure-size-boundary=8 forces gcc to generate a working version of fdisk 2.12a

Still testing....

BTW: Build instructions for anyone wanting to test this..

Get util-linux-2.12a from www.kernel.org.

Unpack... edit MCONFIG and change DISABLE_NLS to say yes.
Run ./configure
Edit the resultant make_include file and add -mstructure-size-boundary=8 to the CCFLAGS line.
Go into the fdisk directory and type 'make fdisk'.

[iamasmith]

OK, now tried a few cards (CF and SD) with the new version of fdisk.

Just fdisk'ed my 512MB SD card so we'll see if it's stable for a week or so then I'll issue a copy for test.

Pop quiz to a moderator here...

What's the position on issuing a single file such as fdisk as a non IPK ?, traditionally the IPK stored fdisk in /usr/sbin. Mr M has put fdisk on Cacko 1.21b in /sbin - basically off the RO root fs instead of via a symlink.

Also the default path uses /sbin before /usr/local/bin /usr/local/sbin which would have been my favourite place to put it as an add-on tool.

Because of the difference in ROMs I would like to allow people to place this version where they like. - If they can't manage that then they shouldn't really be playing with fdisk anyway.

What do you think ? Can I do it as single file that people can copy on and chmod or should I produce 2 IPKs, one for Cacko 1.21b and one for Sharp ROMs... would have to ask someone to test the Sharp one.

Or Mr M., would you take this as part of a hotfix ?

[iamasmith]

Test are going well with the card following partitioning with the new version of fdisk and use of ext3, however, I have discovered that version 1.19 of e2fsck can scratch your ext3 journal from the file system.

Symptoms are that you get a bad block warning for block 8, if you choose to correct it then seems to orphan all the inodes in the journal so you see a shed load more. If you ignore block 8 then you don't get the initial warning and the journal blocks stay allocated.

Have built e2fsck version 1.35 and this seems to fix this problem... still more testing to go...

[iamasmith]

Now maintaining this thread on Sharp ROM Forum under 'New versions of fdisk....'

Have found a couple of issues with the older versions of the tools. I would like people to test with these tools - my motive is that I would like to see far fewer people with duff SD cards and I have certainly seen a few issues that the older tools could cause - please if you have the ability help with the test and by doing so help your fellow Zaurus users.

Regards,

Andy