C3000 Device Specifics

From OESF

Revision as of 17:00, 5 April 2005

Device Description

The SL-C3000 is the first attempt by Sharp to introduce a hard disk drive to the Zaurus range in the form of a 4Gb Hitachi Microdrive.

In placing the 4Gb Microdrive in the unit Sharp have also significantly reduced the amount of NAND flash memory present within the unit to 16Mb which still has some major significance in the operation of the software distribution on the Zaurus.

I'll mention one more thing here because I may not mention it later. If you shutdown -h the SL-C3000 it does NOT power down (or look like it) like the previous models did. The screen will stay on and the device will freeze. To completely drop the device following shutdown you should open the battery compartment and push the recessed reset switch that sits on the side of the battery compartment nearest the centre of the unit – you don't need to remove the battery to do this.

Oh and finally do NOT use the version of fdisk that comes with the Z to repartition the HDD, it suffers from a bug with structure alignment that's actually present in the version of GCC for ARM architectures.

Do a search and find reference to spurious 4th partition entry appearing with 0 bytes length.. this is the error. There are various good versions knocking around including the one I built for the 1.22 Cacko ROM. (search for new versions of fdisk and e2fsprogs on the oesf Sharp forum).

The traditional Zaurus layout

Traditional Zaurus models using NAND flash for user storage tend to use a set of independently mounted NAND regions to produce a software distribution layout that when running a Sharp ROM (or derivative) allow return to a default state by zeroing and reinitializing some of these regions.

Typically the NAND based models have a read only file system carrying a more or less static distribution of the OS, GUI and toolsets which is based upon a jffs2 file system and provides the root mount point for the Linux distribution. The actual files that are targeted at the distribution are kept in directories with names similar to '/usr/bin.rom, /usr/sbin.rom' such that they don't clash with regions typically associated with user modification (equivalent /usr/bin, usr/sbin). The user modification regions that are somewhat scattered across a standard Linux distribution are symbolic links within the root file system that redirect access to areas within the /home part of the file system which is mounted as a read/write area. Typically /usr/bin on the root file system will be a symbolic link pointing at /home/root/usr/bin and within the /home/root/usr/bin directory a set of symbolic links are present for each utility that is present within /usr/bin.rom.

This mechanism allows the root file system to be mounted as read only so that integrity may be preserved (against user changes) whilst allowing the flexibility of user changes to take place within the file system mounted on /home only.

The 'device zero' process or factory erase will wipe the /home partition and the startup scripts of the Zaurus are configured to unpack from the root file system a default set of symbolic links such that the device may be reinitalized to the factory state.

Note that we haven't discussed many of the symbolic links off the root file system, just understand that on a more or less traditional Zaurus configuration if you can write to an area of flash then it will be stored upon the /home directory so that your changes may be purged on device zero.

The SL-C3000 layout

The SL-C3000 differs slightly from the traditional Zaurus models that have adequate space available within NAND flash to maintain user data, user installed applications and the Qtopia environment.

In fact the flash memory on the SL-C3000 is so small that Sharp haven't even bothered trying to place the main operating environment into it.

Instead the distribution layout works as follows.

There is a read only flash region corresponding to the region on the traditional Zaurus. It is only 5Mb in size, whereas on a traditional Zaurus ROM this partition could be up to 50Mb!

There is a /home region of read write flash used to maintain configuration data that may need to be present before the hard drive is available or whilst the hard drive is suspended (i.e. device wakes with critically low power and suspends immediately – this operation should not need to resume the HDD). This region is only 4Mb in size and is a precious resource that should be used sparingly.

The hard drive is presented as a CF device in Slot 1 of the two slot controller with the user CF slot being slot 0 (equivalent to traditional Zaurus).

The user boot environment is entirely dependent upon having a working hard disk image because as previously mentioned the NAND flash is too small to accommodate the Qtopia environment.

Traditional disk partitioning is therefore as follows.

/dev/hda1, mounted as /hdd1 (read only) - default raw size ~100Mb and default ext3 - Sharp ROM uses about 50Mb of this - giving approximately 50Mb of EASY expansion for ROMs without reformatting. - this replaces all those /usr/QtPaltmop.rom type locations with... /hdd1/usr/QtPalmtop.rom, /hdd1/usr/QtPalmtop/bin.rom...sbin.rom etc.

/dev/hda2, mounted as /hdd2 (read write) - default raw size ~400Mb and default ext3 - This is where your user installed apps go - 400Mb of uncompressed space amply replaces the compressed jffs2 that you get on the flash models. /home/zaurus/Application is symbolically linked to /hdd2/Applications using this partition for traditional Qtopia application data.

/dev/hda3, mounted as /hdd3 (read write) - default raw size ~3.3Gb and default to vfat - /home/zaurus/Documents is symlinked to /hdd3/Documents and is the default location for the Samba home share etc..

There are some good things about this configuration and some bad things.

The good things are that basically everything needed to recreate the environment may be stored on the first hdd partition and a reasonable amount of effort has been made to ensure that user data goes onto the 3rd or 2nd partitions.

The bad thing related to this configuration is that if you are building up an environment for use and trying to apply traditional Zaurus applications which are widely used on the other models then you need to be extremely careful as to where these applications install their files.

For example a 'well behaved' application installation for the Zaurus will place all files somewhere downward of /opt/QtPalmtop. This is fine on the SL-C3000 because this region is actually symbolic linked into /hdd2/QtPalmtop.

Badly behaved applications (from the perspective of this environment) may place data into other read/write areas that aren't symbolically linked to the hdd i.e. /home or even /home/zaurus which actually lives in the /home flash partition.

None of these issues are terminal, they can all be worked around, however, a regular user installing some applications without the knowledge of how to adapt the install to the 3000 may have some difficulty.

Some examples of problem applications and resolutions are...

One version of apache-php places the entire install within /home/www. This install is 5Mb in size and because it goes to /home/www it misses the HDD completely and attempts to install in flash – which it doesn't – it fails mid way leaving all the space eaten up on flash. This can be overcome by pre-creating a link to a directory on the HDD. I suggest that /hdd2 locations are good candidates for this kind of activity.. i.e. mkdir /hdd2/www && ln -s /hdd2/www /home/www. If you do this prior to installation then the package will expand across the link and onto the hdd. ScummVM has a looming issue for some users in that the default location for storing saved game files is in ~ or the home directory. For the user 'zaurus' the home directory is in flash. Users using scummvm should change the settings so that saved game files go into /home/zaurus/Documents which is symbolic linked onto partition 3 of the hdd (the big one).

Basically if you do get an application installed. Run it a few times, particularly try saving things and then check if there's anything going into ~.

The device zero process on the SL-C3000 rebuilds the environment to factory state (or ROM install state) in a similar manner to the traditional Zaurus. However, instead of handling just flash, this process now recovers the hdd layout to a factory state too (as long as /hdd1 is intact and that /hdd2 and /hdd3 are partitioned).

The device zero process now performs the following sequence.

/home (on /dev/mtdblock3), /hdd2 (/dev/hda2) and /hdd3 (/dev/hda3) are all unmounted and formatted.

The file systems are then all remounted in those respective locations.

/hdd1/.sys/hdimage2.tgz is a tarball which is extracted at the root level, it contains /hdd2 layout including all symbolic links that point into /hdd1/usr/bin.rom, /hdd2/usr/sbin.rom, /hdd2/usr/QtPalmtop.rom etc. along with a basic structure for /hdd3.

/root/.home_default.tar (from the read only flash location) is then extracted at root. This contains the flash based elements used to populate the /home directory structure on mtdblock3.

At this stage the device is in the factory shipped (or ROM install) state.

Doing a better job for /home ? is it possible.

I had been thinking along the lines of moving /home to a hdd location by the use of mount bind points such that /home/root/etc and other such locations could be bound back to flash regions whilst /home itself resided upon the hdd.

This idea, unfortunately, could lead to stability issues so was abandoned.

The main area of concern for moving /home is the availability of the subordinate locations whilst the hdd is in suspend mode because a critically low battery condition has special handling code that the kernel executes - this immediately places the device back into suspension (without even displaying anything on the screen)... if you want to see this in action run your Z until flat, try to power it on a few times when it's showing completely flat, charge it then go into terminal and look at the kernel ring buffer using dmesg. You will see the emergency suspend stuff logged in there.

Even with mount bind placing the critical regions in flash we have to realise that they are still subordinate to /home and that inode traversal from /home to those subordinate locations (finding the bind point) will potentially trigger a required read of /home which will in turn attempt to use the hdd – not as efficient as maintaining the /home in flash.

We will have to be more inventive to find a better solution for /home in the future.

Zeroing an SL-C3000

To return an SL-C3000 back to factory settings simply remove the power and battery, reinsert the battery, plug in the mains cable and ensure that the battery cover lock is in the locked position then perform the following steps.

Hold down the OK button and keeping it held press the power button.

You will get a Japanese menu.

Select option 3.

You get another Japanese Menu.

Select option 1 – do NOT select option 2 unless you want a good 1hr wait (extensive erase writing /dev/zero over whole disk – did it once, it's not that exciting).

The device will ask you to confirm (in Japanese), hit Y and the device will reboot and recover to the factory/ROM default state.

Examining the flashing process

Firstly it's important to understand that the flashing process is a shell script that runs from /bin/sh under Linux.

You may ask how a flashing process works if the Linux is screwed. The answer is that there are two copies of Linux in flash on all clamshell units.

If you want to check this out then use the D+B device reset and the device will boot into a console of the version of Linux used for flashing and you can get at the utilities in /sbin (tested on SL-C3000 and on SL-C860) :-). (actually you don't get to see quite everything, you get an ext2 initrd mounted rw and I can't find the OK menu stuff in this image).

Reliably entering the D+B (emergency Linux) or maintenance menus (D+M) can be achieved as follows.

Remove power and battery. Ensure the battery lock is in the unlocked position. Hold down key combinarion (D+B or D+M). Whilst holding down the key combination insert the battery and slide the battery lock to the locked position.

Note (and I have tested this). If your fingers slip and you release the keys in between making contact with the battery and moving the slide lock then the device will go into a normal boot sequence so if you feel any sort of 'wobble' whilst doing this remove the battery again completely and then reinsert – it seems the firmware detection kicks in as soon as the battery contact is made.

The updater.sh script is decoded using the tool /sbin/decsh which is only present in this recovery position.

Basically following boot using the OK menus (OK held on device start) the updater.sh is passed through /sbin/decsh and into /bin/sh for execution.

The script then makes use of several other utilities present in /sbin which you really shouldn't mess about with unless you really understand them.

We shall examine now the installer script from the Sharp ROM 1.11JP and make some comments. (actually it's slightly modified to correct a bug in the original). I have inserted some comments to make this more readable... denoted by ##.

SL-C3000 Updater Script

Following script completion the system shall restart. Please note that the default Sharp script shown does NOT erase the /home, /hdd2 or /hdd3 areas.

Preparing a Kernel for use with the Sharp Installer

I'm not going to talk you through building a custom Kernel. If you need that kind of help you probably shouldn't be thinking of prepping a Kernel for flashing.

Basically the approach that I have taken is simply to strip the last 16 bytes off the end of the Sharp Kernel image from the Sharp 1.11JP ROM as follows..

tail -c 16 zImage.bin-sharprom > kerneltail
cat zImage.bin-mykernel kerneltail > zImage-prepped

This is really enough to get the ball rolling.

Creation of a new initrd for the SL-C3000

The initrd on the SL-C3000 is simply a jffs2 image. It is, however, shipped with a 16 byte version header so here's how to successfully extract an image to work from....

modprobe mtdcore
modprobe jffs2
modprobe mtdram total_size=5120 erase_size=16
modprobe mtdchar
mknod /dev/mtdblock0 b 31 0
modprobe mtdblock
losetup -o 16 /dev/loop0 ./initrd.bin
dd if=/dev/loop0 of=/dev/mtdblock0
losetup -d /dev/loop0
mount -t jffs2 /dev/mtdblock0 /mymountpoint
cd /mymountpoint
tar -cf ~/myinitrd.tar *
cd
rmmod mtdblock
rmmod mtdram
rmmod jffs2
rmmod mtdchar
rmmod mtdpart
rmmod mtd_blkdevs
rmmod mtdcore

This will create you a nice initrd tar file that you can extract and work with.

Remember there are multiple permission sets in the initrd so you have to extract/work with the initrd whilst logged in as root.

You should also then probably extract root/.home_default.tar from the initrd as this is the seed for the /home partition and since /lib/modules is actually linked to /home/root/modules then you will have to add symbolic links to that if you add modules to the initrd.

A WORD OF WARNING when repacking the home structure into a new .home_default.tar... the version of tar built into the Sharp ROM is roughly equivalent to version 1.12 of GNU tar, particularly with respect to the format of device files stored in the tar. Since the /dev directory is stored within the initrd it is absolutely CRITICAL that you repack the .home_default.tar in the right format. If you type 'tar –version' this will get you the version number back. If its a later version of tar than 1.12 you need the following format for the tar command...

tar -c –-format=oldgnu -f .home_default.tar home/

Be sure to run the command from a directory containing nothing else but the home directory structure as this will all get expanded to the Z following a 'device zero' condition.

Now, assuming that you have finished working with the initrd in a structure in a directory called initrd you need to get hold of mkfs.jffs2 (get this from the SL-C860 root image if you like off the Sharp developer site... Sharp didn't yet post the SL-C3000 image which is why we had to extract it.) and use the following command to build the jffs2 image.

mkfs.jffs2 -o initrd.jffs2 -e16k -r initrd

You then need to exract the version information from the Sharp initrd.bin....

head -c 16 sharp-initrd.bin > initrdheader

...and finally concatenate as follows...

cat initrdheader initrd.jffs2 > initrd.bin

This will produce an initrd.bin that is compatible with the installer shown in this paper.

Prior to flashing

Please remember the installer shown in this document is 'fixed'. Using the original Sharp installer with custom kernels tends to brick the devices (remember that nice part of the script that erases the Kernel flash area).

You MUST modify the installer in line with the modification shown around the nandflash area in the script otherwise the flaw in the original Sharp installer will assert itself.. seems to kick in if you have a Kernel even slightly bigger than the original (nice one Sharp).

To modify this script you need a copy of a utility called endecsh which you can probably google for but here's the source in case you want to build it.

endecsh.c