OESF Portables Forum
Everything Else => Zaurus Distro Support and Discussion => Distros, Development, and Model Specific Forums => Archived Forums => Sharp ROMs => Topic started by: iamasmith on November 07, 2004, 01:02:55 pm
-
Firstly heres a bit of eye candy...
3Gb (that's right Gb) of static Wikipedia data running through Apache ON THE Z from a 1Gb Microdrive... in a 550Mb squashfs !
Oh, BTW kudos to Maslovsky on this one, I only got to know about squashfs from his posting RE: New version of Cacko.. 1.22. Thanks Anton!
OK, firstly before the outraged observations the 3Gb report was from du -s -k on my reiserfs file system on my desktop. The below screenshot shows just over 2Gb in the squashfs... however, may I ask you now "Can you do that on your Windows CE/PocketPC/Windows Mobile.... whatever you are calling it this quarter ?"
-
Now here's the module, you need to uudecode it before installing and following install simply reboot or run depmod whilst logged in as root.
You can build the mksquashfs binary for your desktop easily from the source at http://squashfs.sourceforge.net (http://squashfs.sourceforge.net)
Officially the patches only go back as far as 2.4.20, however, a slight tweak after applying the patch to the Cacko kernel (Makefile in fs directory) allowed me to build using a Cacko kernel tree.
BTW: I have absolutely NO idea how big the EN version of the Wikipedia database would be with images since there are now around 70,000 images in the EN database and I suspect it would take the best part of the year to download them. The 3Gb mentioned in the first posting was a TEXT ONLY version of the dump built from the en 20041030_cur_table_sql.bz2 file taken from download.wikipedia.org
Also anyone interesting in doing this now can use the latest scripts for wiki2static.pl from http://www.tommasoconforti.com (http://www.tommasoconforti.com) without modification and without the extra steps outlined in the wiki2zaurus project. As these scripts seem to be being maintained here it seems like a better bet for future Zaurus wiki use.
Have fun,
- Andy
-
I ordered a 1Gb CF yesterday, now it looks like i'll fill it as soon as it arrives :-) . Do you think this can work under pdaXrom also?
-
Do you think this can work under pdaXrom also?
No, you would need to build the module for the Kathrin kernel which itself has had some heavy mods.
I haven't actually done a Kathrin kernel build so I would suggest asking one of the PDAXROM gurus for help on this one - as I say applying the patch is fairly straight forward. I used the one for 2.4.20 and then went through and corrected the Makefile.rej that I got in the fs directory of the kernel source.
BTW: Forgot to mention, squashfs is READ ONLY, so it's good for static apps but not for anything else... + better compression than cramfs + mkcramfs will give you a max file size around 1/8 of the size of your RAM +SWAP so on my machine that was about 280Mb, mksquashfs doesn't seem to limit you in this respect.
- Andy
-
BTW: Forgot to mention, squashfs is READ ONLY, so it's good for static apps but not for anything else... + better compression than cramfs + mkcramfs will give you a max file size around 1/8 of the size of your RAM +SWAP so on my machine that was about 280Mb, mksquashfs doesn't seem to limit you in this respect.
... plus squashfs fully supports GID and UID bits, which lets you stores files belonging to any user/group.
-
Good point Anton, forgot to mention that.
BTW: the mksquashfs also allows you to override current ownership with user specified UID and GID.. useful where your Apache server is running on different UID/GIDs on your target !
-
Actually one point to note on the use seems to be storage card related.
I'm currently waiting for a 2Gb SD card and my only storage card over 512Mb at the moment is an old 1Gb IBM Microdrive. I put my 550Mb image (the one carrying the Wikipedia data) on this card and have been occasionally popping the card in, mounting the squashfs, working with that and dismounting.
I have found, however, that with this card if you suspend for more than a few seconds, when you unsuspend the squashfs has a few problems and I think it's related to activity on the squashfs module while the card is 'resuming' following the suspend. Anyway to recover this condition I have found a reboot necessary.
I suspect that such problems also effect cramfs filesystems on cards like this and will report on the success (or lack of) when I get my 2Gb SD card.
- Andy
-
I think the problem is due to that you are using mikro drive
-
I think the problem is due to that you are using mikro drive
Well I'm hoping so.
I so want to have the Wikipedia content always available via my Apache server on my Cacko ROM with stacks of space free too - hence the order for the 2Gb SD card.
Man I can't wait to get hold of that card!
- Andy
-
I've used a cramfs image (zgcc) extensively on a 512 MB SD on my C760 running Cacko 1.21a and haven't experienced a single problem with suspend. If the problem persists on your SD card then it may be specific to squashfs.
-
I believe the problem is related to ANY CF card, because they are reinitialized upon resume on Zaurus. SD cards are better in this respect.
-
I believe the problem is related to ANY CF card, because they are reinitialized upon resume on Zaurus. SD cards are better in this respect.
Thought that may be the case.... could make the SL-C3000 quite unpopular...
-
I think this is just a kernel issue which could(*) be fixed.
*: theoretically, given enough motivation and/or a skilled (kernel) developer
-
Actually it does seem bullet proof when running from SD (a good SD).
Offroadgeek, should this go to the download section ?
-
You can build the mksquashfs binary for your desktop easily from the source at http://squashfs.sourceforge.net (http://squashfs.sourceforge.net)
Has anyone compiled mksquashfs for the Z?
-
You can build the mksquashfs binary for your desktop easily from the source at http://squashfs.sourceforge.net (http://squashfs.sourceforge.net)
Has anyone compiled mksquashfs for the Z?
Here you go.
-
Here you go.
Thanks, I'll give it a go in a while.
How do I mount the file when created?
-
Assuming you created a directory under /mnt called mountpoint and your squashfs is called mysquashfs.sqsh and stored on /mnt/card...
mount -t squashfs /mnt/card/mysquashfs.sqsh /mnt/mountpoint -o loop
I would advise against leaving squashfs filesystems mounted during suspend (and also any other file systems stored in files) if stored on Microdrives.. this may also effect other kinds of CF cards.
SD Cards don't seem to be a problem.
-
You can build the mksquashfs binary for your desktop easily from the source at http://squashfs.sourceforge.net (http://squashfs.sourceforge.net)
Has anyone compiled mksquashfs for the Z?
Here you go.
iamasmith or anyone else,
Would greatly appreciate your input on getting squashfs2 setup on my C3000....
I'm trying to get the new version of dev_img-1.6 up and running. According to that forum post you need to get squashfs2 setup for it to work.
So...not fully clear as to what I should do with the file you made available for dowloading...
What do I do after putting uncompressed files on my SD card?
Do I try and follow squashfs2 install file instructions or ??
Thanks for any help...really appreciate it!
Mark
C3000
-
Sorry ArchiMark the version of the kernel module that was built is for a 2.4.18 kernel.
You could build your own module by downloading the kernel source from the Sharp website and applying the squashfs patch. In fact I would highly recommend it.
I, however, am not going to do this because I don't have a 3000 to test the module with.
I do need to bring to your attention the issue that occurs with older Microdrives and may potentially occur with the HD in the 3000. If the device is subject to the PCMCIA services suspend/resume as the CF slot is on an 860 it may take a while to resume and consequently not resume in time to satisfy uncompleted read requests from a squashfs stored on it. It should be safe to store the squashfs file on an SD card if this is an issue.
Anyway I look forward to hear how you get on with this.
- Andy
-
Sorry ArchiMark the version of the kernel module that was built is for a 2.4.18 kernel.
You could build your own module by downloading the kernel source from the Sharp website and applying the squashfs patch. In fact I would highly recommend it.
THANKS for your help, Andy!
Could you clarify a bit further as to what you mean by "applying the squashfs patch" ?
What environment do I do this in? Desktop PC?
Do I just follow instructions in Install File?
Once I have a better picture of what I need to do, I have no problem trying it...
Thanks,
Mark
-
You will need a desktop Linux system with a cross compiler toolchain to do this.
You basically patch the Linux kernel sources with the patches provided on the squashfs website then build. The kernel module then needs packaging as an IPK for this distribution.
I would suggest that if you haven't built Linux kernels before you should probably leave this to someone who has.
(anyone with an SL-3000 fancy helping out ?)
- Andy
-
Thanks again for clarifying Andy!
Now I get it....
Thanks for putting the call out for volunteers!
Sounds a bit out of my league....
Although would probably be a good learning exercise !
Hope someone can help out...either
A) do this and make available for us C3000 users or
be willing to share some more info on how to do this, so that I can try and do it and then I'd share it with others here....
Thanks,
Mark
-
Hi again,
One more request for a bit more guidance on how to get Squashfs2 on my C3000...
I think if someone could answer the following questions, I think I could possibly figure what to do.....or at least give it a good try!
A) "desktop Linux system with a cross compiler toolchain"
What software is referred to by this?
"You basically patch the Linux kernel sources with the patches provided on the squashfs website then build. The kernel module then needs packaging as an IPK for this distribution."
Does this mean that you take download the Sharp ROM file, add in the patches, then recompile it, and then package this into an IPK?
Is this the process you need to follow?
Thanks for any help!
Mark
C3000 User Anxious to get Dev_Img-1.6 on his Z!
-
I think this is just a kernel issue which could(*) be fixed.
Does anyone have the details of this problem? Is it a kernel bug affecting all CF cards? Is it fixed in any of the alternative kernels? Thanks.
-
Hi again,
One more request for a bit more guidance on how to get Squashfs2 on my C3000...
I think if someone could answer the following questions, I think I could possibly figure what to do.....or at least give it a good try!
A) "desktop Linux system with a cross compiler toolchain"
What software is referred to by this?
Thanks for any help!
Mark
C3000 User Anxious to get Dev_Img-1.6 on his Z!
You can get the cross-compiler toolchain from ZUG as well as detailed instructions on how to set it up.
Check out these links
Compiler Setup (http://www.zaurususergroup.com/modules.php?op=modload&name=phpWiki&file=index&pagename=Compiler%20Setup)
Compiling the Kernel (http://www.zaurususergroup.com/modules.php?op=modload&name=phpWiki&file=index&pagename=Compiling%20the%20Kernel)
Special Kernel (by Ced) (http://www.zaurususergroup.com/modules.php?op=modload&name=phpWiki&file=index&pagename=Special%20Kernel%20%28by%20Ced%29) - info on how to patch kernel sources
This (http://www.zaurususergroup.com/modules.php?op=modload&name=FAQ&file=index&myfaq=yes&id_cat=19&categories=Programming&parent_id=0#305) might also be of interest to you
"You basically patch the Linux kernel sources with the patches provided on the squashfs website then build. The kernel module then needs packaging as an IPK for this distribution."
Does this mean that you take download the Sharp ROM file, add in the patches, then recompile it, and then package this into an IPK?
Is this the process you need to follow?
Basically yes, although you will want to get the kernel sources rather than the Rom file. And it isn't actually necessary to create an ipk of the squashfs module, you could just copyit over manually. But an ipk would be nicer if you were planning to release it to others.
Stu
-
Hey, you're da' man, I mean bear, stubear!!
This is exactly the guidance I was looking for. Will try to do my homework now and see if I can make this work!
If so, I'll be glad to share it with others in ZUG...
Will report back on progress later....
THANKS,
Mark
C3000 User Anxious to get Dev_Img-1.6 on his Z!
-
anybody can post the 2.4.18 kernel squashfs ipk again. the original link is broken. thanks
-
anybody can post the 2.4.18 kernel squashfs ipk again. the original link is broken. thanks
Here it is: