Author Topic: Creating Python 2.4 Zaurus Image (Read 21684 times)

kopsis · « **on:** January 05, 2005, 04:15:58 pm »

I decided the shiny new Cacko 1.22 ROM on my Z needed a shiny new version of Python to go with it, so I'm building up a new Zaurus Python image with Python 2.4. I have the base Python image built and working and it includes all the "standard" Python modules (with the exception of crypt and nis for which I'm still missing libraries) including distutils, encodings, curses, readline, xml, etc.

In addition to the ext2 format, I'll also be releasing this new Python image as a cramfs for folks that want to run off SD (or internal flash) without chewing up so much space. Its not easy for folks to add stuff to the cramfs image on their own, so before I package it up, I'd like to find out if there are any "site-packages" that people think are critical to doing Python on the Z. The obvious ones (to me) are PyQT and pysqlite so I'll be including those. If there's anything else you'd like to see (that won't bloat things up too much), now's the time to ask.

LarsAC · « **Reply #1 on:** January 07, 2005, 04:33:29 am »

Would the mySQL API for Python fit into this, too?

I have a number of apps at home for which I might want to write a dedicated client for the Z in favor of porting stuff amde for large screens...

Lars

fpp · « **Reply #2 on:** January 07, 2005, 07:00:08 am »

Does this mean than when using the cramfs version, we would be unable to add packages that install themselves to site-packages in the standard distutil way :
python setup.py --install
... or am I misunderstanding ?
(this is not a problem for me, I'm comfortable with the ext2 image, but I'm not familiar with these fs tricks, so it's just to make sure...)

On a related topic, maybe you've seen this 'Movable Python' project that appeared recently :
http://sourceforge.net/projects/movpy
http://www.voidspace.org.uk/python/movpy

Do you think it might be a useful option for the Z (without the wxPython part obviously), or would it be no smaller than what you've achieved ?

kopsis · « **Reply #3 on:** January 07, 2005, 08:45:52 am »

Quote

Would the mySQL API for Python fit into this, too?

Perhaps. The source code looks pretty small, I just have to see if I can get it to build.

kopsis · « **Reply #4 on:** January 07, 2005, 08:57:58 am »

Quote

Does this mean than when using the cramfs version, we would be unable to add packages that install themselves to site-packages in the standard distutil way

That is correct. A cramfs filesystem is, by nature, "read only". The only way to make changes is mount it, make a complete copy to a writeable filesystem, make the changes there, then use mkcramfs to build a new image from the copy in the writeable filesystem.

I suspect that Python developers will much prefer the ext2 filesystem image, despite the much larger size. With 1GB SD cards going for under $70 these days, 50MB seems a small price to pay

Quote

On a related topic, maybe you've seen this 'Movable Python' project that appeared recently

I took a look and it's basically trying to create Python filesystem images for Windows platforms. A standard Windows install doesn't have the luxury of things like loop mounting and cramfs drivers so they have to use a different (and more complex) approach. The images they're building are comparable in size to a cramfs image (and are also read-only), so there's little to be gained by going down that path.

clofland · « **Reply #5 on:** January 07, 2005, 02:20:02 pm »

Sounds great! I've been waiting for 2.4 for the Z before I start using it on my Windows box.

As long as it supports sqlite 3, I'll be happy.

I'll probably use the cramfs version until I run into something I can't do, and then I'll decide how badly I want to switch.

One thing I do notice is that every time I start up my Z, it says that the Python file system was not cleanly unmounted. It even does that if I manually stop and start it. I wonder if the script isn't quite right? It doesn't seem to cause any problems though, other than forcing a file system check each time.

fpp · « **Reply #6 on:** January 07, 2005, 04:21:26 pm »

Thanks for both clarifications kopsis !

I'll wait till the new 2.4 ext2 image comes out, and probably the first cacko 1.22 hotfix too, then it'll be flashing time :-)

clofland · « **Reply #7 on:** January 10, 2005, 10:31:16 am »

One quick add: Isn't squashfs "better" than cramfs? My understanding is that Cacko ROM 1.22 uses squashfs instead of cramfs, because it makes a smaller faster image.

kopsis · « **Reply #8 on:** January 10, 2005, 12:53:15 pm »

squashfs is a bit better than cramfs, but that would force users to install drivers (which may not even be available for all ROMs). cramfs will work everywhere, and it's a pretty trivial matter for those who want squashfs to turn the cramfs image into a squashfs image (I'll probably throw a short howto on the website).

gasolin · « **Reply #9 on:** January 10, 2005, 05:07:50 pm »

Dear kopsis:

Very appreciate for your python image release.

As I saw this post, I'd suggest to make "palmdb" module preinstalled to read/write/modify Palm PDB and PRC format databases with python.

This module is maintained by rick_price, one of PDesk(the Palm outliner "progect"'s desktop program) developers, and avaliable at sourcefotge
http://sourceforge.net/projects/pythonpalmdb/

The second is cjkcodecs, which contained in python 2.4. To support Chinese, Japenese, and Korean characters. This can make your MP3player display CJK characters in Tags.

doctest (and unittest(opt.)) is one of essential model, too.
http://aspn.activestate.com/ASPN/Cookbook/...n/Recipe/305292
I think this module is helpful for programers to make unit test while writing documents.

And how about add "twisted" module to ease network-enabled program develop curve?

regards

kopsis · « **Reply #10 on:** January 10, 2005, 07:04:17 pm »

The palmdb module sounds like a great idea! I'll look into that one.

The CJK codecs from the standard 2.4 distribution are definitely included, as are doctest and unittest I'll have to take a look at the "twisted" stuff. If it's small enough and builds easily, I'll throw it in.

Database wise, pysqlite for sqlite3 is in there. Hopefully there won't be too many folks still wanting to use sqlite2 database files.

I've also built and tested MySQL-python 1.1.8 for MySQL 3.23.49 (which seems to be the latest Zaurus IPK around). It's not perfect ... the client libs default to the wrong directory for the localhost socket and choke if you force them to read my.cnf. But those problems are easy enough to work around with the right "connect" parameters. I'll be including the MySQL stuff for expert users who need to work with MySQL databases, but you're on your own when it comes to getting MySQL running on the Z. If you don't know why you would want to choose MySQL over sqlite, then you probably shouldn't

The cramfs image is currently running 8.3MiB and the ext2 version will weigh in at around 35MiB (25MiB + 10MiB free space). That's with the .pyo files stripped out (I don't think too many people use the optimize switch) but all the .py and .pyc files retained. With a little luck I'll be able to release in a couple weeks.

Bertram Wooster · « **Reply #11 on:** January 11, 2005, 06:55:52 am »

dear kopsis, thanks for all your work!
do you also plan to release a non-image version, which does not necessarily have to stay on sd/cf but can be installed/copied to harddisk (on a c3000) directly?

kopsis · « **Reply #12 on:** January 11, 2005, 08:13:37 am »

Quote

do you also plan to release a non-image version, which does not necessarily have to stay on sd/cf but can be installed/copied to harddisk (on a c3000) directly?

For all practical purposes, the HD in the SL-3000 is a CF card, so the ext2 image should work from it. In fact, my understanding is that the main storage partition on the internal HD is formated FAT (which doesn't allow symlinks or file permissions) so you'd have to use the ext2 image rather than installing the files directly.

The only thing that may not work is my init scripts to search for the image. That should be easy enough to fix, but I'd need someone with a 3000 to send me the output from running the mount command (as root and without any arguments).

Now, if someone does want to install Python to an ext2/ext3 formatted filesystem (internal memory or an ext2 formatted CF/SD card), it's pretty easy. Just mount the image, copy everything over to the target directory, and then make the appropriate symlinks. If there's demand, I can write up semi-detailed instructions.

Bertram Wooster · « **Reply #13 on:** January 11, 2005, 03:06:42 pm »

it's definitely worth a try, though i'd be thankful for your "semi-detailed instructions"- i'm not new to computers however to linux and especially unix commands are quite unknown to me.

LarsAC · « **Reply #14 on:** January 11, 2005, 03:42:51 pm »

Quote

But those problems are easy enough to work around with the right "connect" parameters. I'll be including the MySQL stuff for expert users who need to work with MySQL databases, but you're on your own when it comes to getting MySQL running on the Z.

Great stuff, I'm really looking forward towards the release.

Regarding MySQL: I'm rather intending to use a database server on the network rather than a MySQL instance running on the Z. Hopefully this is going to work without problems...

Lars

News:

Author Topic: Creating Python 2.4 Zaurus Image (Read 21684 times)