Author Topic: Creating Python 2.4 Zaurus Image  (Read 18120 times)

kopsis

  • Sr. Member
  • ****
  • Posts: 329
    • View Profile
    • http://kopsisengineering.com
Creating Python 2.4 Zaurus Image
« on: January 05, 2005, 04:15:58 pm »
I decided the shiny new Cacko 1.22 ROM on my Z needed a shiny new version of Python to go with it, so I'm building up a new Zaurus Python image with Python 2.4. I have the base Python image built and working and it includes all the "standard" Python modules (with the exception of crypt and nis for which I'm still missing libraries) including distutils, encodings, curses, readline, xml, etc.

In addition to the ext2 format, I'll also be releasing this new Python image as a cramfs for folks that want to run off SD (or internal flash) without chewing up so much space. Its not easy for folks to add stuff to the cramfs image on their own, so before I package it up, I'd like to find out if there are any "site-packages" that people think are critical to doing Python on the Z. The obvious ones (to me) are PyQT and pysqlite so I'll be including those. If there's anything else you'd like to see (that won't bloat things up too much), now's the time to ask.
« Last Edit: January 05, 2005, 04:16:57 pm by kopsis »

LarsAC

  • Newbie
  • *
  • Posts: 4
    • View Profile
Creating Python 2.4 Zaurus Image
« Reply #1 on: January 07, 2005, 04:33:29 am »
Would the mySQL API for Python fit into this, too?

I have a number of apps at home for which I might want to write a dedicated client for the Z in favor of porting stuff amde for large screens...

Lars

fpp

  • Full Member
  • ***
  • Posts: 187
    • View Profile
    • http://
Creating Python 2.4 Zaurus Image
« Reply #2 on: January 07, 2005, 07:00:08 am »
Does this mean than when using the cramfs version, we would be unable to add packages that install themselves to site-packages in the standard distutil way :
python setup.py --install
... or am I misunderstanding ?
(this is not a problem for me, I'm comfortable with the ext2 image, but I'm not familiar with these fs tricks, so it's just to make sure...)

On a related topic, maybe you've seen this 'Movable Python' project that appeared recently :
http://sourceforge.net/projects/movpy
http://www.voidspace.org.uk/python/movpy

Do you think it might be a useful option for the Z (without the wxPython part obviously), or would it be no smaller than what you've achieved ?

kopsis

  • Sr. Member
  • ****
  • Posts: 329
    • View Profile
    • http://kopsisengineering.com
Creating Python 2.4 Zaurus Image
« Reply #3 on: January 07, 2005, 08:45:52 am »
Quote
Would the mySQL API for Python fit into this, too?
Perhaps. The source code looks pretty small, I just have to see if I can get it to build.

kopsis

  • Sr. Member
  • ****
  • Posts: 329
    • View Profile
    • http://kopsisengineering.com
Creating Python 2.4 Zaurus Image
« Reply #4 on: January 07, 2005, 08:57:58 am »
Quote
Does this mean than when using the cramfs version, we would be unable to add packages that install themselves to site-packages in the standard distutil way

That is correct. A cramfs filesystem is, by nature, "read only". The only way to make changes is mount it, make a complete copy to a writeable filesystem, make the changes there, then use mkcramfs to build a new image from the copy in the writeable filesystem.

I suspect that Python developers will much prefer the ext2 filesystem image, despite the much larger size. With 1GB SD cards going for under $70 these days, 50MB seems a small price to pay

Quote
On a related topic, maybe you've seen this 'Movable Python' project that appeared recently

I took a look and it's basically trying to create Python filesystem images for Windows platforms. A standard Windows install doesn't have the luxury of things like loop mounting and cramfs drivers so they have to use a different (and more complex) approach. The images they're building are comparable in size to a cramfs image (and are also read-only), so there's little to be gained by going down that path.

clofland

  • Full Member
  • ***
  • Posts: 245
    • View Profile
    • http://www.lofland.net/
Creating Python 2.4 Zaurus Image
« Reply #5 on: January 07, 2005, 02:20:02 pm »
Sounds great! I've been waiting for 2.4 for the Z before I start using it on my Windows box.

As long as it supports sqlite 3, I'll be happy.

I'll probably use the cramfs version until I run into something I can't do, and then I'll decide how badly I want to switch.

One thing I do notice is that every time I start up my Z, it says that the Python file system was not cleanly unmounted. It even does that if I manually stop and start it. I wonder if the script isn't quite right? It doesn't seem to cause any problems though, other than forcing a file system check each time.
[span style=\'font-size:14pt;line-height:100%\']C-760 pdaXrom 1.1.0beta3[/span]
- Sandisk UltraII 1GB SD card - Socket LP CF LAN card - Socket LP CF WLAN card - Socket CF Bluetooth Card Rev H -

fpp

  • Full Member
  • ***
  • Posts: 187
    • View Profile
    • http://
Creating Python 2.4 Zaurus Image
« Reply #6 on: January 07, 2005, 04:21:26 pm »
Thanks for both clarifications kopsis !

I'll wait till the new 2.4 ext2 image comes out, and probably the first cacko 1.22 hotfix too, then it'll be flashing time :-)

clofland

  • Full Member
  • ***
  • Posts: 245
    • View Profile
    • http://www.lofland.net/
Creating Python 2.4 Zaurus Image
« Reply #7 on: January 10, 2005, 10:31:16 am »
One quick add: Isn't squashfs "better" than cramfs? My understanding is that Cacko ROM 1.22 uses squashfs instead of cramfs, because it makes a smaller faster image.
« Last Edit: January 10, 2005, 10:31:45 am by clofland »
[span style=\'font-size:14pt;line-height:100%\']C-760 pdaXrom 1.1.0beta3[/span]
- Sandisk UltraII 1GB SD card - Socket LP CF LAN card - Socket LP CF WLAN card - Socket CF Bluetooth Card Rev H -

kopsis

  • Sr. Member
  • ****
  • Posts: 329
    • View Profile
    • http://kopsisengineering.com
Creating Python 2.4 Zaurus Image
« Reply #8 on: January 10, 2005, 12:53:15 pm »
squashfs is a bit better than cramfs, but that would force users to install drivers (which may not even be available for all ROMs). cramfs will work everywhere, and it's a pretty trivial matter for those who want squashfs to turn the cramfs image into a squashfs image (I'll probably throw a short howto on the website).

gasolin

  • Newbie
  • *
  • Posts: 1
    • View Profile
Creating Python 2.4 Zaurus Image
« Reply #9 on: January 10, 2005, 05:07:50 pm »
Dear kopsis:

Very appreciate for your python image release.

As I saw this post, I'd suggest to make  "palmdb" module preinstalled to read/write/modify Palm PDB and PRC format databases with python.

This module is maintained by rick_price, one of PDesk(the Palm outliner "progect"'s desktop program) developers, and avaliable at sourcefotge
http://sourceforge.net/projects/pythonpalmdb/


The second is cjkcodecs, which contained in python 2.4. To support Chinese, Japenese, and Korean characters. This can make  your MP3player  display CJK characters in Tags.

doctest (and unittest(opt.)) is one of essential model, too.
http://aspn.activestate.com/ASPN/Cookbook/...n/Recipe/305292
I think this module is helpful for programers to make unit test while writing documents.

And how about add "twisted" module  to ease network-enabled program develop curve?

regards
« Last Edit: January 10, 2005, 05:42:36 pm by gasolin »

kopsis

  • Sr. Member
  • ****
  • Posts: 329
    • View Profile
    • http://kopsisengineering.com
Creating Python 2.4 Zaurus Image
« Reply #10 on: January 10, 2005, 07:04:17 pm »
The palmdb module sounds like a great idea! I'll look into that one.

The CJK codecs from the standard 2.4 distribution are definitely included, as are doctest and unittest  I'll have to take a look at the "twisted" stuff. If it's small enough and builds easily, I'll throw it in.

Database wise, pysqlite for sqlite3 is in there. Hopefully there won't be too many folks still wanting to use sqlite2 database files.

I've also built and tested MySQL-python 1.1.8 for MySQL 3.23.49 (which seems to be the latest Zaurus IPK around). It's not perfect ... the client libs default to the wrong directory for the localhost socket and choke if you force them to read my.cnf. But those problems are easy enough to work around with the right "connect" parameters. I'll be including the MySQL stuff for expert users who need to work with MySQL databases, but you're on your own when it comes to getting MySQL running on the Z. If you don't know why you would want to choose MySQL over sqlite, then you probably shouldn't

The cramfs image is currently running 8.3MiB and the ext2 version will weigh in at around 35MiB (25MiB + 10MiB free space). That's with the .pyo files stripped out (I don't think too many people use the optimize switch) but all the .py and .pyc files retained. With a little luck I'll be able to release in a couple weeks.

Bertram Wooster

  • Newbie
  • *
  • Posts: 39
    • View Profile
    • http://
Creating Python 2.4 Zaurus Image
« Reply #11 on: January 11, 2005, 06:55:52 am »
dear kopsis, thanks for all your work!
do you also plan to release a non-image version, which does not necessarily have to stay on sd/cf but can be installed/copied to harddisk (on a c3000) directly?
c3000, converted sharp rom
sandisk ultraII 512mb sd | d-link dcf-660w | pilot pentopia 3in1 stylus | brando ultra clear protector plus

kopsis

  • Sr. Member
  • ****
  • Posts: 329
    • View Profile
    • http://kopsisengineering.com
Creating Python 2.4 Zaurus Image
« Reply #12 on: January 11, 2005, 08:13:37 am »
Quote
do you also plan to release a non-image version, which does not necessarily have to stay on sd/cf but can be installed/copied to harddisk (on a c3000) directly?
For all practical purposes, the HD in the SL-3000 is a CF card, so the ext2 image should work from it. In fact, my understanding is that the main storage partition on the internal HD is formated FAT (which doesn't allow symlinks or file permissions) so you'd have to use the ext2 image rather than installing the files directly.

The only thing that may not work is my init scripts to search for the image. That should be easy enough to fix, but I'd need someone with a 3000 to send me the output from running the mount command (as root and without any arguments).

Now, if someone does want to install Python to an ext2/ext3 formatted filesystem (internal memory or an ext2 formatted CF/SD card), it's pretty easy. Just mount the image, copy everything over to the target directory, and then make the appropriate symlinks. If there's demand, I can write up semi-detailed instructions.

Bertram Wooster

  • Newbie
  • *
  • Posts: 39
    • View Profile
    • http://
Creating Python 2.4 Zaurus Image
« Reply #13 on: January 11, 2005, 03:06:42 pm »
it's definitely worth a try, though i'd be thankful for your "semi-detailed instructions"- i'm not new to computers however to linux and especially unix commands are quite unknown to me.
c3000, converted sharp rom
sandisk ultraII 512mb sd | d-link dcf-660w | pilot pentopia 3in1 stylus | brando ultra clear protector plus

LarsAC

  • Newbie
  • *
  • Posts: 4
    • View Profile
Creating Python 2.4 Zaurus Image
« Reply #14 on: January 11, 2005, 03:42:51 pm »
Quote
But those problems are easy enough to work around with the right "connect" parameters. I'll be including the MySQL stuff for expert users who need to work with MySQL databases, but you're on your own when it comes to getting MySQL running on the Z.

Great stuff, I'm really looking forward towards the release.

Regarding MySQL: I'm rather intending to use a database server on the network rather than a MySQL instance running on the Z. Hopefully this is going to work without problems...

Lars