OESF | ELSI | pdaXrom | OpenZaurus | Zaurus Themes | Community Links | Ibiblio

IPB

Welcome Guest ( Log In | Register )

2 Pages V   1 2 >  
Reply to this topicStart new topic
> Creating Python 2.4 Zaurus Image, What extra modules are a "must"
kopsis
post Jan 5 2005, 01:15 PM
Post #1





Group: Members
Posts: 329
Joined: 1-July 04
Member No.: 3,880



I decided the shiny new Cacko 1.22 ROM on my Z needed a shiny new version of Python to go with it, so I'm building up a new Zaurus Python image with Python 2.4. I have the base Python image built and working and it includes all the "standard" Python modules (with the exception of crypt and nis for which I'm still missing libraries) including distutils, encodings, curses, readline, xml, etc.

In addition to the ext2 format, I'll also be releasing this new Python image as a cramfs for folks that want to run off SD (or internal flash) without chewing up so much space. Its not easy for folks to add stuff to the cramfs image on their own, so before I package it up, I'd like to find out if there are any "site-packages" that people think are critical to doing Python on the Z. The obvious ones (to me) are PyQT and pysqlite so I'll be including those. If there's anything else you'd like to see (that won't bloat things up too much), now's the time to ask.
Go to the top of the page
 
+Quote Post
LarsAC
post Jan 7 2005, 01:33 AM
Post #2





Group: Members
Posts: 4
Joined: 31-December 04
Member No.: 6,075



Would the mySQL API for Python fit into this, too?

I have a number of apps at home for which I might want to write a dedicated client for the Z in favor of porting stuff amde for large screens...

Lars
Go to the top of the page
 
+Quote Post
fpp
post Jan 7 2005, 04:00 AM
Post #3





Group: Members
Posts: 188
Joined: 1-February 04
Member No.: 1,647



Does this mean than when using the cramfs version, we would be unable to add packages that install themselves to site-packages in the standard distutil way :
python setup.py --install
... or am I misunderstanding ?
(this is not a problem for me, I'm comfortable with the ext2 image, but I'm not familiar with these fs tricks, so it's just to make sure...)

On a related topic, maybe you've seen this 'Movable Python' project that appeared recently :
http://sourceforge.net/projects/movpy
http://www.voidspace.org.uk/python/movpy

Do you think it might be a useful option for the Z (without the wxPython part obviously), or would it be no smaller than what you've achieved ?
Go to the top of the page
 
+Quote Post
kopsis
post Jan 7 2005, 05:45 AM
Post #4





Group: Members
Posts: 329
Joined: 1-July 04
Member No.: 3,880



QUOTE(LarsAC @ Jan 7 2005, 04:33 AM)
Would the mySQL API for Python fit into this, too?

Perhaps. The source code looks pretty small, I just have to see if I can get it to build.
Go to the top of the page
 
+Quote Post
kopsis
post Jan 7 2005, 05:57 AM
Post #5





Group: Members
Posts: 329
Joined: 1-July 04
Member No.: 3,880



QUOTE(fpp @ Jan 7 2005, 07:00 AM)
Does this mean than when using the cramfs version, we would be unable to add packages that install themselves to site-packages in the standard distutil way


That is correct. A cramfs filesystem is, by nature, "read only". The only way to make changes is mount it, make a complete copy to a writeable filesystem, make the changes there, then use mkcramfs to build a new image from the copy in the writeable filesystem.

I suspect that Python developers will much prefer the ext2 filesystem image, despite the much larger size. With 1GB SD cards going for under $70 these days, 50MB seems a small price to pay smile.gif

QUOTE
On a related topic, maybe you've seen this 'Movable Python' project that appeared recently


I took a look and it's basically trying to create Python filesystem images for Windows platforms. A standard Windows install doesn't have the luxury of things like loop mounting and cramfs drivers so they have to use a different (and more complex) approach. The images they're building are comparable in size to a cramfs image (and are also read-only), so there's little to be gained by going down that path.
Go to the top of the page
 
+Quote Post
clofland
post Jan 7 2005, 11:20 AM
Post #6





Group: Members
Posts: 245
Joined: 30-October 03
Member No.: 813



Sounds great! I've been waiting for 2.4 for the Z before I start using it on my Windows box.

As long as it supports sqlite 3, I'll be happy.

I'll probably use the cramfs version until I run into something I can't do, and then I'll decide how badly I want to switch. smile.gif

One thing I do notice is that every time I start up my Z, it says that the Python file system was not cleanly unmounted. It even does that if I manually stop and start it. I wonder if the script isn't quite right? It doesn't seem to cause any problems though, other than forcing a file system check each time.
Go to the top of the page
 
+Quote Post
fpp
post Jan 7 2005, 01:21 PM
Post #7





Group: Members
Posts: 188
Joined: 1-February 04
Member No.: 1,647



Thanks for both clarifications kopsis !

I'll wait till the new 2.4 ext2 image comes out, and probably the first cacko 1.22 hotfix too, then it'll be flashing time :-)
Go to the top of the page
 
+Quote Post
clofland
post Jan 10 2005, 07:31 AM
Post #8





Group: Members
Posts: 245
Joined: 30-October 03
Member No.: 813



One quick add: Isn't squashfs "better" than cramfs? My understanding is that Cacko ROM 1.22 uses squashfs instead of cramfs, because it makes a smaller faster image.
Go to the top of the page
 
+Quote Post
kopsis
post Jan 10 2005, 09:53 AM
Post #9





Group: Members
Posts: 329
Joined: 1-July 04
Member No.: 3,880



squashfs is a bit better than cramfs, but that would force users to install drivers (which may not even be available for all ROMs). cramfs will work everywhere, and it's a pretty trivial matter for those who want squashfs to turn the cramfs image into a squashfs image (I'll probably throw a short howto on the website).
Go to the top of the page
 
+Quote Post
gasolin
post Jan 10 2005, 02:07 PM
Post #10





Group: Members
Posts: 1
Joined: 3-November 04
Member No.: 5,326



Dear kopsis:

Very appreciate for your python image release.

As I saw this post, I'd suggest to make "palmdb" module preinstalled to read/write/modify Palm PDB and PRC format databases with python.

This module is maintained by rick_price, one of PDesk(the Palm outliner "progect"'s desktop program) developers, and avaliable at sourcefotge
http://sourceforge.net/projects/pythonpalmdb/


The second is cjkcodecs, which contained in python 2.4. To support Chinese, Japenese, and Korean characters. This can make your MP3player display CJK characters in Tags.

doctest (and unittest(opt.)) is one of essential model, too.
http://aspn.activestate.com/ASPN/Cookbook/...n/Recipe/305292
I think this module is helpful for programers to make unit test while writing documents.

And how about add "twisted" module to ease network-enabled program develop curve?

regards
Go to the top of the page
 
+Quote Post
kopsis
post Jan 10 2005, 04:04 PM
Post #11





Group: Members
Posts: 329
Joined: 1-July 04
Member No.: 3,880



The palmdb module sounds like a great idea! I'll look into that one.

The CJK codecs from the standard 2.4 distribution are definitely included, as are doctest and unittest smile.gif I'll have to take a look at the "twisted" stuff. If it's small enough and builds easily, I'll throw it in.

Database wise, pysqlite for sqlite3 is in there. Hopefully there won't be too many folks still wanting to use sqlite2 database files.

I've also built and tested MySQL-python 1.1.8 for MySQL 3.23.49 (which seems to be the latest Zaurus IPK around). It's not perfect ... the client libs default to the wrong directory for the localhost socket and choke if you force them to read my.cnf. But those problems are easy enough to work around with the right "connect" parameters. I'll be including the MySQL stuff for expert users who need to work with MySQL databases, but you're on your own when it comes to getting MySQL running on the Z. If you don't know why you would want to choose MySQL over sqlite, then you probably shouldn't smile.gif

The cramfs image is currently running 8.3MiB and the ext2 version will weigh in at around 35MiB (25MiB + 10MiB free space). That's with the .pyo files stripped out (I don't think too many people use the optimize switch) but all the .py and .pyc files retained. With a little luck I'll be able to release in a couple weeks.
Go to the top of the page
 
+Quote Post
Bertram Wooster
post Jan 11 2005, 03:55 AM
Post #12





Group: Members
Posts: 39
Joined: 18-December 04
From: Germany
Member No.: 5,957



dear kopsis, thanks for all your work!
do you also plan to release a non-image version, which does not necessarily have to stay on sd/cf but can be installed/copied to harddisk (on a c3000) directly?
Go to the top of the page
 
+Quote Post
kopsis
post Jan 11 2005, 05:13 AM
Post #13





Group: Members
Posts: 329
Joined: 1-July 04
Member No.: 3,880



QUOTE(Bertram Wooster @ Jan 11 2005, 06:55 AM)
do you also plan to release a non-image version, which does not necessarily have to stay on sd/cf but can be installed/copied to harddisk (on a c3000) directly?

For all practical purposes, the HD in the SL-3000 is a CF card, so the ext2 image should work from it. In fact, my understanding is that the main storage partition on the internal HD is formated FAT (which doesn't allow symlinks or file permissions) so you'd have to use the ext2 image rather than installing the files directly.

The only thing that may not work is my init scripts to search for the image. That should be easy enough to fix, but I'd need someone with a 3000 to send me the output from running the mount command (as root and without any arguments).

Now, if someone does want to install Python to an ext2/ext3 formatted filesystem (internal memory or an ext2 formatted CF/SD card), it's pretty easy. Just mount the image, copy everything over to the target directory, and then make the appropriate symlinks. If there's demand, I can write up semi-detailed instructions.
Go to the top of the page
 
+Quote Post
Bertram Wooster
post Jan 11 2005, 12:06 PM
Post #14





Group: Members
Posts: 39
Joined: 18-December 04
From: Germany
Member No.: 5,957



it's definitely worth a try, though i'd be thankful for your "semi-detailed instructions"- i'm not new to computers however to linux and especially unix commands are quite unknown to me.
Go to the top of the page
 
+Quote Post
LarsAC
post Jan 11 2005, 12:42 PM
Post #15





Group: Members
Posts: 4
Joined: 31-December 04
Member No.: 6,075



QUOTE(kopsis @ Jan 11 2005, 12:04 AM)
But those problems are easy enough to work around with the right "connect" parameters. I'll be including the MySQL stuff for expert users who need to work with MySQL databases, but you're on your own when it comes to getting MySQL running on the Z.


Great stuff, I'm really looking forward towards the release.

Regarding MySQL: I'm rather intending to use a database server on the network rather than a MySQL instance running on the Z. Hopefully this is going to work without problems...

Lars
Go to the top of the page
 
+Quote Post

2 Pages V   1 2 >
Reply to this topicStart new topic
2 User(s) are reading this topic (2 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 22nd November 2014 - 09:54 PM