confused new zaurus owner

[Anonymous]

Hi List,

I have just become the proud owner of a new Zaurus 750 from Shirtpocket, and I\'m very new to the whole linux thing though willing to give it all a go. I have read all about flashing the rom, and think I will probably do that at some point but would like to stick with the out-of-the-box version until I get used to it. So, it says in system info that it\'s running:
Qtopia version 1.5.4, lunux kernel 2-4.18-mk7-embedix, Sharp rom 1.10JP

I tried to install some programmes on it that seemed to be right, for instance qpdf (http://www.softforge.de/qpdf/qpe-qpdf_1.5.0-20020618_arm.ipk ), but while they showed up as packages they didn\'t appear in the package installer and I couldn\'t get them to install using the command line. (They were in the installed_files folder)

I\'ve read lots of posts about checking that they are the right sort of package, and as far as I could tell they were, but now I\'m totally confused by it all! Could anyone point me in the direction of an app that verifiably works with the rom that I have so I can check I\'m doing this right, or tell me what I\'m doing wrong?

Many thanks,

totaln00b

[jchung]

that\'s weird.. that package reader should have recongized that as a package.

Try qpdf2 http://www.killefiz.de/zaurus/showdetail.php?app=1228

[Abichai]

I have encountered similar problems on my C750 (which still has is Sharp ROM). Some packages do appear in the installer while others justdon\'t...
May it have to do with some compatibility of the given ipkg withthe system? (libraries version, kernel version, etc...)
Any idea?
Something else bothers me: Am I silly or is the /etc/ part of a read-only filesystem? I wanted to add some library but cudn\'t run ldconfig???
Any idea fo you power-user guys? My Linux skills ain\'t definitely as good as I wud them to be!

[rikiya]

I can\'t really help but when i tried installing the qpdf it showed as uninstalled so I installed it but I oculdn\'t find it in the applications place but when i uninstalled it and reinstalled it it was there. Also i think it would help if you installed it then you turn off the z on the back button and then turn on again.

[lardman]

You might need to restart Qtopia (from the shutdown applet).

Si

[Anonymous]

Hi list again,

Thanks for the help. Aah, it\'s a learning experience, this. For some reason my win2K machine at work was saving all downloaded ipk files and adding a gz extension to them. In my ignorance I simply gunzipped them and put the resultant file onto my zaurus, which then didn\'t recognise them. My winXP machine at home doesn\'t add the extension, so the files work just fine when I try and install them. I shall be having words with my win2K machine... :wink:

[Stubear]

The gui installer on the C7x0 (at least the 750 and 760 probably also 700 and 860 too) attempts to unpack the ipk to get the version info etc from the control.tar.gz file within.

From my experience there are 4 things that will stop a package from showing up in the install files list

1) ipk are not in /mnt/card/Documents/Install_Files, /mnt/cf/Documents/Install_Files or /home/zaurus/Documents/Install_Files
2) ipk is an ar compressed ipk rather than tgz ipk - there are 2 type of ipk and Sharp install er supports only the older tar gzip style ones
3) Web browser renames ipk to ipk.tar - this occasionally happens to me when using IE (one of the reasons I don\'t use IE if I can avoid it). rename the ipk to just be <package>.ipk rather than <package>.ipk.tar
4) Incomplete download or corrupt download or ipk - very rarely the package just get broke when downloading, try downloading again

You can test 2) and 4) by renaming the <package>.ipk to <package>.ipk.tgz - if you have winzip installed then the icon will change to the winzip icon and you will be able to open the ipk in winzip by double clicking - it will ask if you want to open the tar file (say yes). There should be 3 files in the ipk (debian-binary, control.tar.gz, data.tar.gz). If it doesn\'t open then you have problem 2) or 4) - my bet is on 2) from experience.

If you are using a real OS rather than windows () then you can just run \"file\" on the ipk and it will tell you whether its a gzip tarball (sharp style) or debian package (ar compressed)

Stu

[Miami_Bob]

Stubear & all -

Actually, its very easy to tell the \'good\' \"Sharp\" from the \'ungood\' \"OPIE\" IPKs, no matter what OS or platform you are using.

Both are identical through the first 2 steps.

1) Tar ball files & directories to control.tar & data.tar

2) Gzip the Tar balls to control.tar.gz & data.tar.gz then create the debian-binary one line text file.


The \"Sharp\" IPKs are made by rolling all 3 of those into another Tar ball, then Gziping *that* Tar and giving the .tar.gz file an IPK extension.

We call them TGZ for convienience, but they are still a Tar file package archive inside a Gzip compression archive.

Since the outer \"wrapper\" is a Gzip, they will allways start with the two hex bytes (1F 8B), which is the header signature (\"magic number\") that identifies a Gzip file.


The \"OPIE\" IPKs are made by taking the TGZ files from step 2 and wrapping them in an AR packaging archive layer. The signature header - \"magic number\" for the IPKG version of AR files is:

 (hex)      21 3C 61 72 63 68 3E 0A
 (ACSII)    !  <    A  R   C  H  >


So a hex editor or viewer will tell you what type of IPK you have no matter what the extension, OS or other criteria you are working under.

It gets more complicated, in truth, like DEB and AR-IPK files not really being the same format exactly even though they are *supposed* to both be AR files, for example, but the above rules hold in every case that I\'v seen where the IPKs were not bunged in transfer. Over 200 IPKs examined so far.


On my C860, I\'v found that the GUI \"Add / remove Software\" app does not even look at files unless they have IPK extensions. Rename a \"good\" IPK to IP and it does not show on the installer list.

Hide an IPK behind the \".\" or below a directory path hidden with a \".\", and the IPK also \"disappeares\". More interesting is that having a LOT of IPK files, especially the AR-IPKs, causes the \"Add / remove\" app to take an extremely long time crunching the list.

It seems that if a file has the IPK extension, the GUI app checks to see if it really IS a \"Sharp\" TGZ IPK, which takes time.

Not to be contentious, but I have put IPKs all over my CF & SD cards, as well as the Internal Flash, and the C860 finds them no matter WHERE they are hidden. With the \".\" exceptions stated above. Other Zs may be different here.

On a WhenDose machine, 7-Zip will unwrap the AR-IPKs & TGZ-IPKs both. Since the permissions, symbolic links & so on are \"covered\" by the *first* Tar packaging, working under Win (or OS/2, CP/M etc) doesn\'t seem to cause any problems in \"repackaging\" the IPK.

When I first got my Z, this IPK crud drove me up a wall. And no FAQ or thread was *really* clear and comprehensive about what was going on at the nuts & volts levels, and WHY. So, I started to research the whole issue in depth.

I\'m still discovering little \"got\'chas\" in the whole DPKG - IPKG complex (such as critical file order and termination of file names with the \"/\", for examples) but telling the two \"flavors\" of IPK apart is cake if you just check the Hex.

BTW - MC will fool you. If you try to view a .tar.gz file, MC recognizes the outer most GZ layer and decompresses it to the TAR *before* showing the contents. Unless you turn Parse \'off\'. <G>

I\'m working on a more detailed FAQ on the subject - \"Why Johnny Z Can\'t Read (That IPK File)\". Maybe keep some other newbies from going as nutso as *I* originally did, trying to figure this out.

WHY couldn\'t they just have given the \"new\" IPKs a slightly different extension??? This is exactly the kind of unnecessary obsfucation that turns a lot of folks off Linux, IMHO.


Bob W
Miami FL

[Miami_Bob]

totaln00b -

  << For some reason my win2K machine at work was saving all downloaded ipk files and adding a gz extension to them. >>

  <<  My winXP machine at home doesn\'t add the extension, so the files work just fine when I try and install them. I shall be having words with my win2K machine... >>


Welllllll. actually your W2K box IS telling you the \"truth\", since the \"IPK\" really *is* a \'renamed\' GZIP file. <G> I think this is more of a server side error in the d/l process.

The computer is ALWAYS \"right\", even when we meat machines don\'t see the logic behind the confusion.

First generation cyborgs unite! You have nothing to lose except your wetware! <G>

Bob W
Miami FL

[Mickeyl]

Just for the records... it\'s not the \"OPIE\" format, it\'s the new ipkg format of the projects: familiar linux, OpenSIMpad, OpenZaurus, and OpenEmbedded.

[Anonymous]

Hello Mickey! Glad you commented. Perhaps you can clear up some things for me & any lurkers?

The www.handhelds.org/z/wiki/iPKG file, \"Itsy Package Managment System\", is writted in single first person tense but has multiple authors listed at the end (ie: Chris Thompson <critta>, Carl Worth, Jonathan Chetwynd, \"previous content by Colin Marquardt <stale items removed>\" & Jamey Hicks).

Who IS the alpha of this bunch? Who is active & who background? Who is it that speaks \"first person\" here & can give straight answers to \"geeks\' gotta know\"?

I particularly like the, IMHO, extremely ironic part about how \"I first made the .IPK format equivalent to .DEB (but renamed to avoid frustration and confusion).\" Great idea, no? Note \"equivalent\" instead of \"identical\", too.

Also have real questions about, \"However *I* (emphasis RW) recently changed it to be a tar.gz file rather than an ar archive so that ar would not have to be installed to use ipkg. But with 0.99 *we* [emphasis again RW] changed it back because gzipping compressed data is a waste of CUP & battery).\" Who are *I* & who is *we* here?

After all, it would have been a LOT less confusing to make the top wrapper a TAR instead of AR, so I don\'t think that I can accept the statement as it stands. AR isn\'t even available on some *big* box Linux distros as native install. You have to get it from binutils. There are multiple, incompatible AR file formats. AND there is no easily available equivalent for other OSs. Smells in Denmark, IMHO.

The details of the Gzip format specifications are readily available, in RFC 1952 and elsewhere. The TAR specifications are also easily found. Likewise their \"magic numbers\". But, every post on the Debian fora that I have seen requesting the same info for the paricular AR use for dpkg (and ipkgs?) gets blown off with vague references to \"its in the info in the developers section\".

More specific inquires for more exact citations get smokescreened. All that I have been able to find anywhere is that the \"magic number\" is !<arch> and info about the higher level build files (debian-binary, control.tar.gz & data.tar.gz). Nothing at all about the acutal hex field formats.

Since there are well known multiple incompatible AR file lower level formats, this is far from a trivial question. Even Ian Jackson, in a 2002 Debian Planet interview, admitts:

\"The kind of ar used is a very very old format that every modern ar understands as well as all old ones, but you have to generate it with dpkg-deb, because the new ar\'s want to generate funky new formats :-).\"

which is NOT really accurate sonce the \"really old\" AR formats did NOT use any of the various !<arch> varriants of the \"magic numbers\". The !<arch> varriant seems to have originated with BSD in the 1990\'s, from info that I have found. So \"old\" AR would not recognize it at all. And Solaris looks for !<arch>/n to recongize its ARs, it seems. And so forth. \"Funky new formats\"?? The .deb - .ipk AR seems to BE one of the newest \"funky\" formats in town!

One comment to this interview tjat I found interesting was:

\"Well - not quite the consequences of using ar, but really the consequence of writing a program that wrote out the payload in \'ar\' format. What were you thinking Ian?

Couldnt you have just used \'ar\' itself to write the deb out instead of producing non-portable code? No - I am not referring to porting to Linux on multiple hardware architectures but on porting Debian to other operating systems.

Plus it would have simplified the code so much to just use \'ar\' itself rather than duplicating what it did.\"

HUMMMMMM?

Anyhow, I have a LOT of questions about this subject, Mickey. Who has the expertise to give straight answers to them, in your opinion?

Oh, I originally started using the term \"OPIE\" for these IPKs instead of OZ, as some did, because OPIE is the user interface that runs atop OZ (if I understand corrctly) so IPK manipulation would be an OPIE related issue instead of an OZ one.

And, true, \"familiar linux, OpenSIMpad, OpenZaurus, and OpenEmbedded\" may be joint members in this project, BUT it is the OPIE team (?) decision to use the format for Zaurus while retaining the same extension that has created so much \"frustration & confusion\" HERE. Right?

However, to be more accurate & to avoid joining the blame game, I have already started changing my terms to TGZ-IPK & AR-IPK. Wouldn\'t we ALL have been better off if OPIE folks had done something similar??

BTW - where are the notes for these decisions documented? Google signal to noise for these searches is abysmally low for some reason.

THANKS!

Bob W
Miami FL

\"Data WANT to be free!\"

[Miami_Bob]

Sorry, all. That last long post was mine but the system logged me off while it was being written <yeh, long winded - I know G>.

Bob W
Miami FL

[Mickeyl]

Hi,

the best source of information is actually the cvs repository. There you see who did what - and when. For example head over to http://cvs.handhelds.org/cgi-bin/viewcvs.c...ar/dist/ipkg/C/

Jamey, Florian and Pigi are the authorities in ipkg-land right now and they respond to entries in the familiar bugzilla or to questions on the various mailing lists. I consider familiar@handhelds.org to be the appropriate list for ipkg issues.

I\'m afraid I can\'t answer any details about ipkg since I am just a user of it as well. However I can correct some things here:

Oh, I originally started using the term \"OPIE\" for these IPKs instead of OZ, as some did, because OPIE is the user interface that runs atop OZ (if I understand corrctly) so IPK manipulation would be an OPIE related issue instead of an OZ one.

No. All Opie packagemanagers (as well as the Qtopia/Sharp ones btw.) rely on calling the ipkg binary which does the real work. Hence, Opie really doesn\'t handle packaging, we just have a more or less nice frontend.

And, true, \"familiar linux, OpenSIMpad, OpenZaurus, and OpenEmbedded\" may be joint members in this project, BUT it is the OPIE team (?) decision to use the format for Zaurus while retaining the same extension that has created so much \"frustration & confusion\" HERE. Right?

No. Opie is a GUI environment and as that the Opie team tries to stay out of distribution business. Unfortunately, since \"our\" main distributors (familiar and OpenZaurus) are releasing less often than we\'d like, we sometimes have to package ipkgs ourselves.

When that happens, we (of course) follow the package format decisions made by the main distributors because we want our packages to be compatible.

However, to be more accurate & to avoid joining the blame game, I have already started changing my terms to TGZ-IPK & AR-IPK. Wouldn\'t we ALL have been better off if OPIE folks had done something similar??

My opinion on that is we would\'ve been better off if Sharp weren\'t reluctant to upgrade their ipkg to a version which handles more than just the ancient format. (That goes for a number of things btw., i.e. kernel, gcc, glibc, busybox, qt-embedded, etc.)

BTW - where are the notes for these decisions documented? Google signal to noise for these searches is abysmally low for some reason.

No idea, I guess the familiar folks decided that after discussions happened on #familiar (irc.freenode.net).

[lardman]

You make some interesting points about using ar.

IMO and as stated by the ipkg maintainers, the outer wrapper shouldn\'t be tar.gz as it is space and processor ineffifient to compress compressed files.

However is there a reason for not using tar rather than ar as the outer wrapper? Persumably this goes back to whomever invented the .deb format for debian.

About not having ar available - I was under the impression that static libraries (*.a) are ar archives, so presumably most distros would normally have ar available?

I agree with the Windows thing though, it\'s a real pain not having an implementation of ar on Windows.

I also agree with Mickeyl, why of why couldn\'t Sharp have updated ipkg when they released the 3.xx (5500) ROMs and all of the ROMs on the new machines? By the time they\'d got round to this the writing was on the wall in terms of ar type ipks being better and also there being lots of them around to frustrate standard ROM users.


Si

[Miami_Bob]

Mickey & lardman -

Thanks for the feedback, guys. My comments are intended to be constructive and to arrive at \"real\" facts. So input like yours, from all \"sides\", is extremely welcome!

Mickey\'s points are well taken. I have a lot of technical, academic & scientific experience, but there is a real high angle learning curve for ANY newbie in areas like the Z World, where there simply are no comprehensive \"textbooks\". Gathering data is like pulling molars from a bobcat <G>.

So, if I mistate factoids, or if there are conflicting opions, don\'t hesitate a second on telling me. Thats one of the best ways to learn quickly. I may not *agree* with what is said, but I *do* try to listen & give due consideration.

Suggest strongly that a comprehensive FAQ be set up for this subject alone since it is a question so often asked, and answered only in fragments at best.



Happy Monday, all <G>


Bob W
Miami FL