OESF Portables Forum
Model Specific Forums => Sharp Zaurus => Zaurus - pdaXrom => Topic started by: Bundabrg on July 20, 2006, 10:34:50 pm
-
Hi All,
I'd like to discuss a few things that I think are basically some inherent weaknesses in pdaXrom, or rather its development philosophy. Please don't get me wrong as I actually quite like pdaXrom and I think there’s been an exceptional amount of good work put into it by the developer(s), and I must also applaud those who package and contribute applications. You just need to look at http://mail.pdaxrom.org (http://mail.pdaxrom.org) to see the amount of good work put in.
However my concern is that the design of pdaXrom currently prohibits its expansibility and has some pretty big risks.
What are some of the big questions that are always asked in the forums? They are things like this: -
- Where do I find the feed for (some older version)?
- Please could someone port app from (newer/older) feed to (older/newer) feed?
- Where can I find (appname)? It used to be here, but its not here any more.
Currently, from my perspective, an image of a version of pdaXrom is released periodically through the efforts of sashz and Laze. These may or may not work with existing feeds (including contributed ones), and generally kind developers on the forums scramble to ensure that packages they've released still work and still have the proper dependencies etc..
My worry is that this process has the following flaws: -
- The 'hit by the bus' issue - What if a core developer disappears/gets sick of the project. What happens to the project? Is it capable of surviving? What happens if sashz is burned out and feels that he is obligated now to keep developing since everyone is depending on him? That’s quite a bit of stress!
- Effort to create a package - There seems to be a lot of effort needed to generate packages. Once generated, people are reluctant to necessarily update these packages unless they are actively using them as they need to almost reproduce the effort to create it in the first place unless they're smart enough to keep good logs or have a good make script.
- Effort to create a public package - Due to the above point, it’s possible that people who could package something for the public may decide against doing it as the effort may only work for the current release of the 'rom'.
- Lost Compiling Information/Source code - How someone packaged something (including patches) seems to be lost. This actually breaks GPL. It also means that should this person be 'hit by a bus' etc.. someone else has to re-work out how to package the app.
- Old Version Maintenance - Old versions. Most distributions will 'backport' fixes to old versions for a period of time. Currently as soon as a release is made in pdaXrom, the old feed is 'frozen' and it used to be lost. I'm talking about non-core feeds here as well (i.e. contributed ones that may/may not with other versions of pdaXrom). No back porting is really done.
- Dependancy Change Issue - If a major library changes (IE, say libc, glibc or something everything depends on), too many contributed packages will break! Same old story, who will go through the steps to repackage them (talking about contributed as well as core feeds)? In fact, it may take a while before someone realises the package doesn't work if it’s not too popular a package.
I really don't wish to see a project like this die because of one of these flaws. Something that most distributions have is the following: -
- Version control with full logging and accessible access to the code. All of it!
- A method of generating a new feed (by compiling everything based on rules/recipes) when new versions that may break old feeds come out
The advantages, especially of the latter point are: -
- Rule based packaging - If I want to package something, I create a recipe/rule for it safe in the knowledge that most likely this same rule/recipe will work to recompile the package for future versions. I can submit this somewhere and know that someone else can run the 'rule' and get a package. This takes lots of things out of the equation
- Effort expended only once - Because I only really have to use the effort once to create the rule/recipe I'm more likely to create packages. If I'm bored, I'll just randomly pick one and create it. Then forever more that package (possibly with minor tweaks to the rule/recipe) will be available.
What I'm describing as most of you may already know is already provided in the OpenEmbedded framework. I realise there has also be friendly (and some unfriendly) rivalry between OpenZaurus and pdaXrom, but I think that no one can disagree that OpenEmbedded does a pretty good job and takes a lot of risk away. I'm not saying this is the only way, but a lot of the work is already done here.
I decided the other day to backup my pdaXrom installation, install OZ/GPE and try out OpenEmbedded to compile things. I haven’t before because its scary learning something new and I'd heard that OpenEmbedded was a bit difficult. I took my time tracking app (Gtimer) that I use (and normally use in a debian chroot) as my challenge to package. It’s a bit of a complicated package in that debian provides quite a big patch for it (the author no longer maintains it), it’s still packaged for the older debian system (IE, the original_tar is a tar of a tar.bz2) and uses libgtk1.2. It took me 30 minutes to work out how to use OpenEmbedded (to compile nano as an example...), then 1 hour to muddle my way through to create a recipe that downloads, extracts, patches, compiles and packages gtimer (and the end recipe is only 14 lines long!!). I can submit this and know that gtimer will always be around now (since the whole feed is compiled for each version), and if I get 'hit by a bus', someone else surely can fix a 14 line file. And someone will KNOW it breaks because the WHOLE feed is compiled every version and non-compiling packages will be flagged. I also don't have to figure out how to create a tool-chain or any of the dependency libraries as they are compiled by OE automatically as part of creating my package.
What I would like to ask is if pdaXrom developers please (please!) could seriously consider using this valuable tool (or an equivalent if it exists). You can (and probably should) have a totally separate 'branch' from OpenZaurus dedicated to the pdaXrom style that people could contribute recipes to... and it also means that probably with minimal tweaking, rules can be take from the '.dev' and '.oz354x' branches and vice versa. It means that you're still 'separate' but that everyone can contribute and maintain what is a very good OS.
What do other developers and users think? This is NOT a flame starter and NOT a this-rom is better than that-rom thread! It is simply my concern that I wish to express!
- Bundabrg
-
I think that post is one of the most rational and balanced I've read here for a while. It correctly sums up the issues and position and puts things very well - nice one!
I'm pleased you got OE working and saw a glimpse of its potential. I suspect OE would value feeback to make the getting started process easier and help in improving the documentation. A new user perspective on that is often useful. Also, please be sure to submit the .bb file for inclusion if you haven't already!
Richard
-
What do other developers and users think? This is NOT a flame starter and NOT a this-rom is better than that-rom thread! It is simply my concern that I wish to express!
[div align=\"right\"][a href=\"index.php?act=findpost&pid=135929\"][{POST_SNAPBACK}][/a][/div]
Bundabrg,
Excellent post! It echos many of my thoughts as a software developer. The OE build system brings a substantial learning curve. You can't just grab an ISO, download a tarball, and ./configure; make a working application. But if you can find time to set it up and get educated, it carries some huge benefits.
I agree that it could be beneficial for pdaXrom to switch to the OE build system. But I find myself asking a more fundamental question as a software developer: why do I still want pdaXrom? That may sound like a troll, but it's not meant to be. When I left the Cacko ROM I chose pdaXrom over OZ for some specific reasons: mainly that the kernel was compatible (mostly) with the Sharp ROM kernel and the development tools (gcc) were compatible with Sharp SDK versions. But lately pdaXrom has been going down the OZ path ... first with soft-float that broke binary compatibility with Sharp ROMs and now with a completely custom 2.6 kernel.
Now, there are very good justifications for adopting those technologies, but it doesn't change the fact that if I'm going to run something that breaks all compatibility with the Sharp ROM, why not just run OZ/GPE? With OZ/GPE I get basically the same functionality plus a bigger developer community, a very sophisticated software build system, and a (relatively) consistant stable release across all Zaurus models.
I think Sashz is a brilliant Linux hacker and I hope he continues his research since his breakthroughs can benefit all Zaurus distros. But as a part time Zaurus software developer, I want to target a stable ROM and not a research project. If you're not a developer, you may have a totally different perspective. In any case, I wish the loyal pdaXrom devs and users nothing but the best. We're all after the same thing ... making our handhelds do what we want them to do and not what some big company tells us they should do
-
I think that post is one of the most rational and balanced I've read here for a while. It correctly sums up the issues and position and puts things very well - nice one!
ehe, just a little fact is missing though, that sashz do use a tool to build pdaxrom
-
i think pdaxrom-builder is much easier to hack than OE..
It dumps much more debug code during compilation..
It can create multiple build versions...
the new u-boot can speed up root or ketnel flashing...
maybe the builder need a bit of clean up
-
i think pdaxrom-builder is much easier to hack than OE..
It dumps much more debug code during compilation..
It can create multiple build versions...
the new u-boot can speed up root or ketnel flashing...
maybe the builder need a bit of clean up :)
[div align=\"right\"][a href=\"index.php?act=findpost&pid=136021\"][{POST_SNAPBACK}][/a][/div]
This may be true for quick "get it running" packages, but for a distribution having a system in place to automatically deal with packaging, dependencies, toolchain building, etc, is nice to have.
OE doesn't dump all output to console, this is true, but you normally don't need all that. OE *does* store the output for all commands run in handy log files which you can easily peruse after the fact (or even tail -f while the build is running, I've done this many times).
-
It can create multiple build versions...
[div align=\"right\"][a href=\"index.php?act=findpost&pid=136021\"][{POST_SNAPBACK}][/a][/div]
That was one of OE's design goals, and it can do it perfectly.
-
i think pdaxrom-builder is much easier to hack than OE..
[div align=\"right\"][a href=\"index.php?act=findpost&pid=136021\"][{POST_SNAPBACK}][/a][/div]
It probably is ... but serious software devs don't want hacks. If I hack up something to build in that environment, how much confidence do I have that it will build in the next version? How easy is it for me to get it committed to the software baseline so that it will be automatically built by someone other than me every time a new ROM version is released? How much do I have to tweak my "hacks" when a new upstream release comes out?
For me this looks to be a case where investing a little extra time up-front will save me a lot of time down the road. Can pdaXrom builder eventually get to that point? Sure. Does it make sense to reinvent something that already exists? Maybe for the entertainment/education/professional development of the person doing it, but that's a non-transferrable justification.
-
What do other developers and users think? This is NOT a flame starter and NOT a this-rom is better than that-rom thread! It is simply my concern that I wish to express!
[div align=\"right\"][a href=\"index.php?act=findpost&pid=135929\"][{POST_SNAPBACK}][/a][/div]
Bundabrg,
-------- snip 8< ---------------
I agree that it could be beneficial for pdaXrom to switch to the OE build system. But I find myself asking a more fundamental question as a software developer: why do I still want pdaXrom? That may sound like a troll, but it's not meant to be. When I left the Cacko ROM I chose pdaXrom over OZ for some specific reasons: mainly that the kernel was compatible (mostly) with the Sharp ROM kernel and the development tools (gcc) were compatible with Sharp SDK versions. But lately pdaXrom has been going down the OZ path ... first with soft-float that broke binary compatibility with Sharp ROMs and now with a completely custom 2.6 kernel.
Now, there are very good justifications for adopting those technologies, but it doesn't change the fact that if I'm going to run something that breaks all compatibility with the Sharp ROM, why not just run OZ/GPE? With OZ/GPE I get basically the same functionality plus a bigger developer community, a very sophisticated software build system, and a (relatively) consistant stable release across all Zaurus models.
------- snip 8< --------------------
[div align=\"right\"][a href=\"index.php?act=findpost&pid=136018\"][{POST_SNAPBACK}][/a][/div]
I don't wish to debate OS's in this thread, and I know you are not either. One could say why 'Debian' and 'Ubuntu'? Why Redhat and Fedora? Why slackware?
The simple fact is that there ARE lots of people using pdaXrom, and lots of people using OZ/GPE. And whilst both projects are active (using whatever builder), there always will be.
As a developer, it really doesn't matter what OS you run. The users will choose their OS _they_ like, and by god they will defend it to the death ;-) And I respect that!
However, what it boils down to really is, as a developer, when I add a package, I would prefer that same package to ultimately be useful for more OS's than less. It shouldn't matter what OS I use, but if I create a package, sure it's not going to necessarily be binary compatible on different OS's (Try ubuntu debs versus debian debs. You reach dependency hell quite quickly), BUT how the package is created being standard would be extremely useful. Thats why I advocate OE as the .bb recipe files could VERY easily be used with minor-to-no modifications between dists, BUT the dists themselves could be quite seperate. It also means MUCH less duplication of work so BOTH (or more if anyone creates a new) dists can grow faster.
I do wish to add though that though I mention OE, my main aim is to raise awareness of getting rid of the risks in the first post. I myself don't know the politics of OE itself and that may be a factor, but it seems to me that it would be relatively simple to create a new branch or if there is politics in creating the new branch (I don't know who hands out the keys to where it is currently hosted ;-) then it seems that it would be relatively simple to duplicate OE to a different server and build the branch there.
- Bundabrg
-
I think that post is one of the most rational and balanced I've read here for a while. It correctly sums up the issues and position and puts things very well - nice one!
ehe, just a little fact is missing though, that sashz do use a tool to build pdaxrom
[div align=\"right\"][a href=\"index.php?act=findpost&pid=136019\"][{POST_SNAPBACK}][/a][/div]
I wasn't aware of that tool. I'll investigate further. If it mitigates the risks I've posted then thats great, and if it allows one to create package creation dependencies etc, then excellent.
I just assumed everyone just used the native or the cross-tool-chain to create packages, which loses how the package was created, what patches, the source code etc...
- Bundabrg
-
Dear Bundabrg, koen, knopsis and papercrane,
I found it AMUSING that you guys are always hanging around in Pdaxrom
discussion talking about how great OZ is. OZ is no doubt, great, but why
would you guys try to push it so much around here ??
And you guys, when given a chance, always bashs pdaxrom.
It seemed that OZ is so well developed that you guys have so much time ??
BR,
Felix.
-
i think pdaxrom-builder is much easier to hack than OE..
[div align=\"right\"][a href=\"index.php?act=findpost&pid=136021\"][{POST_SNAPBACK}][/a][/div]
It probably is ... but serious software devs don't want hacks. If I hack up something to build in that environment, how much confidence do I have that it will build in the next version? How easy is it for me to get it committed to the software baseline so that it will be automatically built by someone other than me every time a new ROM version is released? How much do I have to tweak my "hacks" when a new upstream release comes out?
[div align=\"right\"][a href=\"index.php?act=findpost&pid=136199\"][{POST_SNAPBACK}][/a][/div]
I think you hit it right on the head with this part of the quote: -
How easy is it for me to get it committed to the software baseline so that it will be automatically built by someone other than me every time a new ROM version is released?
Having a package compiled, by someone else who is not necesarilly familiar with the package, or automatically on each new release is very important. As I mention in my first post, it means that as a developer I'm more inclined to package it in the first place since I know it will live longer and if I get sick of the whole thing, someone else doesn't have to go through what I did to create the package in the first place.
- Bundabrg
-
Dear Bundabrg, koen, knopsis and papercrane,
I found it AMUSING that you guys are always hanging around in Pdaxrom
discussion talking about how great OZ is. OZ is no doubt, great, but why
would you guys try to push it so much around here ??
And you guys, when given a chance, always bashs pdaxrom.
It seemed that OZ is so well developed that you guys have so much time ??
BR,
Felix.
[div align=\"right\"][a href=\"index.php?act=findpost&pid=136203\"][{POST_SNAPBACK}][/a][/div]
Hi Felix,
I actually run pdaXrom so theres a good reason I'm in this forum ;-)
- Bundabrg
-
OE is great however i have noticed that its main problems stem from people building distros straight from the package sources
this might sound strange howeveras i see it the OE tools are supposed to help you build the programs yourself (and package them, more on this latter) i have tried to set it up a couplo of times and failed so you please try and disprove some of my comments
the problem i see is that the programs are packaged with no formal set of rules. there is a bit of varince in the packages, for exemple some programs compiled with everything turned on and shiping with config files while others are compiled with the bare minimum and no config files at all
there also seems to be a bit of a reliance on the auto magic to make things work but i can understand that due to the size of the user base not everything gets tested and the things that do get tested are normally with the defalt config in a graphical enviroment, not on the commmand line (i dont run X so i have seen some of these errors)
OE and bitbake are halfway between retargetable package/compiling program and distro. and the distros that are bassed of it are not using it correctly in my opinon. this is partially because of the diffrent styles of distros that are rellying on it (openslug.org, openwrt.org and OZ, i have 2 openwrt routors, 2 NSLU "openslugs" and a Z) some of these only have 8MB of flash and 16MB of ram so you can see how this contrasts to the Z's of wihch most have 64MB of ram and most have 64MB of storage
there really needs to be a set standard for packaging and having OE turn itself into a distro in its own right, OR make it retargetable to have a "high" and "low" end target that slightly changes the compiling process, i assume it has somthing like gentoos make flags, (as i said eariler i neven got it to work properly) if so perhaps that should be given more enfisis along with a standard guide and template to packaging
the best thing to happen to it would be if a distro like debian picked it up (now i am dreaming) as you would slowly get alot of packages and some sort of packaging guidelines
after reading that again i feel its all over the place, and it is but i belive some of the points are valid. btw i used to run pdaXrom until OZ got a 2.6 kernel on the spitz, after seeing the overlay support i may have to switch back
-
Note also that this particular subject has come up a couple of times here:
https://www.oesf.org/forums/index.php?showtopic=20021 (https://www.oesf.org/forums/index.php?showtopic=20021)
https://www.oesf.org/forums/index.php?showtopic=19923& (https://www.oesf.org/forums/index.php?showtopic=19923&)
https://www.oesf.org/forums/index.php?showtopic=19146&st=15 (https://www.oesf.org/forums/index.php?showtopic=19146&st=15)
(from a quick search on "openembedded" in the pdaxrom section, I'm pretty sure there are others)
-
well.. the fastest way to end this is to close this thread...
If you think OE fit u better than use OE.
If you think pdaxrom-builder fit u better than use pdaxrom-builder..
Why do you want to compare... which one is better..
We should work together and make programs to run smoothly (fitted nicely) in our Z, whether it is OZ or pdaxrom.
By the way, Sashz already open up for all who want to contribute..
-
Dear Bundabrg,
Sorry for the mentioning in the post.
I just found it very tiring in reading so many posts in comparing
OE and Pdax that I did not recognize.
BR,
Felix.
-
OE is great however i have noticed that its main problems stem from people building distros straight from the package sources
there really needs to be a set standard for packaging and having OE turn itself into a distro in its own right, OR make it retargetable to have a "high" and "low" end target that slightly changes the compiling process, i assume it has somthing like gentoos make flags, (as i said eariler i neven got it to work properly) if so perhaps that should be given more enfisis along with a standard guide and template to packaging
I think you're misunderstanding how OE works and how distros use it. It seems you assume that every recipe generates the same package regardless of distro. That isn't true. You can have different patches per distro, different configure options per distro, different packaging per distro and different config files per distro. You can override anything you want per distro, per machine or both.
For example:
DEPENDS = "gtk+"
SRC_URI = "http://foo.com/bar-${PV}.tar.bz2"
SRC_URI_append_dablitzleetdist = "http://dablitz.com/extraleetness.patch;patch=1"
SRC_URI_append_pdax = "http://obscureurl.com/uglyhacks.patch;patch=1"
inherit autotools
EXTRA_OECONF_openzaurus = " --disable-leetnes"
EXTRA_OECONF_dablitzleetdist = " --enable-leetnes"
EXTRA_OECONF_pdax = " --only-work-at-full-moon"
EXTRA_OECONF_append_collie = " --enable-tinyscreenresolutions"
EXTRA_OECONF_append_akita = " --enable-OMGleetoverlayLOL"
CONFFILES_append_dablitzleetdist = "${sysconfdir}/leetnes.conf"
The above shows how one recipe produces different output for three distributions
and two machines. For the uninitiated: the above will download, patch, build, packages *and* add all dependencies to the resulting .ipk with a mere 'bitbake bar'.
Can you still say with a straight face it isn't retargetable?
the best thing to happen to it would be if a distro like debian picked it up (now i am dreaming) as you would slowly get alot of packages and some sort of packaging guidelines
Check the video of wookey's emdebian presentation at debconf2005[1]
And for the whiners in this thread: OE people will respond if you talk about OE, like pdaX people will respond if you talk about pdaX, get over it.
Pgas: I hope this is what you meant with 'show us examples instead of saying how great it is'
[1] http://meetings-archive.debian.net/pub/deb...emo-Wookey.mpeg (http://meetings-archive.debian.net/pub/debian-meetings/2005/debconf5/mpeg/2005-07-11/05-Embedded_Debian_Demo-Wookey.mpeg)
-
well.. the fastest way to end this is to close this thread...
If you think OE fit u better than use OE.
If you think pdaxrom-builder fit u better than use pdaxrom-builder..
Why do you want to compare... which one is better..
We should work together and make programs to run smoothly (fitted nicely) in our Z, whether it is OZ or pdaxrom.
Fortunately my profession gives me the chance to look at things from a deep technical and a user perspective.
The thing is that redundancy is not a very good way of creating software.
One might say that competition is good- however if competion is applied in a very small market like ours it ends up with going nowhere at all ind ht elong run because have to decide between two average products instead of a rather good one.
From what I have seen pdaxrom has created a good user environment with many interesting applications and a desktop like feeling. This environment is lacking with OE based distributions. Sure there is OPIE, but regarding new applications this looks like a dead end road without X capabilities though the existing applications are the most usable. There is GPE but it lacks the feeling for completeness and usability, enlightenment is complicated and bloated though powerful and fluxbox is well done but not finished without tweaking it manually. Those are lacking the overall feeling of an encapsulated environment pdaxrom seems to provide.
On the other hand there is OE with the possibility to almost automatically address a whole lot of distributions and devices and which makes it possible integrate and manage new software with minimum effort amongst a lot of other features.
OE could well be used to create its very own pdaxrom distribution but without loosing compatibility with OZ. I can see a lot of people wasting a whole lot of efforts by redundantly creating the same thing: A mobile platform suited for daily usage. People which are not competing but engaged in some sort of rivalry and I would like to tell people to stop it. In fact this leads knowhere.
Joining efforts would advance things a whole lot more
Oliver