Author Topic: Pdaxrom Development  (Read 4255 times)

Bundabrg

  • Full Member
  • ***
  • Posts: 183
    • View Profile
Pdaxrom Development
« on: July 20, 2006, 10:34:50 pm »
Hi All,

I'd like to discuss a few things that I think are basically some inherent weaknesses in pdaXrom, or rather its development philosophy. Please don't get me wrong as I actually quite like pdaXrom and I think there’s been an exceptional amount of good work put into it by the developer(s), and I must also applaud those who package and contribute applications. You just need to look at http://mail.pdaxrom.org to see the amount of good work put in.

However my concern is that the design of pdaXrom currently prohibits its expansibility and has some pretty big risks.

What are some of the big questions that are always asked in the forums? They are things like this: -
  - Where do I find the feed for (some older version)?
  - Please could someone port app from (newer/older) feed to (older/newer) feed?
  - Where can I find (appname)? It used to be here, but its not here any more.

Currently, from my perspective, an image of a version of pdaXrom is released periodically through the efforts of sashz and Laze. These may or may not work with existing feeds (including contributed ones), and generally kind developers on the forums scramble to ensure that packages they've released still work and still have the proper dependencies etc..

My worry is that this process has the following flaws: -
  - The 'hit by the bus' issue - What if a core developer disappears/gets sick of the project. What happens to the project? Is it capable of surviving? What happens if sashz is burned out and feels that he is obligated now to keep developing since everyone is depending on him? That’s quite a bit of stress!
  - Effort to create a package - There seems to be a lot of effort needed to generate packages. Once generated, people are reluctant to necessarily update these packages unless they are actively using them as they need to almost reproduce the effort to create it in the first place unless they're smart enough to keep good logs or have a good make script.
  - Effort to create a public package - Due to the above point, it’s possible that people who could package something for the public may decide against doing it as the effort may only work for the current release of the 'rom'.
  - Lost Compiling Information/Source code - How someone packaged something (including patches) seems to be lost. This actually breaks GPL. It also means that should this person be 'hit by a bus' etc.. someone else has to re-work out how to package the app.
  - Old Version Maintenance - Old versions. Most distributions will 'backport' fixes to old versions for a period of time. Currently as soon as a release is made in pdaXrom, the old feed is 'frozen' and it used to be lost. I'm talking about non-core feeds here as well (i.e. contributed ones that may/may not with other versions of pdaXrom). No back porting is really done.
  - Dependancy Change Issue - If a major library changes (IE, say libc, glibc or something everything depends on), too many contributed packages will break! Same old story, who will go through the steps to repackage them (talking about contributed as well as core feeds)? In fact, it may take a while before someone realises the package doesn't work if it’s not too popular a package.


I really don't wish to see a project like this die because of one of these flaws. Something that most distributions have is the following: -
  - Version control with full logging and accessible access to the code. All of it!
  - A method of generating a new feed (by compiling everything based on rules/recipes) when new versions that may break old feeds come out

The advantages, especially of the latter point are: -
  - Rule based packaging - If I want to package something, I create a recipe/rule for it safe in the knowledge that most likely this same rule/recipe will work to recompile the package for future versions. I can submit this somewhere and know that someone else can run the 'rule' and get a package. This takes lots of things out of the equation
  - Effort expended only once - Because I only really have to use the effort once to create the rule/recipe I'm more likely to create packages. If I'm bored, I'll just randomly pick one and create it. Then forever more that package (possibly with minor tweaks to the rule/recipe) will be available.

 
What I'm describing as most of you may already know is already provided in the OpenEmbedded framework. I realise there has also be friendly (and some unfriendly) rivalry between OpenZaurus and pdaXrom, but I think that no one can disagree that OpenEmbedded does a pretty good job and takes a lot of risk away. I'm not saying this is the only way, but a lot of the work is already done here.

I decided the other day to backup my pdaXrom installation, install OZ/GPE and try out OpenEmbedded to compile things. I haven’t before because its scary learning something new and I'd heard that OpenEmbedded was a bit difficult. I took my time tracking app (Gtimer) that I use (and normally use in a debian chroot) as my challenge to package. It’s a bit of a complicated package in that debian provides quite a big patch for it (the author no longer maintains it), it’s still packaged for the older debian system (IE, the original_tar is a tar of a tar.bz2) and uses libgtk1.2. It took me 30 minutes to work out how to use OpenEmbedded (to compile nano as an example...), then 1 hour to muddle my way through to create a recipe that downloads, extracts, patches, compiles and packages gtimer (and the end recipe is only 14 lines long!!). I can submit this and know that gtimer will always be around now (since the whole feed is compiled for each version), and if I get 'hit by a bus', someone else surely can fix a 14 line file. And someone will KNOW it breaks because the WHOLE feed is compiled every version and non-compiling packages will be flagged. I also don't have to figure out how to create a tool-chain or any of the dependency libraries as they are compiled by OE automatically as part of creating my package.

What I would like to ask is if pdaXrom developers please (please!) could seriously consider using this valuable tool (or an equivalent if it exists). You can (and probably should) have a totally separate 'branch' from OpenZaurus dedicated to the pdaXrom style that people could contribute recipes to... and it also means that probably with minimal tweaking, rules can be take from the '.dev' and '.oz354x' branches and vice versa. It means that you're still 'separate' but that everyone can contribute and maintain what is a very good OS.

What do other developers and users think? This is NOT a flame starter and NOT a this-rom is better than that-rom thread! It is simply my concern that I wish to express!


 - Bundabrg
[img]http://www.worldguard.com.au/brendan-ambi.png\" border=\"0\" class=\"linked-sig-image\" /]

rpurdie

  • Newbie
  • *
  • Posts: 26
    • View Profile
Pdaxrom Development
« Reply #1 on: July 21, 2006, 11:06:13 am »
I think that post is one of the most rational and balanced I've read here for a while. It correctly sums up the issues and position and puts things very well - nice one!

I'm pleased you got OE working and saw a glimpse of its potential. I suspect OE would value feeback to make the getting started process easier and help in improving the documentation. A new user perspective on that is often useful. Also, please be sure to submit the .bb file for inclusion if you haven't already!

Richard

kopsis

  • Sr. Member
  • ****
  • Posts: 329
    • View Profile
    • http://kopsisengineering.com
Pdaxrom Development
« Reply #2 on: July 21, 2006, 12:40:59 pm »
Quote
What do other developers and users think? This is NOT a flame starter and NOT a this-rom is better than that-rom thread! It is simply my concern that I wish to express!
[div align=\"right\"][a href=\"index.php?act=findpost&pid=135929\"][{POST_SNAPBACK}][/a][/div]

Bundabrg,

Excellent post! It echos many of my thoughts as a software developer. The OE build system brings a substantial learning curve. You can't just grab an ISO, download a tarball, and ./configure; make a working application. But if you can find time to set it up and get educated, it carries some huge benefits.

I agree that it could be beneficial for pdaXrom to switch to the OE build system. But I find myself asking a more fundamental question as a software developer: why do I still want pdaXrom? That may sound like a troll, but it's not meant to be. When I left the Cacko ROM I chose pdaXrom over OZ for some specific reasons: mainly that the kernel was compatible (mostly) with the Sharp ROM kernel and the development tools (gcc) were compatible with Sharp SDK versions. But lately pdaXrom has been going down the OZ path ... first with soft-float that broke binary compatibility with Sharp ROMs and now with a completely custom 2.6 kernel.

Now, there are very good justifications for adopting those technologies, but it doesn't change the fact that if I'm going to run something that breaks all compatibility with the Sharp ROM, why not just run OZ/GPE? With OZ/GPE I get basically the same functionality plus a bigger developer community, a very sophisticated software build system, and a (relatively) consistant stable release across all Zaurus models.

I think Sashz is a brilliant Linux hacker and I hope he continues his research since his breakthroughs can benefit all Zaurus distros. But as a part time Zaurus software developer, I want to target a stable ROM and not a research project. If you're not a developer, you may have a totally different perspective. In any case, I wish the loyal pdaXrom devs and users nothing but the best. We're all after the same thing ... making our handhelds do what we want them to do and not what some big company tells us they should do

pgas

  • Hero Member
  • *****
  • Posts: 1097
    • View Profile
    • http://
Pdaxrom Development
« Reply #3 on: July 21, 2006, 12:52:26 pm »
Quote
I think that post is one of the most rational and balanced I've read here for a while. It correctly sums up the issues and position and puts things very well - nice one!

ehe, just a little fact is missing though, that sashz do use a tool to build pdaxrom
SLC-860 cacko / senao wifi

willgan

  • Jr. Member
  • **
  • Posts: 84
    • View Profile
    • http://
Pdaxrom Development
« Reply #4 on: July 21, 2006, 01:28:54 pm »
i think pdaxrom-builder is much easier to hack than OE..

It dumps much more debug code during compilation..
It can create multiple build versions...

the new u-boot can speed up root or ketnel flashing...

maybe the builder need a bit of clean up
My Current Z...

Zaurus C1000 - Pdaxii13
Using Pdaxrom-builder (SVN) for building packages

papercrane

  • Jr. Member
  • **
  • Posts: 65
    • View Profile
    • http://reversefold.com
Pdaxrom Development
« Reply #5 on: July 21, 2006, 05:30:51 pm »
Quote
i think pdaxrom-builder is much easier to hack than OE..

It dumps much more debug code during compilation..
It can create multiple build versions...

the new u-boot can speed up root or ketnel flashing...

maybe the builder need a bit of clean up :)
[div align=\"right\"][a href=\"index.php?act=findpost&pid=136021\"][{POST_SNAPBACK}][/a][/div]

This may be true for quick "get it running" packages, but for a distribution having a system in place to automatically deal with packaging, dependencies, toolchain building, etc, is nice to have.

OE doesn't dump all output to console, this is true, but you normally don't need all that. OE *does* store the output for all commands run in handy log files which you can easily peruse after the fact (or even tail -f while the build is running, I've done this many times).

koen

  • Hero Member
  • *****
  • Posts: 1008
    • View Profile
    • http://dominion.thruhere.net/koen/cms/
Pdaxrom Development
« Reply #6 on: July 23, 2006, 07:25:23 am »
Quote
It can create multiple build versions...
[div align=\"right\"][a href=\"index.php?act=findpost&pid=136021\"][{POST_SNAPBACK}][/a][/div]

That was one of OE's design goals, and it can do it perfectly.
Forums are not bugtrackers!!! Smart questions
Ångström release team
iPAQ h2210, iPAQ h5550, iPAQ hx4700, Zaurus SL-C700, Nokia 770, all running some form of GPE
My blog

kopsis

  • Sr. Member
  • ****
  • Posts: 329
    • View Profile
    • http://kopsisengineering.com
Pdaxrom Development
« Reply #7 on: July 23, 2006, 08:16:34 pm »
Quote
i think pdaxrom-builder is much easier to hack than OE..
[div align=\"right\"][a href=\"index.php?act=findpost&pid=136021\"][{POST_SNAPBACK}][/a][/div]

It probably is ... but serious software devs don't want hacks. If I hack up something to build in that environment, how much confidence do I have that it will build in the next version? How easy is it for me to get it committed to the software baseline so that it will be automatically built by someone other than me every time a new ROM version is released? How much do I have to tweak my "hacks" when a new upstream release comes out?

For me this looks to be a case where investing a little extra time up-front will save me a lot of time down the road. Can pdaXrom builder eventually get to that point? Sure. Does it make sense to reinvent something that already exists? Maybe for the entertainment/education/professional development of the person doing it, but that's a non-transferrable justification.

Bundabrg

  • Full Member
  • ***
  • Posts: 183
    • View Profile
Pdaxrom Development
« Reply #8 on: July 23, 2006, 10:37:45 pm »
Quote
Quote
What do other developers and users think? This is NOT a flame starter and NOT a this-rom is better than that-rom thread! It is simply my concern that I wish to express!
[div align=\"right\"][a href=\"index.php?act=findpost&pid=135929\"][{POST_SNAPBACK}][/a][/div]

Bundabrg,

-------- snip 8< ---------------

I agree that it could be beneficial for pdaXrom to switch to the OE build system. But I find myself asking a more fundamental question as a software developer: why do I still want pdaXrom? That may sound like a troll, but it's not meant to be. When I left the Cacko ROM I chose pdaXrom over OZ for some specific reasons: mainly that the kernel was compatible (mostly) with the Sharp ROM kernel and the development tools (gcc) were compatible with Sharp SDK versions. But lately pdaXrom has been going down the OZ path ... first with soft-float that broke binary compatibility with Sharp ROMs and now with a completely custom 2.6 kernel.

Now, there are very good justifications for adopting those technologies, but it doesn't change the fact that if I'm going to run something that breaks all compatibility with the Sharp ROM, why not just run OZ/GPE? With OZ/GPE I get basically the same functionality plus a bigger developer community, a very sophisticated software build system, and a (relatively) consistant stable release across all Zaurus models.

------- snip 8< --------------------

[div align=\"right\"][a href=\"index.php?act=findpost&pid=136018\"][{POST_SNAPBACK}][/a][/div]

I don't wish to debate OS's in this thread, and I know you are not either. One could say why 'Debian' and 'Ubuntu'? Why Redhat and Fedora? Why slackware?
The simple fact is that there ARE lots of people using pdaXrom, and lots of people using OZ/GPE. And whilst both projects are active (using whatever builder), there always will be.

As a developer, it really doesn't matter what OS you run. The users will choose their OS _they_ like, and by god they will defend it to the death ;-) And I respect that!

However, what it boils down to really is, as a developer, when I add a package, I would prefer that same package to ultimately be useful for more OS's than less. It shouldn't matter what OS I use, but if I create a package, sure it's not going to necessarily be binary compatible on different OS's (Try ubuntu debs versus debian debs. You reach dependency hell quite quickly), BUT how the package is created being standard would be extremely useful. Thats why I advocate OE as the .bb recipe files could VERY easily be used with minor-to-no modifications between dists, BUT the dists themselves could be quite seperate. It also means MUCH less duplication of work so BOTH (or more if anyone creates a new) dists can grow faster.

I do wish to add though that though I mention OE, my main aim is to raise awareness of getting rid of the risks in the first post. I myself don't know the politics of OE itself and that may be a factor, but it seems to me that it would be relatively simple to create a new branch or if there is politics in creating the new branch (I don't know who hands out the keys to where it is currently hosted ;-) then it seems that it would be relatively simple to duplicate OE to a different server and build the branch there.

 - Bundabrg
[img]http://www.worldguard.com.au/brendan-ambi.png\" border=\"0\" class=\"linked-sig-image\" /]

Bundabrg

  • Full Member
  • ***
  • Posts: 183
    • View Profile
Pdaxrom Development
« Reply #9 on: July 23, 2006, 10:41:33 pm »
Quote
Quote
I think that post is one of the most rational and balanced I've read here for a while. It correctly sums up the issues and position and puts things very well - nice one!

ehe, just a little fact is missing though, that sashz do use a tool to build pdaxrom
[div align=\"right\"][a href=\"index.php?act=findpost&pid=136019\"][{POST_SNAPBACK}][/a][/div]

I wasn't aware of that tool. I'll investigate further. If it mitigates the risks I've posted then thats great, and if it allows one to create package creation dependencies etc, then excellent.

I just assumed everyone just used the native or the cross-tool-chain to create packages, which loses how the package was created, what patches, the source code etc...

 - Bundabrg
[img]http://www.worldguard.com.au/brendan-ambi.png\" border=\"0\" class=\"linked-sig-image\" /]

iczer3

  • Jr. Member
  • **
  • Posts: 59
    • View Profile
Pdaxrom Development
« Reply #10 on: July 23, 2006, 10:45:22 pm »
Dear Bundabrg, koen, knopsis and papercrane,

I found it AMUSING that you guys are always hanging around in Pdaxrom
discussion talking about how great OZ is. OZ is no doubt, great, but why
would you guys try to push it so much around here ??

And you guys, when given a chance, always bashs pdaxrom.

It seemed that OZ is so well developed that you guys have so much time ??

 

BR,

Felix.

Bundabrg

  • Full Member
  • ***
  • Posts: 183
    • View Profile
Pdaxrom Development
« Reply #11 on: July 23, 2006, 10:45:30 pm »
Quote
Quote
i think pdaxrom-builder is much easier to hack than OE..
[div align=\"right\"][a href=\"index.php?act=findpost&pid=136021\"][{POST_SNAPBACK}][/a][/div]

It probably is ... but serious software devs don't want hacks. If I hack up something to build in that environment, how much confidence do I have that it will build in the next version? How easy is it for me to get it committed to the software baseline so that it will be automatically built by someone other than me every time a new ROM version is released? How much do I have to tweak my "hacks" when a new upstream release comes out?

[div align=\"right\"][a href=\"index.php?act=findpost&pid=136199\"][{POST_SNAPBACK}][/a][/div]

I think you hit it right on the head with this part of the quote: -
Quote
How easy is it for me to get it committed to the software baseline so that it will be automatically built by someone other than me every time a new ROM version is released?

Having a package compiled, by someone else who is not necesarilly familiar with the package, or automatically on each new release is very important. As I mention in my first post, it means that as a developer I'm more inclined to package it in the first place since I know it will live longer and if I get sick of the whole thing, someone else doesn't have to go through what I did to create the package in the first place.

 - Bundabrg
[img]http://www.worldguard.com.au/brendan-ambi.png\" border=\"0\" class=\"linked-sig-image\" /]

Bundabrg

  • Full Member
  • ***
  • Posts: 183
    • View Profile
Pdaxrom Development
« Reply #12 on: July 23, 2006, 10:46:28 pm »
Quote
Dear Bundabrg, koen, knopsis and papercrane,

I found it AMUSING that you guys are always hanging around in Pdaxrom
discussion talking about how great OZ is. OZ is no doubt, great, but why
would you guys try to push it so much around here ??

And you guys, when given a chance, always bashs pdaxrom.

It seemed that OZ is so well developed that you guys have so much time ??

 

BR,

Felix.
[div align=\"right\"][a href=\"index.php?act=findpost&pid=136203\"][{POST_SNAPBACK}][/a][/div]

Hi Felix,

I actually run pdaXrom so theres a good reason I'm in this forum ;-)

 - Bundabrg
[img]http://www.worldguard.com.au/brendan-ambi.png\" border=\"0\" class=\"linked-sig-image\" /]

Da_Blitz

  • Hero Member
  • *****
  • Posts: 1579
    • View Profile
    • http://www.pocketnix.org
Pdaxrom Development
« Reply #13 on: July 24, 2006, 01:26:36 am »
OE is great however i have noticed that its main problems stem from people building distros straight from the package sources

this might sound strange howeveras i see it the OE tools are supposed to help you build the programs yourself (and package them, more on this latter) i have tried to set it up a couplo of times and failed so you please try and disprove some of my comments

the problem i see is that the programs are packaged with no formal set of rules. there is a bit of varince in the packages, for exemple some programs compiled with everything turned on and shiping with config files while others are compiled with the bare minimum and no config files at all

there also seems to be a bit of a reliance on the auto magic to make things work but i can understand that due to the size of the user base not everything gets tested and the things that do get tested are normally with the defalt config in a graphical enviroment, not on the commmand line (i dont run X so i have seen some of these errors)

OE and bitbake are halfway between retargetable package/compiling program and distro. and the distros that are bassed of it are not using it correctly in my opinon. this is partially because of the diffrent styles of distros that are rellying on it (openslug.org, openwrt.org and OZ, i have 2 openwrt routors, 2 NSLU "openslugs" and a Z) some of these only have 8MB of flash and 16MB of ram so you can see how this contrasts to the Z's of wihch most have 64MB of ram and most have 64MB of storage

there really needs to be a set standard for packaging and having OE turn itself into a distro in its own right, OR make it retargetable to have a "high" and "low" end target that slightly changes the compiling process, i assume it has somthing like gentoos make flags, (as i said eariler i neven got it to work properly) if so perhaps that should be given more enfisis along with a standard guide and template to packaging

the best thing to happen to it would be if a distro like debian picked it up  (now i am dreaming) as you would slowly get alot of packages and some sort of packaging guidelines

after reading that again i feel its all over the place, and it is but i belive some of the points are valid. btw i used to run pdaXrom until OZ got a 2.6 kernel on the spitz, after seeing the overlay support i may have to switch back
Personal Blog
Code
Twitter

Gemini Order: #95 (roughly)
Current Device: Samsung Chromebook Gen 3
Current Arm Devices Count: ~30
Looking to acquire: Cavium Thunder X2 Hardware

pgas

  • Hero Member
  • *****
  • Posts: 1097
    • View Profile
    • http://
Pdaxrom Development
« Reply #14 on: July 24, 2006, 01:48:09 am »
Note also that this particular subject has come up a couple of times here:

https://www.oesf.org/forums/index.php?showtopic=20021
https://www.oesf.org/forums/index.php?showtopic=19923&
https://www.oesf.org/forums/index.php?showtopic=19146&st=15

(from a quick search on "openembedded"  in the pdaxrom section, I'm pretty sure there are  others)
« Last Edit: July 24, 2006, 02:25:58 am by pgas »
SLC-860 cacko / senao wifi