Building An Optimized Debian

[Capn_Fish]

I was chatting with one of the folks making Debian (easy to) run (well)  on the N810 on IRC, and another person in the channel brought up speed concerns. After assuring him that it ran fine and as FAR superior to OS2008, I got to thinking: Is it possible to, in QEMU, do a

Code: [Select]

apt-get source --compile most-everything-important
to get a Zaurus-optimized Debian? Or, at least, Zaurus-optimized important parts of Debian? I know I don't need all of Debian's packages, but even doing the top used 1% would be nice (eg, base system + common Z apps + special requests).

What do people think? Is it even possible to specify CFLAGS/CXXFLAGS when doing using "apt-get source --compile?"

EDIT: Oops... I forgot that I already had brought this up here. Still, something should still be possible, either with QEMU or Crosstool.

EDIT2: Crosstool can only build RPMs, it appears, but we should be able to convert those to DEBs, no?

EDIT3: I just saw that Crosstool supports Distcc. Maybe using Distcc running in an ARM QEMU emulation would work best?

[Raul]

EDIT3: I just saw that Crosstool supports Distcc. Maybe using Distcc running in an ARM QEMU emulation would work best?

Hope you will manage it. A little less resource dependent debian would be great.

[radiochickenwax]

EDIT3: I just saw that Crosstool supports Distcc. Maybe using Distcc running in an ARM QEMU emulation would work best?

Remember this "semi" Native Compilation With Distcc, a short howto ?

But instead of using the cross-compiler from that thread use the one from this thread:

https://www.oesf.org/forum/index.php?showto...t=0&start=0

i.e.:
http://www.codesourcery.com/gnu_toolchains...nux-gnu.tar.bz2

The "deb"s should be easily built in this manner, but I haven't tried it yet.
What do you think?

[Capn_Fish]

Could you elaborate on that? It sounds interesting...

I'm all for anything that would make my Debian run faster

[radiochickenwax]

Okay, I'm going to try to write this as I go... and try to compile stock emacs22 from zaurus using distcc and my debian/ubuntu laptop


From debian based "desktop" laptop PC:

Install distcc

Code: [Select]

apt-get install distcc
Install arm cross-compiler

Code: [Select]

wget http://www.codesourcery.com/gnu_toolchains/arm/portal/package2549/public/arm-none-linux-gnueabi/arm-2008q1-126-arm-none-linux-gnueabi-i686-pc-linu
x-gnu.tar.bz2
Install to "/usr/local/"

Code: [Select]

tar -xvjf arm-2008q1-126-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2

Code: [Select]

mv arm-2008q1 /usr/local


 make a directory for the links to the cross gcc,

Code: [Select]

mkdir /home/radiochickenwax/distcc make some links so that distcc uses the laptop's ARM cross compilers

Code: [Select]

ln -s  /usr/local/arm-2008q1/bin/arm-none-linux-gnueabi-gcc /home/radiochickenwax/distcc/gcc
ln -s  /usr/local/arm-2008q1/bin/arm-none-linux-gnueabi-g++ /home/radiochickenwax/distcc/g++
ln -s  /usr/local/arm-2008q1/bin/arm-none-linux-gnueabi-c++ /home/radiochickenwax/distcc/c++

still on laptop,add the paths to the cross-compiler to the system path:

Code: [Select]

export PATH=/home/radiochickenwax/distcc:$PATHthe above should override the default (x86) compilers


I'm going to do this compiliation over USB, since I don't have a wifi card handy.  The important thing is that we know the IP addresses of both zaurus and slave/laptop.  I'm using the setup from https://www.oesf.org/forum/index.php?showtopic=25025
where the laptop is 192.168.129.1
and the zaurus is 192.168.129.10

start the "distcc" server to accept commands from zaurus (192.168.129.10)

Code: [Select]

distccd --daemon -a 192.168.129.10gives me warning:
distccd[13340] (dcc_preferred_user) Warning: no such user as "distcc"



Now, from the zaurus:

Code: [Select]

apt-get install distcc
link the distcc compilers to override the standard zaurus compilers and build over network: (similar to laptop setting)

Code: [Select]

mkdir /root/distcc/
ln -s  /usr/bin/distcc /root/distcc/gcc
ln -s  /usr/bin/distcc /root/distcc/g++
ln -s  /usr/bin/distcc /root/distcc/c++
export PATH=/root/distcc:$PATHIndicate where laptop is

Code: [Select]

export DISTCC_HOSTS=192.168.129.1
get sources, configure, compile over network (check with "top" command on laptop to see if cross compiler is working):

Code: [Select]

cd /usr/src
cvs -z3 -d:pserver:anonymous@cvs.savannah.gnu.org:/cvsroot/emacs co -rEMACS_22_2 emacs
cd emacs
./configure && make bootstrap && make && make install
cross fingers, and see if this works....  

Note, I wanted to rebuild emacs22 from stock source because I'm having trouble getting images to display inline on the repo version.  Before trying this I did

Code: [Select]

apt-get --compile source and that applied a bunch of patches that I didn't want.

  I just thought I'd try/mention this method since you mentioned using distcc.  I'll reply later to this thread what happens with emacs22.

[radiochickenwax]

Okay, all C/C++ compilation is transferred to laptop from zaurus in this manner, but all of the other commands (texinfo, bash scripts,etc).. are done on zaurus... seems to work....still takes a long time to compile emacs even with a faster computer and more RAM.  Maybe I should add more machines to the setup....  

Trouble with emacs is that so much of it is written in lisp, which distcc doesn't really help with.

Not sure what you wanted to compile... hopefully this might help

[Capn_Fish]

Sounds like a good plan. I've got maybe 3 machines I could set up (a P4 2.6 GHz with 1.25 GB RAM, a PM 1.6 GHz with 768 MB RAM, and a C7 1 GHz with 1 GB RAM).

EDIT: Maybe a PD 3 GHz with 3 GB RAM as well.

Maybe we should try to hack together a livecd that runs distcc on the non-Zaurus boxes for easy setup?

[radiochickenwax]

Maybe we should try to hack together a livecd that runs distcc on the non-Zaurus boxes for easy setup?

would that be a lot of work?

I've got maybe 3 machines I could set up (a P4 2.6 GHz with 1.25 GB RAM, a PM 1.6 GHz with 768 MB RAM, and a C7 1 GHz with 1 GB RAM). EDIT: Maybe a PD 3 GHz with 3 GB RAM as well.

That would speed the C/C++ compiling up I'd think

Is it even possible to specify CFLAGS/CXXFLAGS when doing using "apt-get source --compile?"

It's done by default on zDevil's rootfs. I think.

What confuses me at this point is what to compile, and how to optimize.. other than setting the CFLAGS/CXXFLAGS.  I'm not a very experienced programmer, especially in things like bash, C, C++...  

Maybe you can squeeze *a little* more speed out of just rebuilding, but overall, I think more of the code would need to be patched to get *a lot* more speed out of it. As Donald Knuth says,  "if you optimize everything, you'll never be happy"

On this note, I've been trying for a few days to compile the usb-rndis-lite kernel module and having no luck.

[Capn_Fish]

My impression is that you can squeeze a good amount of speed out of apps just by recompiling them with the correct optimizations. If you've tried Angstrom and Debian one after the other, you can see it.

[radiochickenwax]

My impression is that you can squeeze a good amount of speed out of apps just by recompiling them with the correct optimizations. If you've tried Angstrom and Debian one after the other, you can see it.

Oh, I agree, I wasn't saying that you couldn't...  I'm just saying that you could get  *a lot more* speed by modding the sources than by just optimizing the compilation, and that changing the source is a lot harder to do... that's all....

I'm up for helping to build another rootfs... that's what this thread is all about right?  So... where to start?  I'm still stuck on the kernel and u-boot personally, but you could begin compiling using the method described above.

[Capn_Fish]

Let it be known that I have jumped into a bunch of stuff recently, and, unfortunateky, this will likely be the first thing to be cut if I lack time. Also, I lack any way to get this to people, so that would be another roadblock.

Now, my goal was to build packages, not a whole rootfs. I was thinking that we could take the most commonly used packages (base system, X, GTK, Epiphany, etc.), build them with Zaurus-specific CFLAGS, then stick them in a repo. That would allow people to just apt-get upgrade to them, and would be easier and less confusing than just building another rootfs.

I'm thinking I may just build a liveCD that is configured for headless booting (eg, you stick it in a computer, let it boot, and it's ready for distcc-ing), then stop there and let interested parties take it the rest of the way.

[radiochickenwax]

I was thinking that we could take the most commonly used packages (base system, X, GTK, Epiphany, etc.), build them with Zaurus-specific CFLAGS, then stick them in a repo.

At this point, I'd be happy if someone would just optimize an Xorg/Xfbdev build and make it available... or better yet, explain how to do it.

[Capn_Fish]

You mean rebuild it with Z-specific flags, or actually go edit the code?

[radiochickenwax]

You mean rebuild it with Z-specific flags, or actually go edit the code?

rebuild with Z-specific flags.. kdrive, xfbdev... the one in the yonggun/andromeda combo is still from angstrom?  

I haven't tried rebuilding yet, but it's always failed for me in past roms