OESF Portables Forum

General Forums => Off Topic forum => Topic started by: Capn_Fish on June 07, 2009, 12:34:55 pm

Title: Zaurus Cluster How To With Distcc
Post by: Capn_Fish on June 07, 2009, 12:34:55 pm

I can't remember if I've asked this before, but here goes: It it possible/feasable to USB network all of my ARM devices, running a common OS (Ubuntu/Debian, likely) and have a mini-cluster going to speed up native builds? It would seem that with 2 Zaurii and an N810 (and hopefully a Pandora soon), I could get things built 2-3x faster, which would be nice.

I suspect something with distcc could be done, but my impression is that that ignores the "host" CPU (eg, if I started the build on the N810, only the two Zaurii would actually do work), making it less than optimal. Is that correct?

Anyway, thoughts, how-tos, etc.?

EDIT: It looks completely possible. I guess this thread will likely turn into a how-to once I get around to it.

EDIT 2: Changed the thread title, see the next post for the tutorial.

Title: Zaurus Cluster How To With Distcc
Post by: Capn_Fish on June 09, 2009, 07:14:07 pm

OK, it turns out this is really easy. I'm going to do this from the prospective of two Zaurii running Zubuntu, so adapt it however necessary. This is a rough draft, so please don't expect everything to be absolutely correct the first time around.

Note that you COULD do this over traditional networks, but that the latency would slow things down to some extent (and it's pretty awesome to have a mini USB network cluster going).

A generic example can be found here (http://code.google.com/p/distcc/).

0. Get USB networking between all of the involved devices going (not going in-depth on this, look it up):

Sample /etc/network/interfaces:

Code: [Select]

iface usb0 inet static
address 192.168.2.4
netmask 255.255.255.0
gateway 192.168.2.1
Then run:

Code: [Select]

ifup usb0
If all went well, you should be networked.

1. Install the necessary tools (both machines):

Code: [Select]

sudo apt-get install build-essential automake autoconf python2.5-dev
2. Download the latest distcc source (3.1 as of this writing, both machines unless you want to compile and transfer):

Code: [Select]

wget http://distcc.googlecode.com/files/distcc-3.1.tar.gz
3. Build distcc (both machines, once again, unless you want to compile and transfer):

Code: [Select]

tar -xf distcc-3.1.tar.gz
cd distcc-3.1
./configure --prefix=/usr --without-get --without-gnome --disable-Werror
make
sudo make install
4. Add a "distcc" user (both machines):

Code: [Select]

useradd distcc
5. Start the server (both machines, I believe):

Code: [Select]

distccd --daemon --allow "127.0.0.1 xxx.xxx.xxx.xxx"
6. On the machine "doing" the build, export the hosts:

Code: [Select]

export DISTCC_POTENTIAL_HOSTS='127.0.0.1 xxx.xxx.xxx.xxx'
7. Run the build ("host" machine only, NUMBER_OF_THREADS should probably be the number of machines available + 1, IIRC):

Code: [Select]

cd /to/source/dir/
pump make -jNUMBER_OF_THREADS CC=distcc
8. Check to make sure everything is working (on "host" machine only):

Code: [Select]

distccmon-text
If you see something like:

Code: [Select]

object_abc.o [0]: localhost
object_xyz.o [1]: 192.168.2.3
Sample speedups:
Building distcc using distcc (N810 and SL-C3100, using "pump make -j3 cc=distcc"):4 minutes, 49 seconds
Building distcc using only the N810: 4 minutes, 52 seconds

That's not really the speedup I was hoping for, but I'll try to build some other things and see how it pans out.

Anyway, I hope somebody finds this useful.

Title: Zaurus Cluster How To With Distcc
Post by: RX Shorty on June 10, 2009, 04:36:04 am

Thanks, I will get my third arm device soon.
So this is something nice to try indeed