The End Of Ewi546 Is Coming Options

[Hrw]

For nearly year we used ewi546.ewi.utwente.nl as our primary mirror and also as build machine. In next 1-2 weeks machine will get cleaned and moved to other tasks.

I want to thanks Koen Kooi for providing disk space there so we were able to provide feeds and images for our users. Without CPU power of that machine I would not be able to provide OpenZaurus 3.5.4.1/3.5.4.2-rc so easily (my home machine is not reliable enough to do builds).

Now we have to think about new home for our releases. Thanks to Matthias 'CoreDump' Hentges we have it mirrored and available under adress http://feeds.openzaurus.org/ (thx goes to Chris 'kergoth' Larson for providing that CNAME).

But when we release new version bandwidth usage is very big - for example OpenZaurus 3.5.4.1 took about 14GB in first days after release. We have to think how to solve it as we do not have mirrors without transfer limits now.

One of solutions is moving OpenZaurus to LinuxToGo but it would need getting new machine to LTG project so we have to collect money. That will also mean moving website from SourceForge so it will work faster.

Any thoughts/commments/suggestions?

[Da_Blitz]

i am prepared to offer cpu cycles to OE as well as hard drive space

currentlly i am using a dual core amd chip (2Ghz x2) with 4GB of RAM and 2 x 200GB (raid 0, about 60MB/s transfer rate, i rely on ram to cache) of hard drive space on LVM, so setting somthing up wouldnt be too hard.

the only thing i am worried about is that it is a home machine on ADSL, so i wouldnt want to serve content, only provide cpu cycles

any idea how much data is pulled down when builhing the tree?

[simbaB]

How much disk space do you need? I've got a lot, but it's spread out over a few machines. Fortunately, two of those machines are always on and one could be set up to be.

CODE

Filesystem            Size  Used Avail Use% Mounted on
/dev/hdc1              37G  3.4G   34G  10% /
devshm                372M   12K  372M   1% /dev/shm
/dev/sda1             112G   82G   31G  73% /home

Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/Debian-lvol0
                      71G   13G   55G  19% /
tmpfs                 498M     0  498M   0% /lib/init/rw
udev                   10M   68K   10M   1% /dev
tmpfs                 498M     0  498M   0% /dev/shm
df: `/dev/.static/dev': Permission denied
/dev/sda1             236M   15M  209M   7% /boot

Filesystem            Size  Used Avail Use% Mounted on
/dev/hda1             152G   30G  122G  20% /
udev                  237M   44K  237M   1% /dev
devshm                237M     0  237M   0% /dev/shm

The last machine is a MythTV box, so disk space can vary a bit, but most of the time it's fairly constant due to episodes expiring.

CPU cycles--I could set up a distcc network here and offer it for use. It would consist of:

- AMD Athlon XP 3000+ (2.16 GHz)
- Intel Core 2 Duo E6300 (2x1.86 GHz)
- Intel Celeron (2.1 GHz)

And depending on what time zone a developer is in it's entirely possible that that developer could have the machines entirely to themselves. I'm in GMT-5 (well, GMT-4 currently with DST) and I go to bed around 10:30 PM.

Bandwidth: Unfortunately I have 512 Kbps up on a consumer cable connection--consumer broadband ISPs in the US are known for being a bit, ahem, touchy about bandwidth usage despite the constant marketing barrage about "unlimited" connections. So hosting isn't an option.

[Da_Blitz]

i have 1.5mbps down and 512 up with 40G on the downloads only and supposidly unlimited on the up but i dont want to test my ISP i get the feeling that "unlimited" isnt tht over here ethier

[sds]

1. I strongly suggest http://www.linuxtogo.org/ as the better option.
What will Kernel Concepts exactly provide for the OZ team (I think they do mostly Nokia 770)?
In any case, I can donate $50 now for a new machine, and about $20 each month to keep an OZ host running.
I suggest you post a poll and setup a donation scheme to run for at least a week before committing to a decision.


2. Have you checked out with ibiblio.org and/or other space/bandwidth open-source donators?


3. If both of the above does not work, however, I can offer using my home server and ADSL as primary mirror and build:
* 1.5M/768K low-latency, no limit ADSL + 4 fixed IPs
* 2xOpteron 844 K8T + 2GB reg. RAM
* a new 300GB IDE disk just for OZ stuff
It runs SuSE 10.1 on a 2x10K rpm SATA RAID0 config and is always online.
It is used, however, for a couple of other development and entertainment tasks, so performance may slightly fluctuate.

It seems like there is enough "home power" for the development/build on this thread.
For the content serving an option would be to mirror it on several machines and to redirect based on geographical location (IP).


Still, as I said in the beginning, getting professional hosting and building is far superior, and I would suggest trying it first.

[stbrock]

The Linux Mirror Project or LinuxTracker might be able to handle most of the surge of new releases, leaving the others above to handle the day to day volume. This would work to the extent that people could use a torrent client.

[DaemonsGR]

Im looking into it, we have 3-4 new server donations in Hel.L.U.G (I think dual xeon with Gbit line from hol.gr internet provider)

Tasos

[speculatrix]

I find geekhosting to be pretty solid, and price quite reasonable.

[mimeca]

I can contribute with a server with limited monthly transfer. On OpenZaurus release I can add some of the image files to help a bit.

[speculatrix]

have you considered using Google's storage facilities?
http://code.google.com/hosting/

might be worthwhile for the large static files?

This post has been edited by speculatrix: Oct 9 2006, 02:56 AM

[Storm]

i am prepared to offer cpu cycles to OE as well as hard drive space

currentlly i am using a dual core amd chip (2Ghz x2) with 4GB of RAM and 2 x 200GB (raid 0, about 60MB/s transfer rate, i rely on ram to cache) of hard drive space on LVM, so setting somthing up wouldnt be too hard.

the only thing i am worried about is that it is a home machine on ADSL, so i  wouldnt want to serve content, only provide cpu cycles

any idea how much data is pulled down when builhing the tree?

Would it be possible (or even worth it) to set up a distributed build environment for those of us who want to donate CPU cycles? Something between distributed.net or folding@home and a Mosix cluster? Or even something similar to ElectricSheep's render farm?

I don't have any high end machines (My two fastest ones are a 2GHz Pentium-M laptop and a 1.5GHz Athlon XP 1800), but I do have several machines at home (10 or so) and I do have at least 3 dual processors. (2 PIII/966 and an UltraSparc).

I would be willing to donate cycles if it was possible.

[Da_Blitz]

looking into masix bandwidth would be the killer (i just finished an evaluation of it) the best thing for out situation would most likly be distc however by the time the job has transfered your SOL because a local compile would be quicker not to mention keeping toolchains in sync)

looking at the problem though we can see thatwe can paralize the build at the package level (taking into acount dependincies). perhaps we can whip up a deamon that builds only a specified list of packages and distribute part of the tree to each user?

i can see a bash script to de it forming in my head but to me this isreally more of a python job as it would be an internet facing service (even if it only contacts a central server for instructions)

man i would love to get one of the new ultrasparc machines

[koen]

Or you just use the tools already present, like the multithreaded bitbake and the icecc bbclass.

[Hrw]

Thx guys for offers.

I want to write few things:

Mirrors... I contacted CoreDump recently and he told that we have enough mirrors already (he is mirroring maintainer).

Build machines: I recently got access to some boxes and ewi is working still so I can work without problems (and my home machine will move to amd64 in next months).

During OEDEM (OpenEmbedded Developers European Meeting) we discussed about handling mirror/buildpower donations and will provide kind of information soon.

[speculatrix]

there's a *possibility* that I might be able to find an old server and set up a, ahem, free colocate here at work; we've got 16Mbps of bandwidth on bgp failover, I don't know if that would help as a fileserver and build box?