Author Topic: The End Of Ewi546 Is Coming  (Read 7693 times)

Hrw

  • Hero Member
  • *****
  • Posts: 1366
    • View Profile
The End Of Ewi546 Is Coming
« on: September 29, 2006, 03:07:07 am »
For nearly year we used ewi546.ewi.utwente.nl as our primary mirror and also as build machine. In next 1-2 weeks machine will get cleaned and moved to other tasks.

I want to thanks Koen Kooi for providing disk space there so we were able to provide feeds and images for our users. Without CPU power of that machine I would not be able to provide OpenZaurus 3.5.4.1/3.5.4.2-rc so easily (my home machine is not reliable enough to do builds).

Now we have to think about new home for our releases. Thanks to Matthias 'CoreDump' Hentges we have it mirrored and available under adress http://feeds.openzaurus.org/ (thx goes to Chris 'kergoth' Larson for providing that CNAME).

But when we release new version bandwidth usage is very big - for example OpenZaurus 3.5.4.1 took about 14GB in first days after release. We have to think how to solve it as we do not have mirrors without transfer limits now.

One of solutions is moving OpenZaurus to LinuxToGo but it would need getting new machine to LTG project so we have to collect money. That will also mean moving website from SourceForge so it will work faster.

Any thoughts/commments/suggestions?
OpenZaurus 3.5.4x Release Manager
OpenEmbedded, Ångström, Poky developer
My website

Misc embedded hardware.

Da_Blitz

  • Hero Member
  • *****
  • Posts: 1579
    • View Profile
    • http://www.pocketnix.org
The End Of Ewi546 Is Coming
« Reply #1 on: September 30, 2006, 02:31:24 am »
i am prepared to offer cpu cycles to OE as well as hard drive space

currentlly i am using a dual core amd chip (2Ghz x2) with 4GB of RAM and 2 x 200GB (raid 0, about 60MB/s transfer rate, i rely on ram to cache) of hard drive space on LVM, so setting somthing up wouldnt be too hard.

the only thing i am worried about is that it is a home machine on ADSL, so i  wouldnt want to serve content, only provide cpu cycles

any idea how much data is pulled down when builhing the tree?
Personal Blog
Code
Twitter

Gemini Order: #95 (roughly)
Current Device: Samsung Chromebook Gen 3
Current Arm Devices Count: ~30
Looking to acquire: Cavium Thunder X2 Hardware

simbaB

  • Newbie
  • *
  • Posts: 47
    • View Profile
    • http://www.oakcourt.dyndns.org/~andrew/
The End Of Ewi546 Is Coming
« Reply #2 on: September 30, 2006, 05:12:14 pm »
How much disk space do you need? I've got a lot, but it's spread out over a few machines. Fortunately, two of those machines are always on and one could be set up to be.

Code: [Select]
Filesystem            Size  Used Avail Use% Mounted on
/dev/hdc1              37G  3.4G   34G  10% /
devshm                372M   12K  372M   1% /dev/shm
/dev/sda1             112G   82G   31G  73% /home

Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/Debian-lvol0
                       71G   13G   55G  19% /
tmpfs                 498M     0  498M   0% /lib/init/rw
udev                   10M   68K   10M   1% /dev
tmpfs                 498M     0  498M   0% /dev/shm
df: `/dev/.static/dev': Permission denied
/dev/sda1             236M   15M  209M   7% /boot

Filesystem            Size  Used Avail Use% Mounted on
/dev/hda1             152G   30G  122G  20% /
udev                  237M   44K  237M   1% /dev
devshm                237M     0  237M   0% /dev/shm

The last machine is a MythTV box, so disk space can vary a bit, but most of the time it's fairly constant due to episodes expiring.

CPU cycles--I could set up a distcc network here and offer it for use. It would consist of:

- AMD Athlon XP 3000+ (2.16 GHz)
- Intel Core 2 Duo E6300 (2x1.86 GHz)
- Intel Celeron (2.1 GHz)

And depending on what time zone a developer is in it's entirely possible that that developer could have the machines entirely to themselves. I'm in GMT-5 (well, GMT-4 currently with DST) and I go to bed around 10:30 PM.

Bandwidth: Unfortunately I have 512 Kbps up on a consumer cable connection--consumer broadband ISPs in the US are known for being a bit, ahem, touchy about bandwidth usage despite the constant marketing barrage about "unlimited" connections. So hosting isn't an option.
[span style=\'font-size:8pt;line-height:100%\']Collie (not used) | Poodle (OZ/Opie 3.5.4.2-rc2)
Nokia 770 (ITOS 2006) | IBM Thinkpad R51 (Debian)
Intel Core 2 Duo/Desktop Board DG965RY (Debian)[/span]

Da_Blitz

  • Hero Member
  • *****
  • Posts: 1579
    • View Profile
    • http://www.pocketnix.org
The End Of Ewi546 Is Coming
« Reply #3 on: October 01, 2006, 02:01:43 am »
i have 1.5mbps down and 512 up with 40G on the downloads only and supposidly unlimited on the up but i dont want to test my ISP  i get the feeling that "unlimited" isnt tht over here ethier
Personal Blog
Code
Twitter

Gemini Order: #95 (roughly)
Current Device: Samsung Chromebook Gen 3
Current Arm Devices Count: ~30
Looking to acquire: Cavium Thunder X2 Hardware

sds

  • Jr. Member
  • **
  • Posts: 81
    • View Profile
The End Of Ewi546 Is Coming
« Reply #4 on: October 01, 2006, 11:19:45 pm »
1. I strongly suggest http://www.linuxtogo.org/ as the better option.
What will Kernel Concepts exactly provide for the OZ team (I think they do mostly Nokia 770)?
In any case, I can donate $50 now for a new machine, and about $20 each month to keep an OZ host  running.
I suggest you post a poll and setup a donation scheme to run for at least a week before committing to a decision.


2. Have you checked out with ibiblio.org and/or other space/bandwidth open-source donators?


3. If both of the above does not work, however, I can offer using my home server and ADSL as primary mirror and build:
* 1.5M/768K low-latency, no limit ADSL + 4 fixed IPs
* 2xOpteron 844 K8T + 2GB reg. RAM
* a new 300GB IDE disk just for OZ stuff
It runs SuSE 10.1 on a 2x10K rpm SATA RAID0 config and is always online.
It is used, however, for a couple of other development and entertainment tasks, so performance may slightly fluctuate.

It seems like there is enough "home power" for the development/build on this thread.
For the content serving an option would be to mirror it on several machines  and to redirect based on geographical location (IP).


Still, as I said in the beginning, getting professional hosting and building is far superior, and I would suggest trying it first.
C3100 pdaXrom

stbrock

  • Full Member
  • ***
  • Posts: 149
    • View Profile
    • http://
The End Of Ewi546 Is Coming
« Reply #5 on: October 02, 2006, 01:18:59 am »
The Linux Mirror Project or LinuxTracker might be able to handle most of the surge of new releases, leaving the others above to handle the day to day volume. This would work to the extent that people could use a torrent client.

DaemonsGR

  • Full Member
  • ***
  • Posts: 237
    • View Profile
The End Of Ewi546 Is Coming
« Reply #6 on: October 02, 2006, 04:33:11 am »
Im looking into it, we have 3-4 new server donations in Hel.L.U.G (I think dual xeon with Gbit line from hol.gr internet provider)

Tasos
Tasos Varoudis (Tech)
Tasos Varoudis (Architecture)

Zaurus sl-c3100 + Wlan, Lan ,BT ,etc
NOW running... pdaXrom 1.1.0beta3 + OOo + custom apps/setup + debian +.. not yet installed OpenBSD snapshot

WAS:Cacko 1.23 + X/QT + debian

Home-Net
Debian WIFI Router ,OpenBSD Bridge Firewall ,Win 2k Domain Server , :) Sun Fire v120 , :) Gentoo Main Server: Dual PIII

speculatrix

  • Administrator
  • Hero Member
  • *****
  • Posts: 3703
    • View Profile
The End Of Ewi546 Is Coming
« Reply #7 on: October 02, 2006, 05:26:20 pm »
I find geekhosting to be pretty solid, and price quite reasonable.
Gemini 4G/Wi-Fi owner, formerly zaurus C3100 and 860 owner; also owner of an HTC Doubleshot, a Zaurus-like phone.

mimeca

  • Hero Member
  • *****
  • Posts: 682
    • View Profile
    • http://www.innerzaurus.com
The End Of Ewi546 Is Coming
« Reply #8 on: October 08, 2006, 07:19:07 am »
I can contribute with a server with limited monthly transfer. On OpenZaurus release I can add some of the image files to help a bit.
Nunca hagas de un lugar tu hogar. Construye tu hogar en el interior de tu propia cabeza. Encontrare¡s todo lo que necesites para amueblarla: recuerdos, amigos en los que puedes confiar, pasión por el conocimiento y otras cosas por el estilo. De esa forma podrás ir a donde te vayas. Nunca te faltará un hogar..., a menos que pierdas la cabeza, claro...

Innerzaurus => www.innerzaurus.com (Spanish Zaurus Site)

speculatrix

  • Administrator
  • Hero Member
  • *****
  • Posts: 3703
    • View Profile
The End Of Ewi546 Is Coming
« Reply #9 on: October 09, 2006, 06:54:24 am »
have you considered using Google's storage facilities?
http://code.google.com/hosting/

might be worthwhile for the large static files?
« Last Edit: October 09, 2006, 06:56:14 am by speculatrix »
Gemini 4G/Wi-Fi owner, formerly zaurus C3100 and 860 owner; also owner of an HTC Doubleshot, a Zaurus-like phone.

Storm

  • Full Member
  • ***
  • Posts: 156
    • View Profile
    • http://
The End Of Ewi546 Is Coming
« Reply #10 on: October 17, 2006, 11:11:03 pm »
Quote
i am prepared to offer cpu cycles to OE as well as hard drive space

currentlly i am using a dual core amd chip (2Ghz x2) with 4GB of RAM and 2 x 200GB (raid 0, about 60MB/s transfer rate, i rely on ram to cache) of hard drive space on LVM, so setting somthing up wouldnt be too hard.

the only thing i am worried about is that it is a home machine on ADSL, so i  wouldnt want to serve content, only provide cpu cycles

any idea how much data is pulled down when builhing the tree?
[div align=\"right\"][a href=\"index.php?act=findpost&pid=142704\"][{POST_SNAPBACK}][/a][/div]

Would it be possible (or even worth it) to set up a distributed build environment for those of us who want to donate CPU cycles? Something between distributed.net or folding@home and a Mosix cluster? Or even something similar to ElectricSheep's render farm?

I don't have any high end machines (My two fastest ones are a 2GHz Pentium-M laptop and a 1.5GHz Athlon XP 1800), but I do have several machines at home (10 or so) and I do have at least 3 dual processors. (2 PIII/966 and an UltraSparc).

I would be willing to donate cycles if it was possible.
Zaurus SL-5500/Hentges OZ 3.5.4.1
Ambicom WL1100-CF wireless card
Desktop: Debian/GNU Linux (unstable)

Da_Blitz

  • Hero Member
  • *****
  • Posts: 1579
    • View Profile
    • http://www.pocketnix.org
The End Of Ewi546 Is Coming
« Reply #11 on: October 18, 2006, 03:13:26 am »
looking into masix bandwidth would be the killer (i just finished an evaluation of it) the best thing for out situation would most likly be distc however by the time the job has transfered your SOL because a local compile would be quicker not to mention keeping toolchains in sync)

looking at the problem though we can see thatwe can paralize the build at the package level (taking into acount  dependincies). perhaps we can whip up a deamon that builds only a specified list of packages and distribute part of the tree to each user?

i can see a bash script to de it forming in my head but to me this isreally more of a python job as it would be an internet facing service (even if it only contacts a central server for instructions)

man i would love to get one of the new ultrasparc machines
Personal Blog
Code
Twitter

Gemini Order: #95 (roughly)
Current Device: Samsung Chromebook Gen 3
Current Arm Devices Count: ~30
Looking to acquire: Cavium Thunder X2 Hardware

koen

  • Hero Member
  • *****
  • Posts: 1008
    • View Profile
    • http://dominion.thruhere.net/koen/cms/
The End Of Ewi546 Is Coming
« Reply #12 on: October 18, 2006, 04:12:46 am »
Or you just use the tools already present, like the multithreaded bitbake and the icecc bbclass.
Forums are not bugtrackers!!! Smart questions
Ångström release team
iPAQ h2210, iPAQ h5550, iPAQ hx4700, Zaurus SL-C700, Nokia 770, all running some form of GPE
My blog

Hrw

  • Hero Member
  • *****
  • Posts: 1366
    • View Profile
The End Of Ewi546 Is Coming
« Reply #13 on: October 18, 2006, 09:09:34 am »
Thx guys for offers.

I want to write few things:

Mirrors... I contacted CoreDump recently and he told that we have enough mirrors already (he is mirroring maintainer).

Build machines: I recently got access to some boxes and ewi is working still so I can work without problems (and my home machine will move to amd64 in next months).

During OEDEM (OpenEmbedded Developers European Meeting) we discussed about handling mirror/buildpower donations and will provide kind of information soon.
OpenZaurus 3.5.4x Release Manager
OpenEmbedded, Ångström, Poky developer
My website

Misc embedded hardware.

speculatrix

  • Administrator
  • Hero Member
  • *****
  • Posts: 3703
    • View Profile
The End Of Ewi546 Is Coming
« Reply #14 on: October 18, 2006, 10:07:55 am »
there's a *possibility* that I might be able to find an old server and set up a, ahem, free colocate here at work; we've got 16Mbps of bandwidth on bgp failover, I don't know if that would help as a fileserver and build box?
Gemini 4G/Wi-Fi owner, formerly zaurus C3100 and 860 owner; also owner of an HTC Doubleshot, a Zaurus-like phone.