Author Topic: Oesf Forum Archive (Read 41544 times)

speculatrix · « **on:** November 21, 2006, 05:05:59 pm »

I used a program http://www.httrack.com/ to take a snapshot of the entire forum's in lofi format. I spidered the site at a slow speed and bandwidth so as to neither kill the forum nor my ADSL line!

I created a squashfs file system (I had to use an old version of mksquashfs to make it compatible with cacko), which you can store on your memory card and loopback mount on either the Z or a linux box (and probably OSX), e.g.:

Code: [Select]

    mkdir /mnt/oesfforlofi
    mount -t squashfs -o loop /mnt/card/oesfforlofi.squashfs /mnt/oesfforlofi
-or-
    mount -t squashfs -o loop /mnt/card/oesfforlofi.squashfs.nmz /mnt/oesfforlofi

or add to /etc/fstab:

Code: [Select]

/mnt/card/oesfforlofi.squashfs /mnt/oesfforlofi squashfs loop,noauto,noatime,ro 0 0
There are two versions of the squashfs archive, one with and one without the .nmz suffix. The latter has a free-text search function using Namazu .

Please download it from Antikz's mirror at
oesfforlofi.squashfs.nmz -with search, 64MB or oesfforlofi.squashfs without search, 32MB; original files at my downloads

--edit again 20061130
Added nmz details with Antikz's kind mirroring

I originally used cramfs but couldn't mount the archive on my linux/x86 box, so I tried squashfs and that worked on both and was quite a bit smaller than a zipfile; must use squashfs v2 as v3 is incompatible with Cacko's v2.

Antikx · « **Reply #1 on:** November 21, 2006, 05:44:46 pm »

Quote

I'd be grateful for someone with more bandwidth than my cheapie hosting to post a mirror if possible, thanks!
[div align=\"right\"][a href=\"index.php?act=findpost&pid=146864\"][{POST_SNAPBACK}][/a][/div]

I'd be happy to, with your permission.

speculatrix · « **Reply #2 on:** November 22, 2006, 05:26:26 am »

Quote

Quote
I'd be grateful for someone with more bandwidth than my cheapie hosting to post a mirror if possible, thanks!
[div align=\"right\"][a href=\"index.php?act=findpost&pid=146864\"][{POST_SNAPBACK}][/a][/div]
I'd be happy to, with your permission.
[div align=\"right\"][a href=\"index.php?act=findpost&pid=146869\"][{POST_SNAPBACK}][/a][/div]

go right ahead... that data is owned by the denizens of OESF forums, not me, I just packaged it up for them!

Antikx · « **Reply #3 on:** November 22, 2006, 09:23:50 am »

Mirror here: http://www.tyrannozaurus.com/feed/www.oesf.org.20061121.zip

speculatrix · « **Reply #4 on:** November 22, 2006, 10:44:22 am »

Another reason for doing the snapshot is a precaution against the loss of the forum, either through disaster (I'd hope offroadgeek or dz to have decent backups) or (unlikely) a falling-out of the community with those hosting it - even the small risk of this means we should make the effort to have non-live snapshots.

if someone's got bandwidth and disk space to burn, maybe they could unpack the archive on their web server and post a URL?

I was thinking of refreshing the snapshot every two months, anyone want to suggest more or less frequently?

Jon_J · « **Reply #5 on:** November 22, 2006, 12:42:45 pm »

I don't know how to make a cramfs/squashfs image, so I just unzipped the archive and am copying it to my Z thru USB.
I'm copying it to /hdd3/Documents/Web_Files/
It takes about One hour and 45 minutes to copy these files to my Z.
I just hope I can open it in Opera, in a reasonable amount of time to make it usable.
I'll report back when these files are all copied.
Since the forum's search function won't work on these offline pages, what's recommended to search these pages on the Z ?

Antikx · « **Reply #6 on:** November 22, 2006, 01:23:48 pm »

Quote

Since the forum's search function won't work on these offline pages, what's recommended to search these pages on the Z ?
[div align=\"right\"][a href=\"index.php?act=findpost&pid=146967\"][{POST_SNAPBACK}][/a][/div]

I wonder if you could type:

Code: [Select]

find /hdd3/Documents/Web_Files/ | grep searchword

Jon_J · « **Reply #7 on:** November 22, 2006, 01:34:35 pm »

I got it open in Opera, it took awhile to open.
There is a problem with the large number of files in the archive.
When I was copying this archive from my windows box to my Z thru USB, the last file would not copy.
"lofiscreen.css"
I got an error saying this file or directory cannot be created.
I can copy this file to this directory if I move another file out of this directory.
I have my choice of which one of 2 files to leave out.
"lofiscreen.css" or "lofiprint.css"
I placed lofiprint.css" in one directory above the /lofiversion/ directory and copied "lofiscreen.css" to /lofiversion/ directory.
This is on my vfat formatted /hdd3

I just clicked on the General forum and it's taking a very long time (in Opera) to open, about 45 secs. screenshot below.

800 posts

Jon_J · « **Reply #8 on:** November 22, 2006, 01:42:52 pm »

I know this is offtopic, but I now need to defrag this FAT drive thru USB in windows.
I wish there was a better method, this process takes about an hour and half...
EDIT:
After placing the files from this archive onto /hdd3, I decided to use Diskeeper to defrag /hdd3.
Defrag was 4 hours into it's progress and it was only at 11%....
I decided to stop it.
When the drive finally stopped spinning, about 2 minutes after I stopped it, I decided to MOVE the entire contents to my windows machine, then format the drive and MOVE the files back.
This is estimated to take 119 minutes for each move. That's 4 hours total.
The drive looked really fragmented by looking at the graphic in Diskeeper.
The drive spinned constantly. I hope this doesnt wear out my built-in microdrive.
I have it plugged into external power, in case anyone is wondering.

Cresho · « **Reply #9 on:** November 22, 2006, 01:45:52 pm »

I have you bookmarked. Incase of a catastrophy, i know where to go.

Jon_J · « **Reply #10 on:** November 22, 2006, 01:52:00 pm »

This works pretty well. Clicking a thread takes about 30 secs though.
I'm glad to have the forums at my disposal anywhere I go.
I hope I don't have a problem with leaving that last file out of it's proper directory as mentioned above. "lofiprint.css"

Nice gesture Cresho..
Have you downloaded this? I'm offline when I leave my computer overnight, so there isn't a way to connect to me.

Oh BTW, Thank you speculatrix for making this available for us Zaurus users.
Thank you Antikx for hosting this file.

speculatrix · « **Reply #11 on:** November 22, 2006, 01:55:30 pm »

Quote

I got it open in Opera, it took awhile to open.
There is a problem with the large number of files in the archive.
When I was copying this archive from my windows box to my Z thru USB, the last file would not copy.
"lofiscreen.css"
I got an error saying this file or directory cannot be created.
I can copy this file to this directory if I move another file out of this directory.
I have my choice of which one of 2 files to leave out.
"lofiscreen.css" or "lofiprint.css"
I placed lofiprint.css" in one directory above the /lofiversion/ directory and copied "lofiscreen.css" to /lofiversion/ directory.
This is on my vfat formatted /hdd3

I just clicked on the General forum and it's taking a very long time (in Opera) to open, about 45 secs.

800 posts
[div align=\"right\"][{POST_SNAPBACK}][/a][/div]

I'll have a go soon at making a cramfs or squashfs form of it as soon as I can. It'll still be 60+MB, as it probably won't be any more or less efficient than a zip file.

I also had a problem with lofiscreen.css when I first captured the files on windows onto local disk, it would let the two .css files coexist. It wasn't until I moved the files to my fileserver for uploading to web site when I was able to put the two .css files into the main directory and then zip it up. It seems then that there is something odd in windows!

the only free text database that springs to mind for linux is "glimpse": [a href=\"http://webglimpse.net/]http://webglimpse.net/[/url]

Jon_J · « **Reply #12 on:** November 22, 2006, 02:03:00 pm »

Trying to copy lofiprint.css on my Z in Tree!Explorer to /lofiversion/
It seems the /lofiversion/ directory won't open or is taking a very long time to open in Tree!Explorer.
There's no disc activity.
Tree!Explorer seemed to lock up. I get sounds when I click, so I dont think my Z is locked up.
I also have Opera open with the forums loaded.
I just switched to Opera and Tree!Explorer won't open. I'm going to need to kill it or reboot.
One last edit to this post..
I opened sysinfo to the processes tab and I see activity in the treeexplorer entry.
The numbers under memory are fluctuating

speculatrix · « **Reply #13 on:** November 22, 2006, 02:12:49 pm »

I think the 21000 files will break you Z if you attempt to view in a gui. Use the command-line!

Jon_J · « **Reply #14 on:** November 22, 2006, 02:13:51 pm »

I killed it in sysinfo.
Trying again to copy this file results in same situation as above.
Going to try commandline to copy this file, but I first am going to copy/paste this long path so I don't need to remember it or write it down.

I'm in the directory just below /lofiversion

Code: [Select]

cp lofiprint.css /lofiversion/
cp: unable to open '/lofiversion/': Is a directory

cp lofiprint.css /lofiversion
cp: unable to open '/lofiversion/': Read-only file system

speculatrix, do you know what lofiprint.css is used for?
If it's just there to enable printing to a printer, I'll just leave it out.

Another edit, here's lofiprint.css opened in wordpad:

Code: [Select]

/*=============================================*/
/* LO-FI SCREEN CSS FILE                       */
/* (c) 2004 Invision Power Services, Inc.      */
/* This CSS by: Matthew Mecham                 */
/*=============================================*/

@import url(lofiscreen.css);

It looks like it's needed for some reason as it calls lofiscreen.css
lofiscreen.css is too big to post here, but I opened it in wordpad and it has a lot of formatting info in it.

Thanks again,
Jon

News:

Author Topic: Oesf Forum Archive (Read 41544 times)