Nov 10 2006, 02:46 AM
Post
#1
|
|
![]() Group: Admin Posts: 3,281 Joined: 29-July 04 From: Cambridge, England Member No.: 4,149 |
Just as you can get a snapshot of wikipedia and store on your PC (or even PDA), it occurred to me that a snapshot of the OESF forum would be a very useful thing if combined with free text search.
There's a huge amount of wisdom on it (a lot of which should be in the wiki, but isn't), and so an archive would be a real asset. The low-graphics version of course would be best, but provided the archive was created without the attachments, it'd be OK as there'd be only one copy of the site. I did try a speed-throttled wget once, but it wasn't too satisfactory. Any chance of considering being able to do this on the server itself and make a monthly snapshot downloadable in .zip or .tar.bz2? Some of us could burn CDs or DVDs to send to people without broad-band. thanks Paul |
|
|
|
![]() |
Nov 10 2006, 03:42 AM
Post
#2
|
|
![]() Group: Members Posts: 1,002 Joined: 28-April 05 From: Germany Member No.: 7,027 |
This is a terrific idea and I also wish this would be possible.
I think a forum like this is based on a database. it should not be too hard to dump the database content into pute text files (or HTML maybe). If it isn't possible on the server side, there is still the "lo-fi" versoin of the forums which are simpler HTML pages and probalby easier and faster to collect via a tool like wget or plucker. So this may be another idea. Another nice feature would be to install an NNTP gateway or POP/IMAP server inorder to make it possible to download the forum contents to client programs like email clients or usenet clients. If it would be possible to add content (reply, add topics) also from these clients, this would be even better. But this would probablyl be a major effort and since we can be thankful that some nice people drive this forum without profit, I totally understand if this will never happen :-) daniel |
|
|
|
Nov 14 2006, 01:33 PM
Post
#3
|
|
![]() Group: Admin Posts: 3,281 Joined: 29-July 04 From: Cambridge, England Member No.: 4,149 |
QUOTE(daniel3000 @ Nov 10 2006, 12:42 PM) This is a terrific idea and I also wish this would be possible. I think a forum like this is based on a database. it should not be too hard to dump the database content into pute text files (or HTML maybe). If it isn't possible on the server side, there is still the "lo-fi" versoin of the forums which are simpler HTML pages and probalby easier and faster to collect via a tool like wget or plucker. So this may be another idea. I now found a program which will do such a job, and am trying it out... the good thing is that it allows me to choose to only download URLs with "lofiversion" in them and to throttle bandwidth. The problem with wget is that it doesn't fix links, whereas this program does. http://www.httrack.com/ When it's finished, I will see what the results are like and offer a download on my website. QUOTE Another nice feature would be to install an NNTP gateway or POP/IMAP server inorder to make it possible to download the forum contents to client programs like email clients or usenet clients. If it would be possible to add content (reply, add topics) also from these clients, this would be even better. But this would probablyl be a major effort and since we can be thankful that some nice people drive this forum without profit, I totally understand if this will never happen :-) daniel I did ask about an RSS feed, but I think this forum doesn't have the facility. Paul |
|
|
|
speculatrix Static Site Snapshot/mirror Nov 10 2006, 02:46 AM
desertrat QUOTE(speculatrix @ Nov 14 2006, 09:33 PM)The... Nov 14 2006, 05:22 PM
speculatrix well, the first run seems to be complete... create... Nov 14 2006, 03:30 PM
Jon_J speculatrix,
I tried using the program you listed ... Nov 14 2006, 08:39 PM
speculatrix QUOTE(Jon_J @ Nov 15 2006, 05:39 AM)speculatr... Nov 15 2006, 02:21 AM
ShiroiKuma I think it's not a very smart idea to download... Nov 15 2006, 02:33 AM
speculatrix QUOTE(ShiroiKuma @ Nov 15 2006, 11:33 AM)I th... Nov 15 2006, 04:27 AM
speculatrix I've just had a thought... what's the limi... Nov 15 2006, 04:29 AM
ShiroiKuma QUOTE(speculatrix @ Nov 15 2006, 01:29 PM)I... Nov 15 2006, 04:38 AM
matthis Actually these kind of forums store data in a sql ... Nov 15 2006, 07:37 AM
zmiq2 Or maybe a link to a file with the daily activity,... Nov 15 2006, 07:57 AM
speculatrix hmm, well, it's a pretty damn big file when zi... Nov 21 2006, 02:00 PM
speculatrix The snapshot is now up as a mirror... see mainstre... Nov 22 2006, 07:45 AM
speculatrix Can I ask DZ and other admins to please consider a... Feb 21 2007, 10:41 AM
Antikx QUOTE(speculatrix @ Feb 21 2007, 12:41 PM)Can... Feb 21 2007, 02:27 PM
speculatrix QUOTE(Antikx @ Feb 21 2007, 11:27 PM)QUOTE(sp... Feb 21 2007, 03:34 PM
Antikx sorry for stating the obvious. Feb 21 2007, 05:55 PM
speculatrix OK, I'm running whtt again - 2 connections per... Feb 22 2007, 12:28 PM
speculatrix Ok, it completed after 8 hours, about 30,000 pages... Feb 22 2007, 11:22 PM![]() ![]() |
|
Lo-Fi Version | Time is now: 18th June 2013 - 07:12 PM |