Skip to main content.
home | support | download

Back to List Archive

Re: spider a database

From: Aaron Bazar <aaronb(at)>
Date: Fri Nov 04 2005 - 20:27:34 GMT
You will probably get better support from but from the sounds
of it you should use use included spider program to spider your web site...
You would not mess at all with /var/lib/mysql or /home/yc/www... You would
put the web site URL in the spider's config file.

Spidering a database can work too, but is likely a bit more difficult to set
up. You would need to write a program that queries your database and pipes
the results to swish-e. Once you write the program, it is much faster than
using the spider, but, again, it is likely to be a pain.


Aaron Bazar

-----Original Message-----
From: []
On Behalf Of Michael Porcaro
Sent: Friday, November 04, 2005 3:18 PM
To: Multiple recipients of list
Subject: [SWISH-E] Re: spider a database

Please bear with me here and thank you for your patience.  I looked at your
link and searched around.  By searching, I assume that swish-e can spider
databases, I wasn't really sure about this before.  I came across this
document.  Is this the right thing to read, in order to figure out how to
spider my dynamic pages?

Also, I am confused as to where I should direct the config file to spider
the dynamic links.  Let's say I want to spider this particular

Piano-Music-f50.html is actually a php generated file with an html alias,
but I don't know where to direct swish-e to spider this file.
When I spider the files under /home/yc/www/forum (my local site for, all it does is spider the files that run the forum,
not the actual content dynamic pages, such as "Piano-Music-f50.html" or

So I guess my basic question would be, what is the address of my dynamic
files?  A very poor guess is, my database files are located here:


But is this the address to spider?  Or do I spider /home/yc/www/forum

-----Original Message-----
[] On Behalf Of Bill Moseley
Sent: Friday, November 04, 2005 12:56 PM
To: Multiple recipients of list
Subject: [SWISH-E] Re: spider a database

On Fri, Nov 04, 2005 at 09:50:35AM -0800, Michael Porcaro wrote:
> Hi guys,
> I know how to spider physical files on a server, but I am having
> spidering or indexing my dynamic files.  I have a big forum, and it is 
> all run by a mysql database (all dynamic files).  Any insight on how
> spider my dynamic files run by mysql database?  Thank you.

It was once said: if you can use swish you can then learn how to use swish.

Bill Moseley

Unsubscribe from or help with the swish-e list:

Help with Swish-e:
Received on Fri Nov 4 12:27:35 2005