Bill Moseley scribbled on 10/18/06 1:57 PM:
> On Wed, Oct 18, 2006 at 11:03:23AM -0700, brad miele wrote:
>> we currently use fs to index our stuff, this is because we are already
>> dumping xml for every record in our database.
>> my question is, is it faster to break the directory up into sub
>> directories? right now, we have one directory called IDX that has all
>> 900K+ files in it. it takes a very long time for swish-e to move from the
>> "Checking dir" phase to the actual indexing phase. "a very long time" is
>> not really quantifiable right now since we generally don't see it
>> happening and i am just noticing because i am running things manually
>> today. it has been sitting at this stage for about 1.5 hours so far.
>> so should i try breaking the directory up into sub directories?
> Depends on your file system. But I'd probably break it up into
> smaller directories.
besides, must be next to impossible to 'ls' in that dir. ;)
check out SWISH::Prog::DBI on cpan too. That lets you index directly from db via
DBI without the intermediate XML files.
Peter Karman . http://peknet.com/ . peter(at)not-real.peknet.com
Received on Wed Oct 18 12:00:58 2006