>>So the issue is how to run an external program *safely* under Windows.
>>What I'm currently using is IPC::Open3, which under windows is suppose
>>to avoid the shell (although it seems like the data is still messed
>>with by Windows).
>>
>>I'm sure there's a better way to run a program from Perl under
>>Windows, but I've never found anyone that could help.

Using this:

>    So you must find out, in your program, when the child process has
>    finished it's work, and then do a waitpid() on it's process-id to
>    clean the table.

I rummaged through the code and discovered someone had kindly added

$self->{pid} = $pid;

in the windows_fork of Filter.pm. But I didn't find any references to 
"waitpid".

After tracing backwards (getting lost, feeling like I'm wandering the 
labyrinth waiting for the Minotaur to show up and eat me) I finally 
found a location where waitpid could be used properly. ;-)

In "$filter_sub = sub { ... " (Approx. line 1051 in spider.pl), I added 
"waitpid($doc->{pid},0);" just after "my $doc = $filter->convert( .." 
and before "return 1 unless $doc;"

I ran swish-e against my list that showed repeatable errors and it 
completed successfully every time!!

Now to make this platform friendly, I guess we would need to either 
check to see if $doc->{pid} exists or if the platform was using 
IPC::Open2 before calling "waitpid". But I'll leave that to the code 
experts.

I hope this helps. Now I can sleep tonight and not have this bugging the 
heck out of me. ;-)

P.S. Stupid Windows. Bad Windows. No biscuit!!