HI Bill,

I think that the latest code is not well optimized with very long files.
21 MB after pdftotext (Chris data)is too much... 
I need to improve it.

I have tested the unfreed memory issue in some OSs with a very
simple program as you told me. I have tested it in solaris 2.6.

The program is:

int main()
{
char *p;
int pid =getpid();
char ps[100];

sprintf(ps,"ps -p %d -o osz",pid);

system(ps);
p=malloc(100000000); /* ~ 100 MB */
system(ps);
free(p);
system(ps);
p=malloc(10000000); /* ~ 10 MB */
system(ps);
free(p);
system(ps);
}


In solaris I get the following:
  SZ
 103  << Before malloc
  SZ
13311  << After malloc
  SZ
13311   << after first free
  SZ
13311   << second malloc - memory reused
  SZ
13311   << after second  free

So, in Solaris, the memory is not returned to the OS, even if we do
not longer use part of it. 

So, for a very long file, the new code can be using a lot of memory
and, even freed and no longer used by swish, this memory will not
be returned to the OS.

cu
Jose

On 29 Aug 2001, at 7:49, Bill Moseley wrote:

> Jose,
> 
> When I was indexing I had similar upper memory usage between the
> different versions -- just that the newer version got to the upper
> memory limit and stayed there.  But maybe that depends on the type of
> documents that are being indexed?  In other words, you think it's
> possible that Chris has files that are making swish use more memory
> than in the previous version?
> 
> Is it really true that BSD doesn't free the memory?  I just assumed
> that the reported number was not reduced after freeing memory.
> 
>