I think that the latest code is not well optimized with very long files.
21 MB after pdftotext (Chris data)is too much...
I need to improve it.
I have tested the unfreed memory issue in some OSs with a very
simple program as you told me. I have tested it in solaris 2.6.
The program is:
int pid =getpid();
sprintf(ps,"ps -p %d -o osz",pid);
p=malloc(100000000); /* ~ 100 MB */
p=malloc(10000000); /* ~ 10 MB */
In solaris I get the following:
103 << Before malloc
13311 << After malloc
13311 << after first free
13311 << second malloc - memory reused
13311 << after second free
So, in Solaris, the memory is not returned to the OS, even if we do
not longer use part of it.
So, for a very long file, the new code can be using a lot of memory
and, even freed and no longer used by swish, this memory will not
be returned to the OS.
On 29 Aug 2001, at 7:49, Bill Moseley wrote:
> When I was indexing I had similar upper memory usage between the
> different versions -- just that the newer version got to the upper
> memory limit and stayed there. But maybe that depends on the type of
> documents that are being indexed? In other words, you think it's
> possible that Chris has files that are making swish use more memory
> than in the previous version?
> Is it really true that BSD doesn't free the memory? I just assumed
> that the reported number was not reduced after freeing memory.
Received on Wed Aug 29 17:10:38 2001