Skip to main content.
home | support | download

Back to List Archive

Re: fhash.c bug?

From: Peter Karman <peter(at)not-real.peknet.com>
Date: Mon Mar 28 2005 - 17:36:35 GMT
I traced the loop to uncompress1(), then googled for f_getc, which turned up (of 
all things), the Swish-e archive.

I need to search the archive better in the first place. :)

Looks like Patrick O'Lone posted a fix for this in Dec (which matches your Jan 
timeframe), but the ensuing thread makes it sound like his fix wasn't the best 
solution.

http://swish-e.org/archive/2004-12/8695.html

I tried inserting the break and it fixed my immediate problem.

However, it's a bit of a red herring.

In my case, the problem was simply that the code was checking to see if the file 
existed yet in the hash and was getting hung decompressing the key. In Patrick's 
case it was with -L in searching and extracting properties. Same function, but 
hanging in two totally different cases.

I followed the advice Bill and Patrick suggested and inserted this:

     do
     {
         _c = (int) f_getc(fp);


    /* FIX */
         if (_c < 0) {
              progerr("\nfatal err: _c is < 0 in uncompress1\n");
         }

    /* end FIX */

         num <<= 7;
         num |= _c & 127;
         if (!num)
             break;
     }
     while (_c & 128);

around line 158 in compress.c

That fixes the hung loop, but not the error that got me there. So instead of 
hanging, now I just get a fatal error (is that progress?).

Seems to me the error (in my case anyway) is here in fhash.c, around line 191:

         sw_fseek(fp,next,SEEK_SET);
         sw_fread((unsigned char *)&tmp,sizeof(tmp),1,fp);
         next = UNPACKFILEOFFSET(tmp);

because next is coming back as 0, which is what leads to the uncompress1() call.
I don't know enough about the I/O functions being used there to hazard a guess...

In any case, I think progerr() ought be thrown in the uncompress1() call as 
above. I'll check that fix in (though as Bill notes in the thread above, that 
feels a little ugly...)



Dobrica Pavlinusic scribbled on 3/28/05 9:44 AM:
> On Mon, Mar 28, 2005 at 07:02:24AM -0800, Peter Karman wrote:
> 
>>>What is your smallest fileset on which you can demonstrate problem?
>>
>>I can name that tune in 12 files. I have put a tar at 
>>http://peknet.com/swish/12badfiles.tar.gz
>>
>>can you duplicate the error with that set? for me it consistently hangs on 
>>'acl_size.3c.html'.
> 
> 
> Strange. It works for me just fine. I used latest CVS version compiled
> with:
> 
> $ ./configure --disable-docs --with-pcre --enable-incremental --disable-shared
> 
> I indexed it using:
> 
> $ swish-e -S fs -f 1/test -i karman/
> 
> And it worked for me. Then I tried
> 
> $ swish-e -S fs -f 1/test -i karman/ -u
> 
> which also worked. However, than I remembered that I had problems with
> repeated indexing of same data, so I wrote following script (in-lined so
> that list doesn't remove it):
> 
> #!/bin/sh
> 
> ./swish-e -S fs -i karman -f 1/bug 2>/dev/null
> nr=0
> while true ; do
> 	nr=`expr $nr + 1`
> 	echo "LOOP: $nr"
> 	touch karman/* && ../swish-e -S fs -i karman -f 1/bug -u 2>/dev/null || exit
> done
> 
> After about 500 loops it always dumps core on me. It doesn't dump
> core on same loop, however.
> 
> I also noticed that "unique words indexed" count get incremented from
> time to time. Recompiling with --enable-psortarray didn't help.
> 
> Well, now we have semi-reproducible bug. I do need to finish various
> other stuff for tomorrow, so further investigation is pending for now.
> 

-- 
Peter Karman  .  http://peknet.com/  .  peter(at)not-real.peknet.com
Received on Mon Mar 28 09:36:38 2005