Skip to main content.
home | support | download

Back to List Archive

Re: A swish-e performance question

From: Ziegenhorn, Eric <eric.ziegenhorn(at)not-real.ngc.com>
Date: Thu Oct 30 2003 - 17:44:54 GMT
Awesome.  Do you know if it is likely that having 1.5 GB of RAM may become a
limiting factor before the size of the index file with 64-bit support?

Thanks!
Eric

-----Original Message-----
From: jmruiz@boe.es [mailto:jmruiz@boe.es]
Sent: Thursday, October 30, 2003 9:33 AM
To: Ziegenhorn, Eric
Cc: swish-e@sunsite.berkeley.edu
Subject: Re: [SWISH-E] Re: A swish-e performance question



Hi Eric,

I am working now in 64 it support.

Jose

On 29 Oct 2003 at 10:09, Ziegenhorn, Eric wrote:

> On 17 Oct 2003 at 15:33, Patrick O'Lone wrote:
> 
> > All,
> > 
> > > 
> > > > There is a 2GB limit index file. So I think that you will reach
> > > > the limit of swish easily. But, the solution must be easy. You
> > > > can hack the source the code and make it 64 bit. The trick is
> > > > easy. Look for fopen, fclose, fseek, fwrite... routines in the
> > > > code, mainly in db_native.c
> > > >
> > > > I have seen this question before. Anyone in the list has made
> > > > anything? If not, and if I find some time I will try to add this
> > > > feature. Now, I am fighting against the PHP module.
> > > >
> > 
> > Can't you just compile with the -D_FILE_OFFSET_BITS=64
> > -D_LARGEFILE_SOURCE? Shouldn't this just make it 64-bit assuming the
> > file system the index resides on is LFS-aware? You shouldn't have to
> > go through the code and convert fopen() to fopen64().
> > 
> 
> >Nope. There must be also changed the offset types. fseek and ftell
> >uses a long (usually 32 bit). 
> >
> >Basically, fseek anf ftell should be moved to fseeko and ftello. This
> >uses an off_t type instead of a long type. So there are more things
> >to change to allow swish internal data to hold the 64 bit offsets.
> >
> >I am working on this issue now. As Bill points, this may be a feature
> >for a future 2.4.1.
> >
> >This is true for UNIX/LINUX system but Ms Win32 is a different story
> >as Dave told me.
> >
> >cu
> >Jose
> 
> 
> Does anyone know that if Swish-e was changed to handle large files
> over 2GB if memory usage would likely still be a limiting factor?  I'm
> on a machine with 1.5GB RAM, but I'm trying to index 60GB of files. 
> Anyone know if this will be possible in the near future or how I could
> make it possible?  Any guesses on a release date for 2.4.1 with 64-bit
> file support?  I may be able to work on it if it seems acheivable in
> the near term.
> 
> Thanks,
> Eric
Received on Thu Oct 30 17:57:09 2003