Skip to main content.
home | support | download

Back to List Archive

[swish-e] Portable Indexes (WAS: Re: ANNOUNCE: 2.4.7 released)

From: Josh Rabinowitz <joshr-swishe(at)>
Date: Fri Apr 10 2009 - 13:04:40 GMT
Portable Indexes (WAS: Re: [swish-e] ANNOUNCE: 2.4.7 relea
Hello, All

At 9:17 PM -0500 4/9/09, Peter Karman wrote:
no. I just tested both scenarios (build on 32, search on 64 -- build on 64,
search on 32) and both gave that same error.
Others with more insight into the arch issue might have suggestions.

Yeah, we've been discussing for a while that indexes built on 32 or 64bit platforms were not portable between architectures.

I took a crack at making the indexes portable some time ago, by using specific 64bit signed and unsigned integers instead of types like int, unsigned int, and long. I got as far as the code compiling, but it would crash.

I've appended the script I was using to the end of this message.  If people like, I can dust it off and check it into the Swish-e SVN tree somewhere.

The basic issue has to do with the size of basic types like 'int' and 'long' (which change from 32bit to 64bit architectures), and the handling of the bits within.  Changing the types isn't too tricky - making sure that the code always addresses the types as appropriate is a little trickier, but not too bad methinks. It'll probably cause a lot of ripples across the code in things like printf() format strings.

Josh Rabinowitz

######### begin script #############

#!/usr/bin/perl -w
use strict;

# converts swish-e source code to all 64-bit integers.
# --> in progress.
# Copyright 2007 Josh Rabinowitz

# call main()

# main()
sub main {
     # #include <stdint.h> has the uint... typedefs used below
     # note that 1) stdint.h needs to be included in mem.h and swish.h
     # 2) wait() still needs a true 'int' in http
     # and 3) printf() formats need to be changed to match
     # see build warnings for more.
     # this works on 64bit CentosOS5, but not 32bit CentosOS5,
     # on which swish-e -h says:
     #  err: Missing switch character at ''.
Use -h for options.
     my @regexes = (
         q{ s/ \bunsigned\s+long\s+int\b /uint64_t/gx},
         q{ s/ \bunsigned\s+long\b       /uint64_t/gx},
         q( s/ \bunsigned\s+int\b        /uint64_t/gx),
         q( s/ \blong\s+long\s+int\b     /int64_t/gx),
         q( s/ \blong\s+long\b           /int64_t/gx),
         q( s/ \blong\s+int\b            /int64_t/gx),
         q( s/ \bint\b                   /int64_t/gx),
     my @files = glob( "src/*.c src/*.h");
     for my $file (@files) {
         print "$file\n";
         _apply_regexes( $file, @regexes );

# _apply_regexes( $file, @search_and_replace_regexes )
# backs up $file to $file.bak, and
# applies supplied regexes to the lines of a file,
sub _apply_regexes {
     my ($file, @regexes) = @_;
     # changes a file by applying the supplied regexes to each line
     my $tmpfile = "$file.tmp";
     open(my $rfh, "<", $file)    || die "$0: Can't open $file: $!";
     open(my $wfh, ">", $tmpfile) || die "$0: Can't open $tmpfile:
$!";  # clobber old $file.tmp
     print "Applying regexes:to file $file\n" . join("\n", @regexes) . "\n";
     while(<$rfh>) {
         for my $r (@regexes) {
             # $r should operate on $_ !
             eval $r;
             die "$0: Error in regex: $r: $@" if $@;
         print $wfh "$_\n";
     close($rfh) || die "$0: Can't open $file: $!";
     close($wfh) || die "$0: Can't close $tmpfile: $!";
     rename( $file, "$file.bak" );
     rename( $tmpfile, $file ) || die "$0: Can't rename $tmpfile to $file: $!";
-- Josh Rabinowitz                  --
-- --
-- SkateTalk Chat Systems(tm)    --
Users mailing list
Received on Fri Apr 10 09:04:56 2009