Skip to main content.
home | support | download

Back to List Archive

Re: Updated build_swish.sh

From: Greg Fenton <greg_fenton(at)not-real.yahoo.com>
Date: Mon Nov 11 2002 - 16:50:48 GMT
Bloody...grr...

Hope the line-wrapping doesn't get messed up...

Terribly sorry!
greg_fenton.
---------------------------------------------------------------------------
#!/bin/sh

# build_swish.sh
#
# A script to build SWISH-E databases ( http://www.swish-e.org/ )
#
# Distributed under the GNU GPL version 2 or later
# ( http://www.gnu.org/licenses/gpl.html )
#
#
# Description:
# -----------
#
# This script (re)builds a SWISH-E database based on a number of
# configurable attributes.  It has been tested with swish-e 2.2.1
# and 2.2.2rc1
#
# The default configuration settings assume the following directory
# structure (though this is easily changed):
#
#  /var/www        - root of the web tree, usually owned by 'apache' or
'httpd'
#       |
#       +-swish-e - top directory of swish-e config and database files
#         |
#         +-bin   - directory containing swish-e command files
(binaries)
#         |
#         +-conf  - directory containing swish-e configuration file(s)
#         |
#         +-db    - directory containing swish-e database file(s)
#
# If you wanted to have the swish-e directory in another location like,
# for example, /home/httpd then simply modify the settins in the
# Configuration Section accordingly.
#
#
# Revision History:
# ----------------
#  2002/11/08 - GLF - First public release
#  2002/11/11 - GLF - Added REPORT_NO_BUILD option
#             - GLF - Added EXIT_CODE_FOR_NO_BUILD option
#
#
# Authors:
# -------
#  GLF - Greg Fenton - <greg_fenton@_NOSPAM_yahoo.com>
#
# $Revision: 1.6 $
# $Date: 2002/11/11 16:46:50 $

#############################################################################
##
## --------------------- Start of Configuration Section
---------------------
##

# The full path to the basename command.
BASENAME="/bin/basename"
# The full path to the cp command.
CP="/bin/cp"
# The full path to the date command.
DATE="/bin/date"
# The full path to the mail command.
MAIL="/bin/mail"
# The full path to the mkdir command.
MKDIR="/bin/mkdir"
# The full path to the mv command.
MV="/bin/mv"
# The full path to the rm command.
RM="/bin/rm"
# The full path to the touch command.
TOUCH="/bin/touch"


# The full path to a temporary report file.
# The parent directory must already exist.
REPORT="/var/www/swish-e/build.report"

# If set to anything (i.e. not blank) report messages will not be
echoed
# to STDOUT.  For example, if run from a cronjob and you set
FROM_CRON=1
# in the crontab, then REPORT_QUIET=$FROM_CRON will not echo messages
# (echo'ed messages get emailed to the MAILTO value after the cronjob
ends).
REPORT_QUIET="$FROM_CRON"

# The list of email addresses that receive the report (separated by
spaces).
# If left unset, no email will be sent at the end of successfully
completing.
# e.g. REPORT_DEST="webmaster@localhost root@localhost"
REPORT_DEST="webmaster@localhost root@localhost"

# The list of email addresses that receive error messages (separated by
spaces).
# If left unset, no email will be sent upon an error.
# e.g. ERROR_DEST="root@localhost"
ERROR_DEST="root@localhost"

# If set to anything (i.e. not blank), a report message will be sent to
# REPORT_DEST if a build is not needed.
# e.g. REPORT_NO_BUILD="true"
REPORT_NO_BUILD=

# The exit code to use if no build is done.  This value does not
directly
# affect this script, but may affect other entities calling this script
# if they rely on its return code.  If not set (i.e. left blank) most
# environments will treat this as 0...but don't rely on this!
# NOTE: The script currently uses exit codes 1-7 for various errors.
# e.g. EXIT_CODE_FOR_NO_BUILD=-1
EXIT_CODE_FOR_NO_BUILD=0

# The filename to archive the report to when completed.  If left blank
# the REPORT file is left in place after emailing to the appropriate
# destinations. The parent directory must aleady exist.
# (Using `$DATE +"%b%d_%H_%M_%S"` will generate a string similar to:
#  Nov08_11_16_23 )
REPORT_ARCHIVE="/var/www/logs/build_`$DATE +"%b%d_%H_%M_%S"`.log"

# The directory in which the current db exists (but the new will soon
exist!)
# This directory must already exist.
SWISH_DB_DIR="/var/www/swish-e/db"

# We build the new database in a directory separate from the current
db.
# This directory must *not* already exist, but its parent directory
must exist.
SWISH_NEW_DB_DIR="/var/www/swish-e/db_new"

# Temporary name used during swap of current and new databases.
# If the swap is successful, this directory (now the "old" database) is
deleted.
# This directory must not already exist, but its parent directory must
exist.
#
# Note: Typically SWISH_DB_DIR, SWISH_NEW_DB_DIR and SWISH_OLD_DB_DIR
should
#       exist on the same filesystem to make the swap happen quickly.
SWISH_OLD_DB_DIR="/var/www/swish-e/db_old"

# The directory in which the swish command should be run (this is not
# necessarily the directory in which the database is built).  By
# default swish will create temporary files in this directory.  Set
# the TEMPDIR environent variable or the TmpDir setting to have
# temporary files created elsewhere.
# This directory must already exist.
SWISH_RUN_DIR="/var/www/swish-e"

# The full path to the swish-e command.
SWISH_CMD="/var/www/swish-e/bin/swish-e"

# The full path to the configuration file for swish-e.
SWISH_CONF_FILE="/var/www/swish-e/conf/swish.conf"

# The data source that swish-e is to use (see "-S" option in the
swish-e
# help file).
# Must be one of "fs", "http" or "prog".
SWISH_SOURCE="prog"

# The full path to the bulid flag file.  If this file exists then the
# build proceeds, otherwise it aborts (nicely).  Leave its value blank
# to skip the check (force a build).  This is useful if you have a
# process updating your website remotely (say, from a CVS repository).
# Have that process "touch" the flag file to indicate is build is
# necessary. The file is deleted during the build process.
SWISH_BUILD_FLAG="/var/www/swish-e/build.flag"

# The full path to the build lock file.  This file indicates that a
build
# is currently under way.  When the build starts, it checks for this
file
# and if it exists this build will abort (nicely).  If the file does
not
# exist, this build will create one.  The file will be deleted at the
end
# of a successful build.
# The parent directory must already exist.
SWISH_BUILD_LOCK="/var/www/swish-e/build.lock"

# The method of handling errors.  If set to "lock", then when an error
# occurs during a build the SWISH_BUILD_LOCK file will not be deleted
# (thereby halting further builds until errors have been resolved).
# Set this value to anything else (or leave blank) to force the
deletion
# of the lock file. If SWISH_BUILD_LOCK is not set, this value is
ignored.
SWISH_BUILD_ON_ERROR="lock"

##
## ----------------- End of Configuration Section
---------------------------
##
#############################################################################



ME=`$BASENAME $0`

# a function to clean up temp files and email the $REPORT file
#
# param1: the subject of the email
# param2: the destination list (email addresses separated by spaces)
#
function exit_script {

    SUBJECT=$1
    DEST=$2
    EXIT_CODE=$3

    # remove lock file, if requested
    if [ -n "$SWISH_BUILD_LOCK" ]
    then
        if [ $EXIT_CODE = 0 -o "$SWISH_BUILD_ON_ERROR" != "lock" ]
        then
            rm -f "$SWISH_BUILD_LOCK" >> $REPORT 2>&1
        fi
    fi

    # put back the build flag, if need be
    if [ -n "$SWISH_BUILD_FLAG" -a $EXIT_CODE != 0 ]
    then
	$TOUCH "$SWISH_BUILD_FLAG" >> $REPORT 2>&1
    fi

    # send email, if we can
    if [ -n "$DEST" -a -x "$MAIL" ]
    then

        # Send out the report ...
        $MAIL -s "$SUBJECT" $DEST < $REPORT

	if [ -n "$REPORT_ARCHIVE" -a ! -e "$REPORT_ARCHIVE" ]
	then
	    $CP $REPORT $REPORT_ARCHIVE && $RM $REPORT
	fi

    elif [ -n "$DEST" ]
    then
        report "ERROR: cannot execute mail command, MAIL=[$MAIL]"
    fi

    exit $EXIT_CODE
}


# a function to report a given message to STDOUT and to $REPORT
#
# param1: the message to report
#
function report {
    d=`$DATE +"%a %b %d %H:%M:%S"`
    if [ -z "$REPORT_QUIET" ]
    then
        echo "[$d] $1"
    fi
    echo "[$d] $1" >> $REPORT
}


#-------------------------------------------------------------
#
# Prepare the report file
#

$RM -f $REPORT
$TOUCH $REPORT

report "Start  (using [$SHELL])"


#
# Check if a build is necessary
#
if [ -n "$SWISH_BUILD_FLAG" ]
then
    if [ ! -e "$SWISH_BUILD_FLAG" ]
    then
        report "Build flag not found"
	if [ -z "$REPORT_NO_BUILD" ]
	then
	    unset REPORT_DEST
	fi
        exit_script "Build not needed" "$REPORT_DEST"
"$EXIT_CODE_FOR_NO_BUILD"
    else
        report "Found build flag; remove it now"
        $RM "$SWISH_BUILD_FLAG" >> $REPORT 2>&1  # don't worry if this
fails
    fi
fi


#
# Get the build lock
#
if [ -e $SWISH_BUILD_LOCK ]
then
    report "Build lock already exists"
    exit_script "Warning: Build already underway?" "$ERROR_DEST" 1
fi

$TOUCH $SWISH_BUILD_LOCK >> $REPORT 2>&1
if [ $? != 0 ]
then
    report "Could not create build lock file"
    exit_script "Error: could not get build lock" "$ERROR_DEST" 2
fi

report "Got build lock"


#
# Make a directory to build the new database in.
#
$MKDIR $SWISH_NEW_DB_DIR >> $REPORT 2>&1

if [ $? != 0 ]
then
    report "Failed to create new db dir [$SWISH_NEW_DB_DIR]"
    exit_script "Error: swish-e build failed" "$ERROR_DEST" 3
fi


#
# Run swish-e to index the pages/documents
#
report "Start the database build"
( cd $SWISH_RUN_DIR && $SWISH_CMD -S $SWISH_SOURCE -c $SWISH_CONF_FILE
) \
    >> $REPORT 2>&1

if [ $? != 0 ]
then
    report "Failed to build new database"
    exit_script "Error: swish-e build failed" "$ERROR_DEST" 4
fi

report "Database build complete.  Now swap databases."


#
# Swap the old db out and the new db in
#
$MV -v $SWISH_DB_DIR $SWISH_OLD_DB_DIR >> $REPORT 2>&1
if [ $? != 0 ]
then
    report "Failed to swap out old database"
    exit_script "Error: failed to swap out old database" "$ERROR_DEST"
5
fi

$MV -v $SWISH_NEW_DB_DIR $SWISH_DB_DIR >> $REPORT 2>&1
if [ $? != 0 ]
then
    $MV -v $SWISH_OLD_DB_DIR $SWISH_NEW_DB_DIR >> $REPORT 2>&1
    report "Failed to swap new database into place"
    exit_script "Error: failed to swap in new database" "$ERROR_DEST" 5
fi

report "Database swap complete.  Now remove old database."

#
# Delete the old database
#
$RM -rf $SWISH_OLD_DB_DIR >> $REPORT 2>&1

if [ $? != 0 ]
then
    report "Failed to remove old database"
    exit_script "Warning: failed to remove old database" "$ERROR_DEST"
6
fi


#
# Remove the lock file
#
$RM -f $SWISH_BUILD_LOCK >> $REPORT 2>&1

if [ $? != 0 ]
then
    report "Failed to remove build lock"
    exit_script "Warning: failed to remove build lock" "$ERROR_DEST" 7
fi


#
# Success!
#
report "New database built and installed"
exit_script "Swish build successful" "$REPORT_DEST" 0

--------------------------------------------------------------------------

=====
Greg Fenton
greg_fenton@yahoo.com

__________________________________________________
Do you Yahoo!?
U2 on LAUNCH - Exclusive greatest hits videos
http://launch.yahoo.com/u2
Received on Mon Nov 11 16:50:59 2002