Skip to main content.
home | support | download

Back to List Archive

Re: Re: Any way to restrict URLs with http method since

From: Ron Klatchko <ron(at)not-real.library.ucsf.edu>
Date: Thu Apr 29 1999 - 21:55:53 GMT
At 05:34 PM 4/29/99 -0400, john.leth@gulfaero.com wrote:
>o Stop indexing of anything that might possibly be under construction,
>confidential, or an outright security risk.

Wouldn't you want those excluded by robots.txt anyway?  Otherwise
commerical search engines will index them.
	
moo
----------------------------------------------------------------------
          Ron Klatchko - Manager, Advanced Technology Group           
           UCSF Library and Center for Knowledge Management           
                        ron@library.ucsf.edu                
Received on Thu Apr 29 14:52:31 1999