Received: by taz.hyperreal.com (8.6.12/8.6.5) id UAA14768;
 Mon, 22 Jul 1996 20:27:03 -0700
Received: from paris.ics.uci.edu by taz.hyperreal.com (8.6.12/8.6.5) with SMTP
 id UAA14748; Mon, 22 Jul 1996 20:26:59 -0700
Received: from liege.ics.uci.edu by paris.ics.uci.edu id aa13474;
          22 Jul 96 20:16 PDT
To: new-httpd@hyperreal.com
Subject: Re: robot denial 
In-reply-to: Your message of "Fri, 19 Jul 1996 10:34:07 MDT."
             <199607191634.AA196144048@ooo.lanl.gov>
Date: Mon, 22 Jul 1996 20:16:25 -0700
From: "Roy T. Fielding" <fielding@liege.ICS.UCI.EDU>
Message-ID: <9607222016.aa13474@paris.ics.uci.edu>
Sender: owner-new-httpd@apache.org
Precedence: bulk
Reply-To: new-httpd@hyperreal.com

> So if "/robot" is bad (Koster seems to think it's ok), then what would
> be valid?

What I mentioned -- "indexer", "link checker", "pre-fetcher", etc.
Each type of robot has different characteristics -- lumping them into
one category is a bad thing.

....Roy