httpd-docs mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "William A. Rowe, Jr." <>
Subject Re: Auto-docs rewrite
Date Sat, 06 Aug 2005 15:37:25 GMT
At 04:53 AM 8/6/2005, André Malo wrote:
>> >I'd consider query?directive not as that easy as it could be. Think of
>> > the users that type them manually.
>> Yes, I'm thinking of manual entry; and i want something that is
>> clearly distinct from normal filesystem entries, for the sake of
>> various query, spiders, and caching approaches.
>HTTP doesn't know about file systems, so I'm wondering why a spider would 
>care. URLs with query string are indexed as any other URL for a long time 

You seem to be missing my point; I don't want spiders crawling that
section, it will be lookup intensive.  If users google they can hit
the very same information from the normal pages.  

That said, obviously we can use robots.txt to exclude query* and be
done with it.  Not sure if all spiders would grok excluding //*.
We could certainly use /411/xxxx if that's quicker than /411?xxxx


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message