Return-Path: Delivered-To: apmail-apache-bugdb-archive@apache.org Received: (qmail 26415 invoked by uid 500); 25 Aug 2000 09:00:19 -0000 Mailing-List: contact apache-bugdb-help@apache.org; run by ezmlm Precedence: bulk X-No-Archive: yes Reply-To: apache-bugdb@apache.org list-help: list-unsubscribe: list-post: Delivered-To: mailing list apache-bugdb@apache.org Received: (qmail 26144 invoked by uid 501); 25 Aug 2000 09:00:05 -0000 Resent-Date: 25 Aug 2000 09:00:05 -0000 Resent-Message-ID: <20000825090005.26143.qmail@locus.apache.org> Resent-From: gnats-admin@bugz.apache.org (GNATS Management) Resent-To: apache-bugdb@apache.org Resent-Cc: apache-bugdb@apache.org Resent-Reply-To: gnats-admin@bugz.apache.org, shibin@public3.bta.net.cn Received: (qmail 25581 invoked by uid 501); 25 Aug 2000 08:58:39 -0000 Message-Id: <20000825085839.25580.qmail@locus.apache.org> Date: 25 Aug 2000 08:58:39 -0000 From: Xiao Shibin Reply-To: shibin@public3.bta.net.cn To: submit@bugz.apache.org X-Send-Pr-Version: 3.110 Subject: other/6466: some advised functions >Number: 6466 >Category: other >Synopsis: some advised functions >Confidential: no >Severity: non-critical >Priority: medium >Responsible: apache >State: open >Quarter: >Keywords: >Date-Required: >Class: change-request >Submitter-Id: apache >Arrival-Date: Fri Aug 25 02:00:02 PDT 2000 >Closed-Date: >Last-Modified: >Originator: shibin@public3.bta.net.cn >Release: Apache2.x >Organization: apache >Environment: All OSes >Description: When I spider a web site which has more than 1 million pages, only few pages changed in the most time, but I have to spider almost all the pages. If Web Server can return only the new pages urls which newer than a certain date, the spider will do very fast. Can Apache add this function in 2.0 version? >How-To-Repeat: >Fix: >Release-Note: >Audit-Trail: >Unformatted: [In order for any reply to be added to the PR database, you need] [to include in the Cc line and make sure the] [subject line starts with the report component and number, with ] [or without any 'Re:' prefixes (such as "general/1098:" or ] ["Re: general/1098:"). If the subject doesn't match this ] [pattern, your message will be misfiled and ignored. The ] ["apbugs" address is not added to the Cc line of messages from ] [the database automatically because of the potential for mail ] [loops. If you do not include this Cc, your reply may be ig- ] [nored unless you are responding to an explicit request from a ] [developer. Reply only with text; DO NOT SEND ATTACHMENTS! ]