www-apache-bugdb mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xiao Shibin <shi...@public3.bta.net.cn>
Subject other/6466: some advised functions
Date Fri, 25 Aug 2000 08:58:39 GMT

>Number:         6466
>Category:       other
>Synopsis:       some advised functions
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    apache
>State:          open
>Class:          change-request
>Submitter-Id:   apache
>Arrival-Date:   Fri Aug 25 02:00:02 PDT 2000
>Originator:     shibin@public3.bta.net.cn
>Release:        Apache2.x
All OSes
When I spider a web site which has more than 1 million pages, only few pages changed in the
most time, but I have to spider almost all the pages.

If Web Server can return only the new pages urls which newer than a certain date, the spider
will do very fast.

Can Apache add this function in 2.0 version?


 [In order for any reply to be added to the PR database, you need]
 [to include <apbugs@Apache.Org> in the Cc line and make sure the]
 [subject line starts with the report component and number, with ]
 [or without any 'Re:' prefixes (such as "general/1098:" or      ]
 ["Re: general/1098:").  If the subject doesn't match this       ]
 [pattern, your message will be misfiled and ignored.  The       ]
 ["apbugs" address is not added to the Cc line of messages from  ]
 [the database automatically because of the potential for mail   ]
 [loops.  If you do not include this Cc, your reply may be ig-   ]
 [nored unless you are responding to an explicit request from a  ]
 [developer.  Reply only with text; DO NOT SEND ATTACHMENTS!     ]

View raw message