forrest-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Crossley <cross...@apache.org>
Subject Re: web.archive.org stoped indexing our site
Date Thu, 10 May 2007 09:33:49 GMT
Thorsten Scherler wrote:
> Hi all,
> 
> when I was trying to find the origin of the dead link reported by Bart I
> found out that the waybackmaschine has stop indexing our web site.
> http://web.archive.org/web/*/http://forrest.apache.org
> 
> The last entry is Apr 24, 2006, which makes me think whether we changed
> the .htaccess or robot.txt to exclude crawlers. Is this by purpose?

I am not aware anything changed at our end.

Try our dev archives, Ferdinand and i talked
about this before.

You get a similar result for www.apache.org

I wonder if the results are deliberately old,
i.e. they don't show the last six months results
so that people don't use it as a daily tool.

-David

Mime
View raw message