forrest-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dave Brondsema <d...@brondsema.net>
Subject Re: svn commit: rev 23083 - in forrest/trunk/src/forrestbot/webapp/src/webapp: . common
Date Tue, 20 Jul 2004 16:09:09 GMT
On Tue, 20 Jul 2004, Juan Jose Pablos wrote:

> brondsem@apache.org escribió:
> > Author: brondsem
> > Date: Tue Jul 20 05:40:54 2004
> > New Revision: 23083
> >
> > Modified:
> >    forrest/trunk/src/forrestbot/webapp/src/webapp/common/layout.jsp
> >    forrest/trunk/src/forrestbot/webapp/src/webapp/common/simple.jsp
> >    forrest/trunk/src/forrestbot/webapp/src/webapp/common/viewlog_frame.jsp
> >    forrest/trunk/src/forrestbot/webapp/src/webapp/viewlog_header.jsp
> > Log:
> > robot meta tags to discourage indexing the generated websites
>
> what about a robots.txt file?
>

I thought about that, but that only works at domain.com/robots.txt and not
at a directory like: domaind.com/forrestbot/robots.txt.  Meta tags are not
as widely supported as robots.txt but googlebot, msnbot and inktomi slurp
support it so I think that's good enough.

-- 
Dave Brondsema : dave@brondsema.net
http://www.brondsema.net : personal
http://www.splike.com : programming
http://csx.calvin.edu : student org

Mime
View raw message