httpd-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dharmendra.T" <dha...@nsecure.net>
Subject RE: [users@httpd] robots.txt
Date Thu, 02 Jan 2003 10:49:47 GMT



On Thu, 2003-01-02 at 15:56, Jeff Cohen wrote:
> That's the problem, I don't have such file robots.txt. I guess the
> content should be text isn't? :)
> But Chris says not to be worried, so I'm not.
> Thanks Chris.
> 
> Jeff Cohen
> 
> > -----Original Message-----
> > From: Dharmendra.T [mailto:dharmu@nsecure.net]
> > Sent: Thursday, January 02, 2003 4:09 AM
> > To: users@httpd.apache.org
> > Subject: Re: [users@httpd] robots.txt
> > 
> > On Thu, 2003-01-02 at 14:07, Jeff Cohen wrote:
> > > Hi All,
> > >
> > > Does anyone of you get the robots.txt file requests from the
> internet??
> > > It seems that I'm starting to get it almost every 3-4 hours a day,
> every
> > > time from a different IP.
> > > Is there a Virus going outta there that I'm not aware of?
> > > How can I block these requests for this file on the root directory?
> > >
> > > Help would be very appreciated,
> > >
> > > Jeff Cohen
> > 
> > What is the format of the file? Try
> > #file robots.txt
> > 
> > What does the file contains?
> > --
> > Dharmendra.T
> > Linux Enth
> > 
> > 
> > ---------------------------------------------------------------------
> > The official User-To-User support forum of the Apache HTTP Server
> Project.
> > See <URL:http://httpd.apache.org/userslist.html> for more info.
> > To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> >    "   from the digest: users-digest-unsubscribe@httpd.apache.org
> > For additional commands, e-mail: users-help@httpd.apache.org
> 
> 
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
>    "   from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
> 

This link should help you:

http://www.searchengineworld.com/robots/robots_tutorial.htm


-- 
Dharmendra.T
Linux Enthu


---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


Mime
View raw message