httpd-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From André Warnier ...@ice-sa.com>
Subject Re: [users@httpd] Weird Ip in access log, need help to understand
Date Thu, 19 Jun 2008 14:37:11 GMT


Artem Kuchin wrote:
> 
> 
> André Warnier пишет:
>>
>>
>> Artem Kuchin wrote:
>>> I recently upgraded to 2.2 and now i see thing which i have never 
>>> seen mefore in
>>> the access log:
>>>
>>> 74.6.29.159, 74.6.8.118 - - [11/Jun/2008:06:28:55 +0400] "GET 
>>> /Go/ViewProduct/id=883 HTTP/1.1" 200 7195 "-" "Mozilla/5.0 
>>> (compatible; Yahoo! Slurp; http://he
>>> lp.yahoo.com/help/us/ysearch/slurp)"
>>
>> Following the link to :
>> http://help.yahoo.com//help/us/ysearch/slurp
>> may help understand
>>
> 
> no, it does not. The problem is not with slurp and is not related to any 
> particular client. It is not even
> a problem in a common meaning. The question is WHY in this record are 2 
> (TWO) ip address in
> the first field of the record: 74.6.29.159, 74.6.8.118
> How it is possible at all? One request comes only from one ip. Where 
> apache takes these
> addresses?
> 
Well ok.
Now both IP's are indeed  from yahoo's crawlers (of which there are 
probably many hundreds) :
dent:~# nslookup 74.6.29.159
Non-authoritative answer:
159.29.6.74.in-addr.arpa        name = lj512625.crawl.yahoo.net.
dent:~# nslookup 74.6.8.118
Non-authoritative answer:
118.8.6.74.in-addr.arpa name = llf520026.crawl.yahoo.net.

What I would guess is that Apache somehow regroups identical and 
simultaneous requests into a single log line.  And I would guess that 
Yahoo crawlers work "in teams", so that it is not surprising that 
several requests would arrive simultaneously to your server.
I'm just guessing here.

But it's interesting anyway.  I have not really looked specifically, but 
I don't recall seeing such lines on any of our servers.

A question : is the URL in those accesses, a real URL that exists on 
your site ?  I see a 200 OK return code, but some sites return a page 
even for an invalid URL.
If it is not, then there is one item on the Yahoo page that may explain 
why there are such quasi-simultaneous requests.  I think it's called 
"why is yahoo trying to get strange urls from my site ?"

André




---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


Mime
View raw message