tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From André Warnier ...@ice-sa.com>
Subject Re: Filtering HTTP OPTIONS request method from logs?
Date Tue, 17 Sep 2013 09:05:46 GMT
Jim Barber wrote:
> All,
> 
> On 16/09/2013 10:52 PM, Christopher Schultz wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA256
>>
>> Felix,
>>
>> On 9/16/13 10:25 AM, Felix Schumacher wrote:
>>> Am Montag, den 16.09.2013, 10:02 -0400 schrieb Christopher
>>> Schultz:
>>>> Jim,
>>>>
>>>> On 9/16/13 3:42 AM, Jim Barber wrote:
>>>>> I'm hoping someone on this list can help me since I've been
>>>>> reading docs, mailing lists, FAQs, and so on for hours now, and
>>>>> I'm not having much luck finding an answer to my question.
>>>>>
>>>>> I am using Tomcat version 7.0.42 as packaged in Debian Linux.
>>>>> In front of my Tomcat servers, I am using haproxy for load
>>>>> balancing. The haproxy load balancers are using the HTTP
>>>>> OPTIONS request method to check if the Tomcat servers are alive
>>>>> and healthy.
>>>>>
>>>>> This results in log entries like the following in the Tomcat
>>>>> accesslog file:
>>>>>
>>>>> 10.122.32.4 - - [16/Sep/2013:17:12:49 +1000] "OPTIONS /
>>>>> HTTP/1.0" 200 - 10.122.32.4 - - [16/Sep/2013:17:12:51 +1000]
>>>>> "OPTIONS / HTTP/1.0" 200 - 10.122.32.4 - -
>>>>> [16/Sep/2013:17:12:53 +1000] "OPTIONS / HTTP/1.0" 200 -
>>>>> 10.122.32.4 - - [16/Sep/2013:17:12:55 +1000] "OPTIONS /
>>>>> HTTP/1.0" 200 - 10.122.32.4 - - [16/Sep/2013:17:12:57 +1000]
>>>>> "OPTIONS / HTTP/1.0" 200 - 10.122.32.4 - -
>>>>> [16/Sep/2013:17:12:59 +1000] "OPTIONS / HTTP/1.0" 200 -
>>>>> 10.122.32.4 - - [16/Sep/2013:17:13:01 +1000] "OPTIONS /
>>>>> HTTP/1.0" 200 - 10.122.32.4 - - [16/Sep/2013:17:13:03 +1000]
>>>>> "OPTIONS / HTTP/1.0" 200 - 10.122.32.4 - -
>>>>> [16/Sep/2013:17:13:05 +1000] "OPTIONS / HTTP/1.0" 200 -
>>>>> 10.122.32.4 - - [16/Sep/2013:17:13:07 +1000] "OPTIONS /
>>>>> HTTP/1.0" 200 - 10.122.32.4 - - [16/Sep/2013:17:13:09 +1000]
>>>>> "OPTIONS / HTTP/1.0" 200 - 10.122.32.4 - -
>>>>> [16/Sep/2013:17:13:11 +1000] "OPTIONS / HTTP/1.0" 200 -
>>>>>
>>>>> At the moment I'm getting one of these every 2seconds, but I
>>>>> haven't enabled the second load balancer for HA purposes yet.
>>>>> When I do that, I'll be getting twice as many hits of this
>>>>> type.
>>>>>
>>>>> This is going to result in rather large log files full of
>>>>> noise that I'm not interested in.
>>>>
>>>> Playing the devil's advocate here a bit...
>>>>
>>>> Why wouldn't you be interested in getting these logs? They are
>>>> requests being handled by your web server. They require (a small
>>>> amount of) time and resources to process, and indicate that your
>>>> lb is still reaching-out to determine the status of the app
>>>> server.
>>>>
>>>> My recommendation would be to leave those logs in there (they
>>>> accurately describe a real request) and filter them out if you
>>>> want to do some kind of analytics against your log files and
>>>> consider those OPTIONS requests to be noise.
>>> I have had one case where I wanted to get rid of those requests
>>> too, so I can understand the OP. But I have to admint I had a scary
>>> feeling about it.
>>>
>>>>
>>>>> <Valve className="org.apache.catalina.valves.AccessLogValve"
>>>>> directory="logs" prefix="localhost_access_log." suffix=".txt"
>>>>> pattern="%h %l %u %t &quot;%r&quot; %s %b" />
>>>>>
>>>>> Specifically adding the condition="<VALUE>" attribute, but I
>>>>> have no idea what to set <VALUE> to.
>>>>
>>>> It's not that simple: if you want to use "condition", then you
>>>> have a write a Valve (can't be a Filter, since it must run
>>>> *before* the AccessLogValve) that tests the request and sets a
>>>> request attribute that will then trigger this condition.
>>>
>>> That is not true, you can use a filter, since the logging will
>>> happen *after* the request and can and will check the request
>>> attribute then.
>>
>> Thanks for pointing that out. After I had sent my message, I realized
>> that and decided not to post a followup after reading yours.
>>
>> - -chris
> 
> Thank you all for your responses.
> It looks like I'll just have to put up with these messages.
> There seems to be no easy way to filter them out.
> 

Actually, based on earlier responses and on the URLrewrite filter 
(http://http://tuckey.org/urlrewrite/), there may be a solution after all, which does not

involve additional Java programming, as long as you are willing to do some research by 
yourself.

(Short intro : the URLrewrite filter is a bit of a workhorse, simioar to mod_rewrite for 
Apache httpd, and which can do a multitude of things when it comes to filter/modify HTTP 
requests in Tomcat)

First, get the URLrewrite User's Manual at 
http://urlrewritefilter.googlecode.com/svn/trunk/src/doc/manual/4.0/index.html

and then search for :

<condition> element  (see "method" and "remote-addr")

and

<set> element  (see "request (default)") : The same as request.setAttribute([name],

[value]) (note, name must be set).

So, the requests that you want to "not log" look like this :
 >>>>> 10.122.32.4 - - [16/Sep/2013:17:12:49 +1000] "OPTIONS /
 >>>>> HTTP/1.0" 200 - 10.122.32.4 - - [16/Sep/2013:17:12:51 +1000]

and thus they address the "/" (default) webapp, originate from the client IP 10.122.32.4,

and have the method "OPTIONS".

And on the other hand, the AccessLogValve has an attribute which allows you to specify 
that if the request has an attribute (e.g.) "nolog" set, it will not be logged.

So, by configuring your ROOT (default) webapp to include the URLrewrite filter, and 
configuring this filter properly, you should be able to attain your goal.

Enough tips now, get to work..


(And apart from your current issue, it really does not hurt to get to know the URLrewrite

filter, because it is really a very useful and powerful piece of software).


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message