tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher Schultz <ch...@christopherschultz.net>
Subject Re: how to block the duplicated requests?
Date Tue, 19 May 2015 13:16:29 GMT
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

To whom it may concern,

On 5/19/15 8:09 AM, javalishixml wrote:
> Just understood you. Really appreciate for your feedback.
> 
> 
> How do we judge it's a robot? item1: we find the request IP is
> always the same one. item2: our page may contains several
> keep-alive connections. But the attack connection only focus on
> connection.

Based upon the first request, how can you tell that the robot is going
to make later keep-alive requests?

> Based on these 2 items, we think the client is a robot.

Can you write some pseudo-code that shows the algorithm in its
simplest form?

> I think maybe putting these 2 items together to consider it as a 
> robot is a bit complex. Let's do it from the simple point.
> 
> If we always find there is a same IP request our website the same
> url for many times, can I block this request at httpd level?

This sounds like a job for mod_qos, mod_evasive, or mod_security.

- -chris

> At 2015-05-19 20:01:00, "David kerber" <dckerber@verizon.net>
> wrote:
>> On 5/19/2015 7:53 AM, javalishixml wrote:
>>> 
>>> 
>>> 
>>>> I doubt you're going to be able to do this in httpd, unless
>>>> you have a very simple, straight forward way of identifying
>>>> the robots.
>>> Yes. I just want to have a way to block the duplicated requests
>>> at httpd level. After all, my website has to face the the big
>>> concurrency issue.
>> 
>> I understand that's what you want.  What we're telling you is
>> that you probably won't be able to do that.
>> 
>> Let me ask the question again, that Chris asked before:  how do
>> you tell that a given request is from a robot?
>> 
>> The answer to that question will determine if you can block it
>> with httpd.
>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> At 2015-05-19 19:35:26, "David kerber" <dckerber@verizon.net>
>>> wrote:
>>>> On 5/19/2015 1:03 AM, javalishixml wrote:
>>>>> Thanks a lot for your information.
>>>>> 
>>>>> 
>>>>> This solution is based on tomcat level.  If I always handle
>>>>> this issue at java level, I'm afraid it has performance
>>>>> issue. Because this web site afford a very big concurrency
>>>>> access.
>>>>> 
>>>>> 
>>>>> Taking a consideration on its basic architect
>>>>> tomcat+apache, I think the best way to move this solution
>>>>> from tomcat to apache. So do you have some good solution at
>>>>> apache's configuration?  I understand this is a mail list
>>>>> for tomcat.. but just want to get any information
>>>> 
>>>> I doubt you're going to be able to do this in httpd, unless
>>>> you have a very simple, straight forward way of identifying
>>>> the robots.
>>>> 
>>>> 
>>>> 
>>>>> 
>>>>> 
>>>>> Thanks,
>>>>> 
>>>>> 
>>>>> At 2015-05-19 04:00:28, "Christopher Schultz"
>>>>> <chris@christopherschultz.net> wrote:
> To whom it may concern,
> 
> On 5/18/15 11:44 AM, javalishixml wrote:
>>>>>>>> I have a website. It is built by apache + tomcat.
>>>>>>>> 
>>>>>>>> Now we make a lottery activity at this website. But
>>>>>>>> we find that some robots always raise the duplicated
>>>>>>>> requests to hit this lottery activity. It causes that
>>>>>>>> robots almost get all the awards.
>>>>>>>> 
>>>>>>>> So we just want to block these kind of duplicated
>>>>>>>> requests at every interval unit. For example, we set
>>>>>>>> the interval unit is 3 seconds. The if the robot want
>>>>>>>> to hit the lottery activity in 3 seconds, the website
>>>>>>>> could block this action.
>>>>>>>> 
>>>>>>>> So how to do it? I suppose if we do it at tomcat
>>>>>>>> level, is it a very low performance? Can I do it at
>>>>>>>> apache level? how to do it? If I could not do it
>>>>>>>> apache level, can I do it by setting sth at tomcat?
> 
> If you have a way to identify a "duplicate" request (e.g. using a 
> fingerprint of the request that you can check during that 3-second 
> interval), then this is conceptually very easy.
> 
> It may not be great for performance, but you'll have to weigh that 
> against your own requirements. (For example, which is worse: poor 
> performance, or a site where only robots ever win the lottery?)
> 
> This will not be something you can configure in Apache httpd or 
> Tomcat. This will have to be an application thing (unless you can 
> describe the fingerprint technique to some httpd module such as 
> mod_security or mod_qos and then allow it to discard duplicates).
> 
> Back to the solution:
> 
> 1. Take a fingerprint of the request 2. Lookup the fingerprint in a
> database of previous requests ( fingerprint -> latest timestamp ) 
> 3. If the fingerprint appears in your database and the timestamp
> is less than 3 seconds ago, discard the request 4. Otherwise, store
> the current timestamp and fingerprint in the databas e
> 
> For a database, I might recommend something like memcached or
> another in-memory-style database. An in-memory key-value store is
> really what you are looking for. Memcached has a nice feature where
> values can automatically time-out (e.g. they are invalid after 3
> seconds), so you can make your application code a bit simpler
> because you'll never have a value in the database that is not
> valid.
> 
> Hope that helps, -chris
>>>>>> 
>>>>>> -----------------------------------------------------------------
- ----
>>>>>>
>>>>>> 
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>> For additional commands, e-mail:
>>>>>> users-help@tomcat.apache.org
>>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> -------------------------------------------------------------------
- --
>>>>
>>>> 
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>> For additional commands, e-mail:
>>>> users-help@tomcat.apache.org
>>>> 
>>> 
>> 
>> 
>> ---------------------------------------------------------------------
>>
>> 
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
Comment: GPGTools - http://gpgtools.org

iQIcBAEBCAAGBQJVWzetAAoJEBzwKT+lPKRYh1sP+wQYPO7BY9Stg2XdvzK9GDA0
/cqRMxIZ2Thq84GzHRKcg2pyC2iN3M+LMCxXCodKdp6+Cl0DfU6H0ijKucy4yHET
FrwhHT/7A1A7bQ4eT6IYhu1R7dtLhM0o5YRwYDDClKSfvsACTmuLOEmin40bmTeH
uuREu7EDz1fdNgOpjpDBNw1bC4ZkyYMlhQZ3Ox/jopnxKqRCXcjUx+kZH1UktJu9
NBBs7rQgvSXFpNtB2DPFBS6pgy97jNSgKgWqoqX0WaQwQkfU1dLIyrH6I2v4/22s
ldCE7sZDQrRR6PUAyMQVVj36H5WHH+xlxr0zSrkdxvy5bFFqjNoCb8fRh3+J8sGC
yd0hesq3QW+uSwkwJOg4LsDkGFHCTYNiZZMLBRULGBabFbnHCXKIRjie3h0JGSfb
uL8diC2sNydKH7Re8WifQ8wqPvqIqkN+oakHed+oLg6BhToJvZJY2mTYuJRdb6gc
iRYRZ1+dH78PrgwLgovRDQHrnNVQRfUTpEyQOfygBi46o5Hh1t1fIADDMMtQ+yLX
C/Fg7JO5+vN2AoG3A1UOHUmoTGi7bAOlqp4RUwJXdN9pho++ullP8X8SfAfW52VZ
A5mZIb3FuI2GstOAcZPIzg63m1dr69d4CY3QCQdYUu5GnmlR9ws/LHwrI0Hbl4DH
FiXD3fsPSgRYvjCXPISV
=PWnL
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message