tephra-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andreas Neumann (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (TEPHRA-269) Protect the Transaction Manager against misconfigured clients using rate limits
Date Mon, 02 Apr 2018 22:42:00 GMT

    [ https://issues.apache.org/jira/browse/TEPHRA-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16423226#comment-16423226
] 

Andreas Neumann commented on TEPHRA-269:
----------------------------------------

Nice design! A couple of comments:
 * would it make sense to blacklist a misbehaving client, and require to explicitly un-blacklist
it before it can start transactions again? If you unthrottle it after its bucket is empty,
it will just start misbehaving again. 
 * if we have a way to generate a report, would it make sense to include other metrics such
as number or percentage of invalid transactions compared to all of its transactions? Th.at
would help ops to identify systemically misbehaving clients
 * An alternative to the _long nextStartTxnMillis(String clientId);_ could be _void startPermitted(clientId)
throws RateLImitExceededException_, and that exception can include the number of outstanding
operations as well as the required delay. 
 * In the LeakyBucket implementation, the nextOperationMillis state variable appears unnecessary,
as it is derived from outstandingOps.
 * When cleaning up the LeakyBuckets, I don't think the delta is required before expiring
an entry. Or why is it needed?
 * The getRateLimits() client API should return a Java object and not a String. Even if you
use Json format to serialize this over the Thrift wire, that should not be exposed to the
client. 

> Protect the Transaction Manager against misconfigured clients using rate limits
> -------------------------------------------------------------------------------
>
>                 Key: TEPHRA-269
>                 URL: https://issues.apache.org/jira/browse/TEPHRA-269
>             Project: Tephra
>          Issue Type: New Feature
>          Components: core
>            Reporter: Poorna Chandra
>            Assignee: Poorna Chandra
>            Priority: Major
>             Fix For: 0.14.0-incubating
>
>         Attachments: Ensure QoS in Transaction Manager by rate limiting client requests.pdf
>
>
> We have seen cases where misconfigured clients can overwhelm the system by making expensive
requests (like invalidating transactions). By the time admins figure out there is a misconfigured
client and take corrective action, thousands of invalid transactions can be created in a very
short period of time. It would be useful to have a way to limit the number of invalid transactions
that a client can create in given time.
> I'll send out a design document for this soon.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message