hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matteo Bertozzi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11598) Add simple rpc throttling
Date Thu, 31 Jul 2014 20:12:41 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081384#comment-14081384
] 

Matteo Bertozzi commented on HBASE-11598:
-----------------------------------------

{quote}I think just doing r+w is a good start – the question is my ind from a user pov is
how would the admin commands be for specifying r-only and w-only throttles? Do the would we
have to add yet more admin commands for "types"/"whats" to the throttle command?{quote}
from the java api, nothing will change since there is a ThrottleType. 
on the ruby side you have tons of possible way of doing that.. new "read/write" flag and so
on.

{quote}From a user's point of view would it make sense to say throttle_table or throttle_namespace
as opposed to throttle 'table' or throttle 'namespace'? (currently it would seem you would
need throttle 'tableReads' and throttle 'tableWrites' as as new types etc.. ){quote}
as above, you have tons of possible way of doing that. I didn't want to add 100 new commands
just to set a flag. but I'm open to any suggestion.

{quote}I didn't see quotaScope in the admin commands code so I'm wondering how that would
fit in from a users's point of view. Where would it go wrt to the current ruby commands you
are proposing to add?{quote}
yeah, at the moment there is only QuotaScope.MACHINE because we don't have a good notification
system to do the aggregation/notification. but again is a flag "throttle ... {QUOTA_SCOPE
=> CLUSTER}"

{quote}1: Let's say you the rs can handled a total of 10MB/s of throughput (the available
capacity). User 'jon' is restricted to 1mb/s. This is not a reservation of 1MB/s throughput
for jon – jon gets to use 1mb/s and then gets an exception despite there being 9MB of capacity
remaiing..{quote}
At the moment this is what the patch is doing.

{quote}2: Let's say you the rs can handled a total of 10MB/s of throughput (the available
capacity). There are many users – u001-u100 each restricted to 1mb/s. This is not a reservation
for 1MB/s throughput for each user – each user would be fighting for a chunk of the 10MB/s
capacity.{quote}
The first problem that we have today is that we don't know the "available capacity", we can
guess it after a while with some stats available and that's why the ThrottleRequest in Master.proto
has a "float share" field that can be used instead of the "uint limit". so in the future you
may be able to specify 25% of the available quota/capacity/whatever... 

> Add simple rpc throttling
> -------------------------
>
>                 Key: HBASE-11598
>                 URL: https://issues.apache.org/jira/browse/HBASE-11598
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Matteo Bertozzi
>            Assignee: Matteo Bertozzi
>            Priority: Minor
>             Fix For: 1.0.0, 2.0.0
>
>
> Add a simple version of rpc throttling.
> (by simple I mean something that requires less changes as possible to the core)
> The idea is to add a hbase:quota table to store the user/table quota information.
> Add a couple of API on the client like throttleUser() and throttleTable()
> and on the server side before executing the request we check the quota, if not an exception
is thrown.
> The quota will be per-machine. There will be a flag "QuotaScope" that will be used in
the future to specify the quota at "cluster level" instead of per machine. (A limit of 100req/min
means that each machine can execute 100req/min with a scope per-machine).
> This will be the first cut, simple solution that requires verify few changes to the core.
> Later on we can make the client aware of the ThrottlingException and deal with it in
a smarter way.
> Also we need to change a bit the RPC code to be able to yield the operation if the quota
will be 
> available not to far in the future, and avoid going back to the client for "few seconds".
> REVIEW BOARD: https://reviews.apache.org/r/23981



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message