hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Hsieh (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-11598) Add simple rpc throttling
Date Thu, 31 Jul 2014 19:40:40 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081243#comment-14081243
] 

Jonathan Hsieh edited comment on HBASE-11598 at 7/31/14 7:40 PM:
-----------------------------------------------------------------

(jmhsieh: minor wiki reformaitting)
{quote}Can you write a quick how a user/admin/hbase dba would interact with hbase to use this
feature doc?{quote}
I still don't have any review, so it is quite early for an "how to interact" doc. Until we
have something agreed and defined the core is described in the jira description. you can throttle
user/table/ns if you can execute the operation you execute otherwise you get an exception.


{quote}Is the reads, write or combing size or ops throughput?{quote}
You have all the combinations in the implementation. (Exposed at the moment you have only
req/time and req-size, I was waiting for comments)
 * req/time (num of read & write reqs)
 * write/time (num of write reqs)
 * read/time (num of read reqs)
 * write-size (size of write reqs data)
 * read-size (size of read reqs response data)
 * req-size (size of read & write reqs data)

{quote}it the mechanism pessimistic and only allows the throttled throughput regardless of
available capacity?{quote}
What is "available capacity"? you send your request and when you have consumed the quota you
get an exception.

{quote}Should probably explain that this i per user/table/namespace per regionserver currently.
(not global across the entire cluster){quote}
What do you mean by global across the cluster? you mean something like 1 global limit max
10/sec?
or you mean "Across the cluster" as in QuotaScope.CLUSTER? In the description I said that
you have stuff like throttleUser() and throttleTable() and the quota will be per-machine.
There will be a flag "QuotaScope" that will be used in the future to specify the quota at
"cluster level" instead of per machine. (A limit of 100req/min means that each machine can
execute 100req/min with a scope per-machine).

{quote}how do overlapping throttles interact? (eg. – if I have a user throttled to 10/s
but a table limited to less (or more) which value wins?{quote}
min limit wins with the hierarchy: Namespace > Table > User
If your Namespace has a lower limit then a Table limit which is lower then a User limit the
NS wins.
If your table has a higher limit than your user your user limit wins.

{quote}Does the idea of making this a coproc make any sense at all? {quote}


was (Author: mbertozzi):
{quota}Can you write a quick how a user/admin/hbase dba would interact with hbase to use this
feature doc?{quota}
I still don't have any review, so it is quite early for an "how to interact" doc. Until we
have something agreed and defined the core is described in the jira description. you can throttle
user/table/ns if you can execute the operation you execute otherwise you get an exception.


{quota}Is the reads, write or combing size or ops throughput?{quota}
You have all the combinations in the implementation. (Exposed at the moment you have only
req/time and req-size, I was waiting for comments)
 * req/time (num of read & write reqs)
 * write/time (num of write reqs)
 * read/time (num of read reqs)
 * write-size (size of write reqs data)
 * read-size (size of read reqs response data)
 * req-size (size of read & write reqs data)

{quota}it the mechanism pessimistic and only allows the throttled throughput regardless of
available capacity?{quota}
What is "available capacity"? you send your request and when you have consumed the quota you
get an exception.

{quota}Should probably explain that this i per user/table/namespace per regionserver currently.
(not global across the entire cluster){quota}
What do you mean by global across the cluster? you mean something like 1 global limit max
10/sec?
or you mean "Across the cluster" as in QuotaScope.CLUSTER? In the description I said that
you have stuff like throttleUser() and throttleTable() and the quota will be per-machine.
There will be a flag "QuotaScope" that will be used in the future to specify the quota at
"cluster level" instead of per machine. (A limit of 100req/min means that each machine can
execute 100req/min with a scope per-machine).

{quota}how do overlapping throttles interact? (eg. – if I have a user throttled to 10/s
but a table limited to less (or more) which value wins?{quota}
min limit wins with the hierarchy: Namespace > Table > User
If your Namespace has a lower limit then a Table limit which is lower then a User limit the
NS wins.
If your table has a higher limit than your user your user limit wins.

Does the idea of making this a coproc make any sense at all?

> Add simple rpc throttling
> -------------------------
>
>                 Key: HBASE-11598
>                 URL: https://issues.apache.org/jira/browse/HBASE-11598
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Matteo Bertozzi
>            Assignee: Matteo Bertozzi
>            Priority: Minor
>             Fix For: 1.0.0, 2.0.0
>
>
> Add a simple version of rpc throttling.
> (by simple I mean something that requires less changes as possible to the core)
> The idea is to add a hbase:quota table to store the user/table quota information.
> Add a couple of API on the client like throttleUser() and throttleTable()
> and on the server side before executing the request we check the quota, if not an exception
is thrown.
> The quota will be per-machine. There will be a flag "QuotaScope" that will be used in
the future to specify the quota at "cluster level" instead of per machine. (A limit of 100req/min
means that each machine can execute 100req/min with a scope per-machine).
> This will be the first cut, simple solution that requires verify few changes to the core.
> Later on we can make the client aware of the ThrottlingException and deal with it in
a smarter way.
> Also we need to change a bit the RPC code to be able to yield the operation if the quota
will be 
> available not to far in the future, and avoid going back to the client for "few seconds".
> REVIEW BOARD: https://reviews.apache.org/r/23981



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message