hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matteo Bertozzi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11737) Document callQueue improvements from HBASE-11355
Date Thu, 14 Aug 2014 20:22:22 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097579#comment-14097579
] 

Matteo Bertozzi commented on HBASE-11737:
-----------------------------------------

{quote}
ipc.server.callqueue.handler.factor
A value between <literal>0</literal> and <literal>1</literal> gives
each handler
a percentage of a queue. For instance, a value of <literal>.5</literal> shares
one
queue between each two handlers.</para>
{quote}
Is this correct? I mean the example is correct, but "gives each handler a percentage of a
queue"
to me sounds like the other way around where 0 means share nothing and 1 share all. 
buy maybe is just me not able to read it correctly.

You can also add that the benefit of having multiple queues (e.g. 1 per handler) means that
there is less contention when the task is added to/select from the queue, which result is
better performance.
but it also means that if you have 2 queues, and 1 ends up with task that takes long you end
up with one handler waiting to receive the next call instead of executing the pending ones
in the other queue.


read.share was renamed to read.ratio (no need to doc the change seen no release was released
with .share)
I've added also more examples after a discussion with jon, which you should include.
{noformat}
The specified interval (which should be between 0.0 and 1.0) will be multiplied by the number
of call queues.
A value of 0 indicate to not split the call queues, meaning that both read and write requests
will be pushed to the same set of queues.
A value lower than 0.5 means that there will be less read queues than write queues.
A value of 0.5 means there will be the same number of read and write queues.
A value greater than 0.5 means that there will be more read queues than write queues.
A value of 1.0 means that all the queues except one are used to dispatch read requests.

Example: Given the total number of call queues being 10
a read.ratio of 0 means that: the 10 queues will contain both read/write requests.
a read.ratio of 0.3 means that: 3 queues will contain only read requests and 7 queues will
contain only write requests.
a read.ratio of 0.5 means that: 5 queues will contain only read requests and 5 queues will
contain only write requests.
a read.ratio of 0.8 means that: 8 queues will contain only read requests and 2 queues will
contain only write requests.
a read.ratio of 1 means that: 9 queues will contain only read requests and 1 queues will contain
only write requests.
{noformat}

Also, add something like: separating the number of read/write queues can be used to "prioritize"
read vs writes, less queues you have more "throttling" you have on that operation.
but separating read and write queues also means that reads will never be stuck waiting a write
operation to complete. (dumb example, 2 handler 1 queue you have a seq of WRITE, WRITE, READ
the read must wait the writes to complete, if you have the 2 separate queue and 1 handler
is processing only the write queue and the other only the read queue at any point in time
you are executing a read and a write)

There is also a new scan.ratio property that splits the read queues in long-read and short-read
{noformat}
the scan.ratio property will split the read call queues into small-read and long-read queues.
A value lower than 0.5 means that there will be less long-read queues than short-read queues.
A value of 0.5 means that there will be the same number of short-read and long-read queues.
A value greater than 0.5 means that there will be more long-read queues than short-read queues
A value of 0 or 1 indicate to use the same set of queues for gets and scans.

Example: Given the total number of read call queues being 8
a scan.ratio of 0 or 1 means that: 8 queues will contain both long and short read requests.
a scan.ratio of 0.3 means that: 2 queues will contain only long-read requests and 6 queues
will contain only short-read requests.
a scan.ratio of 0.5 means that: 4 queues will contain only long-read requests and 4 queues
will contain only short-read requests.
a scan.ratio of 0.8 means that: 6 queues will contain only long-read requests and 2 queues
will contain only short-read requests.
{noformat}
and again, by dividing long-reads from short-reads you can "prioritize" what you need.
(same stuff as read/write but with long/short reads)

said that, this property are meant mainly for perf testing, unless you really know what you
are doing
since they "fixed" for the RS and if you want to change them you have to restart the RS.
The idea is to have them dynamically configurable by user/table/namespace once we have quotas
and maybe at some point autotunables based on the workload stats.

> Document callQueue improvements from HBASE-11355
> ------------------------------------------------
>
>                 Key: HBASE-11737
>                 URL: https://issues.apache.org/jira/browse/HBASE-11737
>             Project: HBase
>          Issue Type: Sub-task
>          Components: documentation
>            Reporter: Misty Stanley-Jones
>            Assignee: Misty Stanley-Jones
>             Fix For: 0.99.0, 0.98.4
>
>         Attachments: HBASE-11737.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message