hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16859) Use Bytebuffer pool for non java clients specifically for scans/gets
Date Wed, 15 Mar 2017 07:22:41 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15925662#comment-15925662

ramkrishna.s.vasudevan commented on HBASE-16859:

Sorry can you tell more here.
bq.CellBlock in use or not, we tend to use BBs from reservoir here
You mean from here in the from the above piece of code? I feel that for cellblock case the
reservoir has already been used and we have formed the cellblock using it.
bq.When CellBlock not in place, this 1st possible BB is null onlt and we start try getting
from reservoir.
You  mean for the non java client case where there is no cellblock right? Yes. True.
bq.And also we have to check remaining BB size need against minSizeForReservoirUse.
Yes. We are doing it inside allocateByteBufferArrayToReadInto.
nq.Not just 1st at top level. Like one BB size is say 100 and the need is 101 bytes, we get
a BB from pool and remaining size need is just 1 byte and better not waste BB for that.
So if the remaining after allocating 100 bytes is 1 byte and minSizeForReservoirUse is greater
than that we don't allocate from the BB Pool we only get it ondemand.

So basically this change is for the header and message. For non  java cleint all data is in
message only and so we are ensuring while writing the message we use the BB Pool. 
Not sure if am gettng your concern here on the unification part.

> Use Bytebuffer pool for non java clients specifically for scans/gets
> --------------------------------------------------------------------
>                 Key: HBASE-16859
>                 URL: https://issues.apache.org/jira/browse/HBASE-16859
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>             Fix For: 2.0.0
>         Attachments: HBASE-16859_V1.patch, HBASE-16859_V2.patch, HBASE-16859_V2.patch,
HBASE-16859_V4.patch, HBASE-16859_V5.patch, HBASE-16859_V6.patch, HBASE-16859_V7.patch
> In case of non java clients we still write the results and header into a on demand  byte[].
This can be changed to use the BBPool (onheap or offheap buffer?).
> But the basic problem is to identify if the response is for scans/gets. 
> - One easy way to do it is use the MethodDescriptor per Call and use the   name of the
MethodDescriptor to identify it is a scan/get. But this will pollute RpcServer by checking
for scan/get type response.
> - Other way is always set the result to cellScanner but we know that isClientCellBlockSupported
is going to false for non PB clients. So ignore the cellscanner and go ahead with the results
in PB. But this is not clean
> - third one is that we already have a RpccallContext being passed to the RS. In case
of scan/gets/multiGets we already set a Rpccallback for shipped call. So here on response
we can check if the callback is not null and check for isclientBlockSupported. In this case
we can get the BB from the pool and write the result and header to that BB. May be this looks

This message was sent by Atlassian JIRA

View raw message