hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15437) Response size calculated in RPCServer for warning tooLarge responses does NOT count CellScanner payload
Date Tue, 06 Dec 2016 11:11:23 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15724821#comment-15724821

Anoop Sam John commented on HBASE-15437:

Ya seems like now addSize() calls are not duplicated.
bq.long responseSize = result.getSerializedSize() + call.getResponseCellSize();
I can see in the patch we added addSize() in 2 places. One within isClientCellBlockSupport
check and one not.  Also the existing calls to addSize() seems not with any check on isClientCellBlockSupport.
 That means when client is not supporting cell block, the results are PBed into 'result' PB
object and its getSerializedSize includes every thing. Adding again call.getResponseCellSize()
seems to duplicate the size in some cases.
We can better change the above one liner to consider isClientCellBlockSupport and do + call.getResponseCellSize()?

Also as per the patch we add
RpcCallContext context = RpcServer.getCurrentCall();
into mutate() method.  And below we have addResult(builder, r, controller); existing call
which in turn again trying to read the context from ThreadLocal.  We can override addResult()
passing an extra boolean 'clientCellBlockSupported'?

> Response size calculated in RPCServer for warning tooLarge responses does NOT count CellScanner
> -------------------------------------------------------------------------------------------------------
>                 Key: HBASE-15437
>                 URL: https://issues.apache.org/jira/browse/HBASE-15437
>             Project: HBase
>          Issue Type: Bug
>          Components: IPC/RPC
>            Reporter: deepankar
>            Assignee: Jerry He
>         Attachments: HBASE-15437-v1.patch, HBASE-15437-v2.patch, HBASE-15437-v3.patch,
HBASE-15437-v4.patch, HBASE-15437.patch
> After HBASE-13158 where we respond back to RPCs with cells in the payload , the protobuf
response will just have the count the cells to read from payload, but there are set of features
where we log warn in RPCServer whenever the response is tooLarge, but this size now is not
considering the sizes of the cells in the PayloadCellScanner. Code form RPCServer
> {code}
>       long responseSize = result.getSerializedSize();
>       // log any RPC responses that are slower than the configured warn
>       // response time or larger than configured warning size
>       boolean tooSlow = (processingTime > warnResponseTime && warnResponseTime
> -1);
>       boolean tooLarge = (responseSize > warnResponseSize && warnResponseSize
> -1);
>       if (tooSlow || tooLarge) {
>         // when tagging, we let TooLarge trump TooSmall to keep output simple
>         // note that large responses will often also be slow.
>         logResponse(new Object[]{param},
>             md.getName(), md.getName() + "(" + param.getClass().getName() + ")",
>             (tooLarge ? "TooLarge" : "TooSlow"),
>             status.getClient(), startTime, processingTime, qTime,
>             responseSize);
>       }
> {code}
> Should this feature be not supported any more or should we add a method to CellScanner
or a new interface which returns the serialized size (but this might not include the compression
codecs which might be used during response ?) Any other Idea this could be fixed ?

This message was sent by Atlassian JIRA

View raw message