hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karan Mehta (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17998) Improve HBase RPC write throttling size estimation
Date Tue, 09 May 2017 01:02:04 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001881#comment-16001881

Karan Mehta commented on HBASE-17998:

I did not find really feasible for this thing to happen because of following reasons.
1. Calling {{getSerializedSize()}} on a Protobuf to estimate its size will require to traverse
over the complete protobuf data which we don't wanna do in the first place.
2. Size of Puts can be estimated based on the number of bytes received for the RPC Request.
This information can be passed around with {{HBaseRpcController}} class. However, it is difficult
to estimate the size of only {{Puts}} in case of a  {{MultiRequest}} since we just have a
total buffer size which may include {{Gets}} and {{Scans}}. I am not sure if we should really
be doing code changes for just this support.

Rather than randomly estimating the size, a slightly better approach might be to use a moving
average based on the past requests that have been seen as suggested by [~vincentpoon]. Lets
discuss if there are any other ways to accomplish this task.

> Improve HBase RPC write throttling size estimation
> --------------------------------------------------
>                 Key: HBASE-17998
>                 URL: https://issues.apache.org/jira/browse/HBASE-17998
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Karan Mehta
>            Assignee: Karan Mehta
> Currently when RPC throttling, the size of each put is estimated using a hardcoded value
100 bytes. This can be improved by using the protobuf size as an estimate, without having
to deserialize or do a big refactoring.

This message was sent by Atlassian JIRA

View raw message