hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-18043) Institute a hard limit for individual cell size that cannot be overridden by clients
Date Sat, 13 May 2017 16:17:04 GMT

    [ https://issues.apache.org/jira/browse/HBASE-18043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16009395#comment-16009395
] 

Andrew Purtell edited comment on HBASE-18043 at 5/13/17 4:16 PM:
-----------------------------------------------------------------

bq. Do we have some checks before accepting RPC requests itself on the size?

As I said the check is on the client side first, and this is best, because rejecting the RPC
after we receive it is expensive. However we cannot fully trust the client so still need enforcement
on the server. I have placed the checks on the server side at points in the code where it
seems least expensive to make them, right where we convert the PB representation into the
on heap object. Whether to do this before conversion from PB to object or after, I looked
at how much code it would add. We don't have code that walks over all the different PBs and
looks at the size of each cell (or if we do I didn't see it -please advise) but we do have
code that can walk a Mutation and do this, the CellScanner and so on. The oversize cell case
is expected to be the abnormal case because there is still a client side check in place, so
it seems ok to do it this way for the sake of minimizing new code/complexity.


was (Author: apurtell):
bq. Do we have some checks before accepting RPC requests itself on the size?

As I said the check is on the client side first, and this is best, because rejecting the RPC
after we receive it is expensive. However we cannot fully trust the client so still need enforcement
on the server. I have placed the checks on the server side at points in the code where it
seems least expensive to make them, right where we convert the PB representation into the
on heap object. Whether to do this before conversion, I looked at how much code it would add.
We don't have code that walks over all the different PBs and looks at the size of each cell
(or if we do I didn't see it -please advise) but we do have code that can walk a Mutation
and do this, the CellScanner and so on. The oversize cell case is expected to be the abnormal
case because there is still a client side check in place, so it seems ok to do it this way
for the sake of minimizing new code/complexity.

> Institute a hard limit for individual cell size that cannot be overridden by clients
> ------------------------------------------------------------------------------------
>
>                 Key: HBASE-18043
>                 URL: https://issues.apache.org/jira/browse/HBASE-18043
>             Project: HBase
>          Issue Type: Improvement
>          Components: IPC/RPC, regionserver
>    Affects Versions: 2.0.0
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>             Fix For: 2.0.0, 1.4.0
>
>         Attachments: HBASE-18043-branch-1.patch, HBASE-18043-branch-1.patch, HBASE-18043.patch,
HBASE-18043.patch
>
>
> For sake of service protection we should not give absolute trust to clients regarding
resource limits that can impact stability, like cell size limits. We should add a server side
configuration that sets a hard limit for individual cell size that cannot be overridden by
the client. We can keep the client side check, because it's expensive to reject a RPC that
has already come in. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message