hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "cuijianwei (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12930) Check single row size not exceed configured max row size across families for Get/Scan
Date Sat, 11 Apr 2015 12:11:12 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14490988#comment-14490988

cuijianwei commented on HBASE-12930:

Thanks for the your comment, and sorry to relay late [~jonathan.lawlor]. I studied [HBASE-11544
| https://issues.apache.org/jira/browse/HBASE-11544], it provides a good solution to avoid
returning large rows to the client, and could cover this case in my opinion:). In [HBASE-11544
| https://issues.apache.org/jira/browse/HBASE-11544], HRegion#nextInternal adds a parameter
'remainingResultSize' to limit the returned size of row, as:
+    private NextState nextInternal(List<Cell> results, int batchLimit, long remainingResultSize)
+        throws IOException {
In the master branch, I notice the 'remainingResultSize' is removed from the parameters of
HRegion#nextInternal, is there some reason to do this? 

> Check single row size not exceed configured max row size across families for Get/Scan
> -------------------------------------------------------------------------------------
>                 Key: HBASE-12930
>                 URL: https://issues.apache.org/jira/browse/HBASE-12930
>             Project: HBase
>          Issue Type: Improvement
>          Components: Scanners
>            Reporter: cuijianwei
>            Priority: Minor
>             Fix For: 2.0.0
> StoreScanner#next will check the 'totalBytesRead' not exceed configured ‘hbase.table.max.rowsize’
for each family. However, if there are several families, the single row will also achieve
unexpected big size even if 'totalBytesRead' of each family not exceed 'hbase.table.max.rowsize'.
This may cause the region server fail because of OOM. What about checking single row size
across families in StoreScanner#next(List<Cell>, int)?
> {code}
>     long totalBytesRead = 0;
>     // ==> compute the size of cells have been read
>     for (Cell cell : outResult) {
>       totalBytesRead += CellUtil.estimatedSerializedSizeOf(old);
>     }
>     LOOP: while((cell = this.heap.peek()) != null) {
>     ...
> {code}

This message was sent by Atlassian JIRA

View raw message