hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Lawlor (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12930) Check single row size not exceed configured max row size across families for Get/Scan
Date Mon, 13 Apr 2015 16:36:13 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14492605#comment-14492605

Jonathan Lawlor commented on HBASE-12930:

[~cuijianwei] Recently there was a change made (HBASE-13421) to the solution initially conceived
in HBASE-11544. The remaining result size limit still exists, but now it is carried within
the new class, ScannerContext (see ScannerContext#incrementaSizeProgress and ScannerContext#checkSizeLimit(...)).
ScannerContext was introduced to allow us to reduce the number of object creations in the
scanner hot code paths and also provides a nice encapsulation of limits and limit progress.
Please let me know if you have any questions :)

> Check single row size not exceed configured max row size across families for Get/Scan
> -------------------------------------------------------------------------------------
>                 Key: HBASE-12930
>                 URL: https://issues.apache.org/jira/browse/HBASE-12930
>             Project: HBase
>          Issue Type: Improvement
>          Components: Scanners
>            Reporter: cuijianwei
>            Priority: Minor
>             Fix For: 2.0.0
> StoreScanner#next will check the 'totalBytesRead' not exceed configured ‘hbase.table.max.rowsize’
for each family. However, if there are several families, the single row will also achieve
unexpected big size even if 'totalBytesRead' of each family not exceed 'hbase.table.max.rowsize'.
This may cause the region server fail because of OOM. What about checking single row size
across families in StoreScanner#next(List<Cell>, int)?
> {code}
>     long totalBytesRead = 0;
>     // ==> compute the size of cells have been read
>     for (Cell cell : outResult) {
>       totalBytesRead += CellUtil.estimatedSerializedSizeOf(old);
>     }
>     LOOP: while((cell = this.heap.peek()) != null) {
>     ...
> {code}

This message was sent by Atlassian JIRA

View raw message