hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Lawlor (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13303) Fix size calculation of results on the region server
Date Thu, 26 Mar 2015 20:05:53 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14382554#comment-14382554
] 

Jonathan Lawlor commented on HBASE-13303:
-----------------------------------------

[~apurtell] Apologies for the late catch, but while working on the addendum to HBASE-11544
I just came across a calculation that was missed with this patch in branch-1+. Specifically
in {{StoreScanner#next(List, int, long)}} the calculation used for totalHeapSize still uses
to {{CellUtil#estimatedHeapSizeOf}}. Same goes for {{RegionScannerImpl#calculateResultSize}}.


These places should use the new "estimate without tags" as well because size calculations
are performed once down in StoreScanner and the calculation propagates up to RSRpcServices
through the NextState (thus the change in RSRpcServices alone won't be sufficient since {{skipResultSizeCalculation}}
would be true). Would this be fixed as an addendum here or should I file elsewhere? Or, given
that the point of the addendum to HBASE-11544 is to get rid of NextState, would it be more
appropriate to fix there?

> Fix size calculation of results on the region server
> ----------------------------------------------------
>
>                 Key: HBASE-13303
>                 URL: https://issues.apache.org/jira/browse/HBASE-13303
>             Project: HBase
>          Issue Type: Sub-task
>          Components: Client
>            Reporter: Lars Hofhansl
>            Assignee: Andrew Purtell
>             Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.12
>
>         Attachments: HBASE-13303-0.98.patch, HBASE-13303-0.98.patch, HBASE-13303-0.98.patch,
HBASE-13303.patch, HBASE-13303.patch, HBASE-13303.patch, TestResultSizeEstimation.java
>
>
> One of the problems in the parent is due to different size calculation between client
and server when HFilev3 is used.
> Since tags are _never_ shipped to the client in a scan, we can have special size function
(or a flag on the current one) that does not include the tags and the tags meta information
(the length is what causes the issue), so that client and server will always calculate the
same size.
> I'll make a patch within the hour, unless somebody beats me to it.
> [~apurtell], FYI.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message