hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12313) Redo the hfile index length optimization so cell-based rather than serialized KV key
Date Mon, 27 Oct 2014 03:58:34 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14184796#comment-14184796
] 

stack commented on HBASE-12313:
-------------------------------

bq. Basically the diff btw estimatedSizeOf and estimatedLengthOf was the former is having
and INT size extra. 

I think in the end we want serialized size and heap size and maybe an estimated size that
would be cheaper to calculate than either of the former for places where it is not that important.

The patch makes a start on it.

The sizings that are in this patch as I see it cause no problem; perhaps a slight overcount
but its for metrics only -- not for anything important (You agree?)

bq. Am I missing some thing? We need fix this?

No you are right but 'do we need to fix it?'  It is ok that the size calculated is 'rough',
approx, in the context, or do you think otherwise?



> Redo the hfile index length optimization so cell-based rather than serialized KV key
> ------------------------------------------------------------------------------------
>
>                 Key: HBASE-12313
>                 URL: https://issues.apache.org/jira/browse/HBASE-12313
>             Project: HBase
>          Issue Type: Sub-task
>          Components: regionserver, Scanners
>            Reporter: stack
>            Assignee: stack
>         Attachments: 0001-HBASE-12313-Redo-the-hfile-index-length-optimization.patch,
0001-HBASE-12313-Redo-the-hfile-index-length-optimization.patch, 0001-HBASE-12313-Redo-the-hfile-index-length-optimization.patch,
0001-HBASE-12313-Redo-the-hfile-index-length-optimization.patch, 0001-HBASE-12313-Redo-the-hfile-index-length-optimization.patch,
12313v5.txt
>
>
> Trying to remove API that returns the 'key' of a KV serialized into a byte array is thorny.
> I tried to move over the first and last key serializations and the hfile index entries
to be cell but patch was turning massive.  Here is a smaller patch that just redoes the optimization
that tries to find 'short' midpoints between last key of last block and first key of next
block so it is Cell-based rather than byte array based (presuming Keys serialized in a certain
way).  Adds unit tests which we didn't have before.
> Also remove CellKey.  Not needed... at least not yet.  Its just utility for toString.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message