hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12313) Redo the hfile index length optimization so cell-based rather than serialized KV key
Date Wed, 29 Oct 2014 20:35:35 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14188952#comment-14188952
] 

Hudson commented on HBASE-12313:
--------------------------------

SUCCESS: Integrated in HBase-TRUNK #5717 (See [https://builds.apache.org/job/HBase-TRUNK/5717/])
HBASE-12313 Redo the hfile index length optimization so cell-based rather than serialized
KV key (stack: rev 889333a6fd854cf27b552cf25ff711f2c50f8c08)
* hbase-server/src/main/java/org/apache/hadoop/hbase/client/ClientSideRegionScanner.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/Increment.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockIndex.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockCompatibility.java
* hbase-prefix-tree/src/main/java/org/apache/hadoop/hbase/codec/prefixtree/decode/PrefixTreeCell.java
* hbase-prefix-tree/src/main/java/org/apache/hadoop/hbase/codec/prefixtree/decode/PrefixTreeArrayScanner.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestSeekTo.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV3.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV2.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java
* hbase-common/src/test/java/org/apache/hadoop/hbase/TestCellComparator.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileWriter.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFilePrettyPrinter.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileEncryption.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/mapred/TestDriver.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/CellKey.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
* hbase-client/src/test/java/org/apache/hadoop/hbase/ipc/TestIPCUtil.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/Cell.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlock.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestChecksum.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/CellComparator.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV3.java
* hbase-common/src/test/java/org/apache/hadoop/hbase/TestCellUtil.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/ReplicationProtbufUtil.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlockIndex.java


> Redo the hfile index length optimization so cell-based rather than serialized KV key
> ------------------------------------------------------------------------------------
>
>                 Key: HBASE-12313
>                 URL: https://issues.apache.org/jira/browse/HBASE-12313
>             Project: HBase
>          Issue Type: Sub-task
>          Components: regionserver, Scanners
>            Reporter: stack
>            Assignee: stack
>             Fix For: 2.0.0, 0.99.2
>
>         Attachments: 0001-HBASE-12313-Redo-the-hfile-index-length-optimization.patch,
0001-HBASE-12313-Redo-the-hfile-index-length-optimization.patch, 0001-HBASE-12313-Redo-the-hfile-index-length-optimization.patch,
0001-HBASE-12313-Redo-the-hfile-index-length-optimization.patch, 0001-HBASE-12313-Redo-the-hfile-index-length-optimization.patch,
12313v10.txt, 12313v5.txt, 12313v6.txt, 12313v8.txt
>
>
> Trying to remove API that returns the 'key' of a KV serialized into a byte array is thorny.
> I tried to move over the first and last key serializations and the hfile index entries
to be cell but patch was turning massive.  Here is a smaller patch that just redoes the optimization
that tries to find 'short' midpoints between last key of last block and first key of next
block so it is Cell-based rather than byte array based (presuming Keys serialized in a certain
way).  Adds unit tests which we didn't have before.
> Also remove CellKey.  Not needed... at least not yet.  Its just utility for toString.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message