hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phabricator (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5521) Move compression/decompression to an encoder specific encoding context
Date Tue, 06 Mar 2012 19:28:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223578#comment-13223578

Phabricator commented on HBASE-5521:

heyongqiang has commented on the revision "HBASE-5521 [jira] Move compression/decompression
to an encoder specific encoding context".

good suggestion. i will change to use class for checking.

  Yes, we need to enforce that contains right now. Basically we want each encoder to use its
own context object which is returned by newDataBlockEncodingContext()

  src/main/java/org/apache/hadoop/hbase/io/encoding/DataBlockEncoder.java:58 will remove the
original method, and change all testcases to use the new api.

  src/main/java/org/apache/hadoop/hbase/io/encoding/EncodedDataBlock.java:53 are you suggesting
adding a new field Encoding to each encoder?
  i also think it is a good thing to do, but not sure the reason of why the current code not
doing that.
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:693 Good catch! this is a
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:811 good catch here. will
assign to null.
  Initially i am doing that is because they are all defined as 'final'. will remove the final.
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:1253 default is used for
nondata block. I can not make them singleton as they need to maintain some reusable objects
internally. By reusable objects, it is basically some buffers allocated once and can be reused
across multiple operations.

  src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:1327-1328 added back
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFileDataBlockEncoderImpl.java:194-195 good
catch. will add a class field to reuse it



> Move compression/decompression to an encoder specific encoding context
> ----------------------------------------------------------------------
>                 Key: HBASE-5521
>                 URL: https://issues.apache.org/jira/browse/HBASE-5521
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: He Yongqiang
>            Assignee: He Yongqiang
>         Attachments: HBASE-5521.1.patch, HBASE-5521.D2097.1.patch
> As part of working on HBASE-5313, we want to add a new columnar encoder/decoder. It makes
sense to move compression to be part of encoder/decoder:
> 1) a scanner for a columnar encoded block can do lazy decompression to a specific part
of a key value object
> 2) avoid an extra bytes copy from encoder to hblock-writer. 
> If there is no encoder specified for a writer, the HBlock.Writer will use a default compression-context
to do something very similar to today's code.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message