hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-7414) Convert HFile to pb
Date Thu, 20 Dec 2012 23:41:14 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13537499#comment-13537499

Andrew Purtell commented on HBASE-7414:

I think at a minimum the file trailer should be PB and extensible without requiring a new
major HFile version[1] beyond the necessary "HFileV3" to introduce it. We have some metadata
extensibility already in HFile V2, so meta blocks would be a second place to look. 

[1] For example consider a possible scheme for encrypting HFiles. We'd want to put in that
capability in the same way as compression, but not replacing compression (adding a new encryption
codec as a "compression" codec would be cheating). It would have to integrate early on the
read side so decryption can be set up before readers have a go at the data or index blocks.
There's no place to do this in the current fixed trailer. To work around this and avoiding
introduction of "HFileV3" simply for encryption, I have some half baked ideas about pluggable
low level transforms of HFile blocks registered on a per-block-type basis as directed by settings
read out of HFileV2 metadata sections, but nothing ready to go. Maybe I can piggyback on this
work if we're already introducing a "HFileV3" for other reasons.
> Convert HFile to pb
> -------------------
>                 Key: HBASE-7414
>                 URL: https://issues.apache.org/jira/browse/HBASE-7414
>             Project: HBase
>          Issue Type: Task
>          Components: HFile
>            Reporter: stack
>              Labels: noob
> See HBASE-7201
> Convertion should be in a manner that does not prevent our being able to read old style
hfiles with Writable metadata.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message