hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-7414) Convert HFile to pb
Date Thu, 20 Dec 2012 23:41:14 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13537499#comment-13537499
] 

Andrew Purtell commented on HBASE-7414:
---------------------------------------

I think at a minimum the file trailer should be PB and extensible without requiring a new
major HFile version[1] beyond the necessary "HFileV3" to introduce it. We have some metadata
extensibility already in HFile V2, so meta blocks would be a second place to look. 

[1] For example consider a possible scheme for encrypting HFiles. We'd want to put in that
capability in the same way as compression, but not replacing compression (adding a new encryption
codec as a "compression" codec would be cheating). It would have to integrate early on the
read side so decryption can be set up before readers have a go at the data or index blocks.
There's no place to do this in the current fixed trailer. To work around this and avoiding
introduction of "HFileV3" simply for encryption, I have some half baked ideas about pluggable
low level transforms of HFile blocks registered on a per-block-type basis as directed by settings
read out of HFileV2 metadata sections, but nothing ready to go. Maybe I can piggyback on this
work if we're already introducing a "HFileV3" for other reasons.
                
> Convert HFile to pb
> -------------------
>
>                 Key: HBASE-7414
>                 URL: https://issues.apache.org/jira/browse/HBASE-7414
>             Project: HBase
>          Issue Type: Task
>          Components: HFile
>            Reporter: stack
>              Labels: noob
>
> See HBASE-7201
> Convertion should be in a manner that does not prevent our being able to read old style
hfiles with Writable metadata.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message