hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-3857) Change the HFile Format
Date Fri, 13 May 2011 20:41:47 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13033270#comment-13033270

stack commented on HBASE-3857:

Thanks lads for the answers.  Helps.

High level, do you lads think this format 'brittle'?  Will corruption or an error logic writing
any piece of the file render the file as a whole unreadable or large swaths of the file unreadable?
A corrupt root index makes the file totally unreadable I suppose.   A corrupt intermediate
index will render all subbranches unreadable?  So, this file format seems more 'brittle' than
V1 because of the chaining between the index parts (root to intermediate, etc.)?  What do
you think?  Its unavoidable I suppose if we want the nice feature that Liyin describes where
we dump out index as we cross over an index size threshold (And yes Mikhail, in V1, there
is not code that makes use of the 'magic' to skip bad bits of the file.  And does 'magic'
for a parser to pick up the parse again even make sense on a filesystem that is checksummed?
 Or, in your words ' I am not sure what are the specific data corruption cases [magic] might
help fix.')


I forgot we were vint'ing already.  Its probably not a bad idea having root keep same format
as old v1 index.

> Change the HFile Format
> -----------------------
>                 Key: HBASE-3857
>                 URL: https://issues.apache.org/jira/browse/HBASE-3857
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Liyin Tang
>            Assignee: Mikhail Bautin
>         Attachments: hfile_format_v2_design_draft_0.1.pdf
> In order to support HBASE-3763 and HBASE-3856, we need to change the format of the HFile.
The new format proposal is attached here. Thanks for Mikhail Bautin for the documentation.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message