Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Fri, 13 May 2011 20:41:47 +0000 (UTC)
From: "stack (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: 
 <1790438715.11219.1305319307552.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <696831602.25250.1304616123588.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HBASE-3857) Change the HFile Format
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-3857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13033270#comment-13033270 ] 

stack commented on HBASE-3857:
------------------------------

Thanks lads for the answers.  Helps.

High level, do you lads think this format 'brittle'?  Will corruption or an error logic writing any piece of the file render the file as a whole unreadable or large swaths of the file unreadable? A corrupt root index makes the file totally unreadable I suppose.   A corrupt intermediate index will render all subbranches unreadable?  So, this file format seems more 'brittle' than V1 because of the chaining between the index parts (root to intermediate, etc.)?  What do you think?  Its unavoidable I suppose if we want the nice feature that Liyin describes where we dump out index as we cross over an index size threshold (And yes Mikhail, in V1, there is not code that makes use of the 'magic' to skip bad bits of the file.  And does 'magic' for a parser to pick up the parse again even make sense on a filesystem that is checksummed?  Or, in your words ' I am not sure what are the specific data corruption cases [magic] might help fix.')

@Mikhail

I forgot we were vint'ing already.  Its probably not a bad idea having root keep same format as old v1 index.


> Change the HFile Format
> -----------------------
>
>                 Key: HBASE-3857
>                 URL: https://issues.apache.org/jira/browse/HBASE-3857
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Liyin Tang
>            Assignee: Mikhail Bautin
>         Attachments: hfile_format_v2_design_draft_0.1.pdf
>
>
> In order to support HBASE-3763 and HBASE-3856, we need to change the format of the HFile. The new format proposal is attached here. Thanks for Mikhail Bautin for the documentation. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira