hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-5453) Switch on-disk formats (reference files, HFile meta fields, etc) to PB
Date Wed, 16 May 2012 17:01:04 GMT

     [ https://issues.apache.org/jira/browse/HBASE-5453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

stack updated HBASE-5453:
-------------------------

    Attachment: 5453v9.txt

Address Gregory's comments.

I've changed the format of .regioninfo and .tableinfo.  Now instead of serialized Writable
followed by toString of the serialized object, instead its just the serialized pb.

This removes our having a human readable .regioninfo/.tablinfo file but my guess no one relied
on this anyways.

Having just serialized content in the file means a check of file length should be enough figuring
whether the file properly serialized.    If ever a chance that a Writable + its toString +
two '\n' characters was equal to a serialized pb, I'd think this likely a pathological state.
 If this state is not cleared up 'naturally' by splits or a schema change, then lets deal
if it happens.

I only need this length-checking in one place on region open.  I want to avoid reading the
.regioninfo file on region open.  The alternative means more load on NN and DNs at region
open time which could be problematic at big-bang cluster start (Thinking 500 nodes w/ 80k
regions, an actual known case).

Otherwise, Gregory's comments led to me to check and I was missing convertion of fs files
to pb in all cases.  This should be fixed now.

There are some failing tests still but running by hadoopqa to see what it says anyways.  Also
putting up on rb to get feedback if problem w/ this approach.
                
> Switch on-disk formats (reference files, HFile meta fields, etc) to PB
> ----------------------------------------------------------------------
>
>                 Key: HBASE-5453
>                 URL: https://issues.apache.org/jira/browse/HBASE-5453
>             Project: HBase
>          Issue Type: Sub-task
>          Components: ipc, master, migration, regionserver
>            Reporter: Todd Lipcon
>            Assignee: stack
>         Attachments: 5453.txt, 5453v2.txt, 5453v3.txt, 5453v6.txt, 5453v9.txt
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message