hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-2366) We should store table metadata in a single location, for HDFS table recovery, not on every region
Date Tue, 22 Mar 2011 01:32:05 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13009503#comment-13009503

stack commented on HBASE-2366:

Cristian 's idea above of a .tableinfo instead of a .regioninfo is a big improvement over
what we currently do.

> We should store table metadata in a single location, for HDFS table recovery, not on
every region
> -------------------------------------------------------------------------------------------------
>                 Key: HBASE-2366
>                 URL: https://issues.apache.org/jira/browse/HBASE-2366
>             Project: HBase
>          Issue Type: Bug
>          Components: regionserver
>         Environment: CentOS 5.4, x86_64
>            Reporter: Cristian Ivascu
>             Fix For: 0.92.0
> Right now, every region has a .regioninfo file that describes what the region contains.
This is used by the add_table tool to recover a table from HDFS, when there are problems with
the .META. table. Right now the .regioninfo file stores both region-specific as well as table-specific
metadata (the columns), which can cause problems.
> One such problem appears when some of the .regioninfo files are not updates. Workflow
to reproduce this is:
> 1. Create a new table with a list of columns.
> 2. Push some data into the table, so at least one region exists.
> 3. Do an ALTER on the table and add some new columns.
> 3.1 Look at one of the .regioninfo files. They still contain the old list of columns.
> 4. Push some more data into the table, containing values for the new columns.
> 5. Run add_table.rb on this table. 
> 6. Try to push the data at point 4 again. You will get errors - NoSuchColumnFamilyExists
> in our specific case, 10 out of the 140 regions of a table had the initial list of columns,
and the others were updated (as part of splits, as we deduced). We had to run add_table, which
replaced the .META. entries with the ones on disk, and as a result we couldn't get to our
data anymore. Re-writing the .regioninfo with the correct column list (as done in HRegion.java)
and re-adding the table fixed it for us,.
> It should be possible to store the table information in a single location (e.g. /hbase/table_name/.tableinfo
maybe?) and work on it on table create / alter which would prevent this case from happening

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message