Mailing-List: contact hbase-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hbase-dev@hadoop.apache.org
Message-ID: <348220847.1254461785817.JavaMail.jira@brutus>
Date: Thu, 1 Oct 2009 22:36:25 -0700 (PDT)
From: "Woosuk Suh (JIRA)" <jira@apache.org>
To: hbase-dev@hadoop.apache.org
Subject: [jira] Commented: (HBASE-1867) Tool to regenerate an hbase table
 from the data files
In-Reply-To: <2064909871.1253836215975.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-1867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12761478#action_12761478 ] 

Woosuk Suh commented on HBASE-1867:
-----------------------------------

Your script worked perfectly with the HBase cluster with 4 running machines having 1011 regions.
I used your script because our .META. region evaporated for some unknown reason.
FYI, each server has 5GB memory and intel quad core CPU.

But sometimes I was not able to run the script because the errors happened.
So I`m going to give you all the process that I took to make the script work.

1. Our table had the structure on HDFS like this.
/hbase/TABLENAME

2. So I moved hbase to hbase_backup
/hbase_backup/TABLENAME

3. And then started hbase so the broken .META. table would regenerate cleanly. After starting hbase,
/hbase/
/hbase_backup/TABLENAME

4. And then I ran the script like this
bin/hbase org.jruby.Main add_table.rb hdfs://our.server.addr:port/hbase_backup/TABLENAME
Then I got the error from line 105, statuses were "nil" objects. Unable to iterate nil objects.

5. I`m not familiar to ruby but python, so I think it was impossible to iterate through None objects.
I printed the tableDir with LOG.info(tableDir.toString()) and I got following.
our.server.addr:port/hbase_backup/TABLENAME

6. So, I tried to copy the hbase_backup/TABLENAME to hbase/TABLENAME like following
bin/hadoop dfs -cp hbase_backup/TABLENAME hbase/TABLENAME

7. After a long time, copy process finished. And I tried to run the script again with following command.
bin/hbase org.jruby.Main add_table.rb hdfs://our.server.addr:port/hbase/TABLENAME
And it worked without any error or problem and all the regions were restored!

I hope this usage information helps your code improved.
Thanks for fabulous script!

> Tool to regenerate an hbase table from the data files
> -----------------------------------------------------
>
>                 Key: HBASE-1867
>                 URL: https://issues.apache.org/jira/browse/HBASE-1867
>             Project: Hadoop HBase
>          Issue Type: New Feature
>          Components: util
>    Affects Versions: 0.20.0
>            Reporter: elsif 
>            Priority: Minor
>         Attachments: add_table.rb
>
>
> The purpose of this JIRA is provide a place to coordinate the development of a utility that will regenerate an hbase table from the data files.
> Here are some comments from stack on this subject from the hbase-user mailing list:
> Well, in the bin directory, there are scripts that do various things with
> the .META. (copy a table, move a table, load a table whose source is hfiles
> written by a mapreduce job; i.e. hbase-48).
> So, to 'regenerate an hbase table from the data files', you'd need to do
> something like the following:
> + delete all exisiting table references from .META.
> + move the backuped up table into position under hbase.rootdir
> + per region under hbase.rootdir, add an entry to .META.  Do this by opening
> the .regioninfo file.  Its content is needed to generate the rowid for
> .META. and its value becomes the info:regioninfo cell value.
> HBase does not need to be down.  On next .META. scan, the newly added
> regions will be noticed. They won't have associated info:server and
> info:startcode entries so master will go ahead and assign them and  you
> should be up and running.
> Code-wise, a study of copy_table.rb (this uses old api ... needs updating
> but the concepts are the same) and loadtable.rb would probably be fruitful.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.