hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: OfflineMetaRepair?
Date Fri, 06 Jan 2012 00:50:03 GMT
Are you sure you have fs.default.name set properly to hdfs://yournn/
in your hbase-site.xml?

You shouldn't *have* to do this, but I bet it will fix the issue.

-Todd

On Thu, Jan 5, 2012 at 4:26 PM, Jonathan Hsieh <jon@cloudera.com> wrote:
> Hey Vlad,
>
> I wrote the tool -- and I've used it to repair a fairly messed up META
> table.  I must of used on a local filesystem copy of META (just got all the
> .regioninfo files in their directory paths), and then shipped the repaired
> version of the .META. dir to the customer.
>
> This is definitely a bug.  FIle the jira and I'll try to fix in the next
> few days.
>
> Jon.
>
> On Thu, Jan 5, 2012 at 4:16 PM, Vladimir Rodionov
> <vrodionov@carrieriq.com>wrote:
>
>> Ted,
>>
>> "fs.default.name" is a standard config property name which is described
>> here:
>> http://hadoop.apache.org/common/docs/current/core-default.html
>>
>> It is not CDH -specific. If you are right than this tool has never been
>> tested.
>>
>> Best regards,
>> Vladimir Rodionov
>> Principal Platform Engineer
>> Carrier IQ, www.carrieriq.com
>> e-mail: vrodionov@carrieriq.com
>>
>> ________________________________________
>> From: Ted Yu [yuzhihong@gmail.com]
>> Sent: Thursday, January 05, 2012 4:06 PM
>> To: dev@hbase.apache.org
>> Subject: Re: OfflineMetaRepair?
>>
>> Vlad:
>> In the future, please drop unrelated discussion from bottom of your email.
>>
>> I think what you saw was caused by FS default name not being set correctly.
>> In hbck:
>>        conf.set("fs.defaultFS", conf.get(HConstants.HBASE_DIR));
>> But cdh3 uses:
>>    conf.set("fs.default.name", "hdfs://localhost:0");
>> ./src/test/org/apache/hadoop/hdfs/server/namenode/TestCheckpoint.java
>>
>> You can try adding the following line after line 77 of
>> OfflineMetaRepair.java:
>>    conf.set("fs.default.name", path);
>> and rebuilding hbase 0.90.6 (tip of 0.92 branch)
>>
>> If the above works, please file a JIRA.
>>
>> Thanks
>>
>> On Thu, Jan 5, 2012 at 3:30 PM, Vladimir Rodionov
>> <vrodionov@carrieriq.com>wrote:
>>
>> > 0.90.5
>> >
>> > I am trying to repair .META. table using this tool
>> >
>> > 1.  HBase cluster was shutdown
>> >
>> > Then I ran:
>> >
>> > 2. [name01 bin]$ hbase
>> org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair
>> > -base hdfs://us01-ciqps1-name01.carrieriq.com:9000/hbase -details
>> >
>> >
>> > This is waht I got:
>> >
>> > 12/01/05 23:23:15 INFO util.HBaseFsck: Loading HBase regioninfo from
>> > HDFS...
>> > 12/01/05 23:23:30 ERROR util.HBaseFsck: Bailed out due to:
>> > java.lang.IllegalArgumentException: Wrong FS: hdfs://
>> >
>> us01-ciqps1-name01.carrieriq.com:9000/hbase/M2M-INTEGRATION-MM_TION-1325190318714/0003d2ede27668737e192d8430dbe5d0/.regioninfo
>> ,
>> > expected: file:///
>> >        at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:352)
>> >        at
>> >
>> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:47)
>> >        at
>> >
>> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:368)
>> >        at
>> >
>> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
>> >        at
>> >
>> org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:126)
>> >        at
>> > org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:284)
>> >        at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:398)
>> >        at
>> > org.apache.hadoop.hbase.util.HBaseFsck.loadMetaEntry(HBaseFsck.java:256)
>> >        at
>> > org.apache.hadoop.hbase.util.HBaseFsck.loadTableInfo(HBaseFsck.java:284)
>> >        at
>> > org.apache.hadoop.hbase.util.HBaseFsck.rebuildMeta(HBaseFsck.java:402)
>> >        at
>> >
>> org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair.main(OfflineMetaRepair.java:90)
>> >
>> >
>> > Q: What am I doing wrong?
>> >
>> > Best regards,
>> > Vladimir Rodionov
>> > Principal Platform Engineer
>> > Carrier IQ, www.carrieriq.com
>> > e-mail: vrodionov@carrieriq.com
>> >
>> >
>>
>> Confidentiality Notice:  The information contained in this message,
>> including any attachments hereto, may be confidential and is intended to be
>> read only by the individual or entity to whom this message is addressed. If
>> the reader of this message is not the intended recipient or an agent or
>> designee of the intended recipient, please note that any review, use,
>> disclosure or distribution of this message or its attachments, in any form,
>> is strictly prohibited.  If you have received this message in error, please
>> immediately notify the sender and/or Notifications@carrieriq.com and
>> delete or destroy any copy of this message and its attachments.
>>
>
>
>
> --
> // Jonathan Hsieh (shay)
> // Software Engineer, Cloudera
> // jon@cloudera.com



-- 
Todd Lipcon
Software Engineer, Cloudera

Mime
View raw message