hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Hbase exportSnapshot fails in HA missing hbase-prefix-tree.jar
Date Fri, 14 Apr 2017 16:47:22 GMT
I can debug this next week when I have access to an HA cluster. 

Which release of Hadoop are you using ?

Cheers

> On Apr 14, 2017, at 9:21 AM, Vasco Pinho <vasco@hotjar.com> wrote:
> 
> I'm not sure I follow. This file already exists under the install dir at:
> 
> /home/ubuntu/hbase-1.2.4/hbase-prefix-tree/target/hbase-prefix-tree-1.2.4.jar
> 
> The class is also present at:
> 
> /home/ubuntu/hbase-1.2.4/hbase-prefix-tree/target/classes/org/apache/hadoop/hbase/codec/prefixtree/PrefixTreeCodec.class
> 
> Why would I rename it? Is my classpath wrong for some reason? In any case,
> renaming it to something like
> "hbase-prefix-tree/target/hbase-prefix-tree-1.2.4.jar.bak " yields another
> exception of the same sort (full trace: https://pastebin.com/mzcPthZR):
> 
> java.io.FileNotFoundException: File does not exist:
> hdfs://clusterID/root/.m2/repository/org/apache/htrace/htrace-core/3.1.0-incubating/htrace-core-3.1.0-incubating.jar
> 
> So it looks like something else is wrong with the installation or there is
> a bug that is making these searches attempt to look in "hdfs://" instead of
> the filesystem?
> 
> Thanks,
> Vasco Pinho
> 
>> On Fri, Apr 14, 2017 at 4:32 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>> 
>> Here is how hbase-prefix-tree dependency is detected:
>> 
>>    try {
>> 
>>      prefixTreeCodecClass =
>> 
>>          Class.forName(
>> "org.apache.hadoop.hbase.codec.prefixtree.PrefixTreeCodec");
>> 
>>    } catch (ClassNotFoundException e) {
>> 
>>      // this will show up in unit tests but should not show in real
>> deployments
>> 
>>      LOG.warn("The hbase-prefix-tree module jar containing PrefixTreeCodec
>> is not present." +
>> 
>>          "  Continuing without it.");
>> 
>>    }
>> 
>> As a workaround, consider temporarily renaming hbase-prefix-tree jar on the
>> node where ExportSnapshot is run.
>> 
>> Remember to rename hbase-prefix-tree jar back after the job.
>> 
>> FYI
>> 
>>> On Fri, Apr 14, 2017 at 5:02 AM, Vasco Pinho <vasco@hotjar.com> wrote:
>>> 
>>> Sure Ted,
>>> 
>>> Here's the full trace: https://pastebin.com/AGimZ8wZ
>>> And here's the hbase-site.xml: https://pastebin.com/yhRWCsaU
>>> 
>>> This is a working Hbase HA setup which has been happily chugging along. 3
>>> node HDFS JN, 3 node zk, 2 of them NN+backup NN. Also two of them
>>> HMaster+HMaster backup. Then several data nodes.
>>> 
>>> Thanks for taking a look!
>>> 
>>> Vasco
>>> 
>>>> On Fri, Apr 14, 2017 at 1:41 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>>>> 
>>>> Can you show the complete stack trace ?
>>>> 
>>>> Please pastebin contents of hbase/site.xml
>>>> 
>>>> Thanks
>>>> 
>>>>> On Apr 14, 2017, at 3:51 AM, Vasco Pinho <vasco@hotjar.com> wrote:
>>>>> 
>>>>> When running:
>>>>> 
>>>>> bin/hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot
>>>>> -Dfs.s3a.buffer.dir=/tmp/hbase_snap_tmp -snapshot
>>>> TestTable-20170413-143020
>>>>> -copy-to s3a://bucket-backup/hbase/snapshots/
>>>>> 
>>>>> 
>>>>> The operation fails with:
>>>>> 
>>>>> 
>>>>> 2017-04-13 15:03:24,947 ERROR [main] snapshot.ExportSnapshot:
>> Snapshot
>>>>> export failed
>>>>> java.io.FileNotFoundException: File does not exist:
>>>>> hdfs://clusterID/home/ubuntu/hbase-1.2.4/hbase-prefix-tree/
>>>> target/hbase-prefix-tree-1.2.4.jar
>>>>>       at
>>>>> org.apache.hadoop.hdfs.DistributedFileSystem$17.
>>>> doCall(DistributedFileSystem.java:1072)
>>>>> 
>>>>> 
>>>>> I assume it's looking for the
>>>>> "/home/ubuntu/hbase-1.2.4/hbase-prefix-tree/target/
>>>> hbase-prefix-tree-1.2.4.jar"
>>>>> which does exist but obviously not under hdfs://. I don't know why
>> this
>>>>> file is needed or why it's being looked for in the hdfs:// instead of
>>> the
>>>>> local filesystem, where it does exist. I confirmed that it looks
>> under
>>>>> "fs.defaultFS" + <hbase install path> + <relative path to
>>>> hbase-prefix-tree
>>>>> jar", although I have no idea how to fix this. Any ideas?
>>>>> 
>>>>> 
>>>>> Thanks,
>>>>> Vasco Pinho
>> 

Mime
View raw message