hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vasco Pinho <va...@hotjar.com>
Subject Re: Hbase exportSnapshot fails in HA missing hbase-prefix-tree.jar
Date Wed, 19 Apr 2017 08:27:20 GMT
Anyone have any ideas what might be causing this? Wrong classpath? Some
package missing?

On Sun, Apr 16, 2017 at 9:25 AM, Vasco Pinho <vasco@hotjar.com> wrote:

> Running Hadoop 2.7.2, Hbase 1.2.4.
>
> On Fri, Apr 14, 2017 at 6:47 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>
>> I can debug this next week when I have access to an HA cluster.
>>
>> Which release of Hadoop are you using ?
>>
>> Cheers
>>
>> > On Apr 14, 2017, at 9:21 AM, Vasco Pinho <vasco@hotjar.com> wrote:
>> >
>> > I'm not sure I follow. This file already exists under the install dir
>> at:
>> >
>> > /home/ubuntu/hbase-1.2.4/hbase-prefix-tree/target/hbase-
>> prefix-tree-1.2.4.jar
>> >
>> > The class is also present at:
>> >
>> > /home/ubuntu/hbase-1.2.4/hbase-prefix-tree/target/classes/
>> org/apache/hadoop/hbase/codec/prefixtree/PrefixTreeCodec.class
>> >
>> > Why would I rename it? Is my classpath wrong for some reason? In any
>> case,
>> > renaming it to something like
>> > "hbase-prefix-tree/target/hbase-prefix-tree-1.2.4.jar.bak " yields
>> another
>> > exception of the same sort (full trace: https://pastebin.com/mzcPthZR):
>> >
>> > java.io.FileNotFoundException: File does not exist:
>> > hdfs://clusterID/root/.m2/repository/org/apache/htrace/htrac
>> e-core/3.1.0-incubating/htrace-core-3.1.0-incubating.jar
>> >
>> > So it looks like something else is wrong with the installation or there
>> is
>> > a bug that is making these searches attempt to look in "hdfs://"
>> instead of
>> > the filesystem?
>> >
>> > Thanks,
>> > Vasco Pinho
>> >
>> >> On Fri, Apr 14, 2017 at 4:32 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>> >>
>> >> Here is how hbase-prefix-tree dependency is detected:
>> >>
>> >>    try {
>> >>
>> >>      prefixTreeCodecClass =
>> >>
>> >>          Class.forName(
>> >> "org.apache.hadoop.hbase.codec.prefixtree.PrefixTreeCodec");
>> >>
>> >>    } catch (ClassNotFoundException e) {
>> >>
>> >>      // this will show up in unit tests but should not show in real
>> >> deployments
>> >>
>> >>      LOG.warn("The hbase-prefix-tree module jar containing
>> PrefixTreeCodec
>> >> is not present." +
>> >>
>> >>          "  Continuing without it.");
>> >>
>> >>    }
>> >>
>> >> As a workaround, consider temporarily renaming hbase-prefix-tree jar
>> on the
>> >> node where ExportSnapshot is run.
>> >>
>> >> Remember to rename hbase-prefix-tree jar back after the job.
>> >>
>> >> FYI
>> >>
>> >>> On Fri, Apr 14, 2017 at 5:02 AM, Vasco Pinho <vasco@hotjar.com>
>> wrote:
>> >>>
>> >>> Sure Ted,
>> >>>
>> >>> Here's the full trace: https://pastebin.com/AGimZ8wZ
>> >>> And here's the hbase-site.xml: https://pastebin.com/yhRWCsaU
>> >>>
>> >>> This is a working Hbase HA setup which has been happily chugging
>> along. 3
>> >>> node HDFS JN, 3 node zk, 2 of them NN+backup NN. Also two of them
>> >>> HMaster+HMaster backup. Then several data nodes.
>> >>>
>> >>> Thanks for taking a look!
>> >>>
>> >>> Vasco
>> >>>
>> >>>> On Fri, Apr 14, 2017 at 1:41 PM, Ted Yu <yuzhihong@gmail.com>
wrote:
>> >>>>
>> >>>> Can you show the complete stack trace ?
>> >>>>
>> >>>> Please pastebin contents of hbase/site.xml
>> >>>>
>> >>>> Thanks
>> >>>>
>> >>>>> On Apr 14, 2017, at 3:51 AM, Vasco Pinho <vasco@hotjar.com>
wrote:
>> >>>>>
>> >>>>> When running:
>> >>>>>
>> >>>>> bin/hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot
>> >>>>> -Dfs.s3a.buffer.dir=/tmp/hbase_snap_tmp -snapshot
>> >>>> TestTable-20170413-143020
>> >>>>> -copy-to s3a://bucket-backup/hbase/snapshots/
>> >>>>>
>> >>>>>
>> >>>>> The operation fails with:
>> >>>>>
>> >>>>>
>> >>>>> 2017-04-13 15:03:24,947 ERROR [main] snapshot.ExportSnapshot:
>> >> Snapshot
>> >>>>> export failed
>> >>>>> java.io.FileNotFoundException: File does not exist:
>> >>>>> hdfs://clusterID/home/ubuntu/hbase-1.2.4/hbase-prefix-tree/
>> >>>> target/hbase-prefix-tree-1.2.4.jar
>> >>>>>       at
>> >>>>> org.apache.hadoop.hdfs.DistributedFileSystem$17.
>> >>>> doCall(DistributedFileSystem.java:1072)
>> >>>>>
>> >>>>>
>> >>>>> I assume it's looking for the
>> >>>>> "/home/ubuntu/hbase-1.2.4/hbase-prefix-tree/target/
>> >>>> hbase-prefix-tree-1.2.4.jar"
>> >>>>> which does exist but obviously not under hdfs://. I don't know
why
>> >> this
>> >>>>> file is needed or why it's being looked for in the hdfs:// instead
>> of
>> >>> the
>> >>>>> local filesystem, where it does exist. I confirmed that it looks
>> >> under
>> >>>>> "fs.defaultFS" + <hbase install path> + <relative path
to
>> >>>> hbase-prefix-tree
>> >>>>> jar", although I have no idea how to fix this. Any ideas?
>> >>>>>
>> >>>>>
>> >>>>> Thanks,
>> >>>>> Vasco Pinho
>> >>
>>
>
>
>
> --
> Vasco Pinho
> DevOps Engineer
> www.hotjar.com
> Connect with me on LinkedIn
> <https://pt.linkedin.com/in/vasco-pinho-58770534>
> IMPORTANT CONFIDENTIALITY NOTICE: This message is confidential and
> intended for the use only of the person to whom this message is addressed.
> If you are not the intended recipient you are strictly prohibited from
> reading, disseminating, copying or using this message, or its contents in
> any way and must contact the sender immediately.
>



-- 
Vasco Pinho
DevOps Engineer
www.hotjar.com
Connect with me on LinkedIn
<https://pt.linkedin.com/in/vasco-pinho-58770534>
IMPORTANT CONFIDENTIALITY NOTICE: This message is confidential and intended
for the use only of the person to whom this message is addressed. If you
are not the intended recipient you are strictly prohibited from reading,
disseminating, copying or using this message, or its contents in any way
and must contact the sender immediately.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message