hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vasco Pinho <va...@hotjar.com>
Subject Re: Hbase exportSnapshot fails in HA missing hbase-prefix-tree.jar
Date Fri, 14 Apr 2017 16:21:42 GMT
I'm not sure I follow. This file already exists under the install dir at:

/home/ubuntu/hbase-1.2.4/hbase-prefix-tree/target/hbase-prefix-tree-1.2.4.jar

The class is also present at:

/home/ubuntu/hbase-1.2.4/hbase-prefix-tree/target/classes/org/apache/hadoop/hbase/codec/prefixtree/PrefixTreeCodec.class

Why would I rename it? Is my classpath wrong for some reason? In any case,
renaming it to something like
"hbase-prefix-tree/target/hbase-prefix-tree-1.2.4.jar.bak " yields another
exception of the same sort (full trace: https://pastebin.com/mzcPthZR):

java.io.FileNotFoundException: File does not exist:
hdfs://clusterID/root/.m2/repository/org/apache/htrace/htrace-core/3.1.0-incubating/htrace-core-3.1.0-incubating.jar

So it looks like something else is wrong with the installation or there is
a bug that is making these searches attempt to look in "hdfs://" instead of
the filesystem?

Thanks,
Vasco Pinho

On Fri, Apr 14, 2017 at 4:32 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> Here is how hbase-prefix-tree dependency is detected:
>
>     try {
>
>       prefixTreeCodecClass =
>
>           Class.forName(
> "org.apache.hadoop.hbase.codec.prefixtree.PrefixTreeCodec");
>
>     } catch (ClassNotFoundException e) {
>
>       // this will show up in unit tests but should not show in real
> deployments
>
>       LOG.warn("The hbase-prefix-tree module jar containing PrefixTreeCodec
> is not present." +
>
>           "  Continuing without it.");
>
>     }
>
> As a workaround, consider temporarily renaming hbase-prefix-tree jar on the
> node where ExportSnapshot is run.
>
> Remember to rename hbase-prefix-tree jar back after the job.
>
> FYI
>
> On Fri, Apr 14, 2017 at 5:02 AM, Vasco Pinho <vasco@hotjar.com> wrote:
>
> > Sure Ted,
> >
> > Here's the full trace: https://pastebin.com/AGimZ8wZ
> > And here's the hbase-site.xml: https://pastebin.com/yhRWCsaU
> >
> > This is a working Hbase HA setup which has been happily chugging along. 3
> > node HDFS JN, 3 node zk, 2 of them NN+backup NN. Also two of them
> > HMaster+HMaster backup. Then several data nodes.
> >
> > Thanks for taking a look!
> >
> > Vasco
> >
> > On Fri, Apr 14, 2017 at 1:41 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> >
> > > Can you show the complete stack trace ?
> > >
> > > Please pastebin contents of hbase/site.xml
> > >
> > > Thanks
> > >
> > > > On Apr 14, 2017, at 3:51 AM, Vasco Pinho <vasco@hotjar.com> wrote:
> > > >
> > > > When running:
> > > >
> > > > bin/hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot
> > > > -Dfs.s3a.buffer.dir=/tmp/hbase_snap_tmp -snapshot
> > > TestTable-20170413-143020
> > > > -copy-to s3a://bucket-backup/hbase/snapshots/
> > > >
> > > >
> > > > The operation fails with:
> > > >
> > > >
> > > > 2017-04-13 15:03:24,947 ERROR [main] snapshot.ExportSnapshot:
> Snapshot
> > > > export failed
> > > > java.io.FileNotFoundException: File does not exist:
> > > > hdfs://clusterID/home/ubuntu/hbase-1.2.4/hbase-prefix-tree/
> > > target/hbase-prefix-tree-1.2.4.jar
> > > >        at
> > > > org.apache.hadoop.hdfs.DistributedFileSystem$17.
> > > doCall(DistributedFileSystem.java:1072)
> > > >
> > > >
> > > > I assume it's looking for the
> > > > "/home/ubuntu/hbase-1.2.4/hbase-prefix-tree/target/
> > > hbase-prefix-tree-1.2.4.jar"
> > > > which does exist but obviously not under hdfs://. I don't know why
> this
> > > > file is needed or why it's being looked for in the hdfs:// instead of
> > the
> > > > local filesystem, where it does exist. I confirmed that it looks
> under
> > > > "fs.defaultFS" + <hbase install path> + <relative path to
> > > hbase-prefix-tree
> > > > jar", although I have no idea how to fix this. Any ideas?
> > > >
> > > >
> > > > Thanks,
> > > > Vasco Pinho
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message