hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lex Toumbourou <...@scrunch.com>
Subject Export HBase snapshot to S3 creates empty root directory (prefix)
Date Thu, 05 May 2016 03:53:34 GMT
Hi all,

I'm having a couple of problems with exporting HBase snapshots to S3. I am
running HBase version 1.2.0.

I have a table called "domain"

And I have created a snapshot for it:

hbase(main):003:0> snapshot 'domain', 'domain-aws-test'
0 row(s) in 0.3310 seconds

---

I am attempting to export it to S3 using the following command:

hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot
"domain-aws-test" -copy-to s3://my-hbase-snapshots/domain-aws-test

Now, when I view the snapshot metadata in the S3 bucket, there's nothing
there:

> aws s3 ls my-hbase-snapshots/domain-aws-snapshots

But there is data under:

> aws s3 ls my-hbase-snapshots/\/domain-aws-test/
                           PRE .hbase-snapshot/
2016-05-05 13:38:12          1 .hbase-snapshot

It seems what's happening is, HBase is creating a directory/prefix with no
name and placing the snapshot data under there.

That wouldn't be a problem, except that when I try to import the snapshot
on my destination cluster, it seems unable to deal with the empty directory:

sudo -u hbase hbase snapshot export -D
hbase.rootdir=s3://my-hbase-snapshots/\/domain-aws-test -snapshot
my-aws-test -copy-to hdfs://hbaseClusterDNSName:8020/user/hbase -mappers 2

Caused by: java.io.FileNotFoundException: No such file or directory
's3://my-hbase-snapshots/domain-aws-test/.hbase-snapshot/domain-aws-test'

---

Any one come across this before?

Lex

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message