Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
MIME-Version: 1.0
Date: Thu, 5 May 2016 13:53:34 +1000
Message-ID: <CAO1bPwkc_q+jjxV3wZ1ZWFsQQ2pse0JvQ8sirAW3cC1F26sLvg@mail.gmail.com>
Subject: Export HBase snapshot to S3 creates empty root directory (prefix)
From: Lex Toumbourou <lex@scrunch.com>
To: user@hbase.apache.org
Content-Type: multipart/alternative; boundary=047d7bd76b9e4e86e205321049dc
archived-at: Thu, 05 May 2016 03:53:40 -0000

--047d7bd76b9e4e86e205321049dc
Content-Type: text/plain; charset=UTF-8

Hi all,

I'm having a couple of problems with exporting HBase snapshots to S3. I am
running HBase version 1.2.0.

I have a table called "domain"

And I have created a snapshot for it:

hbase(main):003:0> snapshot 'domain', 'domain-aws-test'
0 row(s) in 0.3310 seconds

---

I am attempting to export it to S3 using the following command:

hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot
"domain-aws-test" -copy-to s3://my-hbase-snapshots/domain-aws-test

Now, when I view the snapshot metadata in the S3 bucket, there's nothing
there:

> aws s3 ls my-hbase-snapshots/domain-aws-snapshots

But there is data under:

> aws s3 ls my-hbase-snapshots/\/domain-aws-test/
                           PRE .hbase-snapshot/
2016-05-05 13:38:12          1 .hbase-snapshot

It seems what's happening is, HBase is creating a directory/prefix with no
name and placing the snapshot data under there.

That wouldn't be a problem, except that when I try to import the snapshot
on my destination cluster, it seems unable to deal with the empty directory:

sudo -u hbase hbase snapshot export -D
hbase.rootdir=s3://my-hbase-snapshots/\/domain-aws-test -snapshot
my-aws-test -copy-to hdfs://hbaseClusterDNSName:8020/user/hbase -mappers 2

Caused by: java.io.FileNotFoundException: No such file or directory
's3://my-hbase-snapshots/domain-aws-test/.hbase-snapshot/domain-aws-test'

---

Any one come across this before?

Lex

--047d7bd76b9e4e86e205321049dc--