hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lex Toumbourou <...@scrunch.com>
Subject Deleting and cleaning old snapshots exported to S3
Date Tue, 21 Nov 2017 03:45:28 GMT
Hi all,

Wondering if I could get some help figuring out how to clean out old
snapshots that have been exported to S3?

We've been exporting snapshots to S3 using the export snapshot command:

bin/hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot
some-snapshot -copy-to s3a://some-bucket/hbase


Now the size of the S3 bucket is getting a little out of control and I'd
like to remove the old snapshots and let HBase garbage collect blocks no
longer referenced.

One idea I had was to spin up an entirely new cluster that uses the S3
bucket as the hbase.rootdir then just delete the snapshots as normal and
maybe use cleaner_run to clean up the old files but it feels like overkill
having to spin up an entire cluster.

So my question is: what's the best approach for deleting snapshots exported
to an s3 bucket and cleaning old store files no longer referenced? We are
using HBase 1.3.1 on EMR.

Thanks!

Lex ToumbourouCTO at scrunch.com <http://scrunch.com/>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message