Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CAFb6HiKC2LKdyWWe62hwueB9k+K0o6dHNqj7Eb95CnQD=+7fdw@mail.gmail.com>
References: 
 <CAEpFk2coQmUmpZELBqwnDpgQsrTuzJnmvebr_uKDdJFN6oQ5KA@mail.gmail.com>
	<CAFb6HiKC2LKdyWWe62hwueB9k+K0o6dHNqj7Eb95CnQD=+7fdw@mail.gmail.com>
Date: Fri, 10 Jul 2015 15:46:11 -0700
Message-ID: 
 <CAEpFk2e+J1u47vD2yoANgjrworNKE7=vqKapeX9n3EYrCHU4Dw@mail.gmail.com>
Subject: Re: hbase hdfs snapshots
From: rahul gidwani <rahul.gidwani@gmail.com>
To: dev@hbase.apache.org
Content-Type: multipart/alternative; boundary=bcaec52c5c1775fe4a051a8d2324

--bcaec52c5c1775fe4a051a8d2324
Content-Type: text/plain; charset=UTF-8

Hi Matteo,

We do SKIP_FLUSH.  We have 1200+ regionservers with a single table with 60k
regions and 4 column families.  It takes around 30 minutes to snapshot this
table using manifests compared to just seconds doing this with hdfs.
Cloning this table takes considerably longer.

For cases where someone would want to run Map/Reduce over snapshots this
could be much faster as we could take an hdfs snapshot and bypass the clone.

rahul


On Thu, Jul 9, 2015 at 12:20 PM, Matteo Bertozzi <theo.bertozzi@gmail.com>
wrote:

> On Thu, Jul 9, 2015 at 12:12 PM, rahul gidwani <rahul.gidwani@gmail.com>
>  wrote:
>
> > Even with manifests (Snapshot V2) for our larger tables it can take hours
> > to Snapshot and Clone a table.
> >
>
> on snapshot time the only thing that can take hours, is "flush".
> if you don't need that (which is what you get with hdfs snapshots) you can
> specify SKIP_FLUSH => true
>
>
> Matteo
>
>
> On Thu, Jul 9, 2015 at 12:12 PM, rahul gidwani <rahul.gidwani@gmail.com>
> wrote:
>
> > HBase snapshots are a very useful feature. but it was implemented back
> > before there was the ability to snapshot via HDFS.
> >
> > Newer versions of Hadoop support HDFS snapshots.  I was wondering if the
> > community would be interested in something like a Snapshot V3 where we
> use
> > HDFS to take these snapshots.
> >
> > Even with manifests (Snapshot V2) for our larger tables it can take hours
> > to Snapshot and Clone a table.
> >
> > Would this feature be of use to anyone?
> >
> > thanks
> > rahul
> >
>

--bcaec52c5c1775fe4a051a8d2324--