hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: no-flush based snapshot policy?
Date Thu, 03 Apr 2014 01:19:14 GMT
HBASE-9426 (Make custom distributed barrier procedure pluggable) has been
back ported to 0.98
So porting the work from HBASE-7912 to 0.98 would be relatively easy.

I am not aware of this going into 0.94

Cheers


On Wed, Apr 2, 2014 at 6:13 PM, Varun Sharma <varun@pinterest.com> wrote:

> Seems like those JIRAs are 1.0 - did not see a 0.94 version # there ?
>
>
> On Wed, Apr 2, 2014 at 1:40 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>
> > Tianying:
> > Have you seen the design doc attached to HBASE-7912 'HBase Backup/Restore
> > Based on HBase Snapshot' ?
> >
> > Cheers
> >
> > >
> > > > On Tue, Mar 25, 2014 at 2:38 PM, Tianying Chang <tychang@gmail.com>
> > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I need a new snapshot policy. Basically, I cannot disable the
> table,
> > > but
> > > > I
> > > > > also don't need the snapshot to be that "consistent" where all RS
> > > > > coordinated to flush the region before taking the snapshot since
it
> > > slow
> > > > > down production cluster when flush take too long. It is OK for me
> if
> > > the
> > > > > snapshot missed the data in memstore because I will use WALPlayer
> to
> > > fill
> > > > > the data gap that is not in the snapshot but has been persisted (in
> > > WAL).
> > > > > So I should have no data loss.
> > > > >
> > > > > As a quick hack way to test this in my hbase backup workflow, I
> just
> > > add
> > > > a
> > > > > config key, and skip the flushcache() in file
> > > > > *regionserver/snapshot/FlushSnapshotSubprocedure.java*, something
> > like
> > > > > below.  It seems works fine for me, where all data are recovered
> in a
> > > new
> > > > > cluster after running WALPlayer.
> > > > >
> > > > > Does anyone see any problem like data corruption, etc?
> > > > >
> > > > >
> > > > > LOG.debug("Flush Snapshotting region " + region.toString() + "
> > > > > started...");
> > > > > if (noFlushNeeded)
> > > > > {
> > > > >    LOG.debug("No flush before taking snapshot");
> > > > > } else
> > > > > {
> > > > >     region.flushcache();
> > > > > }
> > > > >
> > > > > If there is no data corruption issue with this policy, I can add
an
> > > > > parameter from hbase shell, so that people can dynamically decide
> > when
> > > to
> > > > > use no-flush snapshot.
> > > > >
> > > > > Thanks
> > > > > Tian-Ying
> > > > >
> > > > > On Tue, Mar 25, 2014 at 2:08 PM, Tianying Chang <tychang@gmail.com
> >
> > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I need a new snapshot policy which sits in between the disabled
> and
> > > > > > flushed version. So, basically:
> > > > > > I cannot disable the table, but I also don't need the snapshot
to
> > be
> > > > that
> > > > > > "consistent" where all RS coordinated to flush the region before
> > > taking
> > > > > the
> > > > > > snapshot.
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message