Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
MIME-Version: 1.0
In-Reply-To: <CAAg3a2rorS7AsZicPgofxfHdfFgN7pCey5bhX424gHYO-UmP1w@mail.gmail.com>
References: <CAMUu0w9mX-=zb9exu5J_hoBBei8Hwb3vAKBa_DD+KNp+X7V7sw@mail.gmail.com>
 <CAAg3a2oRGKvq0XSe300JKtBm7YtE2WN5_4dJCC3+8gqWsiorAg@mail.gmail.com>
 <CAAg3a2rz_zDh5ph+thqtcoB+FGD+KPvBGE8=V+kjUiA7ufMgDw@mail.gmail.com>
 <CA+DCeTE5368DTdaG65D1QxXDrmBE9M=LEvxz0TLfzFkf55dD4Q@mail.gmail.com>
 <CAAg3a2ryV9XxS0KCYn7=Hai1drcpfg9O-BHXtq=1Ok8h50EqLw@mail.gmail.com>
 <FE336A68-469F-4B61-AB0B-E75007C14017@gmail.com> <CAAg3a2ohJnGY9J=m3dPPE+rcMCizA54NOyugukpwQsp7cvgcRQ@mail.gmail.com>
 <CALte62wZ+Yg8U5E8AoUF2r4evU69uHaORgrNxYvDLR08=DDtjQ@mail.gmail.com>
 <CAFb6HiKnev-ynTDa8mnCGN_ZX_PrNUsk1G_wQp7jvAkyAJojVg@mail.gmail.com>
 <CAMUu0w85XY4To2oebHhVG+jSTqkwVfB=jo3KOPZ_vXJiRSYjUg@mail.gmail.com>
 <CAAg3a2pKBG_G7BDZ7VxFEu9Dr0udasUZZvTQrTiqmQtKuKLvnw@mail.gmail.com>
 <CAMUu0w8ctU1ECURRKaGjwo1di9jcixmhjgXXsuXnMSJFAJHybg@mail.gmail.com>
 <CAAg3a2oTeoP3PQg+-5HAkjMYq8tcppBwZ1dFVznG0fZvcfR5Tg@mail.gmail.com>
 <CAAg3a2q3G04E5zzjXVRFYBszSpNh-ztPGQv4Hc1KKa-fgC71cw@mail.gmail.com>
 <CALte62zAJLy9EKuoP+KQ1xRFXfV1Boipe+mf4pN5yPHqTGBC+Q@mail.gmail.com>
 <CA+DCeTGMKSeFF6wY3XETnQPDoWywGi5y=g6p25VonBKEY=UPiw@mail.gmail.com>
 <CALte62wX3JAqsbYvkCyQ_YwsspxDnhPETwgEnU=-U89tAEOwAA@mail.gmail.com>
 <92C83CD0-100D-401F-8B2C-D5C0D80DBC91@gmail.com> <CAAg3a2rZS5JUMOMFDyU43BOsyJLjsoJXARgOUL=3-wgcNLaw0g@mail.gmail.com>
 <CAGHyZ6LgzV=eLm0Bq_aCSZZOYsS8m3xABS=_Mi+KbuZFurZHxg@mail.gmail.com>
 <CALte62y3N=nOoK3dD6byBTh4XoX=w+jjxpTOAa86KOSaH=1vcQ@mail.gmail.com>
 <CAAg3a2rK=xnFpseiLmCvw9R13H4k8WN6urvwKdvr35e_a_uoNQ@mail.gmail.com>
 <CA+RK=_CuaRcC_Sef2CuMbRWg2Yy7TJVj76eVBmaFEd0Ukj3d5A@mail.gmail.com>
 <CAGHyZ6JHD6gd3QyNvSP0kWfoZJPkoyOLbH76hacdHxYCMP1joA@mail.gmail.com>
 <CALte62yWsvofQbPx__0o6sdw0GbWMmAv0sjWuDg0jx2zCXPWtA@mail.gmail.com> <CAAg3a2rorS7AsZicPgofxfHdfFgN7pCey5bhX424gHYO-UmP1w@mail.gmail.com>
From: Vladimir Rodionov <vladrodionov@gmail.com>
Date: Fri, 2 Sep 2016 08:51:11 -0700
Message-ID: <CAAg3a2qkzwGzn0z3=b3YW6bSd-eYGfFwvD4105o3Rm+9FSVnCQ@mail.gmail.com>
Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912
To: "dev@hbase.apache.org" <dev@hbase.apache.org>
Content-Type: multipart/alternative; boundary=001a113d9d16bee50f053b884c8c
archived-at: Fri, 02 Sep 2016 15:51:19 -0000

--001a113d9d16bee50f053b884c8c
Content-Type: text/plain; charset=UTF-8

Snapshot robustness is better now with introduction of region splits/merges
on/off feature. Region splits during snapshots was the major problem.

-Vlad

On Fri, Sep 2, 2016 at 8:12 AM, Vladimir Rodionov <vladrodionov@gmail.com>
wrote:

> >>Are they independent enough that we can get backup/restore tolerant to
> >>failures prior to merge to master? Prior to backport to branch-1?
>
> As we stated already, snapshots are not part of the feature, snapshots has
> been merged into the master long time ago
> and as far as I understood - without requiring them to be 100% robust and
> fault tolerant and they are widely used in many production systems
> nevertheless. https://issues.apache.org/jira/browse/HBASE-14415 relies on
> Snapshots v2 but we can reconsider it, there are some thoughts how to make
> backups snapshotless.
>
> Backups are fault tolerant to some extent - in case of failure (and
> failures can happen) we clean everything up and do not leave system table
> in inconsistent state. Would it be enough, Sean Busbey?
>
> -Vlad
>
> On Fri, Sep 2, 2016 at 7:38 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>
>> We're continuing to make backup / restore more robust.
>> Work in progress (both are close to being integrated):
>>
>> HBASE-15565 Rewrite restore with Procedure V2
>> HBASE-15449 Support physical table layout change
>>
>> Since snapshot is dependency in the full backup, backup / restore wouldn't
>> be more robust than snapshot is.
>>
>> On Fri, Sep 2, 2016 at 7:03 AM, Sean Busbey <busbey@cloudera.com> wrote:
>>
>> > right, they're separate features but when asked about "robust
>> > backup/restore" (which is what I care about for this feature getting
>> > merged) things were pawned off on snapshots.
>> >
>> > Are they independent enough that we can get backup/restore tolerant to
>> > failures prior to merge to master? Prior to backport to branch-1?
>> >
>> > On Thu, Sep 1, 2016 at 1:11 PM, Andrew Purtell <apurtell@apache.org>
>> > wrote:
>> > > I agree these are separate features FWIW
>> > >
>> > > On Thu, Sep 1, 2016 at 11:09 AM, Vladimir Rodionov <
>> > vladrodionov@gmail.com>
>> > > wrote:
>> > >
>> > >> >> Do we have JIRA issue(s) covering making snapshots robust in the
>> face
>> > >> >> of monkeys?
>> > >>
>> > >> I would like to mention that "robust snapshots" and "table
>> > backup/restore"
>> > >> are totally separate features, but we have separate JIRA for fault
>> > >> tolerance (HBASE-14413).
>> > >>
>> > >> -Vlad
>> > >>
>> > >> On Thu, Sep 1, 2016 at 9:28 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>> > >>
>> > >> > Sean:
>> > >> > Please see HBASE-14413 for the last question.
>> > >> >
>> > >> > FYI
>> > >> >
>> > >> > On Thu, Sep 1, 2016 at 9:24 AM, Sean Busbey <busbey@cloudera.com>
>> > wrote:
>> > >> >
>> > >> > > On Sat, Aug 20, 2016 at 12:59 PM, Vladimir Rodionov
>> > >> > > <vladrodionov@gmail.com> wrote:
>> > >> > > > Not sure what do you mean, Andrew by "trying out the branch via
>> > the
>> > >> > IT",
>> > >> > > > but we do not recommend running this with monkey enabled.
>> > >> > > > It has not been tested in a such scenario yet and frankly
>> > speaking it
>> > >> > is
>> > >> > > > not supposed to work (snapshots will fail anyway and we
>> depends on
>> > >> > > > snapshots)
>> > >> > > >
>> > >> > > >
>> > >> > >
>> > >> > > Also won't have time to test out the branch this week, but if
>> we're
>> > >> > > not going to handle failures do we have tools or guidance on
>> > >> > > recovering in the case of things falling over?
>> > >> > >
>> > >> > > Do we have JIRA issue(s) covering making snapshots robust in the
>> > face
>> > >> > > of monkeys?
>> > >> > >
>> > >> > > --
>> > >> > > busbey
>> > >> > >
>> > >> >
>> > >>
>> > >
>> > >
>> > >
>> > > --
>> > > Best regards,
>> > >
>> > >    - Andy
>> > >
>> > > Problems worthy of attack prove their worth by hitting back. - Piet
>> Hein
>> > > (via Tom White)
>> >
>> >
>> >
>> > --
>> > busbey
>> >
>>
>
>

--001a113d9d16bee50f053b884c8c--