Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
MIME-Version: 1.0
Sender: saint.ack@gmail.com
In-Reply-To: <7eb0abd3-e544-4242-3561-52ba582f99c4@apache.org>
References: <CADcMMgFkPnZZ2vJsDqgJAV4VRdeqS=HSBmT7=evgz2ZY1_E_Ng@mail.gmail.com>
 <855dbd28-bafe-4880-9f0d-22b208ef0bd4@apache.org> <CADcMMgGESCh7XLhm8qBsFiu32iDCKJBWdoYoHXWQ5A-DBYuCYA@mail.gmail.com>
 <CAAg3a2p5q=-7C=+aK3cpG=1+S=JFHZkGV8W-_gSXLaasnJUxsg@mail.gmail.com>
 <CAAg3a2obou04nF1ySa0CUj657geGA7n891ekyfNHtE7h41Lf2A@mail.gmail.com>
 <CADcMMgGCqr7hoioBV3waEr3SO_sBhEbhg9aMR3kY_HH_arx=Jg@mail.gmail.com>
 <6da8f37f-2f7a-186a-0216-b55d71f71d9b@apache.org> <CAAg3a2pkrKBEXyvtCNpxQVzcXZXt+xAReeieofTdWJ8nkgAyCQ@mail.gmail.com>
 <CAAjhxrpWD1eoFLWXV4_gpUvR0n3f6_qDGovx0D0=N3SU6r16UA@mail.gmail.com>
 <CAAjhxrotkgn=qprcprSM3_Ag+FxbqjTMYGWRTQ7bBvjDzDwN6A@mail.gmail.com>
 <CAAg3a2oGr8PLrAmBPnKTgEx3oKnaqZ+-tv2aq9hoqz+xCQgH2Q@mail.gmail.com>
 <CADcMMgGx2QaY1EqPeY_8Y2DbHAE57H_9q283i1GgUgAAjVDRSg@mail.gmail.com> <7eb0abd3-e544-4242-3561-52ba582f99c4@apache.org>
From: Stack <stack@duboce.net>
Date: Thu, 2 Nov 2017 09:20:01 -0700
Message-ID: <CADcMMgH3-3NsKz9ATDrDvG=T9YqHK5qLh4qnjfojPh_eA6MYZg@mail.gmail.com>
Subject: Re: [DISCUSSION] Items to purge from branch-2 before we cut hbase-2.0.0-beta1.
To: HBase Dev List <dev@hbase.apache.org>
Content-Type: multipart/alternative; boundary="089e08244168448e0d055d025cf5"
archived-at: Thu, 02 Nov 2017 16:20:08 -0000

--089e08244168448e0d055d025cf5
Content-Type: text/plain; charset="UTF-8"

On Thu, Nov 2, 2017 at 5:51 AM, Josh Elser <elserj@apache.org> wrote:

> On 11/1/17 11:33 PM, Stack wrote:
>
>> On Wed, Nov 1, 2017 at 5:08 PM, Vladimir Rodionov<vladrodionov@gmail.com>
>> wrote:
>>
>> There is no way to validate correctness of backup in a general case.
>>>
>>> You can restore backup into temp table, but then what? Read rows
>>> one-by-one
>>> from temp table and look them up
>>>
>>
>>
>> in a primary table? Won't work, because rows can be deleted or modified
>>> since the last backup was done.
>>>
>>>
>>> Replication has a verity table tool.
>>
>> You can ask a cluster not delete rows.
>>
>> You can read at a specific timestamp.
>>
>> Or you could create backups during an extended ITBLL. When ITBLL
>> completes,
>> verify it on src cluster. Create a table from the increment backups.
>> Verify
>> in the restore.
>>
>> Etc.
>>
>> St.Ack
>>
>
> I can definitely see a benefit of a tool which verifies the data collected
> for a backup which:
>
> 1. Is batch in nature
> 2. Is ad-hoc (not intrinsically run for every backup session)
> 3. Relies/is-built on existing tooling (snapshots or other
> verification-like code)
>
> Thanks Stack. I think this is some good teasing of requirements from an
> otherwise very broad and untenable problem statement that we started with
> (which lead to the knee-jerk).
>

To be clear, I wasn't listing requirements. I was having trouble with the
absolute "There is no way to validate correctness of backup in a general
case." which is then seemingly being used to beat down any request for
verification tooling/testing that shows backup/restore works properly.
Good on you Josh,
S

--089e08244168448e0d055d025cf5--