hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Rodionov <vladrodio...@gmail.com>
Subject Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912
Date Sun, 10 Sep 2017 00:58:58 GMT
That was I thought. Thanks. Can you tell me that you are not considering AM
v2 as an unfinished and untested feature? The question to Stack as well.

On Sat, Sep 9, 2017 at 5:53 PM, Andrew Purtell <andrew.purtell@gmail.com>
wrote:

> No all I have to do is pay attention to words you have written yourself in
> emails and on JIRA. Don't argue with us not to believe our lying eyes,
> consider finishing the work. I'll be happy to try it out when you indicate
> it can work if anything happens to fail on the cluster at the time. Until
> then there are a lot of other things need doing first.
>
>
> On Sep 9, 2017, at 5:25 PM, Vladimir Rodionov <vladrodionov@gmail.com>
> wrote:
>
> >>> but the impression we have is it is unfinished and untested.
> > To make a conclusion that "feature is not finished and tested"  you have
> > had to test it at least.
> > Andrew, If you have discovered issues, why wouldn't you open bug JIRAs?
> >
> > -Vlad
> >
> > On Sat, Sep 9, 2017 at 4:40 PM, Andrew Purtell <andrew.purtell@gmail.com
> >
> > wrote:
> >
> >> For what it's worth, I think AMv2 is the main reason to have a 2.0 in
> the
> >> first place, so I would both agree it needs a lot more testing and yet I
> >> would want us to have a 2.0 release as the vehicle for getting that to
> >> happen. For other features without testing from a number of parties or
> at
> >> scale the value proposition is less clear and it's fine by me for the
> RM to
> >> set them aside for future releases.
> >>
> >> Also, I can relay that there is some interest where I work in utilizing
> >> HBASE-7912 but the impression we have is it is unfinished and untested.
> So
> >> for now we are ignoring it and continuing with home grown solutions.
> Part
> >> of the problem is fault tolerance was left to the last phase(s) and yet
> it
> >> is an essential property for adoption for serious work. The best way to
> >> resolve this IMHO is for the developers of this feature to complete
> those
> >> unfinished JIRAs, especially concerning resilience to failures.
> >>
> >>
> >>> On Sep 9, 2017, at 4:11 PM, Vladimir Rodionov <vladrodionov@gmail.com>
> >> wrote:
> >>>
> >>> Hmm, the next on your list (of kicked out from branch v2) should be AM
> >> v2 I
> >>> presume?
> >>>
> >>> -Vlad
> >>>
> >>>> On Sat, Sep 9, 2017 at 4:04 PM, stack <saint.ack@gmail.com> wrote:
> >>>>
> >>>> In spite of repeated requests for eng summary of state of this feature
> >> --
> >>>> summary of what is in 2.0, what is not, what the capabilities are, how
> >> well
> >>>> it has been tested and at what scale -- all I get, when the requests
> are
> >>>> not ignored, are pointers to lists of ill-describing jiras and some
> >> pending
> >>>> user facing doc update.
> >>>>
> >>>> For other features, mob or region server groups, I know that they have
> >> been
> >>>> running at scale in production for as much as a year and more. I have
> >> some
> >>>> confidence these items basically work.  For backup/restore I have no
> >> such
> >>>> sense even after spending time in review and trying to use the
> feature.
> >>>>
> >>>> As release manager, I have say over what makes it into a release.
> >> Unless
> >>>> the work is done to convince me that backup/restore is more than a
> lump
> >> of
> >>>> code and a few unit tests that can pass on some fellows laptop, I am
> >> going
> >>>> to kick it out of branch-2.  Let the feature harden more in master
> >> branch
> >>>> before it ships in a release.
> >>>>
> >>>> S
> >>>>
> >>>> On Sep 8, 2017 10:59 PM, "Vladimir Rodionov" <vladrodionov@gmail.com>
> >>>> wrote:
> >>>>
> >>>>>>> Have I grasped the state of things correctly, Vlad?
> >>>>>
> >>>>> Josh, the only thing which is still pending is doc update. All other
> >>>>> features are good to have but not a blockers for 2.0 release.
> >>>>>
> >>>>> -Vlad
> >>>>>
> >>>>> On Fri, Sep 8, 2017 at 10:42 PM, Vladimir Rodionov <
> >>>> vladrodionov@gmail.com
> >>>>>>
> >>>>> wrote:
> >>>>>
> >>>>>>>> What testing and at what
> >>>>>>>> scale has testing been done?
> >>>>>>
> >>>>>> Do we have have that for other features?
> >>>>>>
> >>>>>>
> >>>>>> On Fri, Sep 8, 2017 at 10:41 PM, Vladimir Rodionov <
> >>>>> vladrodionov@gmail.com
> >>>>>>> wrote:
> >>>>>>
> >>>>>>>>> It asks: "How do I figure what of backup/restore
feature is going
> >>>> to
> >>>>>>> be in
> >>>>>>>>> hbase-2.0.0?
> >>>>>>>
> >>>>>>> Hmm, wait for doc update.
> >>>>>>>
> >>>>>>>
> >>>>>>>> On Fri, Sep 8, 2017 at 2:39 PM, Stack <stack@duboce.net>
wrote:
> >>>>>>>>
> >>>>>>>> HBASE-14414 is a JIRA with a list of random seeming
issues w/
> >>>>>>>> non-descript
> >>>>>>>> summaries: "Add nonce support to TableBackupProcedure,
BackupID
> must
> >>>>>>>> include backup set name, ...". The last comment in that
issue is
> >> from
> >>>>>>>> July.
> >>>>>>>> It asks: "How do I figure what of backup/restore feature
is going
> to
> >>>> be
> >>>>>>>> in
> >>>>>>>> hbase-2.0.0? Thanks Vladimir Rodionov
> >>>>>>>> <https://issues.apache.org/jira/secure/ViewProfile.jspa?
> >>>> name=vrodionov
> >>>>>>>>> ."
> >>>>>>>> to which there is no answer.  Doc update is TODO.
> >>>>>>>>
> >>>>>>>> Where is the summary of the capability in hbase-2? What
testing
> and
> >>>> at
> >>>>>>>> what
> >>>>>>>> scale has testing been done? Is this 'stable or experimental'?
If
> I
> >>>>> can't
> >>>>>>>> get basic info on this feature though I ask repeatedly,
what hope
> >>>> does
> >>>>>>>> the
> >>>>>>>> poor old operator have?
> >>>>>>>>
> >>>>>>>> St.Ack
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Fri, Sep 8, 2017 at 1:59 PM, Vladimir Rodionov <
> >>>>>>>> vladrodionov@gmail.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> HBASE-14414
> >>>>>>>>>
> >>>>>>>>>> On Fri, Sep 8, 2017 at 1:14 PM, Stack <stack@duboce.net>
wrote:
> >>>>>>>>>>
> >>>>>>>>>> Where do I go to get the current status of this
feature? Looking
> >>>> in
> >>>>>>>> JIRA
> >>>>>>>>> I
> >>>>>>>>>> see loads of issues open against backup including
some against
> >>>>>>>>> hbase-2.0.0
> >>>>>>>>>> and no progress being made that I can discern.
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>> S
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> On Wed, Nov 23, 2016 at 8:52 AM, Stack <stack@duboce.net>
> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, Nov 22, 2016 at 6:48 PM, Stack <stack@duboce.net>
> >>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> On Tue, Nov 22, 2016 at 3:17 PM, Vladimir
Rodionov <
> >>>>>>>>>>>> vladrodionov@gmail.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>>>> and/or he answered most
of the review feedback
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> No, questions are still open, but
I do not see any blockers
> >>>> and
> >>>>>>>> we
> >>>>>>>>> have
> >>>>>>>>>>>>> HBASE-16940 to address these questions.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>> Agree. No blockers but stuff that should
be dealt with (No one
> >>>>>>>> will
> >>>>>>>>> pay
> >>>>>>>>>>>> me any attention once merge goes in
-- smile).
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>> Let me clarify the above. I want review
addressed before merge
> >>>>>>>> happens.
> >>>>>>>>>>> Sorry if any confusion.
> >>>>>>>>>>> St.Ack
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>> St.Ack
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>> On Tue, Nov 22, 2016 at 3:04 PM,
Devaraj Das <
> >>>>>>>> ddas@hortonworks.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi Stack, hats off to you for
spending so much time on
> >>>> this!
> >>>>>>>>> Thanks!
> >>>>>>>>>>>>> From
> >>>>>>>>>>>>>> my understanding, Vlad has raised
follow-up jiras for the
> >>>>>>>> issues
> >>>>>>>>> you
> >>>>>>>>>>>>>> raised, and/or he answered most
of the review feedback. So,
> >>>>> do
> >>>>>>>> you
> >>>>>>>>>>>>> think we
> >>>>>>>>>>>>>> could do a merge vote now?
> >>>>>>>>>>>>>> Devaraj.
> >>>>>>>>>>>>>> ________________________________________
> >>>>>>>>>>>>>> From: Vladimir Rodionov <vladrodionov@gmail.com>
> >>>>>>>>>>>>>> Sent: Monday, November 21, 2016
8:34 PM
> >>>>>>>>>>>>>> To: dev@hbase.apache.org
> >>>>>>>>>>>>>> Subject: Re: [DISCUSSION] Merge
Backup / Restore - Branch
> >>>>>>>>> HBASE-7912
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I have spent a good
bit of time reviewing and testing
> >>>> this
> >>>>>>>>>> feature.
> >>>>>>>>>>>>> I
> >>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>> like my review and concerns
addressed and I'd like it to
> >>>>> be
> >>>>>>>>> clear
> >>>>>>>>>>>>> how;
> >>>>>>>>>>>>>>>> either explicit follow-on
issues, pointers to where in
> >>>> the
> >>>>>>>> patch
> >>>>>>>>>> or
> >>>>>>>>>>>>> doc
> >>>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>> remarks have been catered
to, etc. Until then, I am
> >>>>> against
> >>>>>>>>>> commit.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Stack, mega patch review comments
will be addressed in the
> >>>>>>>>> dedicated
> >>>>>>>>>>>>> JIRA:
> >>>>>>>>>>>>>> HBASE-16940
> >>>>>>>>>>>>>> I have open several other JIRAs
to address your other
> >>>>> comments
> >>>>>>>> (not
> >>>>>>>>>> on
> >>>>>>>>>>>>>> review board).
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Details are here (end of the
thread):
> >>>>>>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-14123
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Let me know what else should
we do to move merge forward.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> -Vlad
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Fri, Nov 18, 2016 at 4:54
PM, Stack <stack@duboce.net>
> >>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Fri, Nov 18, 2016 at
3:53 PM, Ted Yu <
> >>>>> yuzhihong@gmail.com
> >>>>>>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks, Matteo.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> bq. restore is not clear
if given an incremental id it
> >>>>>>>> will do
> >>>>>>>>>> the
> >>>>>>>>>>>>> full
> >>>>>>>>>>>>>>>> restore from full up
to that point or if i need to
> >>>> apply
> >>>>>>>>> manually
> >>>>>>>>>>>>>>>> everything
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> The restore takes into
consideration of the dependent
> >>>>>>>>> backup(s).
> >>>>>>>>>>>>>>>> So there is no need
to apply preceding backup(s)
> >>>>> manually.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I ask this question on the
issue. It is not clear from
> >>>> the
> >>>>>>>> usage
> >>>>>>>>> or
> >>>>>>>>>>>>> doc
> >>>>>>>>>>>>>> how
> >>>>>>>>>>>>>>> to run a restore from incremental.
Can you fix in doc and
> >>>>>>>> usage
> >>>>>>>>> how
> >>>>>>>>>>>>> so I
> >>>>>>>>>>>>>>> can be clear and try it.
Currently I am stuck verifying a
> >>>>>>>> round
> >>>>>>>>>> trip
> >>>>>>>>>>>>>> backup
> >>>>>>>>>>>>>>> restore made of incrementals.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>> S
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Fri, Nov 18, 2016
at 3:48 PM, Matteo Bertozzi <
> >>>>>>>>>>>>>>> theo.bertozzi@gmail.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I did one last pass
to the mega patch. I don't see
> >>>>>>>> anything
> >>>>>>>>>> major
> >>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>> should block the
merge.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> - most of the code
is isolated in the backup package
> >>>>>>>>>>>>>>>>> - all the backup
code is client side
> >>>>>>>>>>>>>>>>> - there are few
changes to the server side, mainly
> >>>> for
> >>>>>>>>>> cleaners,
> >>>>>>>>>>>>> wal
> >>>>>>>>>>>>>>>>> rolling and similar
(which is ok)
> >>>>>>>>>>>>>>>>> - there is a good
number of tests, and an integration
> >>>>>>>> test
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> the code seems to
have still some left overs from the
> >>>>> old
> >>>>>>>>>>>>>>> implementation,
> >>>>>>>>>>>>>>>>> and some stuff needs
a cleanup. but I don't think
> >>>> this
> >>>>>>>> should
> >>>>>>>>>> be
> >>>>>>>>>>>>> used
> >>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>> argument to block
the merge. I think the guys will
> >>>> keep
> >>>>>>>>> working
> >>>>>>>>>>>>> on
> >>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>> they may also get
help of others once the patch is in
> >>>>>>>> master.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I still have my
concerns about the current
> >>>> limitations,
> >>>>>>>> but
> >>>>>>>>>>>>> these are
> >>>>>>>>>>>>>>>>> things already planned
for phase 3, so some of this
> >>>>>>>> stuff may
> >>>>>>>>>>>>> even be
> >>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>> the final 2.0.
> >>>>>>>>>>>>>>>>> but as long as we
have a "current limitations"
> >>>> section
> >>>>>>>> in the
> >>>>>>>>>>>>> user
> >>>>>>>>>>>>>>> guide
> >>>>>>>>>>>>>>>>> mentioning important
stuff like the ones below, I'm
> >>>> ok
> >>>>>>>> with
> >>>>>>>>> it.
> >>>>>>>>>>>>>>>>> - if you write to
the table with
> >>>> Durability.SKIP_WALS
> >>>>>>>> your
> >>>>>>>>>> data
> >>>>>>>>>>>>> will
> >>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>> be in the incremental-backup
> >>>>>>>>>>>>>>>>> - if you bulkload
files that data will not be in the
> >>>>>>>>>> incremental
> >>>>>>>>>>>>>>> backup
> >>>>>>>>>>>>>>>>> (HBASE-14417)
> >>>>>>>>>>>>>>>>> - the incremental
backup will not only contains the
> >>>>>>>> data of
> >>>>>>>>>> the
> >>>>>>>>>>>>>> table
> >>>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>> specified but also
the regions from other tables that
> >>>>>>>> are on
> >>>>>>>>>> the
> >>>>>>>>>>>>> same
> >>>>>>>>>>>>>>> set
> >>>>>>>>>>>>>>>>> of RSs (HBASE-14141)
...maybe a note about security
> >>>>>>>> around
> >>>>>>>>> this
> >>>>>>>>>>>>> topic
> >>>>>>>>>>>>>>>>> - the incremental
backup will not contains just the
> >>>>>>>> "latest
> >>>>>>>>>> row"
> >>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>> backup A and B,
but it will also contains all the
> >>>>> updates
> >>>>>>>>>>>>> occurred in
> >>>>>>>>>>>>>>>>> between. but the
restore does not allow you to
> >>>> restore
> >>>>>>>> up to
> >>>>>>>>> a
> >>>>>>>>>>>>>> certain
> >>>>>>>>>>>>>>>>> point in time, the
restore will always be up to the
> >>>>>>>> "latest
> >>>>>>>>>>>>> backup
> >>>>>>>>>>>>>>>> point".
> >>>>>>>>>>>>>>>>> - you should limit
the number of "incremental" up
> >>>> to N
> >>>>>>>> (or
> >>>>>>>>>> maybe
> >>>>>>>>>>>>>>> SIZE),
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> avoid replay time
becoming the bottleneck.
> >>>>> (HBASE-14135)
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I'll be ok even
with the above not being in the final
> >>>>>>>> 2.0,
> >>>>>>>>>>>>>>>>> but i'd like to
see as blocker for the final 2.0 (not
> >>>>> the
> >>>>>>>>>> merge)
> >>>>>>>>>>>>>>>>> - the backup code
moved in an hbase-backup module
> >>>>>>>>>>>>>>>>> - and some more
work around tools, especially to try
> >>>>> to
> >>>>>>>>> unify
> >>>>>>>>>>>>> and
> >>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>> simple the backup
experience (simple example: in some
> >>>>>>>> case
> >>>>>>>>>> there
> >>>>>>>>>>>>> is a
> >>>>>>>>>>>>>>>>> backup_id argument
in others a backupId argument. or
> >>>>>>>> things
> >>>>>>>>>>>>> like..
> >>>>>>>>>>>>>>>> restore
> >>>>>>>>>>>>>>>>> is not clear if
given an incremental id it will do
> >>>> the
> >>>>>>>> full
> >>>>>>>>>>>>> restore
> >>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>> full up to that
point or if i need to apply manually
> >>>>>>>>>> everything).
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> in conclusion, I
think we can open a merge vote. I'll
> >>>>> be
> >>>>>>>> +1
> >>>>>>>>> on
> >>>>>>>>>>>>> it,
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>> think we should
try to reject -1 with just a "code
> >>>>>>>> cleanup"
> >>>>>>>>>>>>>> motivation,
> >>>>>>>>>>>>>>>>> since there will
still be work going on on the code
> >>>>>>>> after the
> >>>>>>>>>>>>> merge.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Matteo
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Sun, Nov 6, 2016
at 10:54 PM, Devaraj Das <
> >>>>>>>>>>>>> ddas@hortonworks.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Stack and others,
anything else on the patch? Merge
> >>>>> to
> >>>>>>>>> master
> >>>>>>>>>>>>> now?
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message