hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Rodionov <vladrodio...@gmail.com>
Subject Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
Date Tue, 14 Mar 2017 21:50:01 GMT
>> How would a user recover from such a state

User will need to run full backup for every table. No, we have not
encountered this issue during testing, but, of course, it is possible,
especially on a large cluster.

-Vlad

On Tue, Mar 14, 2017 at 2:36 PM, Josh Elser <elserj@apache.org> wrote:

> Thanks for quick reply, Vlad!
>
> How would a user recover from such a state with the backup table in a
> broken state? Have you encountered such a scenario in your testing?
>
> The docs issue seems to be pretty minor too. I remember walking through
> the original patch (HBASE-16574) and it was pretty thorough.
>
>
> Vladimir Rodionov wrote:
>
>> Josh,
>>
>> On a doc side we have a very good manual:
>>
>> https://issues.apache.org/jira/secure/attachment/12829269/
>> Backup-and-Restore-Apache_19Sep2016.pdf
>>
>> The only thing what was changed is the command-line tools args format, but
>> you can start from there - just type  command w/o args.
>>
>> In the current implementation the worst thing which could happen is a
>> failure during backup, which will leave backup system table in incomplete
>> state (read - broken). This won't affect cluster operations but may
>> prevent
>> further backup operations (will require full backup for all table to reset
>> broken system table).
>>
>> I am currently starting working on HBASE-15227. It will probably take a
>> week or two to finish.
>>
>> On a doc side, as I already mention we will need to update command-line
>> tool section in the doc I posted above.
>>
>>
>> -Vlad
>>
>>
>>
>>
>> On Tue, Mar 14, 2017 at 1:57 PM, Josh Elser<elserj@apache.org>  wrote:
>>
>> I took a moment to read through the "blockers" as originally identified by
>>> Vlad, and (to echo Enis' take) I read the majority of them as being
>>> blockers not for the next release, but for a "full-fledged feature". I'm
>>> going to intentional avoid addressing the discussion of shipping partial
>>> features (related, but not relevant at the moment).
>>>
>>> HBASE-15227 is actually the one that bothers me the most, with
>>> HBASE-17133
>>> coming in close behind. Vlad, is there any documentation you can point me
>>> to about what the current issues are with the current implementation? For
>>> example, what happens now if the system has some kind of "partial
>>> completion state"?
>>>
>>> Documentation is one of those that is really hard to judge. We want to
>>> get
>>> this code out for people to use (and to free up our strained dev
>>> resources), but what good is some feature if the docs are
>>> missing/incomplete?
>>>
>>> I think I could stomach the docs being inaccurate (with a clear
>>> disclaimer
>>> that the chapter is incomplete -- that's a 5min task). But, I think I
>>> need
>>> an answer about how the feature handles our common dist-sys category of
>>> problems before I can consider whether I'm ok with the feature hitting
>>> 2.0...
>>>
>>> I'll also try to throw up a few nodes and play with it to address the
>>> problem as an (ignorant) user ;)
>>>
>>>
>>> Andrew Purtell wrote:
>>>
>>> I don't like that issues were identified as "blockers" but now there is
>>>> an
>>>> attempt to walk that back.
>>>>
>>>> I don't like that development of this feature has lingered for a long
>>>> time
>>>> in this unfinished state when this work could have been done by now, now
>>>> that we are trying to get a 2.0 out the door. Because this is a
>>>> volunteer
>>>> project I cannot make any demand that it should be done, but I can
>>>> certainly look at the current state and be nonplussed. This will be yet
>>>> another half finished thing in 2.0 when this merge happens. Promises to
>>>> finish the unfinished work are nice but not currency. Commits are
>>>> currency.
>>>> I hope at least the fault tolerance changes can be completed and
>>>> committed
>>>> before we spin a 2.0 RC, and without causing a 2.0 release to slip
>>>> further.
>>>>
>>>> Also, marking something experimental should be done on the merits of
>>>> that
>>>> evaluation, not simply to justify dropping unfinished work into a
>>>> release
>>>> branch.
>>>>
>>>> I will change my vote to -0.
>>>>
>>>>
>>>> On Mon, Mar 13, 2017 at 4:05 PM, Enis Söztutar<enis@apache.org>
>>>>  wrote:
>>>>
>>>> I think there is some misconception of using the term "blockers" for
>>>>
>>>>> referring to those jiras. My understanding is that those three jiras
>>>>> are
>>>>> blockers for the backup functionality to be more mature and more
>>>>> usable.
>>>>> But they are not release blockers. Let's say we merged the code in, and
>>>>> for
>>>>> some reason those did not get addressed in time. We can still do the
>>>>> 2.0
>>>>> release without having to wait for the commits. We can instead mark the
>>>>> "backup" feature as experimental with known issues and go on with the
>>>>> release. In that sense they are not real release blockers.
>>>>>
>>>>> We are proposing the merge at this time because of the above that
>>>>> maintaining this in a branch is becoming extremely costly and not
>>>>> productive for the HBase community. Realistically, we cannot have the
>>>>> luxury of having to wait another couple of months and doing yet another
>>>>> giant round of reviews because the code base is a moving target.
>>>>>
>>>>> Enis
>>>>>
>>>>> On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das<ddas@hortonworks.com>
>>>>> wrote:
>>>>>
>>>>> Vlad, on the first point, I think what Stack is saying is that creating
>>>>>
>>>>>> the new branch (as Ted did) ignores the feedback incorporated thus
far
>>>>>> in
>>>>>> the iterations of the mega-patch. That's a wrong way to go.
>>>>>> On the separation into a backup module, again, that was reverted
to
>>>>>> ease
>>>>>> reviews of the mega-patch, and was noted as work to be done later.
I
>>>>>>
>>>>>> think
>>>>>
>>>>> Stack just wants to make the list of remaining work more complete by
>>>>>>
>>>>>> citing
>>>>>
>>>>> that as pending work.
>>>>>> ________________________________________
>>>>>> From: Vladimir Rodionov<vladrodionov@gmail.com>
>>>>>> Sent: Monday, March 13, 2017 3:09 PM
>>>>>> To: dev@hbase.apache.org
>>>>>> Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing
>>>>>> 3/11/2017
>>>>>>
>>>>>>    It ignores the feedback
>>>>>>
>>>>>>> If I "ignore" feedback, I put my comment - why? I am always 
open for
>>>>>>>
>>>>>> further discussions. If reviewer does not insist on a particular
>>>>>> request
>>>>>>
>>>>>> -
>>>>>
>>>>> it will be dropped. I think it is fair.
>>>>>>
>>>>>> he list is incomplete because a bunch of
>>>>>>
>>>>>>> follow-ons came of the review cycle (including moving backup/restore
>>>>>>>>
>>>>>>>> out
>>>>>>>
>>>>>> of
>>>>>>
>>>>>> core to live in its own module).
>>>>>>> For those who were not following our lengthy conversation on
a review
>>>>>>>
>>>>>> board, separation of a backup code into a separate module has been
>>>>>> done last year, but has been reverted back by request of a reviewer.
>>>>>>
>>>>>>
>>>>>> -Vladimir
>>>>>>
>>>>>> On Mon, Mar 13, 2017 at 2:23 PM, Stack<stack@duboce.net>  
wrote:
>>>>>>
>>>>>> On Fri, Mar 10, 2017 at 9:09 PM, Stack<stack@duboce.net>  
wrote:
>>>>>>
>>>>>>> On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu<yuzhihong@gmail.com>
>>>>>>>  wrote:
>>>>>>>
>>>>>>>> HBASE-14123 branch has been created, with Vlad's mega patch
v61.
>>>>>>>>
>>>>>>>>> The patch put up for VOTE here was done on a branch.
The call to
>>>>>>>>>
>>>>>>>> merge
>>>>>>>
>>>>>> seems to have been premature given the many cycles of review and
test
>>>>>>
>>>>>>> that
>>>>>>>
>>>>>>> happened subsequent (The cycles burned a bunch of dev resource).
>>>>>>>>
>>>>>>>> The patch as is is now in a state where it is too big for
our infra;
>>>>>>>>
>>>>>>>> rb
>>>>>>>
>>>>>> and JIRA are creaking under the size and # of iterations.
>>>>>>
>>>>>>> Adding finish of new JIRAs to this merge implies a new round
of
>>>>>>>>
>>>>>>>> review
>>>>>>>
>>>>>> and
>>>>>>
>>>>>>> test of an already massive patch. Who is going to do this work?
>>>>>>>>
>>>>>>>> Going back to a new branch seems wrong route to take.
>>>>>>>>
>>>>>>>> St.Ack
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> To be more explicit, this patch was developed on a branch
and then a
>>>>>>>>
>>>>>>> bunch
>>>>>>
>>>>>> of dev resources were burned getting it into a state where it could
be
>>>>>>> merged to master. Going back to a branch to bulk up the merge
so it
>>>>>>> includes more JIRAs than the many it already incorporates is
the
>>>>>>> wrong
>>>>>>> direction for us to be headed in. It ignores the feedback given
and
>>>>>>> the
>>>>>>> work done by Vladimir slimming down an already over-broad scope.
It
>>>>>>> is
>>>>>>>
>>>>>>> also
>>>>>>
>>>>>> predicated on abundant review and testing resource being on tap to
>>>>>>>
>>>>>>> cycle
>>>>>> on
>>>>>>
>>>>>> a feature that is useful, but non-core.
>>>>>>>
>>>>>>> The patch is ready for merge IMO. Geoffrey makes a nice list
of what
>>>>>>> is
>>>>>>> still to do though IIRC, the list is incomplete because a bunch
of
>>>>>>> follow-ons came of the review cycle (including moving backup/restore
>>>>>>>
>>>>>>> out
>>>>>> of
>>>>>>
>>>>>> core to live in its own module).
>>>>>>>
>>>>>>> The patch needs three votes to merge. I am not against merge
but I am
>>>>>>>
>>>>>>> not
>>>>>> voting for the patch because I do have any more time to spend on
this
>>>>>>
>>>>>>> non-core feature and feel that a vote will have me assume a
>>>>>>>
>>>>>>> responsibility
>>>>>>
>>>>>> I will not shirk.
>>>>>>>
>>>>>>> S
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> FYI
>>>>>>>>
>>>>>>>>> On Fri, Mar 10, 2017 at 3:30 PM, Ted Yu<yuzhihong@gmail.com>
>>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>
>>>>>>> Thanks for the feedback, Andrew.
>>>>>>
>>>>>>> How about the following plan:
>>>>>>>>>>
>>>>>>>>>> create branch HBASE-14123 off of master with mega
patch v61 as the
>>>>>>>>>>
>>>>>>>>>> first
>>>>>>>>>
>>>>>>>> commit (reviewed by Stack and Enis)
>>>>>>>>
>>>>>>>>> Vlad and I continue development (the 3 blockers) based
on
>>>>>>>>>>
>>>>>>>>>> HBASE-14123
>>>>>>>>>
>>>>>>>>    branch
>>>>>>>
>>>>>>>> when all of the blockers get +1 and merged into HBASE-14123
>>>>>>>>>>
>>>>>>>>>> branch,
>>>>>>>>>
>>>>>>>> we
>>>>>>
>>>>>> propose to community for merging into branch-2 (master branch, if
>>>>>>>
>>>>>>>> branch-2
>>>>>>>>>
>>>>>>>>> doesn't materialize for whatever reason) again
>>>>>>>>>>
>>>>>>>>>> Cheers
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Fri, Mar 10, 2017 at 3:01 PM, Andrew Purtell<
>>>>>>>>>>
>>>>>>>>>> apurtell@apache.org>
>>>>>>>>>
>>>>>>>> wrote:
>>>>>>>
>>>>>>>> Thanks for the offer but I like that you were honest about
>>>>>>>>>> compiling
>>>>>>>>>>
>>>>>>>>> a
>>>>>>>
>>>>>>> list
>>>>>>>>
>>>>>>>>> of issues that you thought were blockers for release.
Since this
>>>>>>>>>>>
>>>>>>>>>>> proposal
>>>>>>>>>> is a merge into 2.0, and we are trying to release
2.0, I am -1 on
>>>>>>>>>> this
>>>>>>>>>>
>>>>>>>>> merge until those blockers are addressed.
>>>>>>>>
>>>>>>>>> I had a look at the list.
>>>>>>>>>>>
>>>>>>>>>>> I think the documentation issue is important
but not actually a
>>>>>>>>>>>
>>>>>>>>>>> blocker.
>>>>>>>>>> That may be a controversial opinion, but documentation
can be
>>>>>>>>>> back-filled
>>>>>>>>>> worst case. So take HBASE-17133 off the list.
>>>>>>>>>>
>>>>>>>>>>> Remaining are effectively HBASE-14417, HBASE-14141,
and
>>>>>>>>>>>
>>>>>>>>>>> HBASE-15227.
>>>>>>>>>>
>>>>>>>>> They
>>>>>>>
>>>>>>>> all have patches attached to the respective JIRAs so completing
>>>>>>>>>> this
>>>>>>>>>>
>>>>>>>>> work
>>>>>>>
>>>>>>>> won't be onerous. Get these committed and I will lift my
-1. The
>>>>>>>>>> others
>>>>>>>>>>
>>>>>>>>> who
>>>>>>>>
>>>>>>>>> voted +1 on this thread surely can help with that.
>>>>>>>>>>>
>>>>>>>>>>> Thanks.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Mar 10, 2017 at 2:32 PM, Vladimir Rodionov<
>>>>>>>>>>> vladrodionov@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> No problem I will downgrade Blockers to Majors
if it scares
>>>>>>>>>>> you,
>>>>>>>>>>>
>>>>>>>>>> Andrew
>>>>>>
>>>>>>> 🙂
>>>>>>>>>>
>>>>>>>>>>> Sent from my iPhone
>>>>>>>>>>>>
>>>>>>>>>>>> On Mar 10, 2017, at 1:52 PM, Andrew Purtell<
>>>>>>>>>>>> apurtell@apache.org
>>>>>>>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>
>>>>>>>> ​I know the merge of this feature has lagged substantially.
I
>>>>>>>>>>>> think
>>>>>>>>>>>>
>>>>>>>>>>> that
>>>>>>>>
>>>>>>>>> is
>>>>>>>>>>>>
>>>>>>>>>>>> regrettable but on another thread we are
lamenting that 2.0
>>>>>>>>>>>>>
>>>>>>>>>>>>> is
>>>>>>>>>>>>
>>>>>>>>>>> already
>>>>>>
>>>>>>> late. Unless I misunderstand, this is a proposal to merge
>>>>>>>>>>
>>>>>>>>>>> something
>>>>>>>>>>>>
>>>>>>>>>>> with
>>>>>>>>
>>>>>>>>> known blockers into trunk before we branch it for 2.0
which
>>>>>>>>>>>> will
>>>>>>>>>>>>
>>>>>>>>>>> effectively prevent that release because these
blockers will
>>>>>>>
>>>>>>>> be
>>>>>>>>>>>>
>>>>>>>>>>> there. I
>>>>>>
>>>>>>> am
>>>>>>>>>>>>
>>>>>>>>>>>> inclined to veto. Probably we should not
propose branch
>>>>>>>>>>>>>
>>>>>>>>>>>>> merges
>>>>>>>>>>>>
>>>>>>>>>>> into
>>>>>>
>>>>>>> code
>>>>>>>>
>>>>>>>>> we
>>>>>>>>>>>>
>>>>>>>>>>>> are trying to get out the door with known
blockers. Why not
>>>>>>>>>>>>>
>>>>>>>>>>>>> do
>>>>>>>>>>>>
>>>>>>>>>>> that
>>>>>>
>>>>>>> work
>>>>>>>>
>>>>>>>>> first? It seems an obvious question. Perhaps I am missing
>>>>>>>>>>>> something.
>>>>>>>>>>>>
>>>>>>>>>>> If we can branch for 2.0 now and then merge this,
and not
>>>>>>>>>>
>>>>>>>>>>> into
>>>>>>>>>>>>
>>>>>>>>>>> the
>>>>>>
>>>>>>> 2.0
>>>>>>>>
>>>>>>>>> branch, I would vote +1 for branch merge even with known
>>>>>>>>>>
>>>>>>>>>>> blockers
>>>>>>>>>>>>
>>>>>>>>>>> pending.
>>>>>>>
>>>>>>>> ​
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Mar 10, 2017 at 1:42 PM, Vladimir
Rodionov<
>>>>>>>>>>>>>
>>>>>>>>>>>>> vladrodionov@gmail.com>
>>>>>>>>>>>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> They are not blockers for merge - only
for 2.0. GA
>>>>>>>>>>>>>
>>>>>>>>>>>>>> As I said already the feature is
usable right now
>>>>>>>>>>>>>> We would like to continue working
on master and we would
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> like
>>>>>>>>>>>>>
>>>>>>>>>>>> to
>>>>>>
>>>>>> see
>>>>>>>
>>>>>>>> a
>>>>>>>>>>
>>>>>>>>>>> commitment from community
>>>>>>>>>>>>
>>>>>>>>>>>>> Sent from my iPhone
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mar 10, 2017, at 11:16 AM, Andrew
Purtell<
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> apurtell@apache.org
>>>>>>>>>>>>>
>>>>>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Only BLOCKERs and CRITICALs are guaranteed for HBase
2.0
>>>>>>>>>>>>>
>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> If we have identified blockers, why
merge this before they
>>>>>>>>>>
>>>>>>>>>>> are
>>>>>>>>>>>>>>
>>>>>>>>>>>>> in?
>>>>>>>
>>>>>>>> Otherwise we can't release 2.0, and it is overdue.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Mar 8, 2017 at 1:32 PM,
Vladimir Rodionov<
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> vladrodionov@gmail.com>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hello, HBase folks
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> For your consideration today
is Backup/Restore feature for
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Apache
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> HBAse
>>>>>>>>>>
>>>>>>>>>>> 2.0.
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Backup code is available as a mega
patch in HBASE-14123
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> (v61),
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> applies
>>>>>>>
>>>>>>>> cleanly to the current master, all test PASS, patch has no
>>>>>>>>>>>>
>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> issues.
>>>>>>>>
>>>>>>>>> The patch has gone through numerous rounds of code reviews
>>>>>>>>>>>>>
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> has
>>>>>>>
>>>>>>>> probably
>>>>>>>>>>
>>>>>>>>>>> the most lengthy discussion thread on Apache
JIRA
>>>>>>>>>>>>>>> (HBASE-14123)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> :)
>>>>>>>>
>>>>>>>>> The work has been split into 3 phases (HBASE-14030, 14123,
>>>>>>>>>>
>>>>>>>>>>> 14414)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Two
>>>>>>>>>>
>>>>>>>>>>> first
>>>>>>>>>>>>
>>>>>>>>>>>>> are complete, third one is still in progress.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> *** Summary of work HBASE-14123
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The new feature introduces
new command-line extensions to
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> hbase
>>>>>>>
>>>>>>>> command
>>>>>>>>>>
>>>>>>>>>>> and, from the client side, is accessible through
>>>>>>>>>>>>>>> command-line
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> only
>>>>>>>
>>>>>>>> Operations:
>>>>>>>>>>
>>>>>>>>>>> * Create full backup on a list of tables or backup
set
>>>>>>>>>>>>>>>> * Create incremental backup
image for table list or backup
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> set
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> * Restore list of tables from a given
backup image
>>>>>>>
>>>>>>>> * Show current backup progress
>>>>>>>>>>>>>>>> * Delete backup image and
all related images
>>>>>>>>>>>>>>>> * Show history of backups
>>>>>>>>>>>>>>>> * Backup set operations:
create backup set, add/remove
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> table
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> to/from
>>>>>>
>>>>>>> backup
>>>>>>>>>>>>
>>>>>>>>>>>>> set, etc
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> In the current implementation,
the feature is already
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> usable,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> meaning
>>>>>>>
>>>>>>>> that
>>>>>>>>>>>>
>>>>>>>>>>>>> users can backup tables and restore them
using provided
>>>>>>>>>>>>>>> command-line
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> tools.
>>>>>>>>>>>>
>>>>>>>>>>>>> Both: full and incremental backups are
supported.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> This work is based on original
work of IBM team
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> (HBASE-7912).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The
>>>>>>>
>>>>>>>> full
>>>>>>>>>>
>>>>>>>>>>> list
>>>>>>>>>>>>
>>>>>>>>>>>>> of JIRAs included in this mega patch
can be found in three
>>>>>>>>>>>>>>> umbrella
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> JIRAs:
>>>>>>>>>>
>>>>>>>>>>> HBASE-14030 (Phase 1), HBASE-14123 (Phase 2)
and
>>>>>>>>>>>>>>> HBASE-14414
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> (Phase 3
>>>>>>
>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> all
>>>>>>>>>>>>>
>>>>>>>>>>>>>> resolved ones made it into the patch)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> *** What are the remaining
work items
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> All remaining items can be
found in Phase 3 umbrella JIRA:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> HBASE-14414.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> They are split into 3 groups: BLOCKER,
CRITICAL, MAJOR
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Only BLOCKERs and CRITICALs are guaranteed
for HBase 2.0
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ***** BLOCKER
>>>>>>>>>>
>>>>>>>>>>> * HBASE-14417 Incremental backup and bulk loading
( Patch
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> available)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> * HBASE-14135 HBase Backup/Restore
Phase 3: Merge backup
>>>>>>>>>>>>
>>>>>>>>>>>>> images
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> * HBASE-14141 HBase Backup/Restore
Phase 3: Filter WALs on
>>>>>>>>
>>>>>>>>> backup
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> to
>>>>>>>>>>
>>>>>>>>>>> include only edits from backup tables (Patch
available)
>>>>>>>>>>>>
>>>>>>>>>>>>> * HBASE-17133 Backup documentation
>>>>>>>>>>>>>>>> * HBASE-15227 Fault tolerance
support
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ***** CRITICAL
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> * HBASE-16465 Disable split/merges
during backup
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> We have umbrella JIRA (HBASE-14414)
to track all the
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> remaining
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> work
>>>>>>>
>>>>>>>> All the BLOCKER and CRITICAL JIRAs currently in open state
>>>>>>>>>>
>>>>>>>>>>> will
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> be
>>>>>>>>
>>>>>>>>> implemented by 2.0 release time. Some MAJOR too, but
it
>>>>>>>>>>
>>>>>>>>>>> depends
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> on
>>>>>>>>
>>>>>>>>> resource
>>>>>>>>>>
>>>>>>>>>>> availability
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The former development branch
(HBASE-7912) is obsolete and
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> be
>>>>>>>>
>>>>>>>>> closed/deleted after the merge.
>>>>>>>>>>
>>>>>>>>>>> We want backup to be a GA feature in 2.0
>>>>>>>>>>>>>>>> We are going to support full
backward compatibility for
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> backup
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> tool in
>>>>>>>
>>>>>>>> 2.0
>>>>>>>>>>>>
>>>>>>>>>>>>> and onwards.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> **** Configuration
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Backup is disabled, by default.
To enable it, the
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> following
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> configuration
>>>>>>
>>>>>>> properties must be added to hbase-site.xml:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> hbase.backup.enable=true
>>>>>>>>>>>>>>>> hbase.master.logcleaner.plugins=YOUR_PLUGINS,org.
>>>>>>>>>>>>>>>> apache.hadoop.hbase.backup.master.BackupLogCleaner
>>>>>>>>>>>>>>>> hbase.procedure.master.classes=YOUR_CLASSES,org.
>>>>>>>>>>>>>>>> apache.hadoop.hbase.backup.master.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> LogRollMasterProcedureManager
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> hbase.procedure.regionserver.classes=YOUR_CLASSES,org.
>>>>>>>>
>>>>>>>>> apache.hadoop.hbase.backup.regionserver.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> LogRollRegionServerProcedureMa
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> nager
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I would like to thank IBM
team and Jerry He for original
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> work,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Enis, Ted, Stack, Matteo, Jerry for
time spent on code
>>>>>>>
>>>>>>>> reviews
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Special thanks to Ted Yu for his
co-development work.
>>>>>>>
>>>>>>>> References:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-7912
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> (original
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> IBM,
>>>>>>
>>>>>>> contains
>>>>>>>>
>>>>>>>>> design doc)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-14030
(Phase
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 1)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-14123
(Phase
>>>>>>
>>>>>>> 2)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-14414
(Phase
>>>>>>
>>>>>>> 3)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Please  vote +1/-1 by midnight Pacific
Time (00:00
>>>>>>
>>>>>>> -0800 GMT) on March 11th  ​on whether or not we should
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> merge
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> this
>>>>>>
>>>>>>> into
>>>>>>>>>>
>>>>>>>>>>> the
>>>>>>>>>>>>
>>>>>>>>>>>>> current master.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> -Vladimir Rodionov
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>    - Andy
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If you are given a choice, you
believe you have acted
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> freely. -
>>>>>>>>>>>>>>
>>>>>>>>>>>>> Raymond
>>>>>>>
>>>>>>>> Teller (via Peter Watts)
>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>
>>>>>>>>>>>>>     - Andy
>>>>>>>>>>>>>
>>>>>>>>>>>>> If you are given a choice, you believe
you have acted
>>>>>>>>>>>>>
>>>>>>>>>>>>> freely. -
>>>>>>>>>>>>
>>>>>>>>>>> Raymond
>>>>>>
>>>>>>> Teller (via Peter Watts)
>>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Best regards,
>>>>>>>>>>>
>>>>>>>>>>>      - Andy
>>>>>>>>>>>
>>>>>>>>>>> If you are given a choice, you believe you have
acted freely. -
>>>>>>>>>>>
>>>>>>>>>>> Raymond
>>>>>>>>>>
>>>>>>>>> Teller (via Peter Watts)
>>>>>>>>
>>>>>>>>>
>>>>>>>>>>>
>>>>
>>>>
>>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message