Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 97045200D10 for ; Sun, 10 Sep 2017 04:03:05 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 9554C1609B6; Sun, 10 Sep 2017 02:03:05 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 64BC01609B5 for ; Sun, 10 Sep 2017 04:03:04 +0200 (CEST) Received: (qmail 17180 invoked by uid 500); 10 Sep 2017 02:03:03 -0000 Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hbase.apache.org Delivered-To: mailing list dev@hbase.apache.org Received: (qmail 17167 invoked by uid 99); 10 Sep 2017 02:03:02 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 10 Sep 2017 02:03:02 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 3396F192FEA for ; Sun, 10 Sep 2017 02:03:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.312 X-Spam-Level: X-Spam-Status: No, score=0.312 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, HTML_TAG_BALANCE_BODY=0.712, KAM_NUMSUBJECT=0.5, MIME_QP_LONG_LINE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-2.8, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id ClIvOypTF6WK for ; Sun, 10 Sep 2017 02:02:55 +0000 (UTC) Received: from mail-pg0-f47.google.com (mail-pg0-f47.google.com [74.125.83.47]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id DE2485F5A5 for ; Sun, 10 Sep 2017 02:02:53 +0000 (UTC) Received: by mail-pg0-f47.google.com with SMTP id t3so10181493pgt.0 for ; Sat, 09 Sep 2017 19:02:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:content-transfer-encoding:mime-version:date:subject:message-id :references:in-reply-to:to; bh=91PKERh3DhldTwguFAI2K7Gk9MyW4bRSAYYx5sUHFrY=; b=cJ2EVtrslUU62XLTnsNMkAliqnI4gYGR9V78aRc5DQDgOD8zYVzc5WssgX0bOS4VcW eVcwgCoi1yhMPImHtOch0vtOoRN7muNGl+Oc24RpbapdVU+TuHcPibiJwqGBGWchVawJ khy/tfv2wfUBakzvy41DVEUShMpklfxLkrsbE6dAa9jxlAKrya603JuFCewC+orAcIsQ 8bc2pxRIEQ8jQDkDXeyIln0ZylaV4g9Z9PbxejnGm6gNP03t0HXknmXshbO9B9w5Oa28 kfQLTHEUTxuy6fhFp0tbWpRF19B19js3AjUULGrkZ1rz+JSHcQfeM8CtE7e68iwn5sEH KWFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:content-transfer-encoding:mime-version:date :subject:message-id:references:in-reply-to:to; bh=91PKERh3DhldTwguFAI2K7Gk9MyW4bRSAYYx5sUHFrY=; b=UsvtGalP0PyK9WrRIW7nIwrjURtm12wJrVLksXt1tCdLG6dMXhr0om1s/+YqIn/c2U lA1Fp5lzKwWE8p4KLwDKgHWZEXRpR9KnX+znFXpPyUAGsWgAtb2mJct46fq3H61n01u7 vxcPcVEXHpED8Fro8pfp2gDJ3X9XzcNLqZcACxgKftqRshV4LJKktpgJl6AT3/wJhUL6 zpnrdxIrrbjWQtgNEWWwGM6ZhQ4hjZSyK8X5U0KNh9IDWJPCmmzWDiOouhs+WyaqLbtm T8xbvORAb/TR+y0/B3f43o9mn3vyJF0qqWwG4y1OFi4qzXv7imevqg5VUwW8/OQhAFaE QQNg== X-Gm-Message-State: AHPjjUgRSepOGQuqepOWxomIVnVYWhKO9ZgpysQPSwYh9xVXgIpoAkPT ZuI81vCHyBQMdNdq3Os= X-Google-Smtp-Source: ADKCNb5g/2qozSN+HXk0/7VSsQK/WXZGK9msIYVtOZ63GOBgF621YHkC9QWZa0FNiWGxC4INuKuNtA== X-Received: by 10.84.235.67 with SMTP id g3mr8316738plt.299.1505008972149; Sat, 09 Sep 2017 19:02:52 -0700 (PDT) Received: from ?IPv6:2605:e000:23d0:1000:a897:27c4:7b53:471e? ([2605:e000:23d0:1000:a897:27c4:7b53:471e]) by smtp.gmail.com with ESMTPSA id y4sm8330157pgs.19.2017.09.09.19.02.48 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 09 Sep 2017 19:02:51 -0700 (PDT) From: Andrew Purtell Content-Type: multipart/alternative; boundary=Apple-Mail-502C06C9-CFAD-41C2-AF80-97F41D4D6890 Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (1.0) Date: Sat, 9 Sep 2017 19:02:48 -0700 Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912 Message-Id: <137A2A01-6567-4759-914D-62AC91911028@gmail.com> References: <1479855885768.74262@hortonworks.com> <873DC21B-E957-472F-9379-D2E5425D5579@gmail.com> In-Reply-To: To: dev@hbase.apache.org X-Mailer: iPhone Mail (14G60) archived-at: Sun, 10 Sep 2017 02:03:05 -0000 --Apple-Mail-502C06C9-CFAD-41C2-AF80-97F41D4D6890 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable To your question, I think it is misplaced (at least to me). Go back and read= what I wrote you just today.=20 As for the backup feature, once the work for HBASE-15227 is committed and it= is resolved, I think we'd be interested in trying it out, and until then I a= m not personally interested in its inclusion in a release, but that is just= me.=20 > On Sep 9, 2017, at 5:58 PM, Vladimir Rodionov wro= te: >=20 > That was I thought. Thanks. Can you tell me that you are not considering A= M > v2 as an unfinished and untested feature? The question to Stack as well. >=20 > On Sat, Sep 9, 2017 at 5:53 PM, Andrew Purtell > wrote: >=20 >> No all I have to do is pay attention to words you have written yourself i= n >> emails and on JIRA. Don't argue with us not to believe our lying eyes, >> consider finishing the work. I'll be happy to try it out when you indicat= e >> it can work if anything happens to fail on the cluster at the time. Until= >> then there are a lot of other things need doing first. >>=20 >>=20 >> On Sep 9, 2017, at 5:25 PM, Vladimir Rodionov >> wrote: >>=20 >>>>> but the impression we have is it is unfinished and untested. >>> To make a conclusion that "feature is not finished and tested" you have= >>> had to test it at least. >>> Andrew, If you have discovered issues, why wouldn't you open bug JIRAs? >>>=20 >>> -Vlad >>>=20 >>> On Sat, Sep 9, 2017 at 4:40 PM, Andrew Purtell >>=20 >>> wrote: >>>=20 >>>> For what it's worth, I think AMv2 is the main reason to have a 2.0 in >> the >>>> first place, so I would both agree it needs a lot more testing and yet I= >>>> would want us to have a 2.0 release as the vehicle for getting that to >>>> happen. For other features without testing from a number of parties or >> at >>>> scale the value proposition is less clear and it's fine by me for the >> RM to >>>> set them aside for future releases. >>>>=20 >>>> Also, I can relay that there is some interest where I work in utilizing= >>>> HBASE-7912 but the impression we have is it is unfinished and untested.= >> So >>>> for now we are ignoring it and continuing with home grown solutions. >> Part >>>> of the problem is fault tolerance was left to the last phase(s) and yet= >> it >>>> is an essential property for adoption for serious work. The best way to= >>>> resolve this IMHO is for the developers of this feature to complete >> those >>>> unfinished JIRAs, especially concerning resilience to failures. >>>>=20 >>>>=20 >>>>> On Sep 9, 2017, at 4:11 PM, Vladimir Rodionov = >>>> wrote: >>>>>=20 >>>>> Hmm, the next on your list (of kicked out from branch v2) should be AM= >>>> v2 I >>>>> presume? >>>>>=20 >>>>> -Vlad >>>>>=20 >>>>>> On Sat, Sep 9, 2017 at 4:04 PM, stack wrote: >>>>>>=20 >>>>>> In spite of repeated requests for eng summary of state of this featur= e >>>> -- >>>>>> summary of what is in 2.0, what is not, what the capabilities are, ho= w >>>> well >>>>>> it has been tested and at what scale -- all I get, when the requests >> are >>>>>> not ignored, are pointers to lists of ill-describing jiras and some >>>> pending >>>>>> user facing doc update. >>>>>>=20 >>>>>> For other features, mob or region server groups, I know that they hav= e >>>> been >>>>>> running at scale in production for as much as a year and more. I have= >>>> some >>>>>> confidence these items basically work. For backup/restore I have no >>>> such >>>>>> sense even after spending time in review and trying to use the >> feature. >>>>>>=20 >>>>>> As release manager, I have say over what makes it into a release. >>>> Unless >>>>>> the work is done to convince me that backup/restore is more than a >> lump >>>> of >>>>>> code and a few unit tests that can pass on some fellows laptop, I am >>>> going >>>>>> to kick it out of branch-2. Let the feature harden more in master >>>> branch >>>>>> before it ships in a release. >>>>>>=20 >>>>>> S >>>>>>=20 >>>>>> On Sep 8, 2017 10:59 PM, "Vladimir Rodionov" = >>>>>> wrote: >>>>>>=20 >>>>>>>>> Have I grasped the state of things correctly, Vlad? >>>>>>>=20 >>>>>>> Josh, the only thing which is still pending is doc update. All other= >>>>>>> features are good to have but not a blockers for 2.0 release. >>>>>>>=20 >>>>>>> -Vlad >>>>>>>=20 >>>>>>> On Fri, Sep 8, 2017 at 10:42 PM, Vladimir Rodionov < >>>>>> vladrodionov@gmail.com >>>>>>>>=20 >>>>>>> wrote: >>>>>>>=20 >>>>>>>>>> What testing and at what >>>>>>>>>> scale has testing been done? >>>>>>>>=20 >>>>>>>> Do we have have that for other features? >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> On Fri, Sep 8, 2017 at 10:41 PM, Vladimir Rodionov < >>>>>>> vladrodionov@gmail.com >>>>>>>>> wrote: >>>>>>>>=20 >>>>>>>>>>> It asks: "How do I figure what of backup/restore feature is goin= g >>>>>> to >>>>>>>>> be in >>>>>>>>>>> hbase-2.0.0? >>>>>>>>>=20 >>>>>>>>> Hmm, wait for doc update. >>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>>> On Fri, Sep 8, 2017 at 2:39 PM, Stack wrote: >>>>>>>>>>=20 >>>>>>>>>> HBASE-14414 is a JIRA with a list of random seeming issues w/ >>>>>>>>>> non-descript >>>>>>>>>> summaries: "Add nonce support to TableBackupProcedure, BackupID >> must >>>>>>>>>> include backup set name, ...". The last comment in that issue is >>>> from >>>>>>>>>> July. >>>>>>>>>> It asks: "How do I figure what of backup/restore feature is going= >> to >>>>>> be >>>>>>>>>> in >>>>>>>>>> hbase-2.0.0? Thanks Vladimir Rodionov >>>>>>>>>> >>>>> name=3Dvrodionov >>>>>>>>>>> ." >>>>>>>>>> to which there is no answer. Doc update is TODO. >>>>>>>>>>=20 >>>>>>>>>> Where is the summary of the capability in hbase-2? What testing >> and >>>>>> at >>>>>>>>>> what >>>>>>>>>> scale has testing been done? Is this 'stable or experimental'? If= >> I >>>>>>> can't >>>>>>>>>> get basic info on this feature though I ask repeatedly, what hope= >>>>>> does >>>>>>>>>> the >>>>>>>>>> poor old operator have? >>>>>>>>>>=20 >>>>>>>>>> St.Ack >>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>> On Fri, Sep 8, 2017 at 1:59 PM, Vladimir Rodionov < >>>>>>>>>> vladrodionov@gmail.com> >>>>>>>>>> wrote: >>>>>>>>>>=20 >>>>>>>>>>> HBASE-14414 >>>>>>>>>>>=20 >>>>>>>>>>>> On Fri, Sep 8, 2017 at 1:14 PM, Stack wrote:= >>>>>>>>>>>>=20 >>>>>>>>>>>> Where do I go to get the current status of this feature? Lookin= g >>>>>> in >>>>>>>>>> JIRA >>>>>>>>>>> I >>>>>>>>>>>> see loads of issues open against backup including some against >>>>>>>>>>> hbase-2.0.0 >>>>>>>>>>>> and no progress being made that I can discern. >>>>>>>>>>>>=20 >>>>>>>>>>>> Thanks, >>>>>>>>>>>> S >>>>>>>>>>>>=20 >>>>>>>>>>>>=20 >>>>>>>>>>>>=20 >>>>>>>>>>>>> On Wed, Nov 23, 2016 at 8:52 AM, Stack >> wrote: >>>>>>>>>>>>>=20 >>>>>>>>>>>>> On Tue, Nov 22, 2016 at 6:48 PM, Stack >>>>>> wrote: >>>>>>>>>>>>>=20 >>>>>>>>>>>>>> On Tue, Nov 22, 2016 at 3:17 PM, Vladimir Rodionov < >>>>>>>>>>>>>> vladrodionov@gmail.com> wrote: >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> and/or he answered most of the review feedback >>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>> No, questions are still open, but I do not see any blockers >>>>>> and >>>>>>>>>> we >>>>>>>>>>> have >>>>>>>>>>>>>>> HBASE-16940 to address these questions. >>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> Agree. No blockers but stuff that should be dealt with (No on= e >>>>>>>>>> will >>>>>>>>>>> pay >>>>>>>>>>>>>> me any attention once merge goes in -- smile). >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>=20 >>>>>>>>>>>>> Let me clarify the above. I want review addressed before merge= >>>>>>>>>> happens. >>>>>>>>>>>>> Sorry if any confusion. >>>>>>>>>>>>> St.Ack >>>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>>> St.Ack >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>> On Tue, Nov 22, 2016 at 3:04 PM, Devaraj Das < >>>>>>>>>> ddas@hortonworks.com> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>> Hi Stack, hats off to you for spending so much time on >>>>>> this! >>>>>>>>>>> Thanks! >>>>>>>>>>>>>>> From >>>>>>>>>>>>>>>> my understanding, Vlad has raised follow-up jiras for the >>>>>>>>>> issues >>>>>>>>>>> you >>>>>>>>>>>>>>>> raised, and/or he answered most of the review feedback. So,= >>>>>>> do >>>>>>>>>> you >>>>>>>>>>>>>>> think we >>>>>>>>>>>>>>>> could do a merge vote now? >>>>>>>>>>>>>>>> Devaraj. >>>>>>>>>>>>>>>> ________________________________________ >>>>>>>>>>>>>>>> From: Vladimir Rodionov >>>>>>>>>>>>>>>> Sent: Monday, November 21, 2016 8:34 PM >>>>>>>>>>>>>>>> To: dev@hbase.apache.org >>>>>>>>>>>>>>>> Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch >>>>>>>>>>> HBASE-7912 >>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>>> I have spent a good bit of time reviewing and testing >>>>>> this >>>>>>>>>>>> feature. >>>>>>>>>>>>>>> I >>>>>>>>>>>>>>>> would >>>>>>>>>>>>>>>>>> like my review and concerns addressed and I'd like it to >>>>>>> be >>>>>>>>>>> clear >>>>>>>>>>>>>>> how; >>>>>>>>>>>>>>>>>> either explicit follow-on issues, pointers to where in >>>>>> the >>>>>>>>>> patch >>>>>>>>>>>> or >>>>>>>>>>>>>>> doc >>>>>>>>>>>>>>>> my >>>>>>>>>>>>>>>>>> remarks have been catered to, etc. Until then, I am >>>>>>> against >>>>>>>>>>>> commit. >>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>> Stack, mega patch review comments will be addressed in the >>>>>>>>>>> dedicated >>>>>>>>>>>>>>> JIRA: >>>>>>>>>>>>>>>> HBASE-16940 >>>>>>>>>>>>>>>> I have open several other JIRAs to address your other >>>>>>> comments >>>>>>>>>> (not >>>>>>>>>>>> on >>>>>>>>>>>>>>>> review board). >>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>> Details are here (end of the thread): >>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-14123 >>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>> Let me know what else should we do to move merge forward. >>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>> -Vlad >>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>> On Fri, Nov 18, 2016 at 4:54 PM, Stack >>>>>>>>>> wrote: >>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> On Fri, Nov 18, 2016 at 3:53 PM, Ted Yu < >>>>>>> yuzhihong@gmail.com >>>>>>>>>>>=20 >>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>>> Thanks, Matteo. >>>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>>> bq. restore is not clear if given an incremental id it >>>>>>>>>> will do >>>>>>>>>>>> the >>>>>>>>>>>>>>> full >>>>>>>>>>>>>>>>>> restore from full up to that point or if i need to >>>>>> apply >>>>>>>>>>> manually >>>>>>>>>>>>>>>>>> everything >>>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>>> The restore takes into consideration of the dependent >>>>>>>>>>> backup(s). >>>>>>>>>>>>>>>>>> So there is no need to apply preceding backup(s) >>>>>>> manually. >>>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> I ask this question on the issue. It is not clear from >>>>>> the >>>>>>>>>> usage >>>>>>>>>>> or >>>>>>>>>>>>>>> doc >>>>>>>>>>>>>>>> how >>>>>>>>>>>>>>>>> to run a restore from incremental. Can you fix in doc and >>>>>>>>>> usage >>>>>>>>>>> how >>>>>>>>>>>>>>> so I >>>>>>>>>>>>>>>>> can be clear and try it. Currently I am stuck verifying a >>>>>>>>>> round >>>>>>>>>>>> trip >>>>>>>>>>>>>>>> backup >>>>>>>>>>>>>>>>> restore made of incrementals. >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> S >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>>> On Fri, Nov 18, 2016 at 3:48 PM, Matteo Bertozzi < >>>>>>>>>>>>>>>>> theo.bertozzi@gmail.com> >>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>>>> I did one last pass to the mega patch. I don't see >>>>>>>>>> anything >>>>>>>>>>>> major >>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>>> should block the merge. >>>>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>>>> - most of the code is isolated in the backup package >>>>>>>>>>>>>>>>>>> - all the backup code is client side >>>>>>>>>>>>>>>>>>> - there are few changes to the server side, mainly >>>>>> for >>>>>>>>>>>> cleaners, >>>>>>>>>>>>>>> wal >>>>>>>>>>>>>>>>>>> rolling and similar (which is ok) >>>>>>>>>>>>>>>>>>> - there is a good number of tests, and an integration >>>>>>>>>> test >>>>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>>>> the code seems to have still some left overs from the >>>>>>> old >>>>>>>>>>>>>>>>> implementation, >>>>>>>>>>>>>>>>>>> and some stuff needs a cleanup. but I don't think >>>>>> this >>>>>>>>>> should >>>>>>>>>>>> be >>>>>>>>>>>>>>> used >>>>>>>>>>>>>>>>> as >>>>>>>>>>>>>>>>>> an >>>>>>>>>>>>>>>>>>> argument to block the merge. I think the guys will >>>>>> keep >>>>>>>>>>> working >>>>>>>>>>>>>>> on >>>>>>>>>>>>>>>> this >>>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>> they may also get help of others once the patch is in >>>>>>>>>> master. >>>>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>>>> I still have my concerns about the current >>>>>> limitations, >>>>>>>>>> but >>>>>>>>>>>>>>> these are >>>>>>>>>>>>>>>>>>> things already planned for phase 3, so some of this >>>>>>>>>> stuff may >>>>>>>>>>>>>>> even be >>>>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>>> the final 2.0. >>>>>>>>>>>>>>>>>>> but as long as we have a "current limitations" >>>>>> section >>>>>>>>>> in the >>>>>>>>>>>>>>> user >>>>>>>>>>>>>>>>> guide >>>>>>>>>>>>>>>>>>> mentioning important stuff like the ones below, I'm >>>>>> ok >>>>>>>>>> with >>>>>>>>>>> it. >>>>>>>>>>>>>>>>>>> - if you write to the table with >>>>>> Durability.SKIP_WALS >>>>>>>>>> your >>>>>>>>>>>> data >>>>>>>>>>>>>>> will >>>>>>>>>>>>>>>>> not >>>>>>>>>>>>>>>>>>> be in the incremental-backup >>>>>>>>>>>>>>>>>>> - if you bulkload files that data will not be in the >>>>>>>>>>>> incremental >>>>>>>>>>>>>>>>> backup >>>>>>>>>>>>>>>>>>> (HBASE-14417) >>>>>>>>>>>>>>>>>>> - the incremental backup will not only contains the >>>>>>>>>> data of >>>>>>>>>>>> the >>>>>>>>>>>>>>>> table >>>>>>>>>>>>>>>>>> you >>>>>>>>>>>>>>>>>>> specified but also the regions from other tables that >>>>>>>>>> are on >>>>>>>>>>>> the >>>>>>>>>>>>>>> same >>>>>>>>>>>>>>>>> set >>>>>>>>>>>>>>>>>>> of RSs (HBASE-14141) ...maybe a note about security >>>>>>>>>> around >>>>>>>>>>> this >>>>>>>>>>>>>>> topic >>>>>>>>>>>>>>>>>>> - the incremental backup will not contains just the >>>>>>>>>> "latest >>>>>>>>>>>> row" >>>>>>>>>>>>>>>>> between >>>>>>>>>>>>>>>>>>> backup A and B, but it will also contains all the >>>>>>> updates >>>>>>>>>>>>>>> occurred in >>>>>>>>>>>>>>>>>>> between. but the restore does not allow you to >>>>>> restore >>>>>>>>>> up to >>>>>>>>>>> a >>>>>>>>>>>>>>>> certain >>>>>>>>>>>>>>>>>>> point in time, the restore will always be up to the >>>>>>>>>> "latest >>>>>>>>>>>>>>> backup >>>>>>>>>>>>>>>>>> point". >>>>>>>>>>>>>>>>>>> - you should limit the number of "incremental" up >>>>>> to N >>>>>>>>>> (or >>>>>>>>>>>> maybe >>>>>>>>>>>>>>>>> SIZE), >>>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>> avoid replay time becoming the bottleneck. >>>>>>> (HBASE-14135) >>>>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>>>> I'll be ok even with the above not being in the final >>>>>>>>>> 2.0, >>>>>>>>>>>>>>>>>>> but i'd like to see as blocker for the final 2.0 (not >>>>>>> the >>>>>>>>>>>> merge) >>>>>>>>>>>>>>>>>>> - the backup code moved in an hbase-backup module >>>>>>>>>>>>>>>>>>> - and some more work around tools, especially to try >>>>>>> to >>>>>>>>>>> unify >>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>> make >>>>>>>>>>>>>>>>>>> simple the backup experience (simple example: in some >>>>>>>>>> case >>>>>>>>>>>> there >>>>>>>>>>>>>>> is a >>>>>>>>>>>>>>>>>>> backup_id argument in others a backupId argument. or >>>>>>>>>> things >>>>>>>>>>>>>>> like.. >>>>>>>>>>>>>>>>>> restore >>>>>>>>>>>>>>>>>>> is not clear if given an incremental id it will do >>>>>> the >>>>>>>>>> full >>>>>>>>>>>>>>> restore >>>>>>>>>>>>>>>>> from >>>>>>>>>>>>>>>>>>> full up to that point or if i need to apply manually >>>>>>>>>>>> everything). >>>>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>>>> in conclusion, I think we can open a merge vote. I'll >>>>>>> be >>>>>>>>>> +1 >>>>>>>>>>> on >>>>>>>>>>>>>>> it, >>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>> I >>>>>>>>>>>>>>>>>>> think we should try to reject -1 with just a "code >>>>>>>>>> cleanup" >>>>>>>>>>>>>>>> motivation, >>>>>>>>>>>>>>>>>>> since there will still be work going on on the code >>>>>>>>>> after the >>>>>>>>>>>>>>> merge. >>>>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>>>> Matteo >>>>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>>>> On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das < >>>>>>>>>>>>>>> ddas@hortonworks.com> >>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>>>>> Stack and others, anything else on the patch? Merge >>>>>>> to >>>>>>>>>>> master >>>>>>>>>>>>>>> now? >>>>>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>=20 >>>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>=20 >>>>>>>=20 >>>>>>=20 >>>>=20 >>=20 --Apple-Mail-502C06C9-CFAD-41C2-AF80-97F41D4D6890--