Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of jxiang@cloudera.com
 designates 209.85.216.41 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAHxLZBWnQFUfynponVrcTraYPF-gstQ0b4-RoUEFJcR6mj-7gA@mail.gmail.com>
References: 
 <CAMUu0w8uvb29+QYpZ8VwSO4qEpKRyKGNPzyj1kw7tLRhLPoZkA@mail.gmail.com>
	<CA+RK=_CWMv-kfw2jxM3BB-X2dsa_P8DqA6kB9ei_V0m4TQFOaA@mail.gmail.com>
	<CAAha9a2-79m1_r0XW7UOzxs6kUR+wftjA547u10u0Cz2EyMMCg@mail.gmail.com>
	<CAMUu0w8+Zagj1tqamjovj4mNQwe6Je_rMTm2XgC=AvazJH8E3w@mail.gmail.com>
	<CAAha9a09OmkgSu3LQPKC9GjyhUMytg4EGXuKvYgdNCMBOKao6A@mail.gmail.com>
	<CAMUu0w9w0NvRFUg2=_ftfunJmj8eC5Ho+JRHa2Cugq0+17XDLw@mail.gmail.com>
	<CAAha9a2h0bz2scdM+_d3rGyQtY6pqC8GsYquhXW0OBZ500RRzw@mail.gmail.com>
	<CAHxLZBWnQFUfynponVrcTraYPF-gstQ0b4-RoUEFJcR6mj-7gA@mail.gmail.com>
Date: Thu, 12 Jun 2014 14:18:07 -0700
Message-ID: 
 <CAOmV22vKVcxySn04zFRVbe3K+w53modz4Ue_X1H-X1VSRs4fAQ@mail.gmail.com>
Subject: Re: [VOTE] Merge branch HBASE-10070 to trunk
From: Jimmy Xiang <jxiang@cloudera.com>
To: dev@hbase.apache.org
Content-Type: multipart/alternative; boundary=001a1132f3ace5bfb404fbaa180c

--001a1132f3ace5bfb404fbaa180c
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

zk-less assignment will be off by default.

co-location of meta and master is on by default now. We can turn it off if
needed.

Thanks,
Jimmy


On Thu, Jun 12, 2014 at 2:12 PM, Mikhail Antonov <olorinbant@gmail.com>
wrote:

> So how would the defaults and dependencies look amongst these 3, once
> committed to 0.99?
>
>  - co-location of meta and master
>  - zk-less assignment
> - timeline-consistent replicas?
>
> As I understand -
>  -  All 3 are off by default
>  -  zk-less assignments and region replicas to not be turned on at the sa=
me
> time
>  -  zk-less assignments require meta co-location
>  -  region replicas don't care about meta co-location?
>
> -Mikhail
>
>
> 2014-06-12 14:07 GMT-07:00 Jonathan Hsieh <jon@cloudera.com>:
>
> > sgtm.
> >
> >
> > On Mon, Jun 9, 2014 at 10:46 AM, Enis S=C3=B6ztutar <enis.soz@gmail.com=
>
> wrote:
> >
> > > Thanks Jon.
> > >
> > > I think we'll try the rebase approach first. I'll start the effort
> today
> > > and see how far along I can get with that. Rebasing each patch might
> be a
> > > bit more work actually. If this turns out painful, we'll revert to th=
e
> > > merge approach similar to what you describe.
> > >
> > > Enis
> > >
> > >
> > > On Sat, Jun 7, 2014 at 7:41 AM, Jonathan Hsieh <jon@cloudera.com>
> wrote:
> > >
> > > > On Fri, Jun 6, 2014 at 5:05 PM, Enis S=C3=B6ztutar <enis.soz@gmail.=
com>
> > > wrote:
> > > >
> > > > > My preference would be to do the rebase-then-merge style (last
> style
> > in
> > > > > your comment). For each patch, I am hoping for all the changes
> > between
> > > > > committed version and rebased version to be mechanical. I like
> > having a
> > > > > linear history with explicit commit-to-patch mapping.
> > > > >
> > > > > If the changes are of a mechanical nature and that each patch
> > compiles
> > > > and
> > > > passes unit tests along the way, then rebase-then-merge is perfectl=
y
> > fine
> > > > by me.
> > > >
> > > > Even though snapshots were fairly orthogonal to the rest of the
> > codebase,
> > > > when we did merges we had problems maintaining the compiles and
> passes
> > > with
> > > > every commit invariants when we tried rebase-then-merge approach.
> >  After
> > > > each rebase we would end up doing a bunch of work and still end up
> > > failing
> > > > unit tests.   In that case we (jesse, matteo, myself) went to the
> merge
> > > > approach.   We actually merged into the snapshot branch a few times
> > > fixing
> > > > things due to changes in trunk that broke parts of the snapshots
> > branch.
> > > > Here's a more accurate picture of what the commit history looked li=
ke
> > in
> > > > git (we dev'ed in git and in end had to recommit this all in svn).
> > > >
> > > > m  (essentially empty merge patch).
> > > > |\
> > > > t |  (trunk patch with no impact)
> > > > | s6* (patch to fix newly introduced problem)
> > > > |/|
> > > > t |
> > > > t*|   (trunk patch that broke snapshots again)
> > > > | s5* (branch bug fixes due to merge)
> > > > | s4* (mechanical fixups due to the merge)
> > > > |/|
> > > > t |
> > > > t |
> > > > t |
> > > > | s3
> > > > | s2
> > > > | s1
> > > > |/
> > > > t
> > > > t
> > > >
> > > > This approach was captured in github and had the added benefit of
> > > > preserving exact history, and having known good points preserving t=
he
> > > > compile/unit test invariants on trunk.  (unfortunately, some of thi=
s
> > > > history was lost when we ported the git history over to svn, but th=
at
> > > isn't
> > > > a problem any more).    If you want to see the whole thing, look at
> my
> > > git
> > > > repo:
> > > >
> > > > git remote add jmhsieh git@github.com:jmhsieh/hbase.git
> > > > git log --oneline --graph --color jmhsieh/hbase-7290
> > > >
> > > >
> > > > Can we do history-preserving merge with the first style? I can do t=
he
> > > > > rebases and upload interdiff if that is big so that we can compar=
e
> > on a
> > > > > per-patch basis.
> > > > >
> > > > >
> > > > It is possible.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > > Enis
> > > > >
> > > > >
> > > > > On Fri, Jun 6, 2014 at 4:36 PM, Jonathan Hsieh <jon@cloudera.com>
> > > wrote:
> > > > >
> > > > > > When we merged snapshots branch in we did this:
> > > > > >
> > > > > > t =3D trunk commit
> > > > > > s =3D snapshot branch commit
> > > > > > m =3D merge point.
> > > > > >
> > > > > > During work:
> > > > > > t
> > > > > > t
> > > > > > t
> > > > > > |  s3
> > > > > > |  s2
> > > > > > |  s1
> > > > > > |/
> > > > > > t
> > > > > > t
> > > > > >
> > > > > > During after merge:
> > > > > > m  (essentially empty merge patch).
> > > > > > t \
> > > > > > t |
> > > > > > t |
> > > > > > | s4* (fixups due to the merge)
> > > > > > | s3
> > > > > > | s2
> > > > > > | s1
> > > > > > |/
> > > > > > t
> > > > > > t
> > > > > >
> > > > > > Does your proposal mean you are going to do this instead?
> > > > > >
> > > > > > s3* (modified due to rebase)
> > > > > > s2* (modified due to rebase)
> > > > > > s1
> > > > > > t
> > > > > > t
> > > > > > t
> > > > > > | s (now dead hbase-10070 branch)
> > > > > > | s
> > > > > > | s
> > > > > > | s
> > > > > > |/
> > > > > > t
> > > > > > t
> > > > > >
> > > > > > If it is the then we should probably have a review of the
> modified
> > > > > patches.
> > > > > >  If it the same as the snapshot merge, then we just need a revi=
ew
> > of
> > > > the
> > > > > > merge point delta (as well as a full review).
> > > > > >
> > > > > > Personally I prefer the merge for these large feature branches =
--
> > it
> > > > > > guarantees that each commit is compilable, and reflects what yo=
u
> > guys
> > > > > have
> > > > > > been testing for a while.  If you go with the last approach you
> > might
> > > > > have
> > > > > > stuff broken, and in the mainline commit path.
> > > > > >
> > > > > > Jon.
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Tue, Jun 3, 2014 at 11:16 AM, Andrew Purtell <
> > apurtell@apache.org
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > >  =E2=80=8B
> > > > > > > On Mon, Jun 2, 2014 at 2:24 PM, Enis S=C3=B6ztutar <enis@apac=
he.org
> >
> > > > wrote:
> > > > > > >
> > > > > > > > This VOTE is for merging back the remaining changes in bran=
ch
> > to
> > > > > trunk.
> > > > > > > If
> > > > > > > > passes, we will rebase the branch on top of current trunk, =
in
> > > which
> > > > > we
> > > > > > > will
> > > > > > > > keep the commit-per-issue log history. After that we will d=
o
> a
> > > git
> > > > > > merge
> > > > > > > > for the branch keeping the history clean and not squashing
> the
> > > > > > commits. I
> > > > > > > > expect rebasing to be straightforward, however with some
> manual
> > > > > > conflict
> > > > > > > > resolution. After the merge we'll keep running the tests to
> > make
> > > > sure
> > > > > > > > everything is ok.
> > > > > > > >
> > > > > > >
> > > > > > > =E2=80=8BJust to clarify that would look something like this:
> > > > > > >
> > > > > > >      $ git checkout HBASE-10070
> > > > > > >      $ git rebase --ignore-date master
> > > > > > >      (fixups, git add, git rebase --continue, etc, etc, etc)
> > > > > > >      $ git checkout master
> > > > > > >      $ git merge HBASE-10070
> > > > > > >
> > > > > > > ?
> > > > > > >
> > > > > > > That sounds good to me, the final merge should be a fast
> forward
> > > > merge.
> > > > > > >
> > > > > > > Use of ' --ignore-date' could be mildly controversial. It's n=
ot
> > > > > strictly
> > > > > > > necessary because the commits for 10070 will appear grouped i=
n
> > > > history,
> > > > > > but
> > > > > > > then dates on commits will be discontiguous in that section o=
f
> > > > > history. I
> > > > > > > suggest using that option so the order of commits and dates
> sort
> > > the
> > > > > same
> > > > > > > on master.
> > > > > > >
> > > > > > >
> > > > > > > On Mon, Jun 2, 2014 at 2:24 PM, Enis S=C3=B6ztutar <enis@apac=
he.org
> >
> > > > wrote:
> > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > Last week we started some discussion[4] for merging branch
> > > > > > hbase-10070[1]
> > > > > > > > into trunk. It seems like the consensus there is to do the
> > merge
> > > > > sooner
> > > > > > > > rather than later.
> > > > > > > >
> > > > > > > >
> > > > > > > > We had branched hbase-10070 in Feb out of trunk[5]. The
> branch
> > > > > contains
> > > > > > > 55
> > > > > > > > jiras committed[2]. Out of these 55, 15 has already been
> > > committed
> > > > to
> > > > > > > trunk
> > > > > > > > and backported to hbase-10070 branch[3].
> > > > > > > >
> > > > > > > > This VOTE is for merging back the remaining changes in bran=
ch
> > to
> > > > > trunk.
> > > > > > > If
> > > > > > > > passes, we will rebase the branch on top of current trunk, =
in
> > > which
> > > > > we
> > > > > > > will
> > > > > > > > keep the commit-per-issue log history. After that we will d=
o
> a
> > > git
> > > > > > merge
> > > > > > > > for the branch keeping the history clean and not squashing
> the
> > > > > > commits. I
> > > > > > > > expect rebasing to be straightforward, however with some
> manual
> > > > > > conflict
> > > > > > > > resolution. After the merge we'll keep running the tests to
> > make
> > > > sure
> > > > > > > > everything is ok.
> > > > > > > >
> > > > > > > > An overview of the changes, and the status of the work can =
be
> > > found
> > > > > > under
> > > > > > > > [4], [6] and [7].In summary, with the code in branch, you c=
an
> > > > create
> > > > > > > tables
> > > > > > > > with region replicas, do gets / multi gets and scans using
> > > TIMELINE
> > > > > > > > consistency with high availability. Region replicas
> > periodically
> > > > scan
> > > > > > the
> > > > > > > > files of the primary and pick up flushed / committed files.
> The
> > > RPC
> > > > > > > paths /
> > > > > > > > assignment, balancing etc are pretty stable. However some
> more
> > > > > > > performance
> > > > > > > > analysis and tuning is needed. Phase 2 work is being worked
> on
> > > > under
> > > > > > > > HBASE-11183, and we have some working prototype for
> > > > async-replicating
> > > > > > and
> > > > > > > > region splits. However, we believe even without those
> features,
> > > > this
> > > > > > work
> > > > > > > > is useable (especially for read-only/bulk load tables) , an=
d
> > can
> > > be
> > > > > > > > released as an experimental feature in 1.0.
> > > > > > > >
> > > > > > > > Please indicate your choice:
> > > > > > > >
> > > > > > > > [ ] +1 on yes, merge branch hbase-10070 to trunk.
> > > > > > > > [ ] 0 on don't care
> > > > > > > > [ ] -1 don't merge, because ...
> > > > > > > >
> > > > > > > > I'll keep the vote running for 7 days, and close it Mon 9th
> of
> > > > June,
> > > > > > PDT.
> > > > > > > >
> > > > > > > > Here is my official +1.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Enis
> > > > > > > >
> > > > > > > > [1]
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://git-wip-us.apache.org/repos/asf?p=3Dhbase.git;a=3Dlog;h=3Drefs/he=
ads/hbase-10070
> > > > > > > > [2]
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/HBASE-11214?jql=3DfixVersion%20%3D%=
20hbase-10070%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolved
> > > > > > > > [3]
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/HBASE-10792?jql=3DfixVersion%20%3D%=
20hbase-10070%20and%20fixversion%20%3D%200.99.0%20AND%20project%20%3D%20HBA=
SE%20AND%20status%20%3D%20resolved
> > > > > > > > [4]
> > > > https://www.mail-archive.com/dev@hbase.apache.org/msg25795.html
> > > > > > > > [5]
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hbase/commit/e22c7efeac02efde3451a0c9ff9bdcd272=
5576d0
> > > > > > > > [6]
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://www.slideshare.net/enissoz/hbase-high-availability-for-reads-with-=
time
> > > > > > > > [7] https://issues.apache.org/jira/browse/HBASE-10070
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Best regards,
> > > > > > >
> > > > > > >    - Andy
> > > > > > >
> > > > > > > Problems worthy of attack prove their worth by hitting back. =
-
> > Piet
> > > > > Hein
> > > > > > > (via Tom White)
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > // Jonathan Hsieh (shay)
> > > > > > // HBase Tech Lead, Software Engineer, Cloudera
> > > > > > // jon@cloudera.com // @jmhsieh
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > // Jonathan Hsieh (shay)
> > > > // HBase Tech Lead, Software Engineer, Cloudera
> > > > // jon@cloudera.com // @jmhsieh
> > > >
> > >
> >
> >
> >
> > --
> > // Jonathan Hsieh (shay)
> > // HBase Tech Lead, Software Engineer, Cloudera
> > // jon@cloudera.com // @jmhsieh
> >
>
>
>
> --
> Thanks,
> Michael Antonov
>

--001a1132f3ace5bfb404fbaa180c--