hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jimmy Xiang <jxi...@cloudera.com>
Subject Re: [VOTE] Merge branch HBASE-10070 to trunk
Date Sat, 07 Jun 2014 02:35:48 GMT
Yes, the state for assignment is in the meta row.


On Fri, Jun 6, 2014 at 7:21 PM, Enis Söztutar <enis.soz@gmail.com> wrote:

> On Fri, Jun 6, 2014 at 7:07 PM, Jimmy Xiang <jxiang@cloudera.com> wrote:
>
> > First of all, I have not looked into the patches recently. I remember
> there
> > are some changes to the public interface. I was wondering if it is
> backward
> > compatible. Enis mentioned that it's rolling upgradable. Just want to
> > confirm if it is backward compatible. For existing applications, do they
> > need to recompile?
> >
>
> I think you are referring to the patch for
> https://issues.apache.org/jira/browse/HBASE-10347. It changes HRI to add
> the replicaId and deprecates all client-facing interfaces using HRL.
> RegionLocations replaces that, but locating regions are no longer
> client-facing interfaces. Existing applications might have to re-compile,
> but we do not have to have binary-compatibility guarantee between 0.98 and
> 1.0. AFAIK, non of the latest major releases had that guarantee to the
> previous major version (0.98 -> 0.96 -> 0.94).
>
>
> >
> > Second, there should be some conflict with HBASE-11059 I am working on
> now,
> > which is almost finished. How should we resolve this issue? ZK-less
> region
> > assignment doesn't know region replica yet.
> >
>
> I have not looked at your patch yet. The region replica assignment does not
> change the assignment mechanics on zookeeper. So assigning a secondary
> region works without any changes there. However region replicas do not have
> explicit meta locations (they share the same meta row). I am not sure where
> you keep the state for assignment. Is it in the meta row?
>
>
> >
> > Thanks,
> > Jimmy
> >
> >
> >
> >
> > On Fri, Jun 6, 2014 at 6:12 PM, Jeffrey Zhong <jzhong@hortonworks.com>
> > wrote:
> >
> > >
> > > +1. It brings goodies such as region replica, cross region server
> > > scan&get, anti-affinity of regions, and pave the way for sync region
> > > replication.
> > >
> > > Thanks,
> > > -Jeffrey
> > >
> > > On 6/6/14 2:42 PM, "Devaraj Das" <ddas@hortonworks.com> wrote:
> > >
> > > >+1
> > > >
> > > >On Fri, Jun 6, 2014 at 2:13 PM, Andrew Purtell <apurtell@apache.org>
> > > >wrote:
> > > >> +1, thanks Enis
> > > >>
> > > >>
> > > >> On Fri, Jun 6, 2014 at 1:46 PM, Enis Söztutar <enis.soz@gmail.com>
> > > >>wrote:
> > > >>
> > > >>> Sorry, I was mostly out for HadoopSummit.
> > > >>>
> > > >>> Yes, the git flow would be very similar to what you propose:
> > > >>>
> > > >>>      $ git checkout HBASE-10070
> > > >>>      $ git rebase --ignore-date master
> > > >>>      (fixups, git add, git rebase --continue, etc, etc, etc)
> > > >>>      $ git checkout master
> > > >>>
> > > >>>      $ git push origin HBASE-10070 HBASE-10070-rebase-date
> > > >>>(optionally)
> > > >>>      $ git merge HBASE-10070
> > > >>>
> > > >>> We can either go --ignore-date or not depending on what we want.
If
> > > >>>needed
> > > >>> I am fine with pushing the rebased master branch for review to
main
> > > >>>repo
> > > >>> before the merge to another branch. If not, I can just rebase
the
> > > >>>branch
> > > >>> locally and merge + push to main repo.
> > > >>>
> > > >>> Creating final patches and attaching them to jira might be
> > cumbersome.
> > > >>>If
> > > >>> we do the rebased-branch on repo, we might not need it. But if
we
> > need
> > > >>>that
> > > >>> for review, I can do it.
> > > >>>
> > > >>> Thanks,
> > > >>> Enis
> > > >>>
> > > >>>
> > > >>>
> > > >>> On Wed, Jun 4, 2014 at 10:48 AM, Andrew Purtell <
> apurtell@apache.org
> > >
> > > >>> wrote:
> > > >>>
> > > >>> > I realize this is a vote thread but I need a satisfactory
answer
> to
> > > >>>the
> > > >>> > below inquiries before feeling comfortable casting a vote.
Or
> > perhaps
> > > >>> that
> > > >>> > means we need to cancel this vote and move back to discussion.
> > > >>> >
> > > >>> >
> > > >>> > On Tue, Jun 3, 2014 at 11:17 AM, Andrew Purtell <
> > apurtell@apache.org
> > > >
> > > >>> > wrote:
> > > >>> >
> > > >>> > > Also after the merge process is completed, do you plan
to use
> git
> > > >>> > > format-patch to break out the per-JIRA changes into
updated
> > > >>>patches for
> > > >>> > > those JIRAs representing in effect the final commit?
> > > >>> > >
> > > >>> > >
> > > >>> > > On Tue, Jun 3, 2014 at 11:16 AM, Andrew Purtell
> > > >>><apurtell@apache.org>
> > > >>> > > wrote:
> > > >>> > >
> > > >>> > >>
> > > >>> > >> On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <
> enis@apache.org>
> > > >>> wrote:
> > > >>> > >>
> > > >>> > >> This VOTE is for merging back the remaining changes
in branch
> to
> > > >>> trunk.
> > > >>> > If
> > > >>> > >>> passes, we will rebase the branch on top of
current trunk, in
> > > >>>which
> > > >>> we
> > > >>> > >>> will
> > > >>> > >>> keep the commit-per-issue log history. After
that we will do
> a
> > > >>>git
> > > >>> > merge
> > > >>> > >>> for the branch keeping the history clean and
not squashing
> the
> > > >>> > commits. I
> > > >>> > >>> expect rebasing to be straightforward, however
with some
> manual
> > > >>> > conflict
> > > >>> > >>> resolution. After the merge we'll keep running
the tests to
> > make
> > > >>>sure
> > > >>> > >>> everything is ok.
> > > >>> > >>>
> > > >>> > >>
> > > >>> > >> Just to clarify that would look something like this:
> > > >>> > >>
> > > >>> > >>      $ git checkout HBASE-10070
> > > >>> > >>      $ git rebase --ignore-date master
> > > >>> > >>      (fixups, git add, git rebase --continue, etc,
etc, etc)
> > > >>> > >>      $ git checkout master
> > > >>> > >>      $ git merge HBASE-10070
> > > >>> > >>
> > > >>> > >> ?
> > > >>> > >>
> > > >>> > >> That sounds good to me, the final merge should be
a fast
> forward
> > > >>> merge.
> > > >>> > >>
> > > >>> > >> Use of ' --ignore-date' could be mildly controversial.
It's
> not
> > > >>> strictly
> > > >>> > >> necessary because the commits for 10070 will appear
grouped in
> > > >>> history,
> > > >>> > but
> > > >>> > >> then dates on commits will be discontiguous in that
section of
> > > >>> history.
> > > >>> > I
> > > >>> > >> suggest using that option so the order of commits
and dates
> sort
> > > >>>the
> > > >>> > same
> > > >>> > >> on master.
> > > >>> > >>
> > > >>> > >>
> > > >>> > >> On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar <
> enis@apache.org>
> > > >>> wrote:
> > > >>> > >>
> > > >>> > >>> Hi,
> > > >>> > >>>
> > > >>> > >>> Last week we started some discussion[4] for
merging branch
> > > >>> > hbase-10070[1]
> > > >>> > >>> into trunk. It seems like the consensus there
is to do the
> > merge
> > > >>> sooner
> > > >>> > >>> rather than later.
> > > >>> > >>>
> > > >>> > >>>
> > > >>> > >>> We had branched hbase-10070 in Feb out of trunk[5].
The
> branch
> > > >>> contains
> > > >>> > >>> 55
> > > >>> > >>> jiras committed[2]. Out of these 55, 15 has
already been
> > > >>>committed to
> > > >>> > >>> trunk
> > > >>> > >>> and backported to hbase-10070 branch[3].
> > > >>> > >>>
> > > >>> > >>> This VOTE is for merging back the remaining
changes in branch
> > to
> > > >>> trunk.
> > > >>> > >>> If
> > > >>> > >>> passes, we will rebase the branch on top of
current trunk, in
> > > >>>which
> > > >>> we
> > > >>> > >>> will
> > > >>> > >>> keep the commit-per-issue log history. After
that we will do
> a
> > > >>>git
> > > >>> > merge
> > > >>> > >>> for the branch keeping the history clean and
not squashing
> the
> > > >>> > commits. I
> > > >>> > >>> expect rebasing to be straightforward, however
with some
> manual
> > > >>> > conflict
> > > >>> > >>> resolution. After the merge we'll keep running
the tests to
> > make
> > > >>>sure
> > > >>> > >>> everything is ok.
> > > >>> > >>>
> > > >>> > >>> An overview of the changes, and the status of
the work can be
> > > >>>found
> > > >>> > under
> > > >>> > >>> [4], [6] and [7].In summary, with the code in
branch, you can
> > > >>>create
> > > >>> > >>> tables
> > > >>> > >>> with region replicas, do gets / multi gets and
scans using
> > > >>>TIMELINE
> > > >>> > >>> consistency with high availability. Region replicas
> > periodically
> > > >>>scan
> > > >>> > the
> > > >>> > >>> files of the primary and pick up flushed / committed
files.
> The
> > > >>>RPC
> > > >>> > >>> paths /
> > > >>> > >>> assignment, balancing etc are pretty stable.
However some
> more
> > > >>> > >>> performance
> > > >>> > >>> analysis and tuning is needed. Phase 2 work
is being worked
> on
> > > >>>under
> > > >>> > >>> HBASE-11183, and we have some working prototype
for
> > > >>>async-replicating
> > > >>> > and
> > > >>> > >>> region splits. However, we believe even without
those
> features,
> > > >>>this
> > > >>> > work
> > > >>> > >>> is useable (especially for read-only/bulk load
tables) , and
> > can
> > > >>>be
> > > >>> > >>> released as an experimental feature in 1.0.
> > > >>> > >>>
> > > >>> > >>> Please indicate your choice:
> > > >>> > >>>
> > > >>> > >>> [ ] +1 on yes, merge branch hbase-10070 to trunk.
> > > >>> > >>> [ ] 0 on don't care
> > > >>> > >>> [ ] -1 don't merge, because ...
> > > >>> > >>>
> > > >>> > >>> I'll keep the vote running for 7 days, and close
it Mon 9th
> of
> > > >>>June,
> > > >>> > PDT.
> > > >>> > >>>
> > > >>> > >>> Here is my official +1.
> > > >>> > >>>
> > > >>> > >>> Thanks,
> > > >>> > >>> Enis
> > > >>> > >>>
> > > >>> > >>> [1]
> > > >>> > >>>
> > > >>> > >>>
> > > >>> >
> > > >>>
> > > >>>
> > >
> https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=log;h=refs/heads/h
> > > >>>base-10070
> > > >>> > >>> [2]
> > > >>> > >>>
> > > >>> > >>>
> > > >>> >
> > > >>>
> > > >>>
> > >
> https://issues.apache.org/jira/browse/HBASE-11214?jql=fixVersion%20%3D%2
> > >
> >
> >>>0hbase-10070%20AND%20project%20%3D%20HBASE%20AND%20status%20%3D%20resolv
> > > >>>ed
> > > >>> > >>> [3]
> > > >>> > >>>
> > > >>> > >>>
> > > >>> >
> > > >>>
> > > >>>
> > >
> https://issues.apache.org/jira/browse/HBASE-10792?jql=fixVersion%20%3D%2
> > >
> >
> >>>0hbase-10070%20and%20fixversion%20%3D%200.99.0%20AND%20project%20%3D%20H
> > > >>>BASE%20AND%20status%20%3D%20resolved
> > > >>> > >>> [4]
> > > >>>https://www.mail-archive.com/dev@hbase.apache.org/msg25795.html
> > > >>> > >>> [5]
> > > >>> > >>>
> > > >>> > >>>
> > > >>> >
> > > >>>
> > > >>>
> > >
> https://github.com/apache/hbase/commit/e22c7efeac02efde3451a0c9ff9bdcd27
> > > >>>25576d0
> > > >>> > >>> [6]
> > > >>> > >>>
> > > >>> > >>>
> > > >>> >
> > > >>>
> > > >>>
> > >
> http://www.slideshare.net/enissoz/hbase-high-availability-for-reads-with
> > > >>>-time
> > > >>> > >>> [7] https://issues.apache.org/jira/browse/HBASE-10070
> > > >>> > >>>
> > > >>> > >>
> > > >>> > >>
> > > >>> > >>
> > > >>> > >> --
> > > >>> > >> Best regards,
> > > >>> > >>
> > > >>> > >>    - Andy
> > > >>> > >>
> > > >>> > >> Problems worthy of attack prove their worth by hitting
back. -
> > > >>>Piet
> > > >>> Hein
> > > >>> > >> (via Tom White)
> > > >>> > >>
> > > >>> > >
> > > >>> > >
> > > >>> > >
> > > >>> > > --
> > > >>> > > Best regards,
> > > >>> > >
> > > >>> > >    - Andy
> > > >>> > >
> > > >>> > > Problems worthy of attack prove their worth by hitting
back. -
> > Piet
> > > >>> Hein
> > > >>> > > (via Tom White)
> > > >>> > >
> > > >>> >
> > > >>> >
> > > >>> >
> > > >>> > --
> > > >>> > Best regards,
> > > >>> >
> > > >>> >    - Andy
> > > >>> >
> > > >>> > Problems worthy of attack prove their worth by hitting back.
-
> Piet
> > > >>>Hein
> > > >>> > (via Tom White)
> > > >>> >
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Best regards,
> > > >>
> > > >>    - Andy
> > > >>
> > > >> Problems worthy of attack prove their worth by hitting back. - Piet
> > Hein
> > > >> (via Tom White)
> > > >
> > > >--
> > > >CONFIDENTIALITY NOTICE
> > > >NOTICE: This message is intended for the use of the individual or
> entity
> > > >to
> > > >which it is addressed and may contain information that is
> confidential,
> > > >privileged and exempt from disclosure under applicable law. If the
> > reader
> > > >of this message is not the intended recipient, you are hereby notified
> > > >that
> > > >any printing, copying, dissemination, distribution, disclosure or
> > > >forwarding of this communication is strictly prohibited. If you have
> > > >received this communication in error, please contact the sender
> > > >immediately
> > > >and delete it from your system. Thank You.
> > >
> > >
> > >
> > > --
> > > CONFIDENTIALITY NOTICE
> > > NOTICE: This message is intended for the use of the individual or
> entity
> > to
> > > which it is addressed and may contain information that is confidential,
> > > privileged and exempt from disclosure under applicable law. If the
> reader
> > > of this message is not the intended recipient, you are hereby notified
> > that
> > > any printing, copying, dissemination, distribution, disclosure or
> > > forwarding of this communication is strictly prohibited. If you have
> > > received this communication in error, please contact the sender
> > immediately
> > > and delete it from your system. Thank You.
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message