hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: [VOTE] Merge HDFS-3077 (QuorumJournalManager) branch to trunk
Date Tue, 09 Oct 2012 03:03:13 GMT
Our position on the QJM is we've already "taken delivery" from the feature
branch and will maintain a private HDFS fork of branch-2 if necessary, i.e.
we don't have a significant stake in this discussion except at a meta level
as potential contributors.

This is a case study of why feature branch development is problematic? A
would-be contributor can make a significant effort over months and end up
with a baked result, ready to move on to a perhaps large backlog of other
work. Others can reasonably be expected meanwhile to triage their attention
until the merge point. Then, reconsidering design points will be more
challenging than a design discussion at an earlier time, because there is
already a significant sunk cost in the current design. What does a feature
branch buy here over development in situ in trunk (like the BookKeeper JM)?
Should would-be future contributors consider a feature branch as a viable
route toward contribution or not?

On Tuesday, October 9, 2012, Suresh Srinivas wrote:

> On Mon, Oct 8, 2012 at 6:20 PM, Todd Lipcon <todd@cloudera.com<javascript:;>>
> wrote:
> > On Mon, Oct 8, 2012 at 6:01 PM, Suresh Srinivas <suresh@hortonworks.com<javascript:;>
> > >wrote:
> >
> > > Todd,
> > >
> > > As I indicated in my comments on the jira, I think some of the design
> > > discussions and further simplification of design should happen before
> the
> > > merge. See -
> > >
> > >
> >
> https://issues.apache.org/jira/browse/HDFS-3077?focusedCommentId=13470680&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13470680
> > >
> > >
> > I still haven't heard any actual concrete suggestion for a design
> > improvement. Just abstract notions that "it should be simpler". I could
> say
> > the same about several other features that have been done in the past
> (e.g
> > federation or YARN) but chose not to because I didn't participate in the
> > development. I generally have faith that, if several other people spent
> > several months working on a project, then there must be good reasons for
> > the complexity that I'm probably overlooking.
> >
> I think I have participated enough in this work. Merge time seemed like a
> good time
> to review this more thoroughly given this jira has required quite a
> bit of educating myself and spending a lot of my time on this because of
> need to
> understand the complexities/subtletie how paxos and ZAB applies to the
> namenode
> journals.
> Please do not misunderstand that I am trying to block this work from being
> merged. I am
> supportive of this as you have seen in my previous response in this thread.
> All I want
> to see is if we can conclude the discussions and incorporate some the
> comments that come
> up after it.
> >
> > Several of us (not just I) spent lots of time on this over the last
> several
> > months, with all of the work going on in the open. My issue with this
> > discussion happening now is that no one saw fit to raise these questions
> > months ago when the design was first proposed. For example, this most
> > recent question about whether NewEpoch and PrepareRecovery should be
> > separate RPCs could have been raised in late June when the code in
> question
> > was first introduced as a patch.
> >
> > Nor was anything raised when I gave a "heads up" that the branch was
> > complete and nearly ready for merge ~3 weeks ago. Only once I actually
> > called a merge vote has this discussion started. That doesn't seem like a
> > healthy way to do development.
> >
> Again Todd, you are reading more than what is intended. As I said earlier
> merge
> time is a good time to have complete picture and an opportunity to look at
> final design
> and implementation.
> > What are the criteria, then, for merging? I don't think it's possible to
> > definitively "prove" that a design is "simple". At some level it's a
> matter
> > of taste. So if the current design works, and is tested, and has people
> > signed up to support it and run it, why not merge? Just like any other
> part
> > of HDFS, we can continue to _improve_ it after it is in trunk. There are
> > many features that we commit and then improve later on when someone comes
> > up with a way to simplify it.
> >
> One concern I have is, once this is merged into trunk I feel that any
> proposed
> improvements may be blocked. Given that why not just wait for these
> discussions
> to complete and do the work in this branch?
> What is the need for getting this into trunk immediately?
> If the problem is one of investment of your time, I have already offered to
> help out.
> > To be clear about the current status of the vote: you are officially
> > vetoing the merge?
> For now I am going to vote not to merge until the discussion completes.
> Regards,
> Suresh

Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message