hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: [VOTE] Merge HDFS-3077 (QuorumJournalManager) branch to trunk
Date Tue, 09 Oct 2012 03:09:47 GMT
Sorry, just to be clear "our" and "we" below refer to my employer, nothing
to do with HBase. Please pardon any confusion.

On Tuesday, October 9, 2012, Andrew Purtell wrote:

> Our position on the QJM is we've already "taken delivery" from the feature
> branch and will maintain a private HDFS fork of branch-2 if necessary, i.e.
> we don't have a significant stake in this discussion except at a meta level
> as potential contributors.
>
> This is a case study of why feature branch development is problematic? A
> would-be contributor can make a significant effort over months and end up
> with a baked result, ready to move on to a perhaps large backlog of other
> work. Others can reasonably be expected meanwhile to triage their attention
> until the merge point. Then, reconsidering design points will be more
> challenging than a design discussion at an earlier time, because there is
> already a significant sunk cost in the current design. What does a feature
> branch buy here over development in situ in trunk (like the BookKeeper JM)?
> Should would-be future contributors consider a feature branch as a viable
> route toward contribution or not?
>
> On Tuesday, October 9, 2012, Suresh Srinivas wrote:
>
>> On Mon, Oct 8, 2012 at 6:20 PM, Todd Lipcon <todd@cloudera.com> wrote:
>>
>> > On Mon, Oct 8, 2012 at 6:01 PM, Suresh Srinivas <suresh@hortonworks.com
>> > >wrote:
>> >
>> > > Todd,
>> > >
>> > > As I indicated in my comments on the jira, I think some of the design
>> > > discussions and further simplification of design should happen before
>> the
>> > > merge. See -
>> > >
>> > >
>> >
>> https://issues.apache.org/jira/browse/HDFS-3077?focusedCommentId=13470680&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13470680
>> > >
>> > >
>> > I still haven't heard any actual concrete suggestion for a design
>> > improvement. Just abstract notions that "it should be simpler". I could
>> say
>> > the same about several other features that have been done in the past
>> (e.g
>> > federation or YARN) but chose not to because I didn't participate in the
>> > development. I generally have faith that, if several other people spent
>> > several months working on a project, then there must be good reasons for
>> > the complexity that I'm probably overlooking.
>> >
>>
>> I think I have participated enough in this work. Merge time seemed like a
>> good time
>> to review this more thoroughly given this jira has required quite a
>> bit of educating myself and spending a lot of my time on this because of
>> need to
>> understand the complexities/subtletie how paxos and ZAB applies to the
>> namenode
>> journals.
>>
>> Please do not misunderstand that I am trying to block this work from being
>> merged. I am
>> supportive of this as you have seen in my previous response in this
>> thread.
>> All I want
>> to see is if we can conclude the discussions and incorporate some the
>> comments that come
>> up after it.
>>
>> >
>> > Several of us (not just I) spent lots of time on this over the last
>> several
>> > months, with all of the work going on in the open. My issue with this
>> > discussion happening now is that no one saw fit to raise these questions
>> > months ago when the design was first proposed. For example, this most
>> > recent question about whether NewEpoch and PrepareRecovery should be
>> > separate RPCs could have been raised in late June when the code in
>> question
>> > was first introduced as a patch.
>> >
>> > Nor was anything raised when I gave a "heads up" that the branch was
>> > complete and nearly ready for merge ~3 weeks ago. Only once I actually
>> > called a merge vote has this discussion started. That doesn't seem like
>> a
>> > healthy way to do development.
>> >
>>
>> Again Todd, you are reading more than what is intended. As I said earlier
>> merge
>> time is a good time to have complete picture and an opportunity to look at
>> final design
>> and implementation.
>>
>>
>> > What are the criteria, then, for merging? I don't think it's possible to
>> > definitively "prove" that a design is "simple". At some level it's a
>> matter
>> > of taste. So if the current design works, and is tested, and has people
>> > signed up to support it and run it, why not merge? Just like any other
>> part
>> > of HDFS, we can continue to _improve_ it after it is in trunk. There are
>> > many features that we commit and then improve later on when someone
>> comes
>> > up with a way to simplify it.
>> >
>>
>> One concern I have is, once this is merged into trunk I feel that any
>> proposed
>> improvements may be blocked. Given that why not just wait for these
>> discussions
>> to complete and do the work in this branch?
>>
>> What is the need for getting this into trunk immediately?
>>
>> If the problem is one of investment of your time, I have already offered
>> to
>> help out.
>>
>>
>> > To be clear about the current status of the vote: you are officially
>> > vetoing the merge?
>>
>>
>> For now I am going to vote not to merge until the discussion completes.
>>
>> Regards,
>> Suresh
>>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
>

-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message