hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Colin McCabe <cmcc...@alumni.cmu.edu>
Subject Re: [VOTE] Merge HDFS-4949 to trunk
Date Tue, 29 Oct 2013 01:14:26 GMT
With 3 +1s, the vote passes.  Thanks, all.

best,
Colin

On Fri, Oct 25, 2013 at 4:01 PM, Colin McCabe <cmccabe@alumni.cmu.edu> wrote:
> On Fri, Oct 25, 2013 at 10:07 AM, Suresh Srinivas
> <suresh@hortonworks.com> wrote:
>> I posted a comment in the other thread about feature branch merges.
>>
>> My preference is to make sure the requirements we have for regular patches
>> be applied to feature branch patch as well (3 +1s is the only exception).
>> Also
>> adding details about what functionality is missing (I posted a comment on
>> HDFS-4949)
>> and the changes that deferred that will be done post merge to trunk would
>> be good.
>>
>> It would be better to start the merge vote  when the work is ready instead
>> of
>> trying to optimize 1 week by doing the required work for merging in
>> parallel with
>> the vote.
>
> OK.
>
>>
>> If all the requirements for merging have been met, I am +1 on the merge,
>> without
>> the need for restarting the vote.
>>
>
> I think the requirements are all in place right now.  I'll create a
> JIRA detailing the post-merge subtasks just to make it clearer what
> the plan is from here.
>
> If there are no more comments, I'll commit later tonight.
>
> I wouldn't mind waiting a week if there was a feature someone
> absolutely felt we needed pre-merge, but I also feel like it would be
> two weeks, due to Hadoop Summit next week.
>
> best,
> Colin
>
>
>>
>> On Thu, Oct 24, 2013 at 11:29 PM, Aaron T. Myers <atm@apache.org> wrote:
>>
>>> I don't necessarily disagree with the general questions about the
>>> procedural issues of merge votes. Thanks for bringing that up in the other
>>> thread you mentioned. To some extent it seems like much of this has been
>>> based on custom, and if folks feel that more precisely defining the merge
>>> vote process is warranted, then I think we should take that up over on that
>>> thread.
>>>
>>> With regard to this particular merge vote, I've spoken with Chris offline
>>> about his feelings on this. He said that he is not dead-set on restarting
>>> the vote, though he suspects that others may be. It seems to me the
>>> remaining unfinished asks (e.g. updating the design doc) can reasonably be
>>> done either after this vote but before the merge to trunk proper, or could
>>> even reasonably be done after merging to trunk.
>>>
>>> Given that, I'll lend my +1 to this merge. I've been reviewing the branch
>>> pretty consistently since work started on it, and have personally
>>> run/tested several builds of it along the way. I've also reviewed the
>>> design thoroughly. The implementation, overall design, and API seem to me
>>> plenty stable enough to be merged into trunk. I know that there remains a
>>> handful of javac warnings in the branch that aren't in trunk, but I trust
>>> those will be taken care of before the merge.
>>>
>>> If anyone out there does feel strongly that this merge vote should be
>>> restarted, then please speak up soon. Again, we can restart the vote if
>>> need be, but I honestly think we'll gain very little by doing so.
>>>
>>> Best,
>>> Aaron
>>>
>>>
>>> On Fri, Oct 25, 2013 at 5:45 AM, Chris Nauroth <cnauroth@hortonworks.com
>>> >wrote:
>>>
>>> > Hi Andrew,
>>> >
>>> > I've come to the conclusion that I'm very confused about merge votes.
>>>  :-)
>>> >  It's not just about HDFS-4949.  I'm confused about all merge votes.
>>> >  Rather than muddy the waters here, I've started a separate discussion on
>>> > common-dev.
>>> >
>>> > I do agree with the general plan outlined here, and I will comment
>>> directly
>>> > on the HDFS-4949 jira with a binding +1 when I see that we've completed
>>> > that plan.
>>> >
>>> > Chris Nauroth
>>> > Hortonworks
>>> > http://hortonworks.com/
>>> >
>>> >
>>> >
>>> > On Wed, Oct 23, 2013 at 2:18 PM, Andrew Wang <andrew.wang@cloudera.com
>>> > >wrote:
>>> >
>>> > > Hey Chris,
>>> > >
>>> > > Right now we're on track to have all of those things done by tomorrow.
>>> > > Since the remaining issues are either not technical or do not involve
>>> > major
>>> > > changes, I was hoping we could +1 this merge vote in the spirit of
"+1
>>> > > pending jenkins". We've gotten clean unit test runs on upstream Jenkins
>>> > as
>>> > > well, so the only fixups we should need for test-patch.sh are findbugs
>>> > and
>>> > > javac (which are normally pretty trivial to clean up). Of course, all
>>> of
>>> > > your listed prereqs and test-patch would be taken care of before
>>> actually
>>> > > merging to trunk.
>>> > >
>>> > > So, we can reset the vote if you feel strongly about this, but it seems
>>> > > like the only real result will be delaying the merge by a week.
>>> > >
>>> > > Thanks,
>>> > > Andrew
>>> > >
>>> > >
>>> > > On Wed, Oct 23, 2013 at 1:03 PM, Chris Nauroth <
>>> cnauroth@hortonworks.com
>>> > > >wrote:
>>> > >
>>> > > > I've received some feedback that we haven't handled this merge
vote
>>> the
>>> > > > same as other comparable merge votes, and that the vote should
be
>>> reset
>>> > > > because of this.
>>> > > >
>>> > > > The recent custom is that we only call for the merge vote after
all
>>> > > > pre-requisites have been satisfied.  This would include committing
to
>>> > the
>>> > > > feature branch all patches that the devs deem necessary before
the
>>> code
>>> > > > lands in trunk, posting a test plan, posting an updated design
doc in
>>> > > case
>>> > > > implementation choices diverged from the original design doc,
and
>>> > > getting a
>>> > > > good test-patch run from Jenkins on the merge patch.  This was
the
>>> > > process
>>> > > > followed for other recent major features like HDFS-2802 (snapshots),
>>> > > > HDFS-347 (short-circuit reads via sharing file descriptors), and
>>> > > > HADOOP-8562 (Windows compatibility).  In this thread, we've diverged
>>> > from
>>> > > > that process by calling for a vote on a branch that hasn't yet
>>> > completed
>>> > > > the pre-requisites and stating a plan for work to be done before
the
>>> > > merge.
>>> > > >
>>> > > > I still support this work, but can we please restart the vote
after
>>> the
>>> > > > pre-requisites have landed in the branch?
>>> > > >
>>> > > > Chris Nauroth
>>> > > > Hortonworks
>>> > > > http://hortonworks.com/
>>> > > >
>>> > > >
>>> > > >
>>> > > > On Fri, Oct 18, 2013 at 1:37 PM, Chris Nauroth <
>>> > cnauroth@hortonworks.com
>>> > > > >wrote:
>>> > > >
>>> > > > > +1
>>> > > > >
>>> > > > > Sounds great!
>>> > > > >
>>> > > > > Regarding testing caching+federation, this is another thing
that I
>>> > had
>>> > > > > intended to pick up as part of HDFS-5149.  I'm not sure if
I can
>>> get
>>> > > this
>>> > > > > done in the next 7 days, so I'll keep you posted.
>>> > > > >
>>> > > > > Chris Nauroth
>>> > > > > Hortonworks
>>> > > > > http://hortonworks.com/
>>> > > > >
>>> > > > >
>>> > > > >
>>> > > > > On Fri, Oct 18, 2013 at 11:15 AM, Colin McCabe <
>>> > cmccabe@alumni.cmu.edu
>>> > > > >wrote:
>>> > > > >
>>> > > > >> Hi Chris,
>>> > > > >>
>>> > > > >> I think it's feasible to complete those tasks in the
next 7 days.
>>> > > > >> Andrew is on HDFS-5386.
>>> > > > >>
>>> > > > >> The test plan document is a great idea.  We'll try to
get that up
>>> > > > >> early next week.  We have a lot of unit tests now, clearly,
but
>>> some
>>> > > > >> manual testing is important too.
>>> > > > >>
>>> > > > >> If we discover any issues during testing, then we can
push out the
>>> > > > >> merge timeframe.  For example, one area that probably
needs more
>>> > > > >> testing is caching+federation.
>>> > > > >>
>>> > > > >> I would like to get HDFS-5378 and HDFS-5366 in as well.
>>> > > > >>
>>> > > > >> The other subtasks are "nice to have" but not really
critical,
>>> and I
>>> > > > >> think it would be just as easy to do them in trunk. 
We're hoping
>>> > that
>>> > > > >> having this in trunk will make it easier for us to collaborate
on
>>> > > > >> HDFS-2832 and other ongoing work.
>>> > > > >>
>>> > > > >> > Also, I want to confirm that this vote only covers
trunk.
>>> > > > >> > I don't see branch-2 mentioned, so I assume that
we're
>>> > > > >> > not voting on merge to branch-2 yet.
>>> > > > >>
>>> > > > >> Yeah, this vote is only to merge to trunk.
>>> > > > >>
>>> > > > >> cheers.
>>> > > > >> Colin
>>> > > > >>
>>> > > > >> On Fri, Oct 18, 2013 at 10:48 AM, Chris Nauroth
>>> > > > >> <cnauroth@hortonworks.com> wrote:
>>> > > > >> > I agree that the code has reached a stable point.
 Colin and
>>> > Andrew,
>>> > > > >> thank
>>> > > > >> > you for your contributions and collaboration.
>>> > > > >> >
>>> > > > >> > Throughout development, I've watched the feature
grow by running
>>> > > daily
>>> > > > >> > builds in a pseudo-distributed deployment.  As of
this week, the
>>> > > full
>>> > > > >> > feature set is working end-to-end.  I also think
we've reached a
>>> > > point
>>> > > > >> of
>>> > > > >> > API stability for clients who want to control caching
>>> > > > programmatically.
>>> > > > >> >
>>> > > > >> > There are several things that I'd like to see completed
before
>>> the
>>> > > > >> merge as
>>> > > > >> > pre-requisites:
>>> > > > >> >
>>> > > > >> > - HDFS-5203: Concurrent clients that add a cache
directive on
>>> the
>>> > > same
>>> > > > >> path
>>> > > > >> > may prematurely uncache from each other.
>>> > > > >> > - HDFS-5385: Caching RPCs are AtMostOnce, but do
not persist
>>> > client
>>> > > ID
>>> > > > >> and
>>> > > > >> > call ID to edit log.
>>> > > > >> > - HDFS-5386: Add feature documentation for datanode
caching.
>>> > > > >> > - Standard clean-ups to satisfy Jenkins pre-commit
on the merge
>>> > > patch.
>>> > > > >> >  (For example, I know we've introduced some Javadoc
warnings.)
>>> > > > >> > - Full test suite run on Windows.  (The feature
is not yet
>>> > > implemented
>>> > > > >> on
>>> > > > >> > Windows.  This is just intended to catch regressions.)
>>> > > > >> > - Test plan posted to HDFS-4949, similar in scope
to the
>>> snapshot
>>> > > test
>>> > > > >> plan
>>> > > > >> > that was posted to HDFS-2802.  For my own part,
I've run the new
>>> > > unit
>>> > > > >> > tests, and I've tested end-to-end in a pseudo-distributed
>>> > > deployment.
>>> > > > >>  It's
>>> > > > >> > unlikely that I'll get a chance to test fully distributed
before
>>> > the
>>> > > > >> vote
>>> > > > >> > closes, so I'm curious to hear if you've done this
on your side
>>> > yet.
>>> > > > >> >
>>> > > > >> > Also, I want to confirm that this vote only covers
trunk.  I
>>> don't
>>> > > see
>>> > > > >> > branch-2 mentioned, so I assume that we're not voting
on merge
>>> to
>>> > > > >> branch-2
>>> > > > >> > yet.
>>> > > > >> >
>>> > > > >> > Before I cast my vote, can you please discuss whether
or not
>>> it's
>>> > > > >> feasible
>>> > > > >> > to complete all of the above in the next 7 days?
 For the issues
>>> > > > >> assigned
>>> > > > >> > to me, I do expect to complete them.
>>> > > > >> >
>>> > > > >> > Thanks again for all of your hard work!
>>> > > > >> >
>>> > > > >> > Chris Nauroth
>>> > > > >> > Hortonworks
>>> > > > >> > http://hortonworks.com/
>>> > > > >> >
>>> > > > >> >
>>> > > > >> >
>>> > > > >> > On Thu, Oct 17, 2013 at 3:07 PM, Colin McCabe <
>>> > > cmccabe@alumni.cmu.edu
>>> > > > >> >wrote:
>>> > > > >> >
>>> > > > >> >> +1.  Thanks, guys.
>>> > > > >> >>
>>> > > > >> >> best,
>>> > > > >> >> Colin
>>> > > > >> >>
>>> > > > >> >> On Thu, Oct 17, 2013 at 3:01 PM, Andrew Wang
<
>>> > > > andrew.wang@cloudera.com
>>> > > > >> >
>>> > > > >> >> wrote:
>>> > > > >> >> > Hello all,
>>> > > > >> >> >
>>> > > > >> >> > I'd like to call a vote to merge the HDFS-4949
branch
>>> > (in-memory
>>> > > > >> caching)
>>> > > > >> >> > to trunk. Colin McCabe and I have been
hard at work the last
>>> > 3.5
>>> > > > >> months
>>> > > > >> >> > implementing this feature, and feel that
it's reached a level
>>> > of
>>> > > > >> >> stability
>>> > > > >> >> > and utility where it's ready for broader
testing and
>>> > integration.
>>> > > > >> >> >
>>> > > > >> >> > I'd also like to thank Chris Nauroth at
Hortonworks for code
>>> > > > reviews
>>> > > > >> and
>>> > > > >> >> > bug fixes, and everyone who's reviewed
the HDFS-4949 design
>>> doc
>>> > > and
>>> > > > >> left
>>> > > > >> >> > comments.
>>> > > > >> >> >
>>> > > > >> >> > Obviously, I am +1 for the merge. The vote
will run the
>>> > standard
>>> > > 7
>>> > > > >> days,
>>> > > > >> >> > closing on October 24 at 11:59PM.
>>> > > > >> >> >
>>> > > > >> >> > Thanks,
>>> > > > >> >> > Andrew
>>> > > > >> >>
>>> > > > >> >
>>> > > > >> > --
>>> > > > >> > CONFIDENTIALITY NOTICE
>>> > > > >> > NOTICE: This message is intended for the use of
the individual
>>> or
>>> > > > >> entity to
>>> > > > >> > which it is addressed and may contain information
that is
>>> > > > confidential,
>>> > > > >> > privileged and exempt from disclosure under applicable
law. If
>>> the
>>> > > > >> reader
>>> > > > >> > of this message is not the intended recipient, you
are hereby
>>> > > notified
>>> > > > >> that
>>> > > > >> > any printing, copying, dissemination, distribution,
disclosure
>>> or
>>> > > > >> > forwarding of this communication is strictly prohibited.
If you
>>> > have
>>> > > > >> > received this communication in error, please contact
the sender
>>> > > > >> immediately
>>> > > > >> > and delete it from your system. Thank You.
>>> > > > >>
>>> > > > >
>>> > > > >
>>> > > >
>>> > > > --
>>> > > > CONFIDENTIALITY NOTICE
>>> > > > NOTICE: This message is intended for the use of the individual
or
>>> > entity
>>> > > to
>>> > > > which it is addressed and may contain information that is
>>> confidential,
>>> > > > privileged and exempt from disclosure under applicable law. If
the
>>> > reader
>>> > > > of this message is not the intended recipient, you are hereby
>>> notified
>>> > > that
>>> > > > any printing, copying, dissemination, distribution, disclosure
or
>>> > > > forwarding of this communication is strictly prohibited. If you
have
>>> > > > received this communication in error, please contact the sender
>>> > > immediately
>>> > > > and delete it from your system. Thank You.
>>> > > >
>>> > >
>>> >
>>> > --
>>> > CONFIDENTIALITY NOTICE
>>> > NOTICE: This message is intended for the use of the individual or entity
>>> to
>>> > which it is addressed and may contain information that is confidential,
>>> > privileged and exempt from disclosure under applicable law. If the reader
>>> > of this message is not the intended recipient, you are hereby notified
>>> that
>>> > any printing, copying, dissemination, distribution, disclosure or
>>> > forwarding of this communication is strictly prohibited. If you have
>>> > received this communication in error, please contact the sender
>>> immediately
>>> > and delete it from your system. Thank You.
>>> >
>>>
>>
>>
>>
>> --
>> http://hortonworks.com/download/
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.

Mime
View raw message