hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Nauroth <cnaur...@hortonworks.com>
Subject Re: [VOTE] Merge HDFS-4949 to trunk
Date Wed, 23 Oct 2013 20:03:07 GMT
I've received some feedback that we haven't handled this merge vote the
same as other comparable merge votes, and that the vote should be reset
because of this.

The recent custom is that we only call for the merge vote after all
pre-requisites have been satisfied.  This would include committing to the
feature branch all patches that the devs deem necessary before the code
lands in trunk, posting a test plan, posting an updated design doc in case
implementation choices diverged from the original design doc, and getting a
good test-patch run from Jenkins on the merge patch.  This was the process
followed for other recent major features like HDFS-2802 (snapshots),
HDFS-347 (short-circuit reads via sharing file descriptors), and
HADOOP-8562 (Windows compatibility).  In this thread, we've diverged from
that process by calling for a vote on a branch that hasn't yet completed
the pre-requisites and stating a plan for work to be done before the merge.

I still support this work, but can we please restart the vote after the
pre-requisites have landed in the branch?

Chris Nauroth
Hortonworks
http://hortonworks.com/



On Fri, Oct 18, 2013 at 1:37 PM, Chris Nauroth <cnauroth@hortonworks.com>wrote:

> +1
>
> Sounds great!
>
> Regarding testing caching+federation, this is another thing that I had
> intended to pick up as part of HDFS-5149.  I'm not sure if I can get this
> done in the next 7 days, so I'll keep you posted.
>
> Chris Nauroth
> Hortonworks
> http://hortonworks.com/
>
>
>
> On Fri, Oct 18, 2013 at 11:15 AM, Colin McCabe <cmccabe@alumni.cmu.edu>wrote:
>
>> Hi Chris,
>>
>> I think it's feasible to complete those tasks in the next 7 days.
>> Andrew is on HDFS-5386.
>>
>> The test plan document is a great idea.  We'll try to get that up
>> early next week.  We have a lot of unit tests now, clearly, but some
>> manual testing is important too.
>>
>> If we discover any issues during testing, then we can push out the
>> merge timeframe.  For example, one area that probably needs more
>> testing is caching+federation.
>>
>> I would like to get HDFS-5378 and HDFS-5366 in as well.
>>
>> The other subtasks are "nice to have" but not really critical, and I
>> think it would be just as easy to do them in trunk.  We're hoping that
>> having this in trunk will make it easier for us to collaborate on
>> HDFS-2832 and other ongoing work.
>>
>> > Also, I want to confirm that this vote only covers trunk.
>> > I don't see branch-2 mentioned, so I assume that we're
>> > not voting on merge to branch-2 yet.
>>
>> Yeah, this vote is only to merge to trunk.
>>
>> cheers.
>> Colin
>>
>> On Fri, Oct 18, 2013 at 10:48 AM, Chris Nauroth
>> <cnauroth@hortonworks.com> wrote:
>> > I agree that the code has reached a stable point.  Colin and Andrew,
>> thank
>> > you for your contributions and collaboration.
>> >
>> > Throughout development, I've watched the feature grow by running daily
>> > builds in a pseudo-distributed deployment.  As of this week, the full
>> > feature set is working end-to-end.  I also think we've reached a point
>> of
>> > API stability for clients who want to control caching programmatically.
>> >
>> > There are several things that I'd like to see completed before the
>> merge as
>> > pre-requisites:
>> >
>> > - HDFS-5203: Concurrent clients that add a cache directive on the same
>> path
>> > may prematurely uncache from each other.
>> > - HDFS-5385: Caching RPCs are AtMostOnce, but do not persist client ID
>> and
>> > call ID to edit log.
>> > - HDFS-5386: Add feature documentation for datanode caching.
>> > - Standard clean-ups to satisfy Jenkins pre-commit on the merge patch.
>> >  (For example, I know we've introduced some Javadoc warnings.)
>> > - Full test suite run on Windows.  (The feature is not yet implemented
>> on
>> > Windows.  This is just intended to catch regressions.)
>> > - Test plan posted to HDFS-4949, similar in scope to the snapshot test
>> plan
>> > that was posted to HDFS-2802.  For my own part, I've run the new unit
>> > tests, and I've tested end-to-end in a pseudo-distributed deployment.
>>  It's
>> > unlikely that I'll get a chance to test fully distributed before the
>> vote
>> > closes, so I'm curious to hear if you've done this on your side yet.
>> >
>> > Also, I want to confirm that this vote only covers trunk.  I don't see
>> > branch-2 mentioned, so I assume that we're not voting on merge to
>> branch-2
>> > yet.
>> >
>> > Before I cast my vote, can you please discuss whether or not it's
>> feasible
>> > to complete all of the above in the next 7 days?  For the issues
>> assigned
>> > to me, I do expect to complete them.
>> >
>> > Thanks again for all of your hard work!
>> >
>> > Chris Nauroth
>> > Hortonworks
>> > http://hortonworks.com/
>> >
>> >
>> >
>> > On Thu, Oct 17, 2013 at 3:07 PM, Colin McCabe <cmccabe@alumni.cmu.edu
>> >wrote:
>> >
>> >> +1.  Thanks, guys.
>> >>
>> >> best,
>> >> Colin
>> >>
>> >> On Thu, Oct 17, 2013 at 3:01 PM, Andrew Wang <andrew.wang@cloudera.com
>> >
>> >> wrote:
>> >> > Hello all,
>> >> >
>> >> > I'd like to call a vote to merge the HDFS-4949 branch (in-memory
>> caching)
>> >> > to trunk. Colin McCabe and I have been hard at work the last 3.5
>> months
>> >> > implementing this feature, and feel that it's reached a level of
>> >> stability
>> >> > and utility where it's ready for broader testing and integration.
>> >> >
>> >> > I'd also like to thank Chris Nauroth at Hortonworks for code reviews
>> and
>> >> > bug fixes, and everyone who's reviewed the HDFS-4949 design doc and
>> left
>> >> > comments.
>> >> >
>> >> > Obviously, I am +1 for the merge. The vote will run the standard 7
>> days,
>> >> > closing on October 24 at 11:59PM.
>> >> >
>> >> > Thanks,
>> >> > Andrew
>> >>
>> >
>> > --
>> > CONFIDENTIALITY NOTICE
>> > NOTICE: This message is intended for the use of the individual or
>> entity to
>> > which it is addressed and may contain information that is confidential,
>> > privileged and exempt from disclosure under applicable law. If the
>> reader
>> > of this message is not the intended recipient, you are hereby notified
>> that
>> > any printing, copying, dissemination, distribution, disclosure or
>> > forwarding of this communication is strictly prohibited. If you have
>> > received this communication in error, please contact the sender
>> immediately
>> > and delete it from your system. Thank You.
>>
>
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message