hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: [ANNOUNCEMENT] Yahoo focusing on Apache Hadoop, discontinuing "The Yahoo Distribution of Hadoop"
Date Thu, 07 Apr 2011 23:22:52 GMT
Is there a list available of which patches you've made this decision about?
I'm curious, for example, about MAPREDUCE-2178 -- as of today, the MR
security in trunk has a serious vulnerability. Do we plan on fixing it, or
will the answer be that, if anyone needs security, they must update to "MR
Next Gen"?


On Thu, Apr 7, 2011 at 3:52 PM, Arun C Murthy <acm@yahoo-inc.com> wrote:

> On Feb 14, 2011, at 1:34 PM, Arun C Murthy wrote:
>> As the final installment in this process, I've started a discussion on
>> us contributing a re-factor of Map-Reduce in
>> https://issues.apache.org/jira/browse/MAPREDUCE-279
>> .
> Hi Folks,
> We wanted to share our thoughts around the co-development of the NextGen
> MapReduce branch (Jira MR-279), maintaining the branch-0.20-security and
> merging the work on the security branch with trunk.  We've concluded that it
> does not make sense for us to port a very small subset of the work from the
> branch-0.20-security to the Hadoop mainline.  The JIRAs we don't plan to
> port all effect areas of the mainline that are going to be replaced by work
> in the NextGen MapReduce branch (
> http://svn.apache.org/viewvc/hadoop/mapreduce/branches/MR-279/).
> We've been working on the NextGen MapReduce branch (MAPREDUCE-279) within
> Apache for a while now and are excited about it's progress.  We think that
> this branch will be a huge improvement in scalability, performance and
> functionality.  We are now confident that we can get it ready for release in
> in the next few months.  We believe that the next major release of Apache
> Hadoop we will test at Yahoo will include the work in this branch and we are
> committed to merging the NextGen branch into the mainline after the PMC
> approves the merge.
> Meanwhile, we have continued to find and fix bugs on branch-0.20-security
> and have been working to port that work into the Hadoop mainline.  Most of
> this work is done and we've also brought all the patches in from our github
> branch into apache subversion, so that it is easy for everyone to see the
> work remaining.  What we've found is that some of the work in
> branch-0.20-security is in code sections that have been completely replaced
> / refactored in the NextGen MapReduce branch.  Since we are committed to the
> NextGen branch, we don't think there is any upside in porting this code into
> portions of mainline we expect to discard. All of these JIRAs will be fixed
> in the NextGen MapReduce branch and through there ultimately in trunk
> (assuming the PMC approves the merge).
> So at this point it is our intent to not port the JIRAs listed above to
> trunk, but to wait until we merge NextGen into trunk to resolve these issues
> there.  If you are interested in seeing these issues ported to mainline, let
> us know.  We are happy to help review your patches and explain context to
> anyone who is interested in doing this work.
> Arun and Eric

Todd Lipcon
Software Engineer, Cloudera

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message