hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: [ANNOUNCEMENT] Yahoo focusing on Apache Hadoop, discontinuing "The Yahoo Distribution of Hadoop"
Date Thu, 07 Apr 2011 23:22:52 GMT
Is there a list available of which patches you've made this decision about?
I'm curious, for example, about MAPREDUCE-2178 -- as of today, the MR
security in trunk has a serious vulnerability. Do we plan on fixing it, or
will the answer be that, if anyone needs security, they must update to "MR
Next Gen"?

-Todd

On Thu, Apr 7, 2011 at 3:52 PM, Arun C Murthy <acm@yahoo-inc.com> wrote:

>
> On Feb 14, 2011, at 1:34 PM, Arun C Murthy wrote:
>
>>
>> As the final installment in this process, I've started a discussion on
>> us contributing a re-factor of Map-Reduce in
>> https://issues.apache.org/jira/browse/MAPREDUCE-279
>> .
>>
>
>
>
> Hi Folks,
>
> We wanted to share our thoughts around the co-development of the NextGen
> MapReduce branch (Jira MR-279), maintaining the branch-0.20-security and
> merging the work on the security branch with trunk.  We've concluded that it
> does not make sense for us to port a very small subset of the work from the
> branch-0.20-security to the Hadoop mainline.  The JIRAs we don't plan to
> port all effect areas of the mainline that are going to be replaced by work
> in the NextGen MapReduce branch (
> http://svn.apache.org/viewvc/hadoop/mapreduce/branches/MR-279/).
>
> We've been working on the NextGen MapReduce branch (MAPREDUCE-279) within
> Apache for a while now and are excited about it's progress.  We think that
> this branch will be a huge improvement in scalability, performance and
> functionality.  We are now confident that we can get it ready for release in
> in the next few months.  We believe that the next major release of Apache
> Hadoop we will test at Yahoo will include the work in this branch and we are
> committed to merging the NextGen branch into the mainline after the PMC
> approves the merge.
>
> Meanwhile, we have continued to find and fix bugs on branch-0.20-security
> and have been working to port that work into the Hadoop mainline.  Most of
> this work is done and we've also brought all the patches in from our github
> branch into apache subversion, so that it is easy for everyone to see the
> work remaining.  What we've found is that some of the work in
> branch-0.20-security is in code sections that have been completely replaced
> / refactored in the NextGen MapReduce branch.  Since we are committed to the
> NextGen branch, we don't think there is any upside in porting this code into
> portions of mainline we expect to discard. All of these JIRAs will be fixed
> in the NextGen MapReduce branch and through there ultimately in trunk
> (assuming the PMC approves the merge).
>
> So at this point it is our intent to not port the JIRAs listed above to
> trunk, but to wait until we merge NextGen into trunk to resolve these issues
> there.  If you are interested in seeing these issues ported to mainline, let
> us know.  We are happy to help review your patches and explain context to
> anyone who is interested in doing this work.
>
> Arun and Eric
>



-- 
Todd Lipcon
Software Engineer, Cloudera

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message