hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Kimball <akimbal...@gmail.com>
Subject Re: [VOTE] Abandon mrunit MapReduce contrib
Date Tue, 22 Feb 2011 01:54:41 GMT
I have made some revisions and improvements here too..
- Aaron

On Sat, Feb 19, 2011 at 6:52 PM, Mattmann, Chris A (388J) <
chris.a.mattmann@jpl.nasa.gov> wrote:

> Hi Guys,
>
> I took a pass at the MRUnit proposal [1]. It's far from complete, but just
> letting you know I took a swipe :)
>
> Cheers,
> Chris
>
> [1] http://wiki.apache.org/incubator/MRUnitProposal
>
> On Feb 17, 2011, at 1:52 PM, Mattmann, Chris A (388J) wrote:
>
> > Thanks Patrick.
> >
> > Sounds good. Besides mentors, we need a champion, who must be an ASF
> member. So, either me, you or Nigel can do it. I'm already championing Gora,
> and would be happy to champion MRUnit, but am open to either one of you guys
> doing it too (or even another ASF member). Just let me know. I'll head over
> to the wiki and add my info. Thanks for getting this started Eric+Patrick!
> >
> > Cheers,
> > Chris
> >
> > On Feb 17, 2011, at 1:00 PM, Patrick Hunt wrote:
> >
> >> Chris a page is up (still being created by Eric afaict):
> >> http://wiki.apache.org/incubator/MRUnitProposal
> >>
> >> I took the liberty of listing us as mentors.
> >>
> >> Patrick
> >>
> >> On Thu, Feb 17, 2011 at 11:31 AM, Mattmann, Chris A (388J)
> >> <chris.a.mattmann@jpl.nasa.gov> wrote:
> >>> Hey Guys,
> >>>
> >>> FYI on this: Eric has mentioned he is going to start the Incubator
> proposal for MRUnit. Let's start small and then grow big (as needed). It
> seems like we've achieved enough consensus for the required mentors and
> critical mass to make an MRUnit Incubator proposal and then to have the
> Incubator community weigh in. If that expands to include other testing
> projects/etc., we can address that over the Incubation process, and as
> needed.
> >>>
> >>> Eric: as soon as that wiki page is up, I'd be happy to add my name to
> it as a mentor and /kick the can on this.
> >>>
> >>> Cheers,
> >>> Chris
> >>>
> >>> On Feb 17, 2011, at 11:11 AM, Aaron Kimball wrote:
> >>>
> >>>> The MRUnit community is a specific subset of the Hadoop community:
> Engineers
> >>>> writing Java code running on Hadoop. The Hadoop community also
> includes
> >>>> IT/ops staff who maintain Hadoop clusters, data scientists who use
> tools
> >>>> such as Pig & Hive, as well as those written by the aforementioned
> >>>> engineers, etc.
> >>>>
> >>>> The Hadoop project has long recognized that tools aimed at a specific
> subset
> >>>> of the Hadoop community, with separate release cycles, can more
> successfully
> >>>> reach their aims by splitting into incubator projects. Hive, Pig, and
> HBase,
> >>>> for example, have all gone this path.
> >>>>
> >>>> A "current" version of MRUnit would need to compile against multiple
> >>>> versions of Hadoop itself. This is not possible if it is in the same
> source
> >>>> tree as Hadoop.
> >>>>
> >>>> - Aaron
> >>>>
> >>>> On Thu, Feb 17, 2011 at 5:31 AM, Bernd Fondermann <
> >>>> bernd.fondermann@googlemail.com> wrote:
> >>>>
> >>>>> On Fri, Feb 11, 2011 at 23:10, Aaron Kimball <akimball83@gmail.com>
> wrote:
> >>>>>> The main reason I am interested in removing MRUnit from Hadoop
is
> that I
> >>>>>> believe that MRUnit deserves its own release cycle. I think
this is
> in
> >>>>> the
> >>>>>> best interest of its users.
> >>>>>
> >>>>> Not in mine, at least. (I'm writing MR unit tests.)
> >>>>> Many projects release more than one product. I'd rather get MRUnit
> >>>>> from the same source where I get my MR from.
> >>>>> Separate release cylcles would be ok for me, though.
> >>>>>
> >>>>>> Perhaps more importantly, access to new features in MRUnit should
> not
> >>>>>> require upgrading one's entire Hadoop deployment; this is a
client
> >>>>> library
> >>>>>> that depends only on Hadoop's public APIs.
> >>>>>
> >>>>> +1.
> >>>>>
> >>>>>> My primary concern is to move MRUnit to a place where the community
> can
> >>>>>> derive the most benefit from it. The Apache Incubator could
fulfill
> this
> >>>>>> role; given the presence of individuals willing to mentor this
> project, I
> >>>>>> believe this would be a successful way to release MRUnit more
> quickly and
> >>>>>> continue to work to grow the MRUnit community.
> >>>>>
> >>>>> What are your expectations what MRUnit would become, software-wise?
> >>>>> Wouldn't the MRUnit community be largely the same as the Hadoop-MR
> >>>>> community?
> >>>>>
> >>>>> Bernd
> >>>>>
> >>>
> >>>
> >>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>> Chris Mattmann, Ph.D.
> >>> Senior Computer Scientist
> >>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> >>> Office: 171-266B, Mailstop: 171-246
> >>> Email: chris.a.mattmann@nasa.gov
> >>> WWW:   http://sunset.usc.edu/~mattmann/
> >>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>> Adjunct Assistant Professor, Computer Science Department
> >>> University of Southern California, Los Angeles, CA 90089 USA
> >>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>>
> >>>
> >
> >
> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> > Chris Mattmann, Ph.D.
> > Senior Computer Scientist
> > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> > Office: 171-266B, Mailstop: 171-246
> > Email: chris.a.mattmann@nasa.gov
> > WWW:   http://sunset.usc.edu/~mattmann/
> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> > Adjunct Assistant Professor, Computer Science Department
> > University of Southern California, Los Angeles, CA 90089 USA
> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >
>
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Senior Computer Scientist
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 171-266B, Mailstop: 171-246
> Email: chris.a.mattmann@nasa.gov
> WWW:   http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Assistant Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message