hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: 0.21 Release
Date Fri, 19 Mar 2010 19:46:06 GMT
I want to bring up the issue about shuffler from thread:
Shuffle In Memory OutOfMemoryError

I was expecting improved performance from MAPREDUCE-1182

That would be one of the reasons for people to try out 0.21 release

Regards

On Fri, Mar 19, 2010 at 12:31 PM, Todd Lipcon <todd@cloudera.com> wrote:

> On Fri, Mar 19, 2010 at 12:17 PM, Jeremy Davis <jerdavis@speakeasy.net
> >wrote:
>
> > Thank you for your reply,
> >
> > So what would be your (or anyones) advice on getting, HDFS
> > sync/flush/append functionality?
> > You seem to indicate that 0.21 branch as is might not be the best idea,
> > with your preference being a patch set against 0.20.
> >
> > We definitely have an application for this specific functionality, and I
> > need to provide some direction/answers in this area for my colleagues.
> >
> > For example, I might say: We will install the current 0.21 now, and
> develop
> > against it.. But in X time we will install CDH2 (and get all the goodness
> it
> > brings), and then apply a given patch set against it. Is this in line
> with
> > what you are thinking? If so, could I get your perceived level of effort,
> > maybe a time frame? April/May/June/ etc..
> >
> >
> FYI the 0.20 sync patches mentioned by Stack will be going into CDH3 at
> some
> point this spring. This is mainly for the benefit of HBase but of course
> other applications will benefit as well.
>
> Thanks
> -Todd
>
>
> > I'm sure I'm not the only one that has plans for this feature, as it's in
> >  "Hadoop: the Definitive Guide" by O'Reilly published in September '09.
> >
> > Thanks,
> > -JD
> >
> >
> >
> >
> >
> > On Mar 19, 2010, at 11:51 AM, Stack wrote:
> >
> >  On Fri, Mar 19, 2010 at 9:43 AM, Jeremy Davis <jerdavis@speakeasy.net>
> >> wrote:
> >>
> >>> ...and I also saw release 0.21 as is with stack as the release manager.
> >>> Was
> >>> there a final decision on this off list?
> >>>
> >>>
> >> On the above, I was toying with the idea of being release manager for
> >> releasing as hadoop 0.21.0 what is in current 0.21 hadoop branch but I
> >> subsequently decided against it after chatting with folks and figuring
> >> that the only group that seemed interested in driving a release of the
> >> hadoop 0.21 branch was the hbase crew.  If I were to guess, an hadoop
> >> vouched for by a couple of hbasers with their spotty hdfs and
> >> mapreduce knowledge probably wouldn't have the penetration of a
> >> release backed by, say, a Yahoo.  No one would trust their data to
> >> such a release.  If no data in hadoop 0.21 clusters, hbase wouldn't
> >> have anything to run against.  So I let it go and figured time could
> >> be spent better elsewhere; e.g. helping test the set of patches that
> >> could get us a sync/flush/append on a patched hadoop 0.20 (hdfs-200,
> >> etc.).
> >>
> >> Sorry, I should have added a note to cited thread that I'd wandered...
> >>
> >> St.Ack
> >>
> >
> >
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message