hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sangjin Lee <sjl...@gmail.com>
Subject Re: Thinking ahead to hadoop-2.6
Date Wed, 24 Sep 2014 00:11:06 GMT
I am also +1 for the end of next week. YARN-1492 (shared cache) is near the
final stages of the review, and with a little more time it would make it
easier to put it in.

On Tue, Sep 23, 2014 at 4:48 PM, Subramaniam V K <subru.vk@gmail.com> wrote:

> +1 for end of next week.
>
> We have got all the patches for YARN-1051 committed to the branch. We are
> currently fixing test-patch & plan to call a merge vote soon. Hopefully it
> will go through by end of next week.
>
> Thanks,
> Subru
>
> On Tue, Sep 23, 2014 at 4:31 PM, Karthik Kambatla <kasha@cloudera.com>
> wrote:
>
> > Looking at the patches, we might be able to get most of YARN-1492 in by
> the
> > end of next week. There would be a couple of security items still
> > remaining, but we can may be call the feature alpha-ready without them.
> >
> > On Tue, Sep 23, 2014 at 4:25 PM, Chris Trezzo <ctrezzo@gmail.com> wrote:
> >
> > > I would like to see the shared cache (YARN-1492) make it in as well. We
> > are
> > > going through the final review process (with Karthik and Vinod) and
> > should
> > > be fairly close to complete. This is another feature that has been
> under
> > > development for a while.
> > >
> > > Thanks,
> > > Chris
> > >
> > > On Tue, Sep 23, 2014 at 4:00 PM, Suresh Srinivas <
> suresh@hortonworks.com
> > >
> > > wrote:
> > >
> > > > I actually would like to see both archival storage and single replica
> > > > memory writes to be in 2.6 release. Archival storage is in the final
> > > stages
> > > > of getting ready for branch-2 merge as Nicholas has already indicated
> > on
> > > > the dev mailing list. Hopefully HDFS-6581 gets ready sooner. Both of
> > > these
> > > > features are being in development for sometime.
> > > >
> > > > On Tue, Sep 23, 2014 at 3:27 PM, Andrew Wang <
> andrew.wang@cloudera.com
> > >
> > > > wrote:
> > > >
> > > > > Hey Arun,
> > > > >
> > > > > Maybe we could do a quick run through of the Roadmap wiki and
> > > > add/retarget
> > > > > things accordingly?
> > > > >
> > > > > I think the KMS and transparent encryption are ready to go. We've
> > got a
> > > > > very few further bug fixes pending, but that's it.
> > > > >
> > > > > Two HDFS things that I think probably won't make the end of the
> week
> > > are
> > > > > archival storage (HDFS-6584) and single replica memory writes
> > > > (HDFS-6581),
> > > > > which I believe are under the HSM banner. HDFS-6484 was just merged
> > to
> > > > > trunk and I think needs a little more work before it goes into
> > > branch-2.
> > > > > HDFS-6581 hasn't even been merged to trunk yet, so seems a bit
> > further
> > > > off
> > > > > yet.
> > > > >
> > > > > Just my 2c as I did not work directly on these features. I just
> > > generally
> > > > > shy away from shipping bits quite this fresh.
> > > > >
> > > > > Thanks,
> > > > > Andrew
> > > > >
> > > > > On Tue, Sep 23, 2014 at 3:03 PM, Arun Murthy <acm@hortonworks.com>
> > > > wrote:
> > > > >
> > > > > > Looks like most of the content is in and hadoop-2.6 is shaping
up
> > > > nicely.
> > > > > >
> > > > > > I'll create branch-2.6 by end of the week and we can go from
> there
> > to
> > > > > > stabilize it - hopefully in the next few weeks.
> > > > > >
> > > > > > Thoughts?
> > > > > >
> > > > > > thanks,
> > > > > > Arun
> > > > > >
> > > > > > On Tue, Aug 12, 2014 at 1:34 PM, Arun C Murthy <
> > acm@hortonworks.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Folks,
> > > > > > >
> > > > > > >  With hadoop-2.5 nearly done, it's time to start thinking
ahead
> > to
> > > > > > > hadoop-2.6.
> > > > > > >
> > > > > > >  Currently, here is the Roadmap per the wiki:
> > > > > > >
> > > > > > >         * HADOOP
> > > > > > >                 * Credential provider HADOOP-10607
> > > > > > >         * HDFS
> > > > > > >                 * Heterogeneous storage (Phase 2) - Support
> APIs
> > > for
> > > > > > using
> > > > > > > storage tiers by the applications HDFS-5682
> > > > > > >                 * Memory as storage tier HDFS-5851
> > > > > > >         * YARN
> > > > > > >                 * Dynamic Resource Configuration YARN-291
> > > > > > >                 * NodeManager Restart YARN-1336
> > > > > > >                 * ResourceManager HA Phase 2 YARN-556
> > > > > > >                 * Support for admin-specified labels in
YARN
> > > YARN-796
> > > > > > >                 * Support for automatic, shared cache for
YARN
> > > > > > application
> > > > > > > artifacts YARN-1492
> > > > > > >                 * Support NodeGroup layer topology on YARN
> > YARN-18
> > > > > > >                 * Support for Docker containers in YARN
> YARN-1964
> > > > > > >                 * YARN service registry YARN-913
> > > > > > >
> > > > > > >  My suspicion is, as is normal, some will make the cut
and some
> > > > won't.
> > > > > > > Please do add/subtract from the list as appropriate. Ideally,
> it
> > > > would
> > > > > be
> > > > > > > good to ship hadoop-2.6 in a 6-8 weeks (say, October) to
keep
> up
> > a
> > > > > > cadence.
> > > > > > >
> > > > > > >  More importantly, as we discussed previously, we'd like
> > hadoop-2.6
> > > > to
> > > > > be
> > > > > > > the *last* Apache Hadoop 2.x release which support JDK6.
I'll
> > > start a
> > > > > > > discussion with other communities (HBase, Pig, Hive, Oozie
> etc.)
> > > and
> > > > > see
> > > > > > > how they feel about this.
> > > > > > >
> > > > > > > thanks,
> > > > > > > Arun
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > --
> > > > > > Arun C. Murthy
> > > > > > Hortonworks Inc.
> > > > > > http://hortonworks.com/
> > > > > >
> > > > > > --
> > > > > > CONFIDENTIALITY NOTICE
> > > > > > NOTICE: This message is intended for the use of the individual
or
> > > > entity
> > > > > to
> > > > > > which it is addressed and may contain information that is
> > > confidential,
> > > > > > privileged and exempt from disclosure under applicable law.
If
> the
> > > > reader
> > > > > > of this message is not the intended recipient, you are hereby
> > > notified
> > > > > that
> > > > > > any printing, copying, dissemination, distribution, disclosure
or
> > > > > > forwarding of this communication is strictly prohibited. If
you
> > have
> > > > > > received this communication in error, please contact the sender
> > > > > immediately
> > > > > > and delete it from your system. Thank You.
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > http://hortonworks.com/download/
> > > >
> > > > --
> > > > CONFIDENTIALITY NOTICE
> > > > NOTICE: This message is intended for the use of the individual or
> > entity
> > > to
> > > > which it is addressed and may contain information that is
> confidential,
> > > > privileged and exempt from disclosure under applicable law. If the
> > reader
> > > > of this message is not the intended recipient, you are hereby
> notified
> > > that
> > > > any printing, copying, dissemination, distribution, disclosure or
> > > > forwarding of this communication is strictly prohibited. If you have
> > > > received this communication in error, please contact the sender
> > > immediately
> > > > and delete it from your system. Thank You.
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message