Mailing-List: contact common-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-dev@hadoop.apache.org
MIME-Version: 1.0
In-Reply-To: <0C22896E-D77B-4DE4-99A2-9D81E150779C@hortonworks.com>
References: 
 <CAMyYaRKomwk4L6ZupB1-V8d0EpYZzTDaHsQO4aWyXMPb8=y1Vg@mail.gmail.com>
 <A16F3F9A-2BE4-4FD1-A400-0A7892302C20@hortonworks.com>
 <CACO5Y4wQSkaZqz2CAypo1XVQWsXKxN8tJJeY=jhBqcOTQcF6+Q@mail.gmail.com>
 <CA+qbEUNR_Xg0E=Hc+kx+4NGqJeFh6d502bzf4+bBq=Z81ib7LQ@mail.gmail.com>
 <0C22896E-D77B-4DE4-99A2-9D81E150779C@hortonworks.com>
From: Wangda Tan <wheeleast@gmail.com>
Date: Wed, 11 Nov 2015 13:28:16 -0800
Message-ID: 
 <CAD++eCm=bxxcpJFW4_bocJO19HuWEDj+ScSFZd=PZf1NJoVu9g@mail.gmail.com>
Subject: Re: [DISCUSS] Looking to a 2.8.0 release
To: "mapreduce-dev@hadoop.apache.org" <mapreduce-dev@hadoop.apache.org>
Cc: Hadoop Common <common-dev@hadoop.apache.org>,
	"yarn-dev@hadoop.apache.org" <yarn-dev@hadoop.apache.org>,
	"hdfs-dev@hadoop.apache.org" <hdfs-dev@hadoop.apache.org>,
 "vinodkv@apache.org" <vinodkv@apache.org>
Content-Type: multipart/alternative; boundary=001a11c1303eeb3afd05244a8249

--001a11c1303eeb3afd05244a8249
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Thanks to Vinod for starting this discussion!

+1 to add YARN-1197 (container resizing) to 2.8.0, it is end-to-end tested.
I'd prefer to push it as an Alpha feature before wilder testing.

And also agree to call first release of 2.8 as an Alpha release according
to the number of new features / code changes.

Regards,
Wangda

On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <vinodkv@hortonworks.co=
m
> wrote:

> Agreed on not mixing this with major release discussions.
>
> Okay, I just finished my review of 2.8 content.
>
> A quick summary follows.
>
> Current state of originally planned items
>
>  - Nearly Done / Done and so need to close down quickly
>     =E2=80=94 Support *both* JDK7 and JDK8 runtimes HADOOP-11090
>     =E2=80=94 Supporting non-exclusive node-labels: YARN-3214: Done, can =
push as
> an alpha / beta feature
>     =E2=80=94 Support priorities across applications within the same queu=
e
> YARN-1963: Can push as an alpha / beta feature
>
>  - Definitely have to move out into 2.9 and beyond
>     =E2=80=94 Early work for disk and network isolation in YARN: YARN-213=
9,
> YARN-2140: Early noise, some critical pieces designed, done but not a lot
> of movement of late
>     =E2=80=94 Classpath isolation for downstream clients HADOOP-11656: Lo=
ts of
> chatter a while ago, not much movement of late
>     =E2=80=94 Support for Erasure Codes in HDFS HDFS-7285<
> https://issues.apache.org/jira/browse/HDFS-7285>: Moved out to 2.9 in the
> interest of stability / bake-in
>
> Non-planned features that went into 2.8.0
>
>     =E2=80=94 The overall list of new features:
> https://issues.apache.org/jira/issues/?filter=3D12333994
>     =E2=80=94 HDFS-6200 Create a separate jar for hdfs-client: Compatible
> improvement - no dimension of alpha/betaness here.
>     =E2=80=94 HDFS-8155    Support OAuth2 in WebHDFS: Alpha / Early featu=
re?
>     =E2=80=94 Stability improvements ready to use:
>         =E2=80=94 HDFS-8008    Support client-side back off when the data=
nodes are
> congested
>         =E2=80=94 HDFS-8009    Signal congestion on the DataNode
>     =E2=80=94 YARN-3611 Support Docker Containers In LinuxContainerExecut=
or: Well
> most of it anyways. Can push as an alpha feature.
>     =E2=80=94 YARN-1197 Support changing resources of an allocated contai=
ner: Can
> push as an alpha/beta feature
>
> Items in progress to think about in 2 weeks
>
>     =E2=80=94 YARN Timeline Service v1.5 - YARN-4233: A short term bridge=
 before
> YARN-2928 comes around. I think this should go in given the tremendous
> activity recently.
>     =E2=80=94 YARN Timeline Service Next generation: YARN-2928: Lots of m=
omentum,
> but clearly a work in progress. Two options here
>         =E2=80=94 If it is safe to ship it into 2.8 in a disable manner, =
we can
> get the early code into trunk and all the way int o2.8.
>         =E2=80=94 If it is not safe, it organically rolls over into 2.9
>     =E2=80=94 Compatibility tools to catch backwards, forwards compatibil=
ity
> issues at patch submission, release times. Some of it is captured at
> YARN-3292. This also involves resurrecting jdiff
> (HADOOP-11776/YARN-3426/MAPREDUCE-6310) and/or investing in new tools.
>
> This is my plan of action for now in terms of the release itself
>
>  - Cut a branch about two weeks from now
>  - Do an RC mid next month (leaving ~4weeks since branch-cut)
>  - As with 2.7.x series, the first release will still be called as early =
/
> alpha release in the interest of
>     =E2=80=94 gaining downstream adoption
>     =E2=80=94 wider testing,
>     =E2=80=94 yet reserving our right to fix any inadvertent incompatibil=
ities
> introduced.
>
> If we can get answers on =E2=80=9CItems to think about now=E2=80=9D durin=
g this and next
> week, we will overall be in good shape.
>
> Thoughts?
>
> Thanks
> +Vinod
> PS:As you may have noted above, this time around, I want to do something
> that we=E2=80=99ve always wanted to do, but never explicitly did. I=E2=80=
=99m calling out
> readiness of each feature as they stand today so we can inform our users
> better of what they can start relying on in production clusters.
>
>
> On Oct 5, 2015, at 11:53 AM, Colin P. McCabe <cmccabe@apache.org<mailto:
> cmccabe@apache.org>> wrote:
>
> I think it makes sense to have a 2.8 release since there are a
> tremendous number of JIRAs in 2.8 that are not in 2.7.  Doing a 3.x
> release seems like something we should consider separately since it
> would not have the same compatibility guarantees as a 2.8 release.
> There's a pretty big delta between trunk and 2.8 as well.
>
> cheers,
> Colin
>
> On Sat, Sep 26, 2015 at 1:36 PM, Chris Douglas <cdouglas@apache.org
> <mailto:cdouglas@apache.org>> wrote:
> With two active sustaining branches (2.6, 2.7), what would you think
> of releasing trunk as 3.x instead of pushing 2.8? There are many new
> features (EC, Y1197, etc.), and trunk could be the source of several
> alpha/beta releases before we fork the 3.x line. -C
>
> On Sat, Sep 26, 2015 at 12:49 PM, Vinod Vavilapalli
> <vinodkv@hortonworks.com<mailto:vinodkv@hortonworks.com>> wrote:
> As you may have noted, 2.8.0 got completely derailed what with 2.7.x and
> the unusually long 2.6.1 release.
>
> With 2.6.1 out of the way, and two parallel threads in progress for 2.6.2
> and 2.7.2, it=E2=80=99s time for us to look back at where we are with Had=
oop 2.8.
>
> I=E2=80=99ll do a quick survey of where the individual features are and t=
he amount
> of content already present in 2.8 and kick-start 2.8.0 process again.
>
> +Vinod
>
>
> On Apr 21, 2015, at 2:39 PM, vinodkv@apache.org<mailto:vinodkv@apache.org=
>
> wrote:
>
> With 2.7.0 out of the way, and with more maintenance releases to stabiliz=
e
> it, I propose we start thinking about 2.8.0.
>
> Here's my first cut of the proposal, will update the Roadmap wiki.
> - Support *both* JDK7 and JDK8 runtimes: HADOOP-11090
> - Compatibility tools to catch backwards, forwards compatibility issues a=
t
> patch submission, release times. Some of it is captured at YARN-3292. Thi=
s
> also involves resurrecting jdiff (HADOOP-11776/YARN-3426/MAPREDUCE-6310)
> and/or investing in new tools.
> - HADOOP-11656 Classpath isolation for downstream clients
> - Support for Erasure Codes in HDFS HDFS-7285
> - Early work for disk and network isolation in YARN: YARN-2139, YARN-2140
> - YARN Timeline Service Next generation: YARN-2928. At least branch-merge
> + early peek.
> - Supporting non-exclusive node-labels: YARN-3214
>
> I'm experimenting with more agile 2.7.x releases and would like to
> continue the same by volunteering as the RM for 2.8.x too.
>
> Given the long time we took with 2.7.0, the timeline I am looking at is
> 8-12 weeks. We can pick as many features as they finish along and make a
> more predictable releases instead of holding up releases for ever.
>
> Thoughts?
>
> Thanks
> +Vinod
>
>
>
>

--001a11c1303eeb3afd05244a8249--