Return-Path: X-Original-To: apmail-hadoop-common-dev-archive@www.apache.org Delivered-To: apmail-hadoop-common-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 651A418142 for ; Wed, 25 Nov 2015 22:01:50 +0000 (UTC) Received: (qmail 49005 invoked by uid 500); 25 Nov 2015 22:01:46 -0000 Delivered-To: apmail-hadoop-common-dev-archive@hadoop.apache.org Received: (qmail 48750 invoked by uid 500); 25 Nov 2015 22:01:45 -0000 Mailing-List: contact common-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-dev@hadoop.apache.org Delivered-To: mailing list common-dev@hadoop.apache.org Received: (qmail 48709 invoked by uid 99); 25 Nov 2015 22:01:45 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 25 Nov 2015 22:01:45 +0000 Received: from spacestar.att.net (107-222-184-213.lightspeed.sntcca.sbcglobal.net [107.222.184.213]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id 364DF1A0015; Wed, 25 Nov 2015 22:01:45 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) Subject: Re: [DISCUSS] Looking to a 2.8.0 release From: Vinod Kumar Vavilapalli In-Reply-To: Date: Wed, 25 Nov 2015 14:01:43 -0800 Cc: "hdfs-dev@hadoop.apache.org" , "mapreduce-dev@hadoop.apache.org" , Hadoop Common Content-Transfer-Encoding: quoted-printable Message-Id: References: <0C22896E-D77B-4DE4-99A2-9D81E150779C@hortonworks.com> To: yarn-dev@hadoop.apache.org X-Mailer: Apple Mail (2.2104) Tx for your comments, Andrew! I did talk about it in a few discussions in the past related to this but = yes, we never codified the feature-level alpha/beta tags. Part of the = reason why I never pushed for such a codification is that (a) it is a = subjective decision that the feature contributors usually have the best = say on and (2) voting on the alpha-ness / beta-ness may not be a = productive exercise in non-trivial number of cases (as I have seen with = the release-level tags, some users think an alpha release is of = production quality enough for _their_ use-cases). That said, I agree about noting down our general recommendations on what = an alpha feature means, what a beta feature means etc. Let me file a = JIRA for this. The second point you made is absolutely true. Atleast on YARN / MR side, = I usually end up traversing (some if not all of) alpha features and = making sure the corresponding APIs are explicitly marked private or = public unstable / evolving. I do think that there is a lot of value in = us getting more systematic with this - how about we do this for the = feature list of 2.8 and evolve the process? In general, may be we could have a list of =E2=80=98check-list=E2=80=99 = JIRAs that we always address before every release. Few things already = come to my mind: - Mark which features are alpha / beta and make sure the corresponding = APIs, public interfaces reflect the state - Revise all newly added configuration properties to make sure they = follow our general naming patterns. New contributors sometimes create = non-standard properties that we come to regret supporting. - Generate a list of newly added public entry-points and validate that = they are all indeed meant to be public - [...] Thoughts? +Vinod > On Nov 25, 2015, at 11:47 AM, Andrew Wang = wrote: >=20 > Hey Vinod, >=20 > I'm fine with the idea of alpha/beta marking in the abstract, but had = a > question: do we define these terms in our compatibility policy or > elsewhere? I think it's commonly understood among us developers (alpha > means not fully tested and API unstable, beta means it's not fully = tested > but is API stable), but it'd be good to have it written down. >=20 > Also I think we've only done alpha/beta tagging at the release-level > previously which is a simpler story to tell users. So it's important = for > this release that alpha features set their interface stability = annotations > to "evolving". There isn't a corresponding annotation for "interface > quality", but IMO that's overkill. >=20 > Thanks, > Andrew >=20 > On Wed, Nov 25, 2015 at 11:08 AM, Vinod Kumar Vavilapalli < > vinodkv@apache.org> wrote: >=20 >> This is the current state from the feedback I gathered. >> - Support priorities across applications within the same queue = YARN-1963 >> =E2=80=94 Can push as an alpha / beta feature per Sunil >> - YARN-1197 Support changing resources of an allocated container: >> =E2=80=94 Can push as an alpha/beta feature per Wangda >> - YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well >> most of it anyways. >> =E2=80=94 Can push as an alpha feature. >> - YARN Timeline Service v1.5 - YARN-4233 >> =E2=80=94 Should include per Li Lu >> - YARN Timeline Service Next generation: YARN-2928 >> =E2=80=94 Per analysis from Sangjin, drop this from 2.8. >>=20 >> One open feature status >> - HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature? >>=20 >> Updated the Roadmap wiki with the same. >>=20 >> Thanks >> +Vinod >>=20 >>> On Nov 13, 2015, at 12:12 PM, Sangjin Lee wrote: >>>=20 >>> I reviewed the current state of the YARN-2928 changes regarding its >> impact >>> if the timeline service v.2 is disabled. It does appear that there = are a >>> lot of things that still do get created and enabled unconditionally >>> regardless of configuration. While this is understandable when we = were >>> working to implement the feature, this clearly needs to be cleaned = up so >>> that when disabled the timeline service v.2 doesn't impact other = things. >>>=20 >>> I filed a JIRA for that work: >>> https://issues.apache.org/jira/browse/YARN-4356 >>>=20 >>> We need to complete it before we can merge. >>>=20 >>> Somewhat related is the status of the configuration and what it = means in >>> various contexts (client/app-side vs. server-side, v.1 vs. v.2, = etc.). I >>> know there is an ongoing discussion regarding YARN-4183. We'll need = to >>> reflect the outcome of that discussion. >>>=20 >>> My overall impression of whether this can be done for 2.8 is that it >> looks >>> rather challenging given the suggested timeframe. We also need to >> complete >>> several major tasks before it is ready. >>>=20 >>> Sangjin >>>=20 >>>=20 >>> On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee = wrote: >>>=20 >>>>=20 >>>> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli < >>>> vinodkv@hortonworks.com> wrote: >>>>=20 >>>>> =E2=80=94 YARN Timeline Service Next generation: YARN-2928: Lots = of >> momentum, >>>>> but clearly a work in progress. Two options here >>>>> =E2=80=94 If it is safe to ship it into 2.8 in a disable = manner, we can >>>>> get the early code into trunk and all the way int o2.8. >>>>> =E2=80=94 If it is not safe, it organically rolls over into = 2.9 >>>>>=20 >>>>=20 >>>> I'll review the changes on YARN-2928 to see what impact it has (if = any) >> if >>>> the timeline service v.2 is disabled. >>>>=20 >>>> Another condition for it to make 2.8 is whether the branch will be = in a >>>> shape in a couple of weeks such that it adds value for folks that = want >> to >>>> test it. Hopefully it will become clearer soon. >>>>=20 >>>> Sangjin >>>>=20 >>=20 >>=20