hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinod Kumar Vavilapalli <vino...@apache.org>
Subject Re: [DISCUSS] Looking to a 2.8.0 release
Date Wed, 25 Nov 2015 22:01:43 GMT
Tx for your comments, Andrew!

I did talk about it in a few discussions in the past related to this but yes, we never codified
the feature-level alpha/beta tags. Part of the reason why I never pushed for such a codification
is that (a) it is a subjective decision that the feature contributors usually have the best
say on and (2) voting on the alpha-ness / beta-ness may not be a productive exercise in non-trivial
number of cases (as I have seen with the release-level tags, some users think an alpha release
is of production quality enough for _their_ use-cases).

That said, I agree about noting down our general recommendations on what an alpha feature
means, what a beta feature means etc. Let me file a JIRA for this.

The second point you made is absolutely true. Atleast on YARN / MR side, I usually end up
traversing (some if not all of) alpha features and making sure the corresponding APIs are
explicitly marked private or public unstable / evolving. I do think that there is a lot of
value in us  getting more systematic with this - how about we do this for the feature list
of 2.8 and evolve the process?

In general, may be we could have a list of ‘check-list’ JIRAs that we always address before
every release. Few things already come to my mind:
 - Mark which features are alpha / beta and make sure the corresponding APIs, public interfaces
reflect the state
 - Revise all newly added configuration properties to make sure they follow our general naming
patterns. New contributors sometimes create non-standard properties that we come to regret
supporting.
 - Generate a list of newly added public entry-points and validate that they are all indeed
meant to be public
 - [...]

Thoughts?

+Vinod


> On Nov 25, 2015, at 11:47 AM, Andrew Wang <andrew.wang@cloudera.com> wrote:
> 
> Hey Vinod,
> 
> I'm fine with the idea of alpha/beta marking in the abstract, but had a
> question: do we define these terms in our compatibility policy or
> elsewhere? I think it's commonly understood among us developers (alpha
> means not fully tested and API unstable, beta means it's not fully tested
> but is API stable), but it'd be good to have it written down.
> 
> Also I think we've only done alpha/beta tagging at the release-level
> previously which is a simpler story to tell users. So it's important for
> this release that alpha features set their interface stability annotations
> to "evolving". There isn't a corresponding annotation for "interface
> quality", but IMO that's overkill.
> 
> Thanks,
> Andrew
> 
> On Wed, Nov 25, 2015 at 11:08 AM, Vinod Kumar Vavilapalli <
> vinodkv@apache.org> wrote:
> 
>> This is the current state from the feedback I gathered.
>> - Support priorities across applications within the same queue YARN-1963
>>    — Can push as an alpha / beta feature per Sunil
>> - YARN-1197 Support changing resources of an allocated container:
>>    — Can push as an alpha/beta feature per Wangda
>> - YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
>> most of it anyways.
>>    — Can push as an alpha feature.
>> - YARN Timeline Service v1.5 - YARN-4233
>>    — Should include per Li Lu
>> - YARN Timeline Service Next generation: YARN-2928
>>    — Per analysis from Sangjin, drop this from 2.8.
>> 
>> One open feature status
>> - HDFS-8155    Support OAuth2 in WebHDFS: Alpha / Early feature?
>> 
>> Updated the Roadmap wiki with the same.
>> 
>> Thanks
>> +Vinod
>> 
>>> On Nov 13, 2015, at 12:12 PM, Sangjin Lee <sjlee@apache.org> wrote:
>>> 
>>> I reviewed the current state of the YARN-2928 changes regarding its
>> impact
>>> if the timeline service v.2 is disabled. It does appear that there are a
>>> lot of things that still do get created and enabled unconditionally
>>> regardless of configuration. While this is understandable when we were
>>> working to implement the feature, this clearly needs to be cleaned up so
>>> that when disabled the timeline service v.2 doesn't impact other things.
>>> 
>>> I filed a JIRA for that work:
>>> https://issues.apache.org/jira/browse/YARN-4356
>>> 
>>> We need to complete it before we can merge.
>>> 
>>> Somewhat related is the status of the configuration and what it means in
>>> various contexts (client/app-side vs. server-side, v.1 vs. v.2, etc.). I
>>> know there is an ongoing discussion regarding YARN-4183. We'll need to
>>> reflect the outcome of that discussion.
>>> 
>>> My overall impression of whether this can be done for 2.8 is that it
>> looks
>>> rather challenging given the suggested timeframe. We also need to
>> complete
>>> several major tasks before it is ready.
>>> 
>>> Sangjin
>>> 
>>> 
>>> On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sjlee0@gmail.com> wrote:
>>> 
>>>> 
>>>> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
>>>> vinodkv@hortonworks.com> wrote:
>>>> 
>>>>>   — YARN Timeline Service Next generation: YARN-2928: Lots of
>> momentum,
>>>>> but clearly a work in progress. Two options here
>>>>>       — If it is safe to ship it into 2.8 in a disable manner, we can
>>>>> get the early code into trunk and all the way int o2.8.
>>>>>       — If it is not safe, it organically rolls over into 2.9
>>>>> 
>>>> 
>>>> I'll review the changes on YARN-2928 to see what impact it has (if any)
>> if
>>>> the timeline service v.2 is disabled.
>>>> 
>>>> Another condition for it to make 2.8 is whether the branch will be in a
>>>> shape in a couple of weeks such that it adds value for folks that want
>> to
>>>> test it. Hopefully it will become clearer soon.
>>>> 
>>>> Sangjin
>>>> 
>> 
>> 


Mime
View raw message