hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2792) Have a public Test-only API for creating important records that ecosystem projects can depend on
Date Sat, 01 Nov 2014 12:24:34 GMT

    [ https://issues.apache.org/jira/browse/YARN-2792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14193126#comment-14193126

Steve Loughran commented on YARN-2792:

+1 for this, I've had to do some mock things and then got told off when changes break things.

Despite all the @Private tags, when you do YARN apps you end up using those internals. At
a quick glance through my App Master, {{NMClientAsyncImpl}} is one example, one lifted from
Distributed Shell, I believe. Also, {{ConverterUtils}}, {{WebApps}} and a few more, including
a reference to HDFS's {{DelegationTokenIdentifier}} when stripping tokens. 

Essentially, when you write a complex YARN app, you do end up working with things that are
private or "limited private", with mapreduce called out as within the limits. 

We really need to go through some of the main YARN apps and see what it is they are using
(and why), and then try and offer public alternatives with some guarantees of stability

> Have a public Test-only API for creating important records that ecosystem projects can
depend on
> ------------------------------------------------------------------------------------------------
>                 Key: YARN-2792
>                 URL: https://issues.apache.org/jira/browse/YARN-2792
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Vinod Kumar Vavilapalli
>            Priority: Blocker
> From YARN-2789,
> {quote}
> Sigh.
>     Even though this is a private API, it will be used by downstream projects for testing.
It'll be useful for this to be re-instated, maybe with a deprecated annotation, so that older
versions of downstream projects can build against Hadoop 2.6.
> I am inclined to have a separate test-only public util API that keeps compatibility for
tests. Rather than opening unwanted APIs up. I'll file a separate ticket for this, we need
all YARN apps/frameworks to move to that API instead of these private unstable APIs.
> For now, I am okay keeping a private compat for the APIs changed in YARN-2698.
> {quote}

This message was sent by Atlassian JIRA

View raw message