Mailing-List: contact dev-help@reef.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@reef.apache.org
Date: Wed, 23 Mar 2016 18:04:25 +0000 (UTC)
From: "Ashvin (JIRA)" <jira@apache.org>
To: dev@reef.apache.org
Message-ID: <JIRA.12952896.1458751088000.32481.1458756265531@Atlassian.JIRA>
In-Reply-To: <JIRA.12952896.1458751088000@Atlassian.JIRA>
References: <JIRA.12952896.1458751088000@Atlassian.JIRA>
 <JIRA.12952896.1458751088803@arcas>
Subject: [jira] [Commented] (REEF-1274) Add validation to
 YarnConfigurationConstructor
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/REEF-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208886#comment-15208886 ] 

Ashvin commented on REEF-1274:
------------------------------

Early detection and reporting will be very useful. I think the use cases listed above will cover most scenarios. I have one other use cases. I am not sure if the use cases need separate discussion/jira threads. Many yarn app developers use {{MiniYarnCluster}} to unit test the application. In this case the RM address address in generated at runtime. Which means the address is not part of a config file in classpath. I am wondering if the test RM address could be provided as a configuration to {{YarnConfigurationConstructor}}.


> Add validation to YarnConfigurationConstructor
> ----------------------------------------------
>
>                 Key: REEF-1274
>                 URL: https://issues.apache.org/jira/browse/REEF-1274
>             Project: REEF
>          Issue Type: New Feature
>          Components: REEF-Runtime-YARN
>            Reporter: Markus Weimer
>
> A frequent problem REEF applications encounter is a broken or unavailable YARN Configuration of the cluster. REEF can help with the diagnose of such issues as we always load / instantiate the YARN Configuration via {{o..r.runtime.yarn.util.YarnConfigurationConstructor}}. In that class, we can validate the YARN configuration and report WARNINGs into the log if something is odd. For example:
>   * Is the RM IP set and reachable?
>   * Is the classpath set and non-empty? If not, are the environment variables set?
>   * Can a `FileSystem` instance be created using that configuration?
> There are probably more tests we can perform as we hear from our users.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)