reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Markus Weimer (JIRA)" <j...@apache.org>
Subject [jira] [Created] (REEF-1274) Add validation to YarnConfigurationConstructor
Date Wed, 23 Mar 2016 16:38:25 GMT
Markus Weimer created REEF-1274:
-----------------------------------

             Summary: Add validation to YarnConfigurationConstructor
                 Key: REEF-1274
                 URL: https://issues.apache.org/jira/browse/REEF-1274
             Project: REEF
          Issue Type: New Feature
          Components: REEF-Runtime-YARN
            Reporter: Markus Weimer


A frequent problem REEF applications encounter is a broken or unavailable YARN Configuration
of the cluster. REEF can help with the diagnose of such issues as we always load / instantiate
the YARN Configuration via {{o..r.runtime.yarn.util.YarnConfigurationConstructor}}. In that
class, we can validate the YARN configuration and report WARNINGs into the log if something
is odd. For example:

  * Is the RM IP set and reachable?
  * Is the classpath set and non-empty? If not, are the environment variables set?
  * Can a `FileSystem` instance be created using that configuration?

There are probably more tests we can perform as we hear from our users.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message