ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Ozerov (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (IGNITE-372) improve Ignite node Hadoop classpath composition reliability
Date Tue, 28 Jun 2016 08:42:57 GMT

     [ https://issues.apache.org/jira/browse/IGNITE-372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Vladimir Ozerov resolved IGNITE-372.
------------------------------------
    Resolution: Duplicate

Duplicate to IGNITE-3185.

> improve Ignite node Hadoop classpath composition reliability 
> -------------------------------------------------------------
>
>                 Key: IGNITE-372
>                 URL: https://issues.apache.org/jira/browse/IGNITE-372
>             Project: Ignite
>          Issue Type: Improvement
>    Affects Versions: sprint-1
>            Reporter: Ivan Veselovsky
>            Assignee: Ivan Veselovsky
>            Priority: Minor
>
> User encountered a problem with Hadoop mapred jobs execution when incorrectly set in
environment 
> {code}
> export HADOOP_MAPRED_HOME=${HADOOP_HOME}
> export HADOOP_COMMON_HOME=${HADOOP_HOME}
> export HADOOP_HDFS_HOME=${HADOOP_HOME}
> {code}
> . That caused incorrect GRIDGAIN_HADOOP_CLASSPATH and , as a consequence, job failure
with NoClassDefFoundError.
> Suggestions to improve reliability and tracability of the Hadoop classpath composition:
> 1) Simplify the logic of the classpath composition: do not take into account HADOOP_COMMON_HOME,
HADOOP_MAPRED_HOME, HADOOP_HDFS_HOME, also do not read /etc/default/hadoop .
> Compose the classpath basing only on the value of $HADOOP_HOME.
> 2) when GRIDGAIN_HADOOP_CLASSPATH composed, print out its value if ignite.sh as run with
"-v" option.
> 3) add the Hadoop classpath validation as a part of Node startup sequence: try to find
(load?) at least one class from each of the key Hadoop APIs (FS, MapRed, Yarn API).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message