giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jose Luis Larroque (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (GIRAPH-1030) Stop using mapred.job.id (is deprecated). Instead, start using mapreduce.job.id
Date Sun, 30 Aug 2015 15:21:45 GMT

     [ https://issues.apache.org/jira/browse/GIRAPH-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jose Luis Larroque updated GIRAPH-1030:
---------------------------------------
    Description: 
When i'm starting a new Giraph Job, there is always a warning about a deprecated property,
and that property is being programmatically set by GiraphYarnClient class(GiraphYarnClient.java,
line 196, http://grepcode.com/file/repo1.maven.org/maven2/org.apache.giraph/giraph-core/1.1.0/org/apache/giraph/yarn/GiraphYarnClient.java#196).

It is not being taken from a XML config file and it's very confusing for newbies, because
it can't be easily fixed.

It will be nice if the non-deprecated property (mapreduce.job.id) can be used instead

A brief (and very generic) log with the problem:
15/08/29 12:42:17 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your
platform... using builtin-java classes where applicable
15/08/29 12:42:18 INFO utils.ConfigurationUtils: No edge input format specified. Ensure your
InputFormat does not require one.
15/08/29 12:42:18 INFO utils.ConfigurationUtils: No edge output format specified. Ensure your
OutputFormat does not require one.
15/08/29 12:42:18 INFO yarn.GiraphYarnClient: Final output path is: hdfs://hdnode01:54310/user/hduser/output/caminosNavegacionales1
15/08/29 12:42:18 INFO yarn.GiraphYarnClient: Running Client
15/08/29 12:42:18 INFO client.RMProxy: Connecting to ResourceManager at hdnode01/192.168.0.10:8050
15/08/29 12:42:18 INFO yarn.GiraphYarnClient: Got node report from ASM for, nodeId=hadoop100:51831,
nodeAddress hadoop100:8042, nodeRackName /default-rack, nodeNumContainers 0
15/08/29 12:42:18 INFO yarn.GiraphYarnClient: Got node report from ASM for, nodeId=hadoop101:40809,
nodeAddress hadoop101:8042, nodeRackName /default-rack, nodeNumContainers 0
15/08/29 12:42:18 INFO yarn.GiraphYarnClient: Obtained new Application ID: application_1440862856801_0001
15/08/29 12:42:18 INFO Configuration.deprecation: mapred.job.id is deprecated. Instead, use
mapreduce.job.id
15/08/29 12:42:18 INFO yarn.GiraphYarnClient: Set the environment for the application master
...

  was:
When i'm starting a new Giraph Job, there is always a warning about a deprecated property,
and that property is being programmatically set by GiraphYarnClient class(GiraphYarnClient.java,
line 196, http://grepcode.com/file/repo1.maven.org/maven2/org.apache.giraph/giraph-core/1.1.0/org/apache/giraph/yarn/GiraphYarnClient.java#196).

It is not being taken from a XML config file and it's very confusing for newbies, because
it can't be easily fixed.

It will be nice if the non-deprecated property (mapreduce.job.id) can be used instead

A brief (and very generic) log with the problem:
15/08/29 12:42:17 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your
platform... using builtin-java classes where applicable
15/08/29 12:42:18 INFO utils.ConfigurationUtils: No edge input format specified. Ensure your
InputFormat does not require one.
15/08/29 12:42:18 INFO utils.ConfigurationUtils: No edge output format specified. Ensure your
OutputFormat does not require one.
15/08/29 12:42:18 INFO yarn.GiraphYarnClient: Final output path is: hdfs://hdnode01:54310/user/hduser/output/caminosNavegacionales1
15/08/29 12:42:18 INFO yarn.GiraphYarnClient: Running Client
15/08/29 12:42:18 INFO client.RMProxy: Connecting to ResourceManager at hdnode01/192.168.0.10:8050
15/08/29 12:42:18 INFO yarn.GiraphYarnClient: Got node report from ASM for, nodeId=hadoop100:51831,
nodeAddress hadoop100:8042, nodeRackName /default-rack, nodeNumContainers 0
15/08/29 12:42:18 INFO yarn.GiraphYarnClient: Got node report from ASM for, nodeId=hadoop101:40809,
nodeAddress hadoop101:8042, nodeRackName /default-rack, nodeNumContainers 0
15/08/29 12:42:18 INFO yarn.GiraphYarnClient: Obtained new Application ID: application_1440862856801_0001
15/08/29 12:42:18 INFO Configuration.deprecation: mapred.job.id is deprecated. Instead, use
mapreduce.job.id
15/08/29 12:42:18 INFO yarn.GiraphYarnClient: Set the environment for the application master
15/08/29 12:42:18 INFO yarn.GiraphYarnClient: Environment for AM :{CLASSPATH=${CLASSPATH}:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*}
...


> Stop using mapred.job.id (is deprecated). Instead, start using mapreduce.job.id
> -------------------------------------------------------------------------------
>
>                 Key: GIRAPH-1030
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-1030
>             Project: Giraph
>          Issue Type: Improvement
>          Components: conf and scripts
>    Affects Versions: 1.1.0
>         Environment: Ubuntu 14.04 LTS, Hadoop 2.4.0
>            Reporter: Jose Luis Larroque
>            Priority: Trivial
>              Labels: newbie
>
> When i'm starting a new Giraph Job, there is always a warning about a deprecated property,
and that property is being programmatically set by GiraphYarnClient class(GiraphYarnClient.java,
line 196, http://grepcode.com/file/repo1.maven.org/maven2/org.apache.giraph/giraph-core/1.1.0/org/apache/giraph/yarn/GiraphYarnClient.java#196).
> It is not being taken from a XML config file and it's very confusing for newbies, because
it can't be easily fixed.
> It will be nice if the non-deprecated property (mapreduce.job.id) can be used instead
> A brief (and very generic) log with the problem:
> 15/08/29 12:42:17 WARN util.NativeCodeLoader: Unable to load native-hadoop library for
your platform... using builtin-java classes where applicable
> 15/08/29 12:42:18 INFO utils.ConfigurationUtils: No edge input format specified. Ensure
your InputFormat does not require one.
> 15/08/29 12:42:18 INFO utils.ConfigurationUtils: No edge output format specified. Ensure
your OutputFormat does not require one.
> 15/08/29 12:42:18 INFO yarn.GiraphYarnClient: Final output path is: hdfs://hdnode01:54310/user/hduser/output/caminosNavegacionales1
> 15/08/29 12:42:18 INFO yarn.GiraphYarnClient: Running Client
> 15/08/29 12:42:18 INFO client.RMProxy: Connecting to ResourceManager at hdnode01/192.168.0.10:8050
> 15/08/29 12:42:18 INFO yarn.GiraphYarnClient: Got node report from ASM for, nodeId=hadoop100:51831,
nodeAddress hadoop100:8042, nodeRackName /default-rack, nodeNumContainers 0
> 15/08/29 12:42:18 INFO yarn.GiraphYarnClient: Got node report from ASM for, nodeId=hadoop101:40809,
nodeAddress hadoop101:8042, nodeRackName /default-rack, nodeNumContainers 0
> 15/08/29 12:42:18 INFO yarn.GiraphYarnClient: Obtained new Application ID: application_1440862856801_0001
> 15/08/29 12:42:18 INFO Configuration.deprecation: mapred.job.id is deprecated. Instead,
use mapreduce.job.id
> 15/08/29 12:42:18 INFO yarn.GiraphYarnClient: Set the environment for the application
master
> ...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message