Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 8C0131074F for ; Sat, 29 Aug 2015 18:33:22 +0000 (UTC) Received: (qmail 49909 invoked by uid 500); 29 Aug 2015 18:33:17 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 49750 invoked by uid 500); 29 Aug 2015 18:33:17 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 49739 invoked by uid 99); 29 Aug 2015 18:33:16 -0000 Received: from Unknown (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 29 Aug 2015 18:33:16 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 86C0E1AB0C2 for ; Sat, 29 Aug 2015 18:33:16 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.901 X-Spam-Level: ** X-Spam-Status: No, score=2.901 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=3, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, WEIRD_PORT=0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-us-west.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id T-lBnxyi9Bkr for ; Sat, 29 Aug 2015 18:33:07 +0000 (UTC) Received: from mail-yk0-f195.google.com (mail-yk0-f195.google.com [209.85.160.195]) by mx1-us-west.apache.org (ASF Mail Server at mx1-us-west.apache.org) with ESMTPS id 6A3FE20647 for ; Sat, 29 Aug 2015 18:33:07 +0000 (UTC) Received: by ykbu129 with SMTP id u129so1235125ykb.3 for ; Sat, 29 Aug 2015 11:33:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=kdZZ2d0oFIfJoTuOZSZVvcja6ssNqdsbLopwV9swqLQ=; b=X/0V7Kvk3WidTD001/CXU3UFD2uKbG0ECtS8w0zSerEeRneJRw2zn6etAwjJLtZKLi 90OgM3a3WFwJ7NW/VetlGTeutzj0hVeC0j7OM51uCT3uJGcu/rfate7olUX3C6na/xZ3 5hXMQ2UQMHey6daZvwMEIPf4678KM1Mxmb3PyOb5ugDSS6AY3kdZyyjKNtMnlphApaI+ l/UxkqpmZHuBhlBZgtQB2X/1KOjnRzUlS4yo4YAWdT3sWiZxHVHBp8GVTXlk6KzwBeDT kMCmb5Idvk1xpHTfPYd9YU1uHOqYYRXQpGK7PqGZ00RyY9hRSK4InVYYkh8sXx0e1CRX VdlA== MIME-Version: 1.0 X-Received: by 10.13.203.76 with SMTP id n73mr14105042ywd.150.1440873180363; Sat, 29 Aug 2015 11:33:00 -0700 (PDT) Received: by 10.37.210.197 with HTTP; Sat, 29 Aug 2015 11:33:00 -0700 (PDT) In-Reply-To: References: Date: Sat, 29 Aug 2015 11:33:00 -0700 Message-ID: Subject: Re: mapred.job.id is deprecated. Instead, use mapreduce.job.id From: Ted Yu To: "common-user@hadoop.apache.org" Content-Type: multipart/alternative; boundary=001a114830021500cc051e776eb6 --001a114830021500cc051e776eb6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable A brief search involving mapred.job.id among the GIRAPH JIRAs doesn't result in any hit. Jose: Maybe you can log one. Cheers On Sat, Aug 29, 2015 at 11:21 AM, Varun Saxena wrote: > Hi Jose, > > I guess you are running Giraph job. > This config is being programmatically set by GiraphYarnClient class(refer > to > http://grepcode.com/file/repo1.maven.org/maven2/org.apache.giraph/giraph-= core/1.1.0/org/apache/giraph/yarn/GiraphYarnClient.java > ). > It is not being taken from a XML config file. > > Regards, > Varun Saxena. > > On Sat, Aug 29, 2015 at 9:15 PM, Jos=C3=A9 Luis Larroque < > larroquester@gmail.com> wrote: > >> I'm using hadoop 2.4.0. Here are the logs (this message doesn't appear a= t >> cluster initialization, it appears when a new job is invocated, my mista= ke): >> >> SLF4J: Class path contains multiple SLF4J bindings. >> SLF4J: Found binding in >> [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.= jar!/org/slf4j/impl/StaticLoggerBinder.class] >> SLF4J: Found binding in >> [jar:file:/usr/local/hadoop/share/hadoop/yarn/lib/giraph-examples-1.1.0-= for-hadoop-2.4.0-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBind= er.class] >> SLF4J: Found binding in >> [jar:file:/usr/local/hadoop/share/hadoop/mapreduce/giraph-examples-1.1.0= -for-hadoop-2.4.0-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBin= der.class] >> SLF4J: Found binding in >> [jar:file:/usr/local/hadoop/share/hadoop/mapreduce/procesamiento_grafo_w= ikiquote.jar!/org/slf4j/impl/StaticLoggerBinder.class] >> SLF4J: Found binding in >> [jar:file:/usr/local/hadoop/share/hadoop/mapreduce/wordcount_mio.jar!/or= g/slf4j/impl/StaticLoggerBinder.class] >> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an >> explanation. >> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] >> 15/08/29 12:42:17 WARN util.NativeCodeLoader: Unable to load >> native-hadoop library for your platform... using builtin-java classes wh= ere >> applicable >> 15/08/29 12:42:18 INFO utils.ConfigurationUtils: No edge input format >> specified. Ensure your InputFormat does not require one. >> 15/08/29 12:42:18 INFO utils.ConfigurationUtils: No edge output format >> specified. Ensure your OutputFormat does not require one. >> 15/08/29 12:42:18 INFO yarn.GiraphYarnClient: Final output path is: >> hdfs://hdnode01:54310/user/hduser/output/caminosNavegacionales1 >> 15/08/29 12:42:18 INFO yarn.GiraphYarnClient: Running Client >> 15/08/29 12:42:18 INFO client.RMProxy: Connecting to ResourceManager at >> hdnode01/192.168.0.10:8050 >> 15/08/29 12:42:18 INFO yarn.GiraphYarnClient: Got node report from ASM >> for, nodeId=3Dhadoop100:51831, nodeAddress hadoop100:8042, nodeRackName >> /default-rack, nodeNumContainers 0 >> 15/08/29 12:42:18 INFO yarn.GiraphYarnClient: Got node report from ASM >> for, nodeId=3Dhadoop101:40809, nodeAddress hadoop101:8042, nodeRackName >> /default-rack, nodeNumContainers 0 >> 15/08/29 12:42:18 INFO yarn.GiraphYarnClient: Obtained new Application >> ID: application_1440862856801_0001 >> *15/08/29 12:42:18 INFO Configuration.deprecation: mapred.job.id >> is deprecated. Instead, use mapreduce.job.id >> * >> 15/08/29 12:42:18 INFO yarn.GiraphYarnClient: Set the environment for th= e >> application master >> 15/08/29 12:42:18 INFO yarn.GiraphYarnClient: Environment for AM >> :{CLASSPATH=3D${CLASSPATH}:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/shar= e/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDF= S_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOO= P_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$= HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoo= p/mapreduce/lib/*} >> ... >> >> Cheers! >> >> >> 2015-08-29 12:26 GMT-03:00 Ted Yu : >> >>> Please provide a bit more information: >>> >>> release of hadoop you're using >>> snippet of the log showing the error >>> >>> Normally from Web UI, you can retrieve the complete list of parameters >>> >>> Cheers >>> >>> On Sat, Aug 29, 2015 at 8:19 AM, Jos=C3=A9 Luis Larroque < >>> larroquester@gmail.com> wrote: >>> >>>> Hi guys, i was removing some warnings for deprecated properties from m= y >>>> hadoop cluster, but i find this interesting warning when my cluster it= 's >>>> initializing: >>>> >>>> *mapred.job.id is deprecated. Instead, use >>>> mapreduce.job.id * >>>> >>>> But i can't find this property (*mapred.job.id *= ) >>>> in any of my configuration xml files for replacing it. I'm missing >>>> something here ? This is a minor bug ? >>>> >>>> Thanks for reading! >>>> >>>> Jose >>>> >>> >>> >> > --001a114830021500cc051e776eb6 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
A brief search involving = mapred.job.id=C2=A0among the=C2=A0GIRAPH JIRAs doesn't result in an= y hit.

Jose:
Maybe you can log one.
=
Cheers

On Sat, Aug 29, 2015 at 11:21 AM, Varun Saxena <v= saxena.varun@gmail.com> wrote:
Hi Jose,

I guess you are running Gi= raph job.
It is not being taken from a XML config file.

Regards,
Varun Saxena.

On Sat, Aug 29, 2015 at 9:15 PM, Jos=C3=A9 Luis Larroque <larroquester@gmail.com> wrote:
I'm using hadoop 2.4.0. Here are the logs = (this message doesn't appear at cluster initialization, it appears when= a new job is invocated, my mistake):

SLF4J: Class path contains mul= tiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hadoo= p/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLog= gerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hadoop/sha= re/hadoop/yarn/lib/giraph-examples-1.1.0-for-hadoop-2.4.0-jar-with-dependen= cies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding = in [jar:file:/usr/local/hadoop/share/hadoop/mapreduce/giraph-examples-1.1.0= -for-hadoop-2.4.0-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBin= der.class]
SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/had= oop/mapreduce/procesamiento_grafo_wikiquote.jar!/org/slf4j/impl/StaticLogge= rBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hadoop/share= /hadoop/mapreduce/wordcount_mio.jar!/org/slf4j/impl/StaticLoggerBinder.clas= s]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings= for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Lo= g4jLoggerFactory]
15/08/29 12:42:17 WARN util.NativeCodeLoader: Unable t= o load native-hadoop library for your platform... using builtin-java classe= s where applicable
15/08/29 12:42:18 INFO utils.ConfigurationUtils: No e= dge input format specified. Ensure your InputFormat does not require one.15/08/29 12:42:18 INFO utils.ConfigurationUtils: No edge output format sp= ecified. Ensure your OutputFormat does not require one.
15/08/29 12:42:1= 8 INFO yarn.GiraphYarnClient: Final output path is: hdfs://hdnode01:54310/u= ser/hduser/output/caminosNavegacionales1
15/08/29 12:42:18 INFO yarn.Gir= aphYarnClient: Running Client
15/08/29 12:42:18 INFO client.RMProxy: Con= necting to ResourceManager at hdnode01/192.168.0.10:8050
15/08/29 12:42:18 INFO yarn.Gir= aphYarnClient: Got node report from ASM for, nodeId=3Dhadoop100:51831, node= Address hadoop100:8042, nodeRackName /default-rack, nodeNumContainers 0
= 15/08/29 12:42:18 INFO yarn.GiraphYarnClient: Got node report from ASM for,= nodeId=3Dhadoop101:40809, nodeAddress hadoop101:8042, nodeRackName /defaul= t-rack, nodeNumContainers 0
15/08/29 12:42:18 INFO yarn.GiraphYarnClient= : Obtained new Application ID: application_1440862856801_0001
15/08/2= 9 12:42:18 INFO Configuration.deprecation: mapred.job.id is deprecated. Instead, use mapreduce.job.id
15/= 08/29 12:42:18 INFO yarn.GiraphYarnClient: Set the environment for the appl= ication master
15/08/29 12:42:18 INFO yarn.GiraphYarnClient: Environment= for AM :{CLASSPATH=3D${CLASSPATH}:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME= /share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOO= P_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$= HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/li= b/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/= hadoop/mapreduce/lib/*}
...

Cheers!


2015-08-29 12:26 GMT-0= 3:00 Ted Yu <yuzhihong@gmail.com>:
Please provide a bit more information:

=
release of hadoop you're using
snippet of the log = showing the error

Normally from Web UI, you can re= trieve the complete list of parameters

Cheers

= On Sat, Aug 29, 2015 at 8:19 AM, Jos=C3=A9 Luis Larroque = <larroqueste= r@gmail.com> wrote:
Hi guys, i was removing some warnings for deprecat= ed properties from my hadoop cluster, but i find this interesting warning w= hen my cluster it's initializing:

mapred.job.id is deprecated. Instead, use mapreduce.job.id
But i can't find this property (mapred.job.id) in any of my configuratio= n xml files for replacing it. I'm missing something here ? This is a mi= nor bug ?

Thanks for reading!

Jose




--001a114830021500cc051e776eb6--