Return-Path: X-Original-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 836CD9161 for ; Tue, 8 Nov 2011 22:34:13 +0000 (UTC) Received: (qmail 98756 invoked by uid 500); 8 Nov 2011 22:34:13 -0000 Delivered-To: apmail-hadoop-hdfs-issues-archive@hadoop.apache.org Received: (qmail 98734 invoked by uid 500); 8 Nov 2011 22:34:13 -0000 Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-issues@hadoop.apache.org Delivered-To: mailing list hdfs-issues@hadoop.apache.org Received: (qmail 98718 invoked by uid 99); 8 Nov 2011 22:34:13 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 08 Nov 2011 22:34:13 +0000 X-ASF-Spam-Status: No, hits=-2001.2 required=5.0 tests=ALL_TRUSTED,RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.116] (HELO hel.zones.apache.org) (140.211.11.116) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 08 Nov 2011 22:34:11 +0000 Received: from hel.zones.apache.org (hel.zones.apache.org [140.211.11.116]) by hel.zones.apache.org (Postfix) with ESMTP id D163D3E516 for ; Tue, 8 Nov 2011 22:33:51 +0000 (UTC) Date: Tue, 8 Nov 2011 22:33:51 +0000 (UTC) From: "Ravi Prakash (Updated) (JIRA)" To: hdfs-issues@hadoop.apache.org Message-ID: <1401749087.12294.1320791631859.JavaMail.tomcat@hel.zones.apache.org> In-Reply-To: <1354692230.10018.1318022909915.JavaMail.tomcat@hel.zones.apache.org> Subject: [jira] [Updated] (HDFS-2417) Warnings about attempt to override final parameter while getting delegation token MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HDFS-2417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Prakash updated HDFS-2417: ------------------------------- Resolution: Duplicate Status: Resolved (was: Patch Available) Attached the patch for branch-0.20-security to HADOOP-7664 > Warnings about attempt to override final parameter while getting delegation token > --------------------------------------------------------------------------------- > > Key: HDFS-2417 > URL: https://issues.apache.org/jira/browse/HDFS-2417 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node > Affects Versions: 0.20.205.0 > Reporter: Rajit Saha > Assignee: Ravi Prakash > Attachments: HDFS-2417.patch > > > I am seeing whenever I run any Mapreduce job and its trying to acquire delegation from NN, In JT log following warnings coming about "a attempt to override final parameter:" > The log snippet in JT log > 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration: > /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: > mapred.job.reuse.jvm.num.tasks; Ignoring. > 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration: > /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: mapred.system.dir; > Ignoring. > 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration: > /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: > hadoop.job.history.user.location; Ignoring. > 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration: > /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: mapred.local.dir; > Ignoring. > 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration: > /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m > apred.job.tracker.http.address; Ignoring. > 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration: > /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: d > fs.data.dir; Ignoring. > 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration: > /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: d > fs.http.address; Ignoring. > 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration: > /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m > apreduce.admin.map.child.java.opts; Ignoring. > 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration: > /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: > mapreduce.history.server.http.address; Ignoring. > 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration: > /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m > apreduce.history.server.embedded; Ignoring. > 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration: > /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m > apreduce.jobtracker.split.metainfo.maxsize; Ignoring.2011-10-07 20:29:19,096 WARN > org.apache.hadoop.conf.Configuration: /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to > override final parameter: m > apreduce.admin.reduce.child.java.opts; Ignoring.2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration: > /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: h > adoop.tmp.dir; Ignoring. > 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration: > /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: > mapred.jobtracker.maxtasks.per.job; Ignoring. > 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration: > /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: > mapred.job.tracker; Ignoring. > 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration: > /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: dfs.name.dir; > Ignoring. > 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration: > /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m > apred.temp.dir; Ignoring.2011-10-07 20:29:19,103 INFO > org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: registering token for renewal for service > =:50470 and jobID > = job_201110072015_0005 > 2011-10-07 20:29:19,103 INFO org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: registering token for > renewal for service =:8020 and jobID = job_201110072015_0005 > The STDOUT of distcp job when these warnings logged into JT log > ---------------------------------------------------------------- > $hadoop distcp hftp://:50070/tmp/inp out > 11/10/07 20:29:17 INFO tools.DistCp: > srcPaths=[hftp://:50070/tmp/inp] > 11/10/07 20:29:17 INFO tools.DistCp: destPath=out > 11/10/07 20:29:18 INFO security.TokenCache: Got dt for > hftp://:50070/tmp/inp;uri=:50470;t.service=:50470 > 11/10/07 20:29:18 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 7 for on :8020 > 11/10/07 20:29:18 INFO security.TokenCache: Got dt for > 1318019341/as;uri=:8020;t.service=:8020 > .. > .. > If this is not critical, it may be unnecessary noise in log -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira