hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3265) Reduce log level on MR2 IPC construction, etc
Date Wed, 23 Nov 2011 03:10:41 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13155656#comment-13155656
] 

Arun C Murthy commented on MAPREDUCE-3265:
------------------------------------------

With the patch, the output looks so:

{noformat}
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter
in all the log4j.properties files.
11/11/23 03:05:12 WARN conf.Configuration: mapred.used.genericoptionsparser is deprecated.
Instead, use mapreduce.client.genericoptionsparser.used
Running on 5 nodes to sort from hdfs://hrt12n01.cc1.ygridcore.net:8020/user/hrt_1/in into
hdfs://nn.foobar.com:8020/user/hrt_1/out with 9 reduces.
Job started: Wed Nov 23 03:05:12 UTC 2011
11/11/23 03:05:12 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 379 for hrt_1 on
xx.xx.xx.xx:8020
11/11/23 03:05:12 INFO security.TokenCache: Got dt for hdfs://nn.foobar.com:8020:8020;uri=xx.xx.xx.xx:8020;t.service=xx.xx.xx.xx:8020
11/11/23 03:05:12 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 380 for hrt_1 on
xx.xx.xx.xx:8020
11/11/23 03:05:12 INFO security.TokenCache: Got dt for hdfs://nn.foobar.com:8020:8020;uri=xx.xx.xx.xx:8020;t.service=xx.xx.xx.xx:8020
11/11/23 03:05:12 WARN conf.Configuration: fs.default.name is deprecated. Instead, use fs.defaultFS
11/11/23 03:05:15 INFO input.FileInputFormat: Total input paths to process : 5
11/11/23 03:05:17 INFO mapreduce.JobSubmitter: number of splits:5
11/11/23 03:05:18 INFO mapred.ResourceMgrDelegate: Submitted application application_1322016908667_0002
to ResourceManager at rm.foobar.com/yy.yy.yy.yy:8040
11/11/23 03:05:18 INFO mapreduce.Job: Running job: job_1322016908667_0002
11/11/23 03:05:19 INFO mapreduce.Job:  map 0% reduce 0%
11/11/23 03:05:29 INFO mapred.ClientServiceDelegate: Job tracking-url: rm.foobar.com:8088/proxy/application_1322016908667_0002/
11/11/23 03:05:40 INFO mapreduce.Job:  map 100% reduce 0%

{noformat}
                
> Reduce log level on MR2 IPC construction, etc
> ---------------------------------------------
>
>                 Key: MAPREDUCE-3265
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3265
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Todd Lipcon
>            Assignee: Arun C Murthy
>            Priority: Blocker
>             Fix For: 0.23.1
>
>         Attachments: MAPREDUCE-3265.patch
>
>
> Currently MR's IPC logging is very verbose. For example, I see a lot of:
> 11/10/25 12:14:06 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:06 INFO mapred.ResourceMgrDelegate: Connecting to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:06 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol
interface org.apache.hadoop.yarn.api.ClientRMProtocol
> 11/10/25 12:14:07 INFO mapred.ResourceMgrDelegate: Connected to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connecting to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connected to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol
interface org.apache.hadoop.mapreduce.v2.api.MRClientProtocol
> ... when submitting a job. This should be DEBUG level.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message