hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj Das (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5103) Too many logs saying "Adding new node" on JobClient console
Date Sat, 31 Jan 2009 06:27:02 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12669154#action_12669154
] 

Devaraj Das commented on HADOOP-5103:
-------------------------------------

I just looked at the code of NetworkTopology.add(Node). If we move the LOG.info line inside
the block {noformat}if (clusterMap.add(node)) { }{noformat} this problem should be solved,
right?  I presume that clusterMap.add wouldn't add a node if it is already there and thereby
return false for duplicate nodes. We should then see as many LOG messages as the number of
successful clusterMap.add calls. The number of such LOG messages should be pretty manageable
(and proportional to the number of nodes where the blocks of the splits reside).

> Too many logs saying "Adding new node" on JobClient console
> -----------------------------------------------------------
>
>                 Key: HADOOP-5103
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5103
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.20.0
>            Reporter: Amareshwari Sriramadasu
>             Fix For: 0.20.0
>
>
> JobClient's console has logs saying "Adding a new node <rackname>/<node-ip-addr>:<port>"
for all the hosts, where each split resides.
> For jobs with more #maps, these logs just fill up client's space.
> This is introduced by HADOOP-3293

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message