hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jothi Padmanabhan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5103) Too many logs saying "Adding new node" on JobClient console
Date Wed, 04 Feb 2009 09:33:59 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12670289#action_12670289

Jothi Padmanabhan commented on HADOOP-5103:

Yes, that should help. However, that alone would be insufficient. Currently, the getSplits
in FileInputFormat calls the getSplitHosts  for each file, which in turn uses a new NetworkTopology
object -- it is a local object in that method. We should use a single clusterMap object across
all the Files. 

> Too many logs saying "Adding new node" on JobClient console
> -----------------------------------------------------------
>                 Key: HADOOP-5103
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5103
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.20.0
>            Reporter: Amareshwari Sriramadasu
>             Fix For: 0.20.0
> JobClient's console has logs saying "Adding a new node <rackname>/<node-ip-addr>:<port>"
for all the hosts, where each split resides.
> For jobs with more #maps, these logs just fill up client's space.
> This is introduced by HADOOP-3293

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message