hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1468) Too many log messages generated if a node has a different Hadoop version installed
Date Tue, 09 Feb 2010 01:35:28 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831243#action_12831243
] 

Allen Wittenauer commented on MAPREDUCE-1468:
---------------------------------------------

FWIW, a lot of this is upgrade process too.

One of the best practices is to take a snapshot of dfs and jt info to make sure that the node
counts and lists matches pre/post upgrade.  Following this, it is fairly clear whether nodes
failed to upgrade properly.

> Too many log messages generated if a node has a different Hadoop version installed
> ----------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1468
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1468
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>    Affects Versions: 0.20.1
>            Reporter: Qi Liu
>
> When upgrading or downgrading Hadoop, for a large cluster it is possible that some nodes
are not upgraded/downgraded. In this case, the jobtracker and namenode generates gigabytes
of logs each day, reporting protocol version mismatch. Such log message is generated once
every 3ms.
> Ideally, the misbehaved node should quit retrying after several attempts if there is
a protocol version mismatch. At the very least, the log message should be minimized (maybe
once a day or once per start up).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message