hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-96) name server should log decisions that affect data: block creation, removal, replication
Date Mon, 01 May 2006 18:15:47 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-96?page=comments#action_12377255 ] 

Doug Cutting commented on HADOOP-96:

Overall I like this.  A few nits, however:

1. It would be nice if, before it switches logging to a file, the name of that file were logged.
 That way, when folks upgrade they can figure out where there logs are written.  This will
also help in debugging configuration issues.

2. When I run bin/start-all.sh on my Linux box, the log files end up in my connected directory,
in HADOOP_HOME, not in HADOOP_HOME/logs.  When I add a print statement, it shows the correct
directory for logDir, but that's not where the files are written.  I have not tested this
on Windows, but it would be good to check that it with a simple configuration (i.e., a hadoop-site.xml
that only specifies localhost for the jobtracker and namenode) that, on Windows and Linux,
the files are written where expected.  Perhaps we could even add a unit test for this.


> name server should log decisions that affect data: block creation, removal, replication
> ---------------------------------------------------------------------------------------
>          Key: HADOOP-96
>          URL: http://issues.apache.org/jira/browse/HADOOP-96
>      Project: Hadoop
>         Type: Improvement

>   Components: dfs
>     Versions: 0.1.0
>     Reporter: Yoram Arnon
>     Assignee: Hairong Kuang
>     Priority: Critical
>      Fix For: 0.2
>  Attachments: namenodeLogging.patch
> currently, there's no way to analyze and debug DFS errors where blocks disapear.
> name server should log its decisions that affect data, including block creation, removal,
> - block <b> created, assigned to datanodes A, B, ...
> - datanode A dead, block <b> underreplicated(1), replicating to datanode C
> - datanode B dead, block <b> underreplicated(2), replicating to datanode D
> - datanode A alive, block <b> overreplicated, removing from datanode D
> - block <removed> from datanodes C, D, ...
> that will enable me to track down, two weeks later, a block that's missing from a file,
and to debug the name server.
> extra credit:
> - rotate log file, as it might grow large
> - make this behaviour optional/configurable

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators:
For more information on JIRA, see:

View raw message