hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-96) name server should log decisions that affect data: block creation, removal, replication
Date Thu, 30 Mar 2006 23:17:46 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-96?page=comments#action_12372591 ] 

Doug Cutting commented on HADOOP-96:
------------------------------------

This sounds like a great plan!

Hadoop's current log formatter has an option to log thread id's.  The namenode code can simply
turn this on.   Also, one can configure the JVM to rotate logs using:

http://java.sun.com/j2se/1.4.2/docs/api/java/util/logging/FileHandler.html

In any case, we should add a timestamp to the log file names generated by bin/hadoop-daemon.sh,
where standard out and error are logged, regardless of JVM log configuration.

> name server should log decisions that affect data: block creation, removal, replication
> ---------------------------------------------------------------------------------------
>
>          Key: HADOOP-96
>          URL: http://issues.apache.org/jira/browse/HADOOP-96
>      Project: Hadoop
>         Type: Improvement
>   Components: dfs
>     Versions: 0.1
>     Reporter: Yoram Arnon
>     Priority: Critical

>
> currently, there's no way to analyze and debug DFS errors where blocks disapear.
> name server should log its decisions that affect data, including block creation, removal,
replication:
> - block <b> created, assigned to datanodes A, B, ...
> - datanode A dead, block <b> underreplicated(1), replicating to datanode C
> - datanode B dead, block <b> underreplicated(2), replicating to datanode D
> - datanode A alive, block <b> overreplicated, removing from datanode D
> - block <removed> from datanodes C, D, ...
> that will enable me to track down, two weeks later, a block that's missing from a file,
and to debug the name server.
> extra credit:
> - rotate log file, as it might grow large
> - make this behaviour optional/configurable

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


Mime
View raw message