Mailing-List: contact hadoop-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hadoop-dev@lucene.apache.org
Message-ID: <1771390012.1143198328044.JavaMail.jira@ajax>
Date: Fri, 24 Mar 2006 11:05:28 +0000 (GMT)
From: "Konstantin Shvachko (JIRA)" <jira@apache.org>
To: hadoop-dev@lucene.apache.org
Subject: [jira] Commented: (HADOOP-101) DFSck - fsck-like utility for
 checking DFS volumes
In-Reply-To: <958365348.1143139938715.JavaMail.jira@ajax>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

    [ http://issues.apache.org/jira/browse/HADOOP-101?page=comments#action_12371705 ] 

Konstantin Shvachko commented on HADOOP-101:
--------------------------------------------

Just posted a map-reduce test that checks all blocks of all files.
See HADOOP-95.
The infinite loop in DFSClient is fixed now, so it works.

With respect to some of the previous comments.
Restarting the cluster (a big one) just to check its consistency is not an exciting option.
This means that we will have to wait up to 55 minutes before missing blocks will
be detected by examining just the namenode data.

A drawback of the map-reduce test is that we cannot force the system to check all replicas 
of the block. So corrupted block is reported only if all of its replicas are bad.
But yes this is better than nothing.

> DFSck - fsck-like utility for checking DFS volumes
> --------------------------------------------------
>
>          Key: HADOOP-101
>          URL: http://issues.apache.org/jira/browse/HADOOP-101
>      Project: Hadoop
>         Type: New Feature
>   Components: dfs
>     Versions: 0.2
>     Reporter: Andrzej Bialecki 
>     Assignee: Andrzej Bialecki 
>  Attachments: DFSck.java
>
> This is a utility to check health status of a DFS volume, and collect some additional statistics.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira