Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Wed, 23 May 2012 05:08:40 +0000 (UTC)
From: "J.Andreina (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <1798313931.10614.1337749720922.JavaMail.jiratomcat@issues-vm>
Subject: [jira] [Created] (HDFS-3457) Number of UnderReplicated blocks and
 Number of Files in the cluster, displayed in UI and Fsck report is
 different
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

J.Andreina created HDFS-3457:
--------------------------------

             Summary: Number of UnderReplicated blocks and Number of Files in the cluster, displayed in UI and Fsck report is different
                 Key: HDFS-3457
                 URL: https://issues.apache.org/jira/browse/HDFS-3457
             Project: Hadoop HDFS
          Issue Type: Bug
    Affects Versions: 2.0.0
            Reporter: J.Andreina
            Priority: Minor


Scenario:
=========
Write an HDFS application with the following sequence of operations
1. Create file.
2. Append and sync file.
3. Delete file.
4. Create file.
5. Rename file.
Run the application using 50 threads for 4 hours.
Next Run the same application using 200 threads for the next 4 hours.
Next Run the application using 50 threads for the next 4 hours.

The Number of under-Replicated blocks and Total number of files mentioned in the UI and fsck report differs

Fsck report for the mismatch in Number of under-Replicated blocks :
===================================================================
Status: HEALTHY
 Total size:	5670200922 B
 Total dirs:	2
 Total files:	2015
 Total blocks (validated):	977 (avg. block size 5803685 B)
 Minimally replicated blocks:	977 (100.0 %)
 Over-replicated blocks:	0 (0.0 %)
 Under-replicated blocks:	94 (9.621289 %)
 Mis-replicated blocks:		0 (0.0 %)
 Default replication factor:	2
 Average block replication:	1.9037871
 Corrupt blocks:		0
 Missing replicas:		94 (5.0537634 %)
 Number of data-nodes:		3
 Number of racks:		1
FSCK ended at Mon Mar 19 11:14:41 IST 2012 in 94 milliseconds

The filesystem under path '/' is HEALTHY


Fsck report for the mismatch in Total number of Files :
=======================================================
Status: HEALTHY
 Total size:	19418 B (Total open files size: 42729325 B)
 Total dirs:	2
 Total files:	4226 (Files currently being written: 15)
 Total blocks (validated):	266 (avg. block size 73 B) (Total open file blocks (not validated): 5)
 Minimally replicated blocks:	266 (100.0 %)
 Over-replicated blocks:	0 (0.0 %)
 Under-replicated blocks:	0 (0.0 %)
 Mis-replicated blocks:		0 (0.0 %)
 Default replication factor:	2
 Average block replication:	2.0
 Corrupt blocks:		0
 Missing replicas:		0 (0.0 %)
 Number of data-nodes:		3
 Number of racks:		2
FSCK ended at Wed Apr 04 10:41:44 IST 2012 in 2302 milliseconds
The filesystem under path '/' is HEALTHY

Have attached UI screenshot for both the issues


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira