hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "lohit vijayarenu (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-2703) New files under lease (before close) still shows up as MISSING files/blocks in fsck
Date Tue, 15 Apr 2008 20:32:21 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-2703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

lohit vijayarenu updated HADOOP-2703:
-------------------------------------

    Attachment: HADOOP-2703-3.patch

This updated patch adds few more details. When hadoop fsck is run without any options it along
with total number of open files, it also lists total number of blocks owned by those open
files and also its size if it is greater than 0. When -openforwrite option is used, all of
them are added up and reported as total blocks. 

Sample output when default fsck is invoked while a file is being written

{noformat}
[lohit@ hadoop-core-trunk]$ ./bin/hadoop fsck /
...Status: HEALTHY
 Total size:    420912 B
 Total dirs:    5
 Total files:   3 (Files currently being written: 1)
 Total blocks:  3 (avg. block size 140304 B) (Total open file blocks: 1)
 Minimally replicated blocks:   3 (100.0 %)
 Over-replicated blocks:        0 (0.0 %)
 Under-replicated blocks:       0 (0.0 %)
 Mis-replicated blocks:         0 (0.0 %)
 Default replication factor:    1
 Average block replication:     1.0
 Missing replicas:              0 (0.0 %)
 Number of data-nodes:          1
 Number of racks:               1


The filesystem under path '/' is HEALTHY
[lohit@ hadoop-core-trunk]$

{noformat}

> New files under lease (before close) still shows up as MISSING files/blocks in fsck
> -----------------------------------------------------------------------------------
>
>                 Key: HADOOP-2703
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2703
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.15.3
>            Reporter: Koji Noguchi
>            Assignee: lohit vijayarenu
>            Priority: Minor
>         Attachments: HADOOP-2703-1.patch, HADOOP-2703-2.patch, HADOOP-2703-3.patch
>
>
> On 0.15.3 which includes HADOOP-2540 ("Empty blocks make fsck report corrupt, even when
it isn't") fix, we are still seeing fsck reporting missing blocks when they aren't.   Basically,
 any unclosed small files under lease shows up as MISSING.
> Can we have an option in fsck to skip those files under lease?
> Or  count those files/blocks separately?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message