hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] Assigned: (HADOOP-2540) Empty blocks make fsck report corrupt, even when it isn't
Date Wed, 09 Jan 2008 19:55:34 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-2540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

dhruba borthakur reassigned HADOOP-2540:
----------------------------------------

    Assignee: dhruba borthakur

> Empty blocks make fsck report corrupt, even when it isn't
> ---------------------------------------------------------
>
>                 Key: HADOOP-2540
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2540
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.15.1
>            Reporter: Allen Wittenauer
>            Assignee: dhruba borthakur
>
> If the name node crashes after blocks have been allocated and before the content has
been uploaded, fsck will report the zero sized files as corrupt upon restart:
> /user/rajive/rand0/_task_200712121358_0001_m_000808_0/part-00808: MISSING 1 blocks of
total size 0 B
> ... even though all blocks are accounted for:
> Status: CORRUPT
>  Total size:    2932802658847 B
>  Total blocks:  26603 (avg. block size 110243305 B)
>  Total dirs:    419
>  Total files:   5031
>  Over-replicated blocks:        197 (0.740518 %)
>  Under-replicated blocks:       0 (0.0 %)
>  Target replication factor:     3
>  Real replication factor:       3.0074053
> The filesystem under path '/' is CORRUPT
> In UFS and related filesystems, such files would get put into lost+found after an fsck
and the filesystem would return back to normal.  It would be super if HDFS could do a similar
thing.  Perhaps if all of the nodes stored in the name node's 'includes' file have reported
in, HDFS could automatically run a fsck and store these not-necessarily-broken files in something
like lost+found.  
> Files that are actually missing blocks, however, should not be touched.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message