hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raghu Angadi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3732) Block scanner should read block information during initialization.
Date Wed, 09 Jul 2008 20:45:31 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12612269#action_12612269
] 

Raghu Angadi commented on HADOOP-3732:
--------------------------------------

> May be we should postpone creating the dn block map until the upgrade is done. It is
the same thins as with the bscanner, before the upgrade is finished block information might
not be correct from the new software point of view.

If FSDataset does not scan the directories, datanode won't know what to upgrade. So some where
FSDataset should be able to understand prev layout. 

For blockScanner I will attach a patch that differs the initialization.

> Block scanner should read block information during initialization.
> ------------------------------------------------------------------
>
>                 Key: HADOOP-3732
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3732
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.17.0
>            Reporter: Konstantin Shvachko
>            Assignee: Raghu Angadi
>            Priority: Blocker
>             Fix For: 0.18.0
>
>
> We see a lot of warning messages on each data-node during startup and upgrading from
0.17 to 0.18 saying:
> {code}
> 2008-07-08 22:22:15,711 WARN org.apache.hadoop.dfs.DataNode: Block /grid/3/hadoop/var/hdfs/data/current/blk_3359714082706415785
does not have a metafile!
> {code}
> The message received twice for each block, because the block information is first read
buy the data-node itself and then by the block scanner.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message