hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Watzke (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7648) Verify the datanode directory layout
Date Fri, 04 Mar 2016 12:14:40 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15179806#comment-15179806
] 

David Watzke commented on HDFS-7648:
------------------------------------

Hi guys. I ran into trouble because I used https://github.com/killerwhile/volume-balancer
with Hadoop 2.6.0 and it messed up my datadirs because that software makes invalid assumptions
about what directory moves can it do. Now the DN logs are filled with these:

WARN org.apache.hadoop.hdfs.server.datanode.VolumeScanner: I/O error while finding block BP-680964103-77.234.46.18-1375882473930:blk_5822441067008155275_0
on volume /data/19/cdfs/dn

What can I do to fix this? I don't know what dirs were moved and from where but is there a
reasonable way out of this? Such as editing VERSION file to a previous version when DN is
down so that it fixes the layout by itself - would that work?

Please note that I've lost the other replica due to a filesystem error so I can't just ignore
it. This is literally my only option to recover some missing blocks.

Thanks

> Verify the datanode directory layout
> ------------------------------------
>
>                 Key: HDFS-7648
>                 URL: https://issues.apache.org/jira/browse/HDFS-7648
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: datanode
>            Reporter: Tsz Wo Nicholas Sze
>            Assignee: Rakesh R
>         Attachments: HDFS-7648-3.patch, HDFS-7648-4.patch, HDFS-7648-5.patch, HDFS-7648.patch,
HDFS-7648.patch
>
>
> HDFS-6482 changed datanode layout to use block ID to determine the directory to store
the block.  We should have some mechanism to verify it.  Either DirectoryScanner or block
report generation could do the check.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message