hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rushabh S Shah (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-10627) Volume Scanner mark a block as "suspect" even if the block sender encounters 'Broken pipe' or 'Connection reset by peer' exception
Date Thu, 14 Jul 2016 16:36:20 GMT
Rushabh S Shah created HDFS-10627:
-------------------------------------

             Summary: Volume Scanner mark a block as "suspect" even if the block sender encounters
'Broken pipe' or 'Connection reset by peer' exception
                 Key: HDFS-10627
                 URL: https://issues.apache.org/jira/browse/HDFS-10627
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: hdfs
    Affects Versions: 2.7.0
            Reporter: Rushabh S Shah
            Assignee: Rushabh S Shah


In the BlockSender code,
{code:title=BlockSender.java|borderStyle=solid}
        if (!ioem.startsWith("Broken pipe") && !ioem.startsWith("Connection reset"))
{
          LOG.error("BlockSender.sendChunks() exception: ", e);
        }
        datanode.getBlockScanner().markSuspectBlock(
              volumeRef.getVolume().getStorageID(),
              block);
{code}

Before HDFS-7686, the block was marked as suspect only if the exception message doesn't start
with Broken pipe or Connection reset.
But after HDFS-7686, the block is marked as corrupt irrespectively of the exception message.

In one of our datanode, it took approximately a whole day (22 hours) to go through all the
suspect blocks to scan one corrupt block.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message