hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jing Zhao (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-8498) Blocks can be committed with wrong size
Date Thu, 19 Jan 2017 22:23:27 GMT

     [ https://issues.apache.org/jira/browse/HDFS-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jing Zhao updated HDFS-8498:
----------------------------
    Attachment: HDFS-8498.001.patch

Update the patch to fix bug when block is initialized as null. Also slightly changed TestDFSOutputStream.java
to trigger tests in hadoop-hdfs.

> Blocks can be committed with wrong size
> ---------------------------------------
>
>                 Key: HDFS-8498
>                 URL: https://issues.apache.org/jira/browse/HDFS-8498
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>    Affects Versions: 2.5.0
>            Reporter: Daryn Sharp
>            Assignee: Jing Zhao
>            Priority: Critical
>         Attachments: HDFS-8498.000.patch, HDFS-8498.001.patch
>
>
> When an IBR for a UC block arrives, the NN updates the expected location's block and
replica state _only_ if it's on an unexpected storage for an expected DN.  If it's for an
expected storage, only the genstamp is updated.  When the block is committed, and the expected
locations are verified, only the genstamp is checked.  The size is not checked but it wasn't
updated in the expected locations anyway.
> A faulty client may misreport the size when committing the block.  The block is effectively
corrupted.  If the NN issues replications, the received IBR is considered corrupt, the NN
invalidates the block, immediately issues another replication.  The NN eventually realizes
all the original replicas are corrupt after full BRs are received from the original DNs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message