hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: Review Request: DFSClient.getBlockLocations returns BlockLocations with no indication that the corresponding blocks are corrupt
Date Thu, 04 Nov 2010 00:03:28 GMT
I wrote such a procmail script for review.hbase.org and posted it for the
ASF Infra guys a few weeks ago. We can file a new INFRA JIRA to get them to
install/configure it.

-Todd

On Wed, Nov 3, 2010 at 1:27 PM, Arun C Murthy <acm@yahoo-inc.com> wrote:

> Can we get RB to send these to jira? Having comments here and on jira is
> very confusing...
>
>
> On Nov 3, 2010, at 11:11 AM, Ramkumar Vadali wrote:
>
>
>> -----------------------------------------------------------
>> This is an automatically generated e-mail. To reply, visit:
>> https://reviews.apache.org/r/27/#review27
>> -----------------------------------------------------------
>>
>>
>>
>>
>> http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/DFSUtil.java
>> <https://reviews.apache.org/r/27/#comment17>
>>
>>   It should be sufficient to pass blk.isCorrupt() here. The client can
>> check the number of locations based on the remaining information.
>>
>>
>> - Ramkumar
>>
>>
>> On 2010-11-02 21:12:28, Patrick Kling wrote:
>>
>>>
>>> -----------------------------------------------------------
>>> This is an automatically generated e-mail. To reply, visit:
>>> https://reviews.apache.org/r/27/
>>> -----------------------------------------------------------
>>>
>>> (Updated 2010-11-02 21:12:28)
>>>
>>>
>>> Review request for hadoop-hdfs.
>>>
>>>
>>> Summary
>>> -------
>>>
>>> DFSClient.getBlockLocations returns BlockLocations with no indication
>>> that the corresponding blocks are corrupt
>>>
>>> When there are no uncorrupted replicas of a block,
>>> FSNamesystem.getBlockLocations returns LocatedBlocks corresponding to
>>> corrupt blocks. When DFSClient converts these to BlockLocations, the
>>> information that the corresponding block is corrupt is lost. We should add a
>>> field to BlockLocation to indicate whether the corresponding block is
>>> corrupt in order to warn the client that reading this block will fail. This
>>> would be especially useful for tools such as RAID FSCK, which could then
>>> easily inspect whether data or parity blocks are corrupted without having to
>>> make direct RPC calls
>>>
>>>
>>> This addresses bug HDFS-1483.
>>>   https://issues.apache.org/jira/browse/HDFS-1483
>>>
>>>
>>> Diffs
>>> -----
>>>
>>>
>>> http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/DFSUtil.java1028386
>>>
>>> http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/test/hdfs/org/apache/hadoop/hdfs/TestDFSUtil.javaPRE-CREATION
>>>
>>> Diff: https://reviews.apache.org/r/27/diff
>>>
>>>
>>> Testing
>>> -------
>>>
>>> TestDFSUtil
>>>
>>>
>>> Thanks,
>>>
>>> Patrick
>>>
>>>
>>>
>>
>


-- 
Todd Lipcon
Software Engineer, Cloudera

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message