hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Colin McCabe <cmcc...@alumni.cmu.edu>
Subject Re: DFSClient may read wrong data in local read
Date Mon, 01 Oct 2012 03:26:31 GMT
I'm going to post a patch to HDFS-347 shortly.  From the user's point
of view, the important thing about the new implementation of local
reads is that it will work with security, and will work on files that
are in the process of being written.

cheers,
Colin


On Thu, Sep 27, 2012 at 9:30 PM, jlei liu <liulei412@gmail.com> wrote:
> Hi Colin´╝î thanks for your reply.
>
> Where can I see the new design for BlockReaderLocal class?
>
> Thanks,
>
> LiuLei
>
>
>
> 2012/9/28 Colin McCabe <cmccabe@alumni.cmu.edu>
>>
>> We don't make very strong guarantees about what happens when clients
>> read from a deleted file.  DFSClients definitely may read data from a
>> deleted file even if local reads are not enabled.
>>
>> Incidentally, BlockReaderLocal is being redesigned to pass file
>> descriptors rather than paths, which will be more secure and fix some
>> corner cases surrounding append and local reads.
>>
>> cheers,
>> Colin
>>
>>
>> On Wed, Sep 26, 2012 at 11:19 PM, jlei liu <liulei412@gmail.com> wrote:
>> > In local read, BlockReaderLocal class use "static Map<Integer,
>> > LocalDatanodeInfo> localDatanodeInfoMap" property to store local block
>> > file
>> > path and local meta file path. When I stop HDFS cluster or I kill the
>> > local
>> > DataNode and delete file use "./hadoop dfs -rm path" command , the
>> > DFSClient
>> > still can read the data form local file. I think that may lead to
>> > DFSClient
>> > read wrong data.
>> >
>> > I think we should fix the problem.
>> >
>> >
>> > Thanks,
>> >
>> > LiuLei
>
>

Mime
View raw message