hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jlei liu <liulei...@gmail.com>
Subject Re: DFSClient may read wrong data in local read
Date Fri, 28 Sep 2012 04:30:37 GMT
Hi Colin´╝î thanks for your reply.

Where can I see the new design for BlockReaderLocal class?

Thanks,

LiuLei


2012/9/28 Colin McCabe <cmccabe@alumni.cmu.edu>

> We don't make very strong guarantees about what happens when clients
> read from a deleted file.  DFSClients definitely may read data from a
> deleted file even if local reads are not enabled.
>
> Incidentally, BlockReaderLocal is being redesigned to pass file
> descriptors rather than paths, which will be more secure and fix some
> corner cases surrounding append and local reads.
>
> cheers,
> Colin
>
>
> On Wed, Sep 26, 2012 at 11:19 PM, jlei liu <liulei412@gmail.com> wrote:
> > In local read, BlockReaderLocal class use "static Map<Integer,
> > LocalDatanodeInfo> localDatanodeInfoMap" property to store local block
> file
> > path and local meta file path. When I stop HDFS cluster or I kill the
> local
> > DataNode and delete file use "./hadoop dfs -rm path" command , the
> DFSClient
> > still can read the data form local file. I think that may lead to
> DFSClient
> > read wrong data.
> >
> > I think we should fix the problem.
> >
> >
> > Thanks,
> >
> > LiuLei
>

Mime
View raw message