hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jlei liu <liulei...@gmail.com>
Subject Re: DFSClient may read wrong data in local read
Date Mon, 01 Oct 2012 08:34:46 GMT
Hi Colin, Thanks for your reply.

What is mean that the patch will work on files that are in the process of
being written?

Thanks,
LiuLei


2012/10/1 Colin McCabe <cmccabe@alumni.cmu.edu>

> I'm going to post a patch to HDFS-347 shortly.  From the user's point
> of view, the important thing about the new implementation of local
> reads is that it will work with security, and will work on files that
> are in the process of being written.
>
> cheers,
> Colin
>
>
> On Thu, Sep 27, 2012 at 9:30 PM, jlei liu <liulei412@gmail.com> wrote:
> > Hi Colin´╝î thanks for your reply.
> >
> > Where can I see the new design for BlockReaderLocal class?
> >
> > Thanks,
> >
> > LiuLei
> >
> >
> >
> > 2012/9/28 Colin McCabe <cmccabe@alumni.cmu.edu>
> >>
> >> We don't make very strong guarantees about what happens when clients
> >> read from a deleted file.  DFSClients definitely may read data from a
> >> deleted file even if local reads are not enabled.
> >>
> >> Incidentally, BlockReaderLocal is being redesigned to pass file
> >> descriptors rather than paths, which will be more secure and fix some
> >> corner cases surrounding append and local reads.
> >>
> >> cheers,
> >> Colin
> >>
> >>
> >> On Wed, Sep 26, 2012 at 11:19 PM, jlei liu <liulei412@gmail.com> wrote:
> >> > In local read, BlockReaderLocal class use "static Map<Integer,
> >> > LocalDatanodeInfo> localDatanodeInfoMap" property to store local block
> >> > file
> >> > path and local meta file path. When I stop HDFS cluster or I kill the
> >> > local
> >> > DataNode and delete file use "./hadoop dfs -rm path" command , the
> >> > DFSClient
> >> > still can read the data form local file. I think that may lead to
> >> > DFSClient
> >> > read wrong data.
> >> >
> >> > I think we should fix the problem.
> >> >
> >> >
> >> > Thanks,
> >> >
> >> > LiuLei
> >
> >
>

Mime
View raw message