Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of liulei412@gmail.com designates
 209.85.223.176 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CA+qbEUO9jk62GEQP0WbTjWubnQmWfbemP2A2qbgTNUwx7tuWUA@mail.gmail.com>
References: 
 <CABbGW3zUY3EJB-jWTZqbnkjqKqFYeZ11r_QLd5ek4ei-OU5evg@mail.gmail.com>
	<CA+qbEUO9jk62GEQP0WbTjWubnQmWfbemP2A2qbgTNUwx7tuWUA@mail.gmail.com>
Date: Fri, 28 Sep 2012 12:30:37 +0800
Message-ID: 
 <CABbGW3yJ9h7H0Y69JVAsQXETCCMYBZ6-FGoqXbc4y__S6JNYwQ@mail.gmail.com>
Subject: Re: DFSClient may read wrong data in local read
From: jlei liu <liulei412@gmail.com>
To: user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=e89a8f3b9b5583304104cabb84f2

--e89a8f3b9b5583304104cabb84f2
Content-Type: text/plain; charset=GB2312
Content-Transfer-Encoding: quoted-printable

Hi Colin=A3=AC thanks for your reply.

Where can I see the new design for BlockReaderLocal class?

Thanks,

LiuLei


2012/9/28 Colin McCabe <cmccabe@alumni.cmu.edu>

> We don't make very strong guarantees about what happens when clients
> read from a deleted file.  DFSClients definitely may read data from a
> deleted file even if local reads are not enabled.
>
> Incidentally, BlockReaderLocal is being redesigned to pass file
> descriptors rather than paths, which will be more secure and fix some
> corner cases surrounding append and local reads.
>
> cheers,
> Colin
>
>
> On Wed, Sep 26, 2012 at 11:19 PM, jlei liu <liulei412@gmail.com> wrote:
> > In local read, BlockReaderLocal class use "static Map<Integer,
> > LocalDatanodeInfo> localDatanodeInfoMap" property to store local block
> file
> > path and local meta file path. When I stop HDFS cluster or I kill the
> local
> > DataNode and delete file use "./hadoop dfs -rm path" command , the
> DFSClient
> > still can read the data form local file. I think that may lead to
> DFSClient
> > read wrong data.
> >
> > I think we should fix the problem.
> >
> >
> > Thanks,
> >
> > LiuLei
>

--e89a8f3b9b5583304104cabb84f2
Content-Type: text/html; charset=GB2312
Content-Transfer-Encoding: quoted-printable

Hi Colin=A3=AC thanks for your reply.<br><br>Where can I see the new design=
 for BlockReaderLocal class?<br><br>Thanks,<br><br>LiuLei<br><br><br><div c=
lass=3D"gmail_quote">2012/9/28 Colin McCabe <span dir=3D"ltr">&lt;<a href=
=3D"mailto:cmccabe@alumni.cmu.edu" target=3D"_blank">cmccabe@alumni.cmu.edu=
</a>&gt;</span><br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">We don&#39;t make very strong guarantees abo=
ut what happens when clients<br>
read from a deleted file. &nbsp;DFSClients definitely may read data from a<=
br>
deleted file even if local reads are not enabled.<br>
<br>
Incidentally, BlockReaderLocal is being redesigned to pass file<br>
descriptors rather than paths, which will be more secure and fix some<br>
corner cases surrounding append and local reads.<br>
<br>
cheers,<br>
Colin<br>
<div class=3D"HOEnZb"><div class=3D"h5"><br>
<br>
On Wed, Sep 26, 2012 at 11:19 PM, jlei liu &lt;<a href=3D"mailto:liulei412@=
gmail.com">liulei412@gmail.com</a>&gt; wrote:<br>
&gt; In local read, BlockReaderLocal class use &quot;static Map&lt;Integer,=
<br>
&gt; LocalDatanodeInfo&gt; localDatanodeInfoMap&quot; property to store loc=
al block file<br>
&gt; path and local meta file path. When I stop HDFS cluster or I kill the =
local<br>
&gt; DataNode and delete file use &quot;./hadoop dfs -rm path&quot; command=
 , the DFSClient<br>
&gt; still can read the data form local file. I think that may lead to DFSC=
lient<br>
&gt; read wrong data.<br>
&gt;<br>
&gt; I think we should fix the problem.<br>
&gt;<br>
&gt;<br>
&gt; Thanks,<br>
&gt;<br>
&gt; LiuLei<br>
</div></div></blockquote></div><br>

--e89a8f3b9b5583304104cabb84f2--