Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of magnito@gmail.com designates
 209.85.128.179 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CA+4kjVsWCNnFyGF9b9+GOEABUdObOAVJaY+aDMb1-FBg1tDRbg@mail.gmail.com>
References: 
 <CAHyy+G-Y+8bGvT5v-8Y6hHS+_=iJ8w1nrJ+9PKv5ZP-50F_=nA@mail.gmail.com>
	<CAHyy+G82DQwF=L9VJqJfTdv1ZzY0wWwpuyYNMwR20HA43U3CPA@mail.gmail.com>
	<CADcMMgEFh8+kJ-wwLBtRoaP+p1+jDeNYhLYTijhMMt19_Tfyew@mail.gmail.com>
	<CAHyy+G-MiN_WE=vKLtNRw9VyAxdMWRQ+xYyiiRrSs+K4bWvRqw@mail.gmail.com>
	<CADcMMgHVEexGkpexAoxZh9vXTgDe=QxM7JnHWiEYjG3ikWBKRA@mail.gmail.com>
	<CAHyy+G9dVOz+hiQ7dSs1h6ns5h0d_891Kjzf+cNSJPaMBR8nsQ@mail.gmail.com>
	<CAHyy+G_Q9FQR55c=hwbgx7ZFi9qGRbPJE7VfrLoaqcObqopaqg@mail.gmail.com>
	<CAHyy+G9-4rgfZa1UZsA_2kj3j6TiAqOPNeYp1rAkxsjB7SNd4A@mail.gmail.com>
	<CADcMMgHpJ16BZp7z4UatcrF7T0wGfVxiwdZ_mkX4Zq54ioe8ug@mail.gmail.com>
	<CAHyy+G9N8iGa4bZmAC91LsPep+Eukkg4owav1mOToDAJDVZEbw@mail.gmail.com>
	<CAHyy+G9XNokkBroL6SGYqAOSs1U01zpHxTkPQwip5ri8n0+R+Q@mail.gmail.com>
	<CAHyy+G9vfrW8K1pdKD7bbL=_BtLFKMatHQUQAd2ampowXW=ozA@mail.gmail.com>
	<CADcMMgFs9G4D41xMU-pJVQbgMNd7pv5DO_8FyALJHApK_KHLHQ@mail.gmail.com>
	<CA+4kjVsWCNnFyGF9b9+GOEABUdObOAVJaY+aDMb1-FBg1tDRbg@mail.gmail.com>
Date: Mon, 24 Feb 2014 12:16:55 -0800
Message-ID: 
 <CAHyy+G8nUYhSyar-8YZ9ed5ps-qadVWkWYNaiwx8Bh=u=t4h+Q@mail.gmail.com>
Subject: Re: Question about dead datanode
From: Jack Levin <magnito@gmail.com>
To: "user@hbase.apache.org" <user@hbase.apache.org>
Content-Type: text/plain; charset=ISO-8859-1

I will do that.

-Jack

On Mon, Feb 24, 2014 at 6:23 AM, Steve Loughran <stevel@hortonworks.com> wrote:
> that's a very old version of cloudera's branch you are working with there;
> patching that is not a good way to go, as you are on the slippery slope of
> having your own private branch and all the costs of it.
>
> It looks like dead node logic has -> DFSInputStream, where it is still
> instance-specific:
>
>   /* XXX Use of CocurrentHashMap is temp fix. Need to fix
>    * parallel accesses to DFSInputStream (through ptreads) properly */
>   private final ConcurrentHashMap<DatanodeInfo, DatanodeInfo> deadNodes =
>              new ConcurrentHashMap<DatanodeInfo, DatanodeInfo>();
>
> This implies the problem still exists -and the opportunity to fix it -but
> you will need to modify your patch to apply to hadoop trunk, ideally think
> of a test, then submit a patch to the HDFS project on JIRA.
>
>
> On 19 February 2014 04:48, Stack <stack@duboce.net> wrote:
>
>> On Sat, Feb 15, 2014 at 8:01 PM, Jack Levin <magnito@gmail.com> wrote:
>>
>> > Looks like I patched it in DFSClient.java, here is the patch:
>> > https://gist.github.com/anonymous/9028934
>> >
>> > ....
>>
>>
>> > I moved 'deadNodes' list outside as global field that is accessible by
>> > all running threads, so at any point datanode does go down, each
>> > thread is basically informed that the datanode _is_ down.
>> >
>>
>> We need to add something like this to current versions of DFSClient, a
>> global status, so each stream does not have to discover bad DNs for itself.
>> St.Ack
>>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.