Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of magnito@gmail.com designates
 209.85.220.169 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAHyy+G8wPjzEsiqFuxi8ayK=0XoYm0hkE3dw0sUtib=4TS4+ow@mail.gmail.com>
References: 
 <CAHyy+G-Y+8bGvT5v-8Y6hHS+_=iJ8w1nrJ+9PKv5ZP-50F_=nA@mail.gmail.com>
	<CAHyy+G82DQwF=L9VJqJfTdv1ZzY0wWwpuyYNMwR20HA43U3CPA@mail.gmail.com>
	<CADcMMgEFh8+kJ-wwLBtRoaP+p1+jDeNYhLYTijhMMt19_Tfyew@mail.gmail.com>
	<CAHyy+G-MiN_WE=vKLtNRw9VyAxdMWRQ+xYyiiRrSs+K4bWvRqw@mail.gmail.com>
	<CADcMMgHVEexGkpexAoxZh9vXTgDe=QxM7JnHWiEYjG3ikWBKRA@mail.gmail.com>
	<CAHyy+G9dVOz+hiQ7dSs1h6ns5h0d_891Kjzf+cNSJPaMBR8nsQ@mail.gmail.com>
	<CAHyy+G_Q9FQR55c=hwbgx7ZFi9qGRbPJE7VfrLoaqcObqopaqg@mail.gmail.com>
	<CAHyy+G9-4rgfZa1UZsA_2kj3j6TiAqOPNeYp1rAkxsjB7SNd4A@mail.gmail.com>
	<CADcMMgHpJ16BZp7z4UatcrF7T0wGfVxiwdZ_mkX4Zq54ioe8ug@mail.gmail.com>
	<CAHyy+G9N8iGa4bZmAC91LsPep+Eukkg4owav1mOToDAJDVZEbw@mail.gmail.com>
	<CAHyy+G9XNokkBroL6SGYqAOSs1U01zpHxTkPQwip5ri8n0+R+Q@mail.gmail.com>
	<CAHyy+G9vfrW8K1pdKD7bbL=_BtLFKMatHQUQAd2ampowXW=ozA@mail.gmail.com>
	<CADcMMgFs9G4D41xMU-pJVQbgMNd7pv5DO_8FyALJHApK_KHLHQ@mail.gmail.com>
	<CA+4kjVsWCNnFyGF9b9+GOEABUdObOAVJaY+aDMb1-FBg1tDRbg@mail.gmail.com>
	<CAHyy+G8nUYhSyar-8YZ9ed5ps-qadVWkWYNaiwx8Bh=u=t4h+Q@mail.gmail.com>
	<CAHyy+G8wPjzEsiqFuxi8ayK=0XoYm0hkE3dw0sUtib=4TS4+ow@mail.gmail.com>
Date: Thu, 27 Feb 2014 22:38:59 -0800
Message-ID: 
 <CAHyy+G-NeHFBoSNobYL9zh0WMGnUFDO1UamMc4hkqsXX-DP6hQ@mail.gmail.com>
Subject: Re: Question about dead datanode
From: Jack Levin <magnito@gmail.com>
To: "user@hbase.apache.org" <user@hbase.apache.org>
Content-Type: text/plain; charset=ISO-8859-1

Is this related to JIRA HDFS-378?

On Wed, Feb 26, 2014 at 11:54 AM, Jack Levin <magnito@gmail.com> wrote:
> Submitted JIRA patch: https://issues.apache.org/jira/browse/HDFS-6022
> (with test)
>
> On Mon, Feb 24, 2014 at 12:16 PM, Jack Levin <magnito@gmail.com> wrote:
>> I will do that.
>>
>> -Jack
>>
>> On Mon, Feb 24, 2014 at 6:23 AM, Steve Loughran <stevel@hortonworks.com> wrote:
>>> that's a very old version of cloudera's branch you are working with there;
>>> patching that is not a good way to go, as you are on the slippery slope of
>>> having your own private branch and all the costs of it.
>>>
>>> It looks like dead node logic has -> DFSInputStream, where it is still
>>> instance-specific:
>>>
>>>   /* XXX Use of CocurrentHashMap is temp fix. Need to fix
>>>    * parallel accesses to DFSInputStream (through ptreads) properly */
>>>   private final ConcurrentHashMap<DatanodeInfo, DatanodeInfo> deadNodes =
>>>              new ConcurrentHashMap<DatanodeInfo, DatanodeInfo>();
>>>
>>> This implies the problem still exists -and the opportunity to fix it -but
>>> you will need to modify your patch to apply to hadoop trunk, ideally think
>>> of a test, then submit a patch to the HDFS project on JIRA.
>>>
>>>
>>> On 19 February 2014 04:48, Stack <stack@duboce.net> wrote:
>>>
>>>> On Sat, Feb 15, 2014 at 8:01 PM, Jack Levin <magnito@gmail.com> wrote:
>>>>
>>>> > Looks like I patched it in DFSClient.java, here is the patch:
>>>> > https://gist.github.com/anonymous/9028934
>>>> >
>>>> > ....
>>>>
>>>>
>>>> > I moved 'deadNodes' list outside as global field that is accessible by
>>>> > all running threads, so at any point datanode does go down, each
>>>> > thread is basically informed that the datanode _is_ down.
>>>> >
>>>>
>>>> We need to add something like this to current versions of DFSClient, a
>>>> global status, so each stream does not have to discover bad DNs for itself.
>>>> St.Ack
>>>>
>>>
>>> --
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity to
>>> which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender immediately
>>> and delete it from your system. Thank You.