Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of tianq01@gmail.com designates
 209.85.192.47 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <53C49596.8050505@gmail.com>
References: 
 <CADJ9AzOKn=zoiUMzEZ_9zim4WSXp8n221NOVKu-Sst=JxU_AYg@mail.gmail.com>
	<CADJ9AzPVQJ_Z=+KjXWiCRTnVEF5e0-GO5QoYPk9UZdC0B6wTkA@mail.gmail.com>
	<53C1FF5D.9060408@gmail.com>
	<1405282709523-4061293.post@n3.nabble.com>
	<53C33165.6010400@gmail.com>
	<CAMZUsP7VMASOgEMoS4H1nLf8GfZGbXeyMVZK+TYTGSw6nzmqHw@mail.gmail.com>
	<53C3BCAC.3050604@gmail.com>
	<CAMZUsP5Nd9Emub+yRRK1JAVqJD_kERqY-LKx+8QVrojnFHwwqA@mail.gmail.com>
	<53C49596.8050505@gmail.com>
Date: Tue, 15 Jul 2014 11:06:12 +0800
Message-ID: 
 <CAMZUsP43kfkS2p0f6Bs9+KhtFC7a3hfWyiK7p7HUAN_jFsgr=g@mail.gmail.com>
Subject: Re: hbase region servers refuse connection
From: Qiang Tian <tianq01@gmail.com>
To: "user@hbase.apache.org" <user@hbase.apache.org>
Content-Type: multipart/alternative; boundary=001a11c14eaea4a6f704fe32b0d7

--001a11c14eaea4a6f704fe32b0d7
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi, below is more details.

the read0 stacktrace you see means reader wants to read something from
client RPC call. in Andrew's test it shows it is in reading RPC request
data (reasonable. since other meta data size is quite small).  although
client follows request-receive style, when multiple clients share the
connection(the default case), the synchronization window when writing to
the same channel is quite small. if those request data have big size, there
might be a sudden rush to the transportation layer..might causing RPC
server could not receive the data in time due to congestion control,
without hbase11277, the reader get 0 byte read again and again...

with hbase11277 the problem should be resolved - we get back to complete
non-blocking IO, but it is still worth investigation non-shared connection
under high workload(hbase11306).


On Tue, Jul 15, 2014 at 10:44 AM, Rural Hunter <ruralhunter@gmail.com>
wrote:

> No. My issue is definitely a server side issue. When the problem happens
> I'm not able to telnet the data port of RS.
>
> =E4=BA=8E 2014/7/15 10:24, Qiang Tian =E5=86=99=E9=81=93:
>
>  not sure if you could also try using different connections for your clie=
nt
>> program. see the related issue HBASE-11306. I guess sharing connection  =
is
>> a contributing factor for this issue.
>>
>>
>>
>

--001a11c14eaea4a6f704fe32b0d7--