hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Qiang Tian <tian...@gmail.com>
Subject Re: hbase region servers refuse connection
Date Tue, 15 Jul 2014 03:48:45 GMT
Hi YuMing,

it looks 0.94 is immune to this problem.  if read 0 bytes, just returns

readAndProcess:
        count = channelRead(channel, data);

        if (data.remaining() == 0) {
          dataLengthBuffer.clear();
          data.flip();
          if (headerRead) {
            processData(data.array());
            data = null;
            return count;
          }
          processHeader();
          headerRead = true;
          data = null;
          continue;
        }
        return count;
      }



On Tue, Jul 15, 2014 at 11:06 AM, Qiang Tian <tianq01@gmail.com> wrote:

> Hi, below is more details.
>
> the read0 stacktrace you see means reader wants to read something from
> client RPC call. in Andrew's test it shows it is in reading RPC request
> data (reasonable. since other meta data size is quite small).  although
> client follows request-receive style, when multiple clients share the
> connection(the default case), the synchronization window when writing to
> the same channel is quite small. if those request data have big size, there
> might be a sudden rush to the transportation layer..might causing RPC
> server could not receive the data in time due to congestion control,
> without hbase11277, the reader get 0 byte read again and again...
>
> with hbase11277 the problem should be resolved - we get back to complete
> non-blocking IO, but it is still worth investigation non-shared connection
> under high workload(hbase11306).
>
>
>
> On Tue, Jul 15, 2014 at 10:44 AM, Rural Hunter <ruralhunter@gmail.com>
> wrote:
>
>> No. My issue is definitely a server side issue. When the problem happens
>> I'm not able to telnet the data port of RS.
>>
>> 于 2014/7/15 10:24, Qiang Tian 写道:
>>
>>  not sure if you could also try using different connections for your
>>> client
>>> program. see the related issue HBASE-11306. I guess sharing connection
>>>  is
>>> a contributing factor for this issue.
>>>
>>>
>>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message