hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kohki Nishio <tarop...@icloud.com>
Subject Re: Scan returns incomplete results
Date Fri, 14 Apr 2017 18:43:52 GMT
Here I have trace output, the exact consecutive queries got two different results, not sure
where to look about my region servers.. 


18:30:00,073 o.a.h.h.c.ClientScanner: Scan table=mytable, startRow=AAAAAZJRJgq702gz9Qq-bXvyvi5Vfwg-\x00\x00\x00\x00X\xF1\x06\xDC
18:30:00,075 o.a.h.h.i.AbstractRpcClient: Call: Scan, callTime: 1ms
18:30:00,075 o.a.h.h.c.ScannerCallableWithReplicas: Setting current scanner as id=692411060
associated with replica=0
18:30:00,076 o.a.h.h.i.AbstractRpcClient: Call: Scan, callTime: 1ms
18:30:00,076 o.a.h.h.c.ScannerCallableWithReplicas: Setting current scanner as id=692411060
associated with replica=0
18:30:00,077 o.a.h.h.c.ClientScanner: number results from RPC: 50,partial != null: false,number
of partials so far: 0
18:30:00,078 o.a.h.h.i.AbstractRpcClient: Call: Scan, callTime: 1ms
18:30:00,078 o.a.h.h.c.ScannerCallableWithReplicas: Setting current scanner as id=692411060
associated with replica=0
18:30:00,078 o.a.h.h.c.ClientScanner: number results from RPC: 6,partial != null: false,number
of partials so far: 0
18:30:00,078 o.a.h.h.c.ScannerCallableWithReplicas: Closing scanner id=692411060
18:30:00,079 o.a.h.h.i.AbstractRpcClient: Call: Scan, callTime: 1ms
18:30:00,079 o.a.h.h.c.ClientScanner: Finished {ENCODED => c6e51623c143984e36c21340b182d2c7,
NAME => 'mytable,AAAAAZ-L4MCUr7LCj8ZzmWJdhOqML0fy\x00\x00\x00\x00X\xE8A\x8D,1492193926115.c6e51623c143984e36c21340b182d2c7.',
STARTKEY => 'AAAAAZ-L4MCUr7LCj8ZzmWJdhOqML0fy\x00\x00\x00\x00X\xE8A\x8D', ENDKEY =>
'AAAAAZzZUufbQvQaVdhpt2wUk1CkRcNz\x00\x00\x00\x00X\xE9\xE71'}


18:30:00,458 o.a.h.h.c.ClientScanner: Scan table=mytable, startRow=AAAAAZJRJgq702gz9Qq-bXvyvi5Vfwg-\x00\x00\x00\x00X\xF1\x06\xDC
18:30:00,459 o.a.h.h.i.AbstractRpcClient: Call: Scan, callTime: 1ms
18:30:00,459 o.a.h.h.c.ScannerCallableWithReplicas: Setting current scanner as id=692411074
associated with replica=0
18:30:00,461 o.a.h.h.i.AbstractRpcClient: Call: Scan, callTime: 1ms
18:30:00,461 o.a.h.h.c.ScannerCallableWithReplicas: Setting current scanner as id=692411074
associated with replica=0
18:30:00,461 o.a.h.h.c.ClientScanner: number results from RPC: 44,partial != null: false,number
of partials so far: 0
18:30:00,461 o.a.h.h.c.ScannerCallableWithReplicas: Closing scanner id=692411074
18:30:00,461 o.a.h.h.i.AbstractRpcClient: Call: Scan, callTime: 0ms
18:30:00,461 o.a.h.h.c.ClientScanner: Finished {ENCODED => c6e51623c143984e36c21340b182d2c7,
NAME => 'mytable,AAAAAZ-L4MCUr7LCj8ZzmWJdhOqML0fy\x00\x00\x00\x00X\xE8A\x8D,1492193926115.c6e51623c143984e36c21340b182d2c7.',
STARTKEY => 'AAAAAZ-L4MCUr7LCj8ZzmWJdhOqML0fy\x00\x00\x00\x00X\xE8A\x8D', ENDKEY =>
'AAAAAZzZUufbQvQaVdhpt2wUk1CkRcNz\x00\x00\x00\x00X\xE9\xE71'}



On Apr 13, 2017, at 08:09 PM, Kohki Nishio <taroplus@icloud.com> wrote:

Actually I tried the one with HBASE-15378, but it didn’t fix the problem ..


On Apr 12, 2017, at 8:14 PM, Kohki Nishio <taroplus@icloud.com> wrote:

As far as I can see, the change is in ClientScanner class, which is a part of Hbase-client,
does this work inside Server (region server?) ?

Thanks
-Kohki


On Apr 12, 2017, at 6:03 PM, Ted Yu <yuzhihong@gmail.com> wrote:

Does the release (deployed on servers) contain HBASE-15378 ?

Cheers

On Wed, Apr 12, 2017 at 5:34 PM, Kohki Nishio <taroplus@icloud.com> wrote:

Hello HBase users !

I’m seeing very strange behavior with HBase (1.1.2) and I’d like to ask
some help here. I’m scanning rows but sometimes it returns incomplete
results. Let’s say I’m expecting 60 rows to come back, most of times I get
all of them. However sometimes (1 in 50), I get only 1 or 2 rows.

According to my test, this happens only when there are other activities
(lots of PUT / GET are happening at the same time), I couldn’t reproduce
with a simple standalone scan test case. First we saw this problem with
AsyncHBase client then I tried v1.3 HBase client, both showed the exact
same behavior.

Based on the trace log message, it seems this is not a problem at client
side since two different client implementations showed the same behavior.
However HBase is an established storage solution, it is so strange to see
this problem at this frequency, that leads me to believe that I might be
doing something very strange. But I’m not sure what to do next.

The key symptom is that it returns the correct result for most of times,
but sometimes it returns very small set of results. Any thoughts would be
appreciated (RegionServer is at 1.1.2.x.x)

Thanks
-Kohki



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
    • Unnamed multipart/related (inline, None, 0 bytes)
View raw message