hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: scanner deadlock?
Date Wed, 14 Sep 2011 15:36:39 GMT
On Tue, Sep 13, 2011 at 10:56 PM, Geoff Hendrey <ghendrey@decarta.com> wrote:
> Anything in its logs when regionserver slows down?
> ANSWER: Yes. I see ScannerTimeoutException, and unknown scanner, and then ClosedChannelException.
Stack trace shows the ClosedChannelException occurs when the server tries to write the response
to the scanner. This seems like a bug to me. Once you close the channel you cannot write it,
no? If you try to write it after you close it, you will get ClosedChannelException.

I meant to say in the accompanying datanode logs.

On the CCE, yes.  We're trying to write the response.  You have set
down the timeout so client goes away faster.  Only way to figure
socket gone is to try to use it.  Hence CCE.

> ANSWER: Nope, no long pauses at all. I've periodically run a few greps with a regex to
try to find pauses one second or longer, and I haven't seen any of late. HOWEVER, one thing
I don't understand at all is why ganglia reports HUGE gc pauses (like 1000 seconds!). But
in reality I can *never* find such a pause in the GC log. Is there any known issue with ganglia
graphs being on the wrong vertical scale for GC pauses? I know that sounds odd, but I just
can't correlate the Ganglia graphs for GC to reality.

Why don't you send us some logs that span a slow down.  GC, RS, and
DN.  Send jstacks too.


View raw message