hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Rodionov <vrodio...@carrieriq.com>
Subject RE: Is this a long GC pause, or something else?
Date Tue, 10 Jun 2014 22:38:01 GMT
Unbelievable. Do you see the same with the latest OpenJDK?

Best regards,
Vladimir Rodionov
Principal Platform Engineer
Carrier IQ, www.carrieriq.com
e-mail: vrodionov@carrieriq.com

________________________________________
From: Otis Gospodnetic [otis.gospodnetic@gmail.com]
Sent: Tuesday, June 10, 2014 2:43 PM
To: user@hbase.apache.org
Subject: Re: Is this a long GC pause, or something else?

Does it repeat?
We are seeing this with u60 oracle JVM too!  SPM shows the whole JVM blocking for about 16
minutes every M minutes.

Otis



> On Jun 10, 2014, at 2:05 PM, Tom Brown <tombrown52@gmail.com> wrote:
>
> Last night a regionserver in my cluster stopped responding in a timely
> manner for about 20 minutes. I know that stop-the-world GC can cause this
> type of behavior, but 20 minutes seems excessive.
>
> The server is a 2 core VM with 16GB of RAM, (hbase max heap is 12GB). We
> are using the latest java 7 from oracle. HDFS is provided by an Isilon
> cluster.
>
> The server workload is read/write: the writing process reads all rows it is
> about to write, updates them if they exist, and then writes all the rows
> (replacing ones that were updated).
>
> The last messages before the pause were regarding an HLog roll:
>
> DEBUG org.apache.hadoop.hbase.regionserver.LogRoller: HLog roll requested
> INFO org.apache.hadoop.hbase.util.FSUtils: FileSystem doesn't support
> getDefaultReplication
> INFO org.apache.hadoop.hbase.util.FSUtils: FileSystem doesn't support
> getDefaultBlockSize
>
> During the next 20 minutes there were a handful of sporadic LruBlockCache
> stats messages but nothing else. After 20 minutes, normal operation resumed.
>
> Is 20 minutes for a GC pause expected given the operational load and
> machine specs? Could a GC pause include periodic log messages? If it wasn't
> a GC pause, what else could it be?
>
> --Tom

Confidentiality Notice:  The information contained in this message, including any attachments
hereto, may be confidential and is intended to be read only by the individual or entity to
whom this message is addressed. If the reader of this message is not the intended recipient
or an agent or designee of the intended recipient, please note that any review, use, disclosure
or distribution of this message or its attachments, in any form, is strictly prohibited. 
If you have received this message in error, please immediately notify the sender and/or Notifications@carrieriq.com
and delete or destroy any copy of this message and its attachments.

Mime
View raw message