hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pere Kyle <p...@whisper.sh>
Subject Avoid GC Pauses on Scan MapReduces
Date Fri, 14 Nov 2014 01:54:31 GMT
Hi there,

Recently I have been experiencing instability when scanning our HBASE cluster. The table we
are trying to scan is 1.5B records 1TB, we have 12GB heap and 17 servers. Our GC options are
as so:
-XX:OnOutOfMemoryError=kill -9 %p -Xmx12000m -XX:+UseConcMarkSweepGC -Xmx12g -Xmx12g

The errors I am seeing in the logs:
2014-11-14 00:00:32,656 WARN org.apache.hadoop.hbase.util.Sleeper (regionserver60020): We
slept 17355ms instead of 3000ms, this is likely due to a long garbage collecting pause and
it's usually bad, see http://hbase.apache.org/book.html#trouble.rs.runtime.zkexpired
2014-11-14 00:00:32,663 WARN org.apache.hadoop.hbase.util.Sleeper (regionserver60020.periodicFlusher):
We slept 24734ms instead of 10000ms, this is likely due to a long garbage collecting pause
and it's usually bad, see http://hbase.apache.org/book.html#trouble.rs.runtime.zkexpired

We are using hbase.client.scanner.caching=1000. I suspect this may be a block cache issue.
My question is if/how to disable the block cache for the scan queries? This is taking out
writes and causing instability on the cluster.

Thanks,
Pere
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message