hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gibbon, Robert, VF-Group" <Robert.Gib...@vodafone.com>
Subject RE: question about region files
Date Mon, 11 Oct 2010 16:40:11 GMT

This smells of garbage and low memory.

See for ref a similar problem report here - http://kr.forums.oracle.com/forums/thread.jspa?messageID=2146733

How many rest servers do you have loading all of that data? AFAIK they're stateless and loadbalancable

#
"Gang worker#0 (Parallel GC Threads)" prio=10 tid=0x0000000041260400 nid=0x2412 runnable
#
 
#
"Gang worker#1 (Parallel GC Threads)" prio=10 tid=0x0000000041261c00 nid=0x2413 runnable
#
 
#
"Concurrent Mark-Sweep GC Thread" prio=10 tid=0x00000000412b6c00 nid=0x2414 runnable 

-----Original Message-----
From: Jack Levin [mailto:magnito@gmail.com]
Sent: Sun 10/10/2010 5:24 AM
To: user@hbase.apache.org
Subject: question about region files
 
We are importing lots and lots of data, 130 TB worth.   If we set
compaction limit to say 128, and blocking limit to say 200.  I know we
expect longer read times unless we use a bloomfilter, however, are
they any other detrimental performance issues to be expected?  Our
flush size limit is set to 100MB.  I notice that REST server sometimes
is unable to talk to regions if there are too many store files, is
that expected behavior?  During massive imports it seems that we run
into problems with REST server freezes (e.g. responds to tcp, but
neither to puts or gets), and CPU goes crazy high, and stays high
until we restart the REST server.  Any ideas?
(I've attached link to jstack trace in my previous email).

-Jack


Mime
  • Unnamed multipart/mixed (inline, None, 0 bytes)
View raw message