hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Rawson <ryano...@gmail.com>
Subject Re: GC makes ryan sad
Date Sat, 25 Apr 2009 19:47:32 GMT
The incremental GC doesn't help on a multicore system, it's designed to
break up the giant concurrent phase (which I've seen last 5 minutes) on
systems with 1 or 2 CPUs to prevent the concurrent GC from starving the app
for CPU.  My machines have 8 cores, so this won't really help.

I talked to a friend, I realize the problem is the ParNew is growing too
big... It starts out at about 15 MB and grows 10x to 150MB.  My friend
suggested that I size ParNew to the L2 cache size, which is 6 MB on my Intel
Xeon CPUs - not sure if the L2 size is shared between pairs of cores or
not.

Switching to a much smaller ParNew has several effects:
- GC pauses are small, tiny, 1ms ish
- GC of ParNew happens 1-10x a second
- Overall memory use goes up as more stuff enters the general heap and has
to wait for the CMS to reclaim it. I watched the CMS prune 1 GB of garbage
in 3 seconds.

However, the crippling pauses that were keeping my cluster from performing
are gone.  At the cost of higher ram usage.

On Sat, Apr 25, 2009 at 10:09 AM, Chad Walters
<Chad.Walters@microsoft.com>wrote:

>
> Done any experimentation with the incremental GC?
>
> Chad
>
> On 4/23/09 6:28 PM, "Ryan Rawson" <ryanobjc@gmail.com> wrote:
>
> From a log of 0.20 in action:
>
> 11061.089: [GC 11061.089: [ParNew: 153331K->17024K(153344K), 0.2249580
> secs]
> 3396126K->3309484K(4579960K) icms_dc=0 , 0.2251020 secs] [Times: user=0.66
> sys=0.04, real=0.23 secs]
> 11062.439: [GC 11062.439: [ParNew: 153344K->17024K(153344K), 0.1625480
> secs]
> 3445804K->3335150K(4579960K) icms_dc=0 , 0.1627020 secs] [Times: user=0.37
> sys=0.02, real=0.17 secs]
>
>
> Yes, in 1.4 seconds we had a 225ms pause and a 162ms pause.  This is not
> atypical.  Most pauses are in the 80-150ms area.
>
> -ryan
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message