cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benjamin Roth <benjamin.r...@jaumo.com>
Subject Re: Java GC pauses, reality check
Date Sat, 26 Nov 2016 07:36:46 GMT
Thanks!

But getting back to the original issue:
I think the GC itself is not the root cause for such a long pause. I
remember having had issues with 1 minute GCs in the beginning. I also tried
around with larger and smaller heap sizes and different GCs (G1, CMS),
different settings but what helped in the end (as far as I remember -
please nail me down on that) was to increase memtable flush writers.
I could explain it like that:
If available mem is getting fuller and fuller, the GC has to ran more often
and longer to reclaim the last available bit that is currently required.
Memtables use a considerable amount of it and if they can't be flushed in
time, they grow and use more and more memory putting more and more pressure
on the GC - also known as the GC death spiral. In my case I never ran in an
OOM crash but the node became totally unresponsive.

I don't tell this must be the case here but it is one possible case.

P.S.: In my case memtable flush writers default was 2 AFAIR as I had only
one SSD but the node could easily handle many more with 8 real cores and an
SSD.

2016-11-26 7:52 GMT+01:00 Work <jrothering@codojo.me>:

> I'm not affiliated with them, I've just been impressed by them. They have
> done amazing work in performance measurement. They discovered a major flaw
> in most performance testing ... I've never seen their pricing. But,
> recently, they made their product available for testing by developers. And
> the assured me that pricing is on a sliding scale depending upon
> utilization, and not ridiculous.
>
> - James
>
> Sent from my iPhone
>
> On Nov 25, 2016, at 10:40 PM, Benjamin Roth <benjamin.roth@jaumo.com>
> wrote:
>
> This sounds amazing but also expensive - I don't see pricing on their
> page. Are you able and allowed to tell a rough pricing range?
>
> Am 26.11.2016 04:33 schrieb "Harikrishnan Pillai" <HPillai@walmartlabs.com
> >:
>
>> We are running azul zing in prod with 1 million reads/s and 100 K
>> writes/s with azul .we never had a major gc above 10 ms .
>>
>> Sent from my iPhone
>>
>> > On Nov 25, 2016, at 3:49 PM, Martin Schröder <martin@oneiros.de> wrote:
>> >
>> > 2016-11-25 23:38 GMT+01:00 Kant Kodali <kant@peernova.com>:
>> >> I would also restate the following sentence "java GC pauses are pretty
>> much
>> >> a fact of life" to "Any GC based system pauses are pretty much a fact
>> of
>> >> life".
>> >>
>> >> I would be more than happy to see if someone can counter prove.
>> >
>> > Azul disagrees.
>> > https://www.azul.com/products/zing/pgc/
>> >
>> > Best
>> >   Martin
>>
>


-- 
Benjamin Roth
Prokurist

Jaumo GmbH · www.jaumo.com
Wehrstraße 46 · 73035 Göppingen · Germany
Phone +49 7161 304880-6 · Fax +49 7161 304880-1
AG Ulm · HRB 731058 · Managing Director: Jens Kammerer

Mime
View raw message