accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Turner <>
Subject Re: Completed 2nd long run of Fluo
Date Mon, 06 Jun 2016 19:33:18 GMT
On Fri, Jun 3, 2016 at 6:34 PM, Josh Elser <> wrote:

> Great stuff, Keith! Looking forward to picking it apart in more detail:
> bq. In the previous run Fluo worker processes were constantly being killed
> by YARN for exceeding memory limits
> I remember stumbling across some (surprising) recommendations which were
> to just completely disable the vmem/pmem monitoring to avoid this. Nice
> that you were actually able to work around it "properly".

I am not quite sure why the workers processes are going over the heap usage
by so much, because no native code should be in use.   One possibility is
thread stacks.   The -Xmx jvm option only controls heap usage.  If there
are lots of threads where the stack has been deep in the past, then those
threads may continue to use a good bit of memory.

> Keith Turner wrote:
>> A 2nd multi-day test of Fluo was completed on EC2 a week or two ago.  I
>> just got around to finishing writing it up. The test was run using
>> Accumulo 1.7.1.

View raw message