accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Turner <ke...@deenlo.com>
Subject Re: Completed 2nd long run of Fluo
Date Mon, 06 Jun 2016 19:33:18 GMT
On Fri, Jun 3, 2016 at 6:34 PM, Josh Elser <josh.elser@gmail.com> wrote:

> Great stuff, Keith! Looking forward to picking it apart in more detail:
>
> bq. In the previous run Fluo worker processes were constantly being killed
> by YARN for exceeding memory limits
>
> I remember stumbling across some (surprising) recommendations which were
> to just completely disable the vmem/pmem monitoring to avoid this. Nice
> that you were actually able to work around it "properly".


I am not quite sure why the workers processes are going over the heap usage
by so much, because no native code should be in use.   One possibility is
thread stacks.   The -Xmx jvm option only controls heap usage.  If there
are lots of threads where the stack has been deep in the past, then those
threads may continue to use a good bit of memory.


>
>
> Keith Turner wrote:
>
>> A 2nd multi-day test of Fluo was completed on EC2 a week or two ago.  I
>> just got around to finishing writing it up. The test was run using
>> Accumulo 1.7.1.
>>
>> http://fluo.io/blog/2016/05/17/webindex-long-run-2
>>
>

Mime
View raw message