accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Busbey <busbey+li...@cloudera.com>
Subject Re: Bulk ingest losing tablet server
Date Mon, 13 Jan 2014 19:44:22 GMT
On Mon, Jan 13, 2014 at 12:02 PM, Anthony F <afccri@gmail.com> wrote:

> Yes, system swappiness is set to 0.  I'll run again and gather more logs.
>
> Is there a zookeeper timeout setting that I can adjust to avoid this issue
> and is that advisable?  Basically, the tservers are colocated with HDFS
> datanodes and Hadoop nodemanagers.  The machines are overallocated in terms
> of RAM.  So, I have a feeling that when a map-reduce job is kicked off, it
> causes the tserver to page out to swap space.  Once the map-reduce job
> finishes and the bulk ingest is kicked off, the tserver is paged back in
> and the ZK timeout causes a shutdown.
>
>
>
You should not overallocate the amount of memory on the machines.
Generally, you should provide memory limits under teh assumption that
everything will be on at once.

Many parts of Hadoop (not just Accumulo) will degrade or malfunction in the
presence of memory swapping.

How much of hte 12GB for Accumulo is for native memmaps?

Mime
View raw message