Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (nike.apache.org: domain of brian.jeltema@digitalenvoy.net
 designates 64.88.168.16 as permitted sender)
From: Brian Jeltema <brian.jeltema@digitalenvoy.net>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: quoted-printable
Subject: hadoop inserts blow out heap
Date: Wed, 12 Sep 2012 07:52:39 -0400
Message-Id: <4CB164F1-B76F-47D3-9F1F-7E12D4F1203E@digitalenvoy.net>
To: user@cassandra.apache.org
Mime-Version: 1.0 (Apple Message framework v1278)

I'm a fairly novice Cassandra/Hadoop guy. I have written a Hadoop job =
(using the Cassandra/Hadoop integration API)
that performs a full table scan and attempts to populate a new table =
from the results of the map/reduce. The read
works fine and is fast, but the table insertion is failing with OOM =
errors (in the Cassandra VM). The resulting heap dump from one node =
shows that
2.9G of the heap is consumed by a JMXConfigurableThreadPoolExecutor that =
appears to be full of batch mutations.

I'm using a 6-node cluster, 32G per node, 8G heap, RF=3D3, if any of =
that matters.

Any suggestions would be appreciated regarding configuration changes or =
additional information I might
capture to understand this problem.

Thanks

Brian J=