incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ben Gambley <ben.gamb...@intoscience.com>
Subject Bulk Loader - OutOfMemoryError: Java heap space
Date Thu, 01 Aug 2013 00:25:55 GMT
Hi All

We are using Cassandra 1.2.4 and are seeing the following error loading  a pretty small amount
of data.

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
	at org.apache.cassandra.utils.obs.OpenBitSet.<init>(OpenBitSet.java:76)
	at org.apache.cassandra.utils.obs.OpenBitSet.deserialize(OpenBitSet.java:429)
	at org.apache.cassandra.utils.BloomFilterSerializer.deserialize(BloomFilterSerializer.java:46)
	at org.apache.cassandra.utils.Murmur3BloomFilter$Murmur3BloomFilterSerializer.deserialize(Murmur3BloomFilter.java:38)
	at org.apache.cassandra.utils.FilterFactory.deserialize(FilterFactory.java:73)
	at org.apache.cassandra.db.RowIndexEntry$Serializer.deserialize(RowIndexEntry.java:122)
	at org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:376)
	at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:198)
	at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:151)
	at org.apache.cassandra.io.sstable.SSTableLoader$1.accept(SSTableLoader.java:98)
	at java.io.File.list(File.java:1010)
	at org.apache.cassandra.io.sstable.SSTableLoader.openSSTables(SSTableLoader.java:67)
	at org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:119)
	at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:67)


Source folder is

-rw-r--r-- 2 cassandra cassandra 1.1K Jul 26 17:25 Australia-ActivityGroupSummaryMathletics-ic-1-CompressionInfo.db
-rw-r--r-- 2 cassandra cassandra 1.8M Jul 26 17:25 Australia-ActivityGroupSummaryMathletics-ic-1-Data.db
-rw-r--r-- 2 cassandra cassandra  816 Jul 26 17:25 Australia-ActivityGroupSummaryMathletics-ic-1-Filter.db
-rw-r--r-- 2 cassandra cassandra  20K Jul 26 17:25 Australia-ActivityGroupSummaryMathletics-ic-1-Index.db
-rw-r--r-- 2 cassandra cassandra 4.3K Jul 26 17:25 Australia-ActivityGroupSummaryMathletics-ic-1-Statistics.db

Machine has 

root@data1:/# free -m
                     total       used       free     shared    buffers     cached
Mem:          3968       1525       2442          0         45        161


Is there a way of increasing the default memory available to stableloader, from what I can
see it might be 64MB. I have looked at attempting to increase using java -classpath but to
no avail.



Ben Gambley
Development Manager
Into Science
 
AUS Direct   +61 2 9922 4403
AUS mobile  +61 411 331 166
Email           ben.gambley@intoscience.com


Mime
View raw message