cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mike Heffner (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-7385) sstableloader OutOfMemoryError: Java heap space
Date Thu, 12 Jun 2014 22:19:02 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-7385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14029924#comment-14029924
] 

Mike Heffner commented on CASSANDRA-7385:
-----------------------------------------

Hopefully 100GB of sstables is not too large a job typically. ;-)

I still think that faulting with an untrapped exception here is a pretty bad user experience.
What if sstableloader took a parameter that allowed the user to set the max-heap size larger
if their job size requires it? A hardcoded value here implies that it should be able to handle
jobs of any size. If it could further trap this known exception and tell the user to increase
the size with the parameter that would be even better.

> sstableloader OutOfMemoryError: Java heap space
> -----------------------------------------------
>
>                 Key: CASSANDRA-7385
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7385
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Tools
>            Reporter: Mike Heffner
>
> We hit the following exception with sstableloader while attempting to bulk load about
100GB of SSTs. We are now employing this workaround before starting an sstableloader run:
> sed -i -e 's/-Xmx256M/-Xmx8G/g' /usr/bin/sstableloader
> {code}
> ERROR 19:25:45,060 Error in ThreadPoolExecutor
> java.lang.OutOfMemoryError: Java heap space
> 	at org.apache.cassandra.io.util.FastByteArrayOutputStream.expand(FastByteArrayOutputStream.java:104)
> 	at org.apache.cassandra.io.util.FastByteArrayOutputStream.write(FastByteArrayOutputStream.java:235)
> 	at java.io.DataOutputStream.writeInt(DataOutputStream.java:199)
> 	at org.apache.cassandra.io.compress.CompressionMetadata$ChunkSerializer.serialize(CompressionMetadata.java:412)
> 	at org.apache.cassandra.io.compress.CompressionMetadata$ChunkSerializer.serialize(CompressionMetadata.java:407)
> 	at org.apache.cassandra.streaming.compress.CompressionInfo$CompressionInfoSerializer.serialize(CompressionInfo.java:59)
> 	at org.apache.cassandra.streaming.compress.CompressionInfo$CompressionInfoSerializer.serialize(CompressionInfo.java:46)
> 	at org.apache.cassandra.streaming.PendingFile$PendingFileSerializer.serialize(PendingFile.java:142)
> 	at org.apache.cassandra.streaming.StreamHeader$StreamHeaderSerializer.serialize(StreamHeader.java:67)
> 	at org.apache.cassandra.streaming.StreamHeader$StreamHeaderSerializer.serialize(StreamHeader.java:58)
> 	at org.apache.cassandra.net.MessagingService.constructStreamHeader(MessagingService.java:782)
> 	at org.apache.cassandra.streaming.compress.CompressedFileStreamTask.stream(CompressedFileStreamTask.java:65)
> 	at org.apache.cassandra.streaming.FileStreamTask.runMayThrow(FileStreamTask.java:91)
> 	at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at java.lang.Thread.run(Thread.java:745)
> Exception in thread "Streaming to /10.167.a.b:1" java.lang.OutOfMemoryError: Java heap
space
> 	at org.apache.cassandra.io.util.FastByteArrayOutputStream.expand(FastByteArrayOutputStream.java:104)
> 	at org.apache.cassandra.io.util.FastByteArrayOutputStream.write(FastByteArrayOutputStream.java:235)
> 	at java.io.DataOutputStream.writeInt(DataOutputStream.java:199)
> 	at org.apache.cassandra.io.compress.CompressionMetadata$ChunkSerializer.serialize(CompressionMetadata.java:412)
> 	at org.apache.cassandra.io.compress.CompressionMetadata$ChunkSerializer.serialize(CompressionMetadata.java:407)
> 	at org.apache.cassandra.streaming.compress.CompressionInfo$CompressionInfoSerializer.serialize(CompressionInfo.java:59)
> 	at org.apache.cassandra.streaming.compress.CompressionInfo$CompressionInfoSerializer.serialize(CompressionInfo.java:46)
> 	at org.apache.cassandra.streaming.PendingFile$PendingFileSerializer.serialize(PendingFile.java:142)
> 	at org.apache.cassandra.streaming.StreamHeader$StreamHeaderSerializer.serialize(StreamHeader.java:67)
> 	at org.apache.cassandra.streaming.StreamHeader$StreamHeaderSerializer.serialize(StreamHeader.java:58)
> 	at org.apache.cassandra.net.MessagingService.constructStreamHeader(MessagingService.java:782)
> 	at org.apache.cassandra.streaming.compress.CompressedFileStreamTask.stream(CompressedFileStreamTask.java:65)
> 	at org.apache.cassandra.streaming.FileStreamTask.runMayThrow(FileStreamTask.java:91)
> 	at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at java.lang.Thread.run(Thread.java:745)
> Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
> 	at org.apache.cassandra.io.compress.CompressionMetadata.getChunksForSections(CompressionMetadata.java:210)
> 	at org.apache.cassandra.streaming.StreamOut.createPendingFiles(StreamOut.java:182)
> 	at org.apache.cassandra.streaming.StreamOut.transferSSTables(StreamOut.java:157)
> 	at org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:145)
> 	at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:67)
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message