incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From j...@javajet.com
Subject Re: java.lang.OutOfMemoryError: Map failed
Date Tue, 22 Jun 2010 00:56:46 GMT
<FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2><DIV>Daniel:</DIV><DIV>&nbsp;</DIV><DIV>Thanks.
That thread helped me solve my problem.</DIV><DIV>&nbsp;</DIV><DIV>I
was able to run&nbsp;a 700k&nbsp;MySQL record import without a single memory error.
</DIV><DIV>&nbsp;</DIV><DIV>I changed the following sections in
storage-conf.xml to fix the OutofMemory errors:</DIV><DIV>&nbsp;</DIV><DIV>&nbsp;&lt;DiskAccessMode&gt;standard&lt;/DiskAccessMode&gt;<BR></DIV>
&lt;CommitLogSync&gt;batch&lt;/CommitLogSync&gt;<BR><DIV>&nbsp;</DIV><DIV>&nbsp;&lt;CommitLogSyncBatchWindowInMS&gt;1&lt;/CommitLogSyncBatchWindowInMS&gt;<BR></DIV><DIV><FONT
color=#990099>-----Daniel &lt;danielg@outbrain.com&gt; wrote: -----<BR><BR></FONT><BLOCKQUOTE
style="PADDING-RIGHT: 0px; PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #000000 2px solid;
MARGIN-RIGHT: 0px">To: "user@cassandra.apache.org" &lt;user@cassandra.apache.org&gt;<BR>From:
Daniel &lt;danielg@outbrain.com&gt;<BR>Date: 06/21/2010 02:30PM<BR>Subject:
Re: java.lang.OutOfMemoryError: Map failed<BR><BR>I am no expert in Cassandra,
but it looks like you might get your answer from reading this thread:<BR><A class=moz-txt-link-freetext
href="http://www.mail-archive.com/user@cassandra.apache.org/msg03702.html" target=blank>http://www.mail-archive.com/user@cassandra.apache.org/msg03702.html</A><BR><BR>Daniel.<BR><BR>On
06/21/2010 06:35 PM, <A class=moz-txt-link-abbreviated href="mailto:jeff@javajet.com" target=blank>jeff@javajet.com</A>
wrote: <BLOCKQUOTE style="DIRECTION: ltr" cite=mid:OFFE28214E.9038C2B7-ON85257749.0054DD76-85257749.0055F48E@javajet.com
type="cite"><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2>I
am using Lucandra to write Lucene documents to my cassandra server. I am processing a MySQL
table of about 700k records, 10k at a time. All goes well until I reach about 220k mark. Figure
it has something to do with my lack of correct memory configuration for JVM, keyspace or Cassandra.
</FONT><BR><BR><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif"
size=2>The Map failed is thrown for many *.db files and Cassandra fails to start with java.lang.OutOfMemoryError:
unable to create new native thread message.</FONT> <DIV dir=rtl style="DIRECTION:
ltr" align=right><BR></DIV><P style="DIRECTION: ltr"><FONT face="Default
Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2>ERROR 11:31:49,137 Corrupt file /var/lib/cassandra/data/Lucandra/TermInfo-256-Data.db;
skipped</FONT> <BR><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif"
size=2>java.io.IOException: Map failed</FONT> <BR><FONT face="Default Sans
Serif,Verdana,Arial,Helvetica,sans-serif" size=2>&nbsp; &nbsp; &nbsp; &nbsp;
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:819)</FONT> <BR><FONT
face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2>&nbsp; &nbsp;
&nbsp; &nbsp; at org.apache.cassandra.io.SSTableReader.mmap(SSTableReader.java:208)</FONT>
<BR><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2>&nbsp;
&nbsp; &nbsp; &nbsp; at org.apache.cassandra.io.SSTableReader.&lt;init&gt;(SSTableReader.java:154)</FONT>
<BR><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2>&nbsp;
&nbsp; &nbsp; &nbsp; at org.apache.cassandra.io.SSTableReader.&lt;init&gt;(SSTableReader.java:218)</FONT>
<BR><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2>&nbsp;
&nbsp; &nbsp; &nbsp; at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:123)</FONT>
<BR><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2>&nbsp;
&nbsp; &nbsp; &nbsp; at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:114)</FONT>
<BR><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2>&nbsp;
&nbsp; &nbsp; &nbsp; at org.apache.cassandra.db.ColumnFamilyStore.&lt;init&gt;(ColumnFamilyStore.java:178)</FONT>
<BR><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2>&nbsp;
&nbsp; &nbsp; &nbsp; at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:248)</FONT>
<BR><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2>&nbsp;
&nbsp; &nbsp; &nbsp; at org.apache.cassandra.db.Table.&lt;init&gt;(Table.java:338)</FONT>
<BR><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2>&nbsp;
&nbsp; &nbsp; &nbsp; at org.apache.cassandra.db.Table.open(Table.java:199)</FONT>
<BR><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2>&nbsp;
&nbsp; &nbsp; &nbsp; at org.apache.cassandra.thrift.CassandraDaemon.setup(CassandraDaemon.java:91)</FONT>
<BR><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2>&nbsp;
&nbsp; &nbsp; &nbsp; at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:177)</FONT>
<BR><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2>Caused
by: java.lang.OutOfMemoryError: Map failed</FONT> <BR><FONT face="Default Sans
Serif,Verdana,Arial,Helvetica,sans-serif" size=2>&nbsp; &nbsp; &nbsp; &nbsp;
at sun.nio.ch.FileChannelImpl.map0(Native Method)</FONT> <BR><FONT face="Default
Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2>&nbsp; &nbsp; &nbsp;
&nbsp; at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:816)</FONT> <BR><FONT
face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2>&nbsp; &nbsp;
&nbsp; &nbsp; ... 11 more</FONT> <BR><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif"
size=2>&nbsp;INFO 11:31:49,140 Replaying /var/lib/cassandra/commitlog/CommitLog-1277133857874.log</FONT>
<BR><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2>ERROR
11:31:49,146 Exception encountered during startup.</FONT> <BR><FONT face="Default
Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2>java.lang.OutOfMemoryError: unable
to create new native thread</FONT> <BR><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif"
size=2>&nbsp; &nbsp; &nbsp; &nbsp; at java.lang.Thread.start0(Native Method)</FONT>
<BR><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2>&nbsp;
&nbsp; &nbsp; &nbsp; at java.lang.Thread.start(Thread.java:614)</FONT> <BR><FONT
face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2>&nbsp; &nbsp;
&nbsp; &nbsp; at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:943)</FONT>
<BR><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2>&nbsp;
&nbsp; &nbsp; &nbsp; at java.util.concurrent.ThreadPoolExecutor.prestartAllCoreThreads(ThreadPoolExecutor.java:1549)</FONT>
<BR><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2>&nbsp;
&nbsp; &nbsp; &nbsp; at org.apache.cassandra.concurrent.JMXEnabledThreadPoolExecutor.&lt;init&gt;(JMXEnabledThreadPoolExecutor.java:51)</FONT>
<BR><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2>&nbsp;
&nbsp; &nbsp; &nbsp; at org.apache.cassandra.concurrent.JMXConfigurableThreadPoolExecutor.&lt;init&gt;(JMXConfigurableThreadPoolExecutor.java:34)</FONT>
<BR><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2>&nbsp;
&nbsp; &nbsp; &nbsp; at org.apache.cassandra.concurrent.StageManager.multiThreadedConfigurableStage(StageManager.java:81)</FONT>
<BR><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2>&nbsp;
&nbsp; &nbsp; &nbsp; at org.apache.cassandra.concurrent.StageManager.&lt;clinit&gt;(StageManager.java:53)</FONT>
<BR><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2>&nbsp;
&nbsp; &nbsp; &nbsp; at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:259)</FONT>
<BR><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2>&nbsp;
&nbsp; &nbsp; &nbsp; at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:173)</FONT>
<BR><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2>&nbsp;
&nbsp; &nbsp; &nbsp; at org.apache.cassandra.thrift.CassandraDaemon.setup(CassandraDaemon.java:95)</FONT>
<BR><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2>&nbsp;
&nbsp; &nbsp; &nbsp; at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:177)</FONT>
<BR><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2>Exception
encountered during startup.</FONT> <BR><BR><BR><FONT face="Default
Sans Serif,Verdana,Arial,Helvetica,sans-serif" size=2>Jeffrey Griffin, Developer</FONT>
<BR><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" color=blue
size=2><B>JavaJet Consulting &nbsp;LLC</B></FONT> <BR><FONT
size=3>&nbsp;</FONT><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif"
size=2>o: 1.717.533.5557</FONT> <BR><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif"
size=2>m: 1.717.419.2869</FONT> <BR><FONT face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif"
size=2>skype: javajet.coder</FONT> </P></BLOCKQUOTE><BR></BLOCKQUOTE><BR></DIV></FONT>
Mime
View raw message