lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ahmet Arslan <iori...@yahoo.com>
Subject Re: Delta import throws java heap space exception
Date Wed, 12 Mar 2014 21:32:41 GMT
Hi Richard,

How much ram do you assign to java heap? Try increasing it to 1 gb for example.
Please see : https://wiki.apache.org/solr/ShawnHeisey

Ahmet 



On Wednesday, March 12, 2014 10:53 PM, Richard Marquina Lopez <richard.marquina@gmail.com>
wrote:

Hi,

I have some problems when execute the delta import with 2 million of rows
from mysql database:

java.lang.OutOfMemoryError: Java heap space
        at java.nio.HeapCharBuffer.<init>(HeapCharBuffer.java:57)
        at java.nio.CharBuffer.allocate(CharBuffer.java:331)
        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:777)
        at java.nio.charset.Charset.decode(Charset.java:810)
        at com.mysql.jdbc.StringUtils.toString(StringUtils.java:2010)
        at com.mysql.jdbc.ResultSetRow.getString(ResultSetRow.java:820)
        at com.mysql.jdbc.BufferRow.getString(BufferRow.java:541)
        at com.mysql.jdbc.ResultSetImpl.getStringInternal(
ResultSetImpl.java:5812)
        at com.mysql.jdbc.ResultSetImpl.getString(ResultSetImpl.java:5689)
        at com.mysql.jdbc.ResultSetImpl.getObject(ResultSetImpl.java:4986)
        at com.mysql.jdbc.ResultSetImpl.getObject(ResultSetImpl.java:5175)
        at org.apache.solr.handler.dataimport.JdbcDataSource$
ResultSetIterator.getARow(JdbcDataSource.java:315)
        at org.apache.solr.handler.dataimport.JdbcDataSource$
ResultSetIterator.access$700(JdbcDataSource.java:254)
        at org.apache.solr.handler.dataimport.JdbcDataSource$
ResultSetIterator$1.next(JdbcDataSource.java:294)
        at org.apache.solr.handler.dataimport.JdbcDataSource$
ResultSetIterator$1.next(JdbcDataSource.java:286)
        at org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(
EntityProcessorBase.java:117)
        at org.apache.solr.handler.dataimport.SqlEntityProcessor.
nextModifiedRowKey(SqlEntityProcessor.java:86)
        at org.apache.solr.handler.dataimport.EntityProcessorWrapper.
nextModifiedRowKey(EntityProcessorWrapper.java:267)
        at org.apache.solr.handler.dataimport.DocBuilder.
collectDelta(DocBuilder.java:781)
        at org.apache.solr.handler.dataimport.DocBuilder.doDelta(
DocBuilder.java:338)
        at org.apache.solr.handler.dataimport.DocBuilder.execute(
DocBuilder.java:223)
        at org.apache.solr.handler.dataimport.DataImporter.
doDeltaImport(DataImporter.java:440)
        at org.apache.solr.handler.dataimport.DataImporter.
runCmd(DataImporter.java:478)
        at org.apache.solr.handler.dataimport.DataImporter$1.run(
DataImporter.java:457)
------------------------------------------------------------
--------------------------------------------------------------------------

java.sql.SQLException: Streaming result set
com.mysql.jdbc.RowDataDynamic@47a034e7
is still active.
No statements may be issued when any streaming result sets are open and in
use on a given connection.
Ensure that you have called .close() on any active streaming result sets
before attempting more queries.
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:927)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:924)
        at com.mysql.jdbc.MysqlIO.checkForOutstandingStreamingDa
ta(MysqlIO.java:3361)
        at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2524)
        at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2778)
        at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2828)
        at com.mysql.jdbc.ConnectionImpl.rollbackNoChecks(
ConnectionImpl.java:5204)
        at com.mysql.jdbc.ConnectionImpl.rollback(ConnectionImpl.java:5087)
        at com.mysql.jdbc.ConnectionImpl.realClose(ConnectionImpl.java:4690)
        at com.mysql.jdbc.ConnectionImpl.close(ConnectionImpl.java:1649)
        at org.apache.solr.handler.dataimport.JdbcDataSource.
closeConnection(JdbcDataSource.java:436)
        at org.apache.solr.handler.dataimport.JdbcDataSource.
close(JdbcDataSource.java:421)
        at org.apache.solr.handler.dataimport.DocBuilder.
closeEntityProcessorWrappers(DocBuilder.java:288)
        at org.apache.solr.handler.dataimport.DocBuilder.execute(
DocBuilder.java:277)
        at org.apache.solr.handler.dataimport.DataImporter.
doDeltaImport(DataImporter.java:440)
        at org.apache.solr.handler.dataimport.DataImporter.
runCmd(DataImporter.java:478)
        at org.apache.solr.handler.dataimport.DataImporter$1.run(
DataImporter.java:457)

Currently I have the batchSize parameter stetted to -1

Configuration:
- SOLR 4.4
- Centos 5.5
- 2GB RAM
- 1 Procesosr

Does someone have the same error?
Could someone help me, please?

Thank you,
Richard

Mime
View raw message