lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sunnyfr <>
Subject MySql / Solr 1.3 / Tomcat55 - Full Import for 8,5M of data >> Exception in thread "Thread-33"
Date Mon, 03 Nov 2008 16:36:57 GMT


I've put a batchsize parameter at -1, it works fine, the point is I will
monopolize the MySql's database for 10hours.
And other request on it like update, or other process will be stack. And if
I don't use batchsize -1 I will have an OOM error like below. I tried to put
batchsize 1000 or 1 but it doesn't work either.

What would you reckon ? 
I can't have another server just for that, I have to use MySql production's

> Should I try to generate CSV files from MySql and then make a full import
> from it, and delta import on the database ?? 

> Should I modify the connection code in solr, to try to manage this memory.

> Do you have another idea ? 

If anyone has any suggestions/needs more info, i’d be very greatful.

FYI : Linux server / 8G Mem

TomCat55 :
JAVA_OPTS="-Xms2000m -Xmx4000m -XX:+HeapDumpOnOutOfMemoryError
-XX:+UseParallelGC -XX:+AggressiveOpts -XX:NewRatio=5 -Xloggc:gc.log
-XX:+PrintGCDetails -XX:+PrintGCTimeStamps"

data-config.xml: data-config.xml 

My Error:
Nov  3 16:45:43 solr-test jsvc.exec[29099]:  [PSYoungGen:
227583K->0K(455104K)] [PSOldGen: 3413375K->142574K(1706688K)]
Nov  3 16:45:43 solr-test jsvc.exec[29099]:  [PSPermGen:
20751K->20751K(21504K)], 1.0853010 secs] [Times: user=0.99 sys=0.10,
real=1.09 secs]
Nov  3 16:45:43 solr-test jsvc.exec[29099]: Nov 3, 2008 4:45:43 PM
processChildren SEVERE: Exception invoking periodic operation: 
java.lang.OutOfMemoryError: Java heap space ^Iat
sun.nio.cs.US_ASCII.newDecoder( ^Iat
^Iat java.nio.charset.CharsetEncoder.replaceWith(
^Iat java.nio.charset.CharsetEncoder.<init>( ^Iat
java.nio.charset.CharsetEncoder.<init>( ^Iat
sun.nio.cs.US_ASCII$Encoder.<init>( ^Iat
sun.nio.cs.US_ASCII$Encoder.<init>( ^Iat
sun.nio.cs.US_ASCII.newEncoder( ^Iat
java.lang.StringCoding$StringEncoder.<init>( ^Iat
java.lang.StringCoding$StringEncoder.<init>( ^Iat
java.lang.StringCoding.encode( ^Iat
java.lang.String.getBytes( ^Iat
Nov  3 16:45:43 solr-test jsvc.exec[29099]: ^Iat ^Iat
^Iat org.apache.catalina.startup.HostConfig.deployApps(
^Iat org.apache.catalina.startup.HostConfig.check( ^Iat

View this message in context:
Sent from the Solr - User mailing list archive at

View raw message