lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From accid <ac...@gmx.net>
Subject Re: JVM GC troubles
Date Fri, 15 Oct 2010 08:45:46 GMT
Hi,

I dont run totally OOM (no OOM exceptions in the log) but I constantly
garbage collect. While not collecting, SOLR master handels the updates
pretty well.

Every insert is unique, so I dont have any deletes or optimizes and all
queries are handled by the single slave instance. Is there a way to reduce
the objects held in the old gen space ? It looks like the JVM is trying to
hold as many objects as possible in the cache, to provide fast queries, who
are not needed in my situation.

Regarding the Jboss ... well as I said, its the minimalistic version of it
and we use it due to automation process within our departement. In my
test-env I tried it with a plain tomcat 6.x but without any improvements, so
the Jboss overhead is minimal to nothing.

The JVM parameters I wrote, are the ones I am struggling with at the moment.
I was hoping someone will come up with a hint regarding the solarconfig.xml
itself.

PS: if anyone is questioning the implemented architecture (master -> slave,
configs, schema, etc.)  ... its our architects fault and I have to operate
it ;-)

2010/10/15 Otis Gospodnetic <otis_gospodnetic@yahoo.com>

> Hello,
>
> I hope you are not running JBoss just to run Solr - there are simpler
> containers
> out there, e.g., Jetty.
> Do you OOM?
> Do things look better if you replicate less often (e.g. every 5 minutes
> instead
> of every 60 seconds)?
> Do all/some of those -X__ JVM params actually help?
>
> Otis----
> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
> Lucene ecosystem search :: http://search-lucene.com/
>
>
>
> ----- Original Message ----
> > From: accid <accid@gmx.net>
> > To: solr-user@lucene.apache.org
> > Sent: Thu, October 14, 2010 1:25:34 PM
> > Subject: Re: JVM GC troubles
> >
> > I forgot a few important details:
> >
> > solr version = 1.4.1
> > current index  size = 50gb
> > growth ~600mb / day
> > jboss runs with web settings (same as  minimal)
> > 2010/10/14 <accid@gmx.net>
> >
> > >  Hi,
> > >
> > > as I am new here, I want to say hello and thanks in advance  for your
> help.
> > >
> > >
> > > HW Setup:
> > >
> > > 1x SOLR Master  - Sun Microsystems SUN FIRE X4450 - 4 x 2,93ghz, 64gb
> ram
> > > 1x SOLR Slave  -  Sun Microsystems SUN FIRE X4450 - 4 x 2,93ghz, 64gb
> ram
> > >
> > >  SW Setup:
> > >
> > > Solaris 10 Generic_142901-03
> > > jboss  5.1.0
> > > JDK 1.6 update 18
> > >
> > >
> > > # Specify the exact Java  VM executable to use.
> > > #
> > >  JAVA="/opt/appsrv/java6/bin/amd64/java"
> > >
> > > #
> > > # Specify  options to pass to the Java VM.
> > > #
> > > JAVA_OPTS="-server -Xms6144m  -Xmx6144m -Xmn3072m
> -XX:ThreadStackSize=1024
> > > -XX:MaxPermSize=512m  -Dorg.jboss.resolver.warning=true
> > >  -Dsun.rmi.dgc.client.gcInterval=3600000
> > >  -Dsun.rmi.dgc.server.gcInterval=3600000
> -Dnetworkaddress.cache.ttl=1800
> > >  -XX:+UseConcMarkSweepGC"
> > >
> > >
> > > SOLR Setup:
> > >
> > > #)  the master has to deal an avg. update rate of 50 updates/s and
> peaks of
> > >  400 updates/s
> > >
> > > #) the slave replicates every 60s using the built  in solr replication
> > > method (NOT rsync)
> > >
> > > #) the slave  querys are ~20/sec
> > >
> > >
> > > #) schema.xml
> > >
> > >
> > >  <field name="myname1" type="string" indexed="true" stored="false"
> > >  required="true"/>
> > > <field name="myname2" type="int" indexed="true"  stored="true"
> > > required="true"/>
> > > <field name="myname3"  type="int" indexed="true" stored="true"
> > > required="true"/>
> > >  <field name="myname4" type="long" indexed="true" stored="true"
> > >  required="true"/>
> > > <field name="myname5" type="int" indexed="true"  stored="true"
> > > required="true"/>
> > > <field name="myname6"  type="string" indexed="true" stored="true"
> > > required="true"/>
> > >  <field name="myname7" type="string" indexed="true"  stored="false"/>
> > > <field name="myname8" type="string"  indexed="true" stored="false"/>
> > > <field name="myname9"  type="string" indexed="true" stored="false"/>
> > > <field  name="myname10" type="long" indexed="true" stored="false"/>
> > > <field  name="myname11" type="int" indexed="true" stored="false"/>
> > > <field  name="myname12" type="string" indexed="true" stored="false"/>
> > >  <field name="myname13" type="tdate" indexed="true"  stored="false"/>
> > > <field name="myname14" type="int" indexed="true"  stored="false"
> > > multiValued="true"/>
> > > <field name="myname15"  type="string" indexed="true" stored="false"
> > >  multiValued="true"/>
> > > <field name="myname16" type="int"  indexed="true" stored="false"
> > > multiValued="true"/>
> > > <field  name="myname17" type="string" indexed="true" stored="false"
> > >  multiValued="true"/>
> > > <field name="myname18" type="string"  indexed="true" stored="false"
> > > multiValued="true"/>
> > > <field  name="myname19" type="string" indexed="true" stored="false"
> > >  multiValued="true"/>
> > > <field name="myname20" type="boolean"  indexed="true" stored="false"/>
> > > <field name="myname21" type="int"  indexed="true" stored="false"
> > > required="true"/>
> > > <field  name="myname22" type="date" indexed="true" stored="true"
> > > default="NOW"  multiValued="false"/>
> > >
> > >
> > > #) The solarconfig.xml is  attached
> > >
> > >
> > >
> > > Both, master & slave suffer from  serious performance impacts during
> garbage
> > >  collects
> > >
> > >
> > > I obviously have an GC problem, because ~30min  after startup, the Old
> space
> > > is full and not beeing freed  up.
> > >
> > > Below you find a JMX copy&paste of the Heap AFTER a  garbage collect!!
> As
> > > you can see, even the Eden Space can only free up  to 700mb total,
> which
> > > gives very little time to relax. The system does  GC's 90% of the time.
> > >
> > >
> > >
> > >
> > > Total Memory  Pools: 5
> > >
> > >    Pool: Code Cache (Non-heap  memory)
> > >
> > >        Peak Usage : init:4194304,  used:7679360, committed:7798784,
> > > max:50331648
> > >         Current Usage : init:4194304, used:7677312,  committed:7798784,
> > > max:50331648
> > >
> > >
> > >             |---------| committed:7.44Mb
> > >
> > >
> +---------------------------------------------------------------------+
> > >             |/////////| | max:48Mb
> > >
> > >
> +---------------------------------------------------------------------+
> > >             |---------|  used:7.32Mb
> > >
> > >
> > >    Pool: Par Eden Space (Heap  memory)
> > >
> > >        Peak Usage : init:2577006592,  used:2577006592,
> committed:2577006592,
> > > max:2577006592
> > >         Current Usage : init:2577006592, used:1911083120,
> > >  committed:2577006592, max:2577006592
> > >
> > >
> > >
> > >
> |---------------------------------------------------------------------|
> > >  committed:2.4Gb
> > >
> > >
> +---------------------------------------------------------------------+
> > >              |/////////////////////////////////////////////////// |
> >max:2.4Gb
> > >
> > >
> +---------------------------------------------------------------------+
> > >              |--------------------------------------------------|
> >used:1.78Gb
> > >
> > >
> > >    Pool: Par Survivor Space (Heap  memory)
> > >
> > >        Peak Usage : init:322109440,  used:322109440,
> committed:322109440,
> > > max:322109440
> > >         Current Usage : init:322109440, used:0,  committed:322109440,
> > > max:322109440
> > >
> > >
> > >
> > >
> |---------------------------------------------------------------------|
> > >  committed:307.19Mb
> > >
> > >
> +---------------------------------------------------------------------+
> > >             | | max:307.19Mb
> > >
> > >
> +---------------------------------------------------------------------+
> > >             | used:0b
> > >
> > >
> > >     Pool: CMS Old Gen (Heap memory)
> > >
> > >         Peak Usage : init:3221225472, used:3221225472,
> committed:3221225472,
> > > max:3221225472
> > >         Current Usage : init:3221225472, used:3207930400,
> > > committed:3221225472,  max:3221225472
> > >
> > >
> > >
> > >
> |---------------------------------------------------------------------|
> > >  committed:3Gb
> > >
> > >
> +---------------------------------------------------------------------+
> > >
> > >
> |/////////////////////////////////////////////////////////////////////|
> > >  max:3Gb
> > >
> > >
> +---------------------------------------------------------------------+
> > >
> > >
> |--------------------------------------------------------------------|
> > >  used:2.99Gb
> > >
> > >
> > >    Pool: CMS Perm Gen (Non-heap  memory)
> > >
> > >        Peak Usage : init:21757952,  used:62622896, committed:104484864,
> > > max:536870912
> > >         Current Usage : init:21757952, used:62607888,
>  committed:104484864,
> > > max:536870912
> > >
> > >
> > >             |------------|  committed:99.64Mb
> > >
> > >
> +---------------------------------------------------------------------+
> > >             |//////// | |  max:512Mb
> > >
> > >
> +---------------------------------------------------------------------+
> > >             |-------|  used:59.71Mb
> > >
> > >
> > >
> > >
> > > I am looking forward for  best practice values for both, solrconfig.xml
> and
> > > JVM  settings.
> > >
> > > br
> > >
> > > accid
> > >
> > > --
> > >  GRATIS! Movie-FLAT mit ├╝ber 300 Videos.
> > > Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message