lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Solr 4.2.1 higher memory footprint vs Solr 3.5
Date Tue, 04 Jun 2013 12:06:23 GMT
While the schema may be the same, did you also
closely examine solrconfig.xml, particularly cache
settings?

Memory _should_ have gone down significantly,
especially if you're dealing with much textual data
or string types. So I'm guessing something
innocent in your configs is showing this difference.

Try taking a heap dump of each, you should see
something significantly different....

About shipping data back to the client. Well, you
say the documents are very large Solr 4.2
automatically compresses stored data, you might
be spending a lot if time decompressing the data.
You could try turning that option off. Other than
that, has your environment changed in any way
especially your connection to disk? Check the CPU
usage on the server, if you see lots of CPU then
I'd look at compression first. If you see lots of I/O wait,
then disk access is where I'd start looking.

Best
Erick

On Mon, Jun 3, 2013 at 12:34 PM, SandeepM <skmirch@hotmail.com> wrote:
> Hi,
>
> Using the same schema for both Solr 3.5 and Solr 4.2.1 and posting the same
> data to both these server,  and the memory requirements seem to have gone up
> sharply during request handling.
> . Requests come in at around 200QPS.
> . Document sizes are very large but that did not seem to be a problem with
> 3.5 (Lots of multivalued fields with large array lengths.)
> Could you help me understand what change in SOLR 4.2.1 would attribute to
> this higher memory requirement?
>
> Also, in a different test, I ran a query to just get a list of all unique
> ID's via a single query and no load and I see it complete in <500ms however
> the time it takes to ship the data back to the client seems to be very
> large.  Any idea what could be causing this behavior?
>
> Would appreciate any help.
>
> Regards,
> -- Sandeep
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Solr-4-2-1-higher-memory-footprint-vs-Solr-3-5-tp4067879.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Mime
View raw message