lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Commit Tag Bot (JIRA)" <>
Subject [jira] [Commented] (SOLR-4589) 4.x + enableLazyFieldLoading + large multivalued fields + varying fl = pathological CPU load & response time
Date Wed, 20 Mar 2013 17:23:18 GMT


Commit Tag Bot commented on SOLR-4589:

[branch_4x commit] Chris M. Hostetter

SOLR-4589: Fixed CPU spikes and poor performance in lazy field loading of multivalued fields
(merge r1458887)

> 4.x + enableLazyFieldLoading + large multivalued fields + varying fl = pathological CPU
load & response time
> ------------------------------------------------------------------------------------------------------------
>                 Key: SOLR-4589
>                 URL:
>             Project: Solr
>          Issue Type: Bug
>    Affects Versions: 4.0, 4.1, 4.2
>            Reporter: Hoss Man
>            Assignee: Hoss Man
>             Fix For: 4.3, 5.0, 4.2.1
>         Attachments: SOLR-4589.patch, SOLR-4589.patch, SOLR-4589.patch, SOLR-4589.patch,
SOLR-4589.patch, SOLR-4589.patch, test-just-queries.out__4.0.0_mmap_lazy_using36index.txt,, test.out__3.6.1_mmap_lazy.txt, test.out__3.6.1_mmap_nolazy.txt, test.out__3.6.1_nio_lazy.txt,
test.out__3.6.1_nio_nolazy.txt, test.out__4.0.0_mmap_lazy.txt, test.out__4.0.0_mmap_nolazy.txt,
test.out__4.0.0_nio_lazy.txt, test.out__4.0.0_nio_nolazy.txt, test.out__4.2.0_mmap_lazy.txt,
test.out__4.2.0_mmap_nolazy.txt, test.out__4.2.0_nio_lazy.txt, test.out__4.2.0_nio_nolazy.txt,
> Following up on a [user report of exterme CPU usage in 4.1|],
I've discovered that the following combination of factors can result in extreme CPU usage
and excessively HTTP response times...
> * Solr 4.x (tested 3.6.1, 4.0.0, and 4.2.0)
> * enableLazyFieldLoading == true (included in example solrconfig.xml)
> * documents with a large number of values in multivalued fields (eg: tested ~10-15K values)
> * multiple requests returning the same doc with different "fl" lists
> I haven't dug into the route cause yet, but the essential observations is: if lazyloading
is used in 4.x, then once a document has been fetched with an initial fl list X, subsequent
requests for that document using a differnet fl list Y can be many orders of magnitute slower
(while pegging the CPU) -- even if those same requests using fl Y uncached (or w/o lazy laoding)
would be extremely fast.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message