lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Commit Tag Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-4589) 4.x + enableLazyFieldLoading + large multivalued fields + varying fl = pathological CPU load & response time
Date Wed, 20 Mar 2013 17:23:18 GMT

    [ https://issues.apache.org/jira/browse/SOLR-4589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13607878#comment-13607878
] 

Commit Tag Bot commented on SOLR-4589:
--------------------------------------

[branch_4x commit] Chris M. Hostetter
http://svn.apache.org/viewvc?view=revision&revision=1458938

SOLR-4589: Fixed CPU spikes and poor performance in lazy field loading of multivalued fields
(merge r1458887)

                
> 4.x + enableLazyFieldLoading + large multivalued fields + varying fl = pathological CPU
load & response time
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: SOLR-4589
>                 URL: https://issues.apache.org/jira/browse/SOLR-4589
>             Project: Solr
>          Issue Type: Bug
>    Affects Versions: 4.0, 4.1, 4.2
>            Reporter: Hoss Man
>            Assignee: Hoss Man
>             Fix For: 4.3, 5.0, 4.2.1
>
>         Attachments: SOLR-4589.patch, SOLR-4589.patch, SOLR-4589.patch, SOLR-4589.patch,
SOLR-4589.patch, SOLR-4589.patch, test-just-queries.out__4.0.0_mmap_lazy_using36index.txt,
test-just-queries.sh, test.out__3.6.1_mmap_lazy.txt, test.out__3.6.1_mmap_nolazy.txt, test.out__3.6.1_nio_lazy.txt,
test.out__3.6.1_nio_nolazy.txt, test.out__4.0.0_mmap_lazy.txt, test.out__4.0.0_mmap_nolazy.txt,
test.out__4.0.0_nio_lazy.txt, test.out__4.0.0_nio_nolazy.txt, test.out__4.2.0_mmap_lazy.txt,
test.out__4.2.0_mmap_nolazy.txt, test.out__4.2.0_nio_lazy.txt, test.out__4.2.0_nio_nolazy.txt,
test.sh
>
>
> Following up on a [user report of exterme CPU usage in 4.1|http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201302.mbox/%3C1362019882934-4043543.post@n3.nabble.com%3E],
I've discovered that the following combination of factors can result in extreme CPU usage
and excessively HTTP response times...
> * Solr 4.x (tested 3.6.1, 4.0.0, and 4.2.0)
> * enableLazyFieldLoading == true (included in example solrconfig.xml)
> * documents with a large number of values in multivalued fields (eg: tested ~10-15K values)
> * multiple requests returning the same doc with different "fl" lists
> I haven't dug into the route cause yet, but the essential observations is: if lazyloading
is used in 4.x, then once a document has been fetched with an initial fl list X, subsequent
requests for that document using a differnet fl list Y can be many orders of magnitute slower
(while pegging the CPU) -- even if those same requests using fl Y uncached (or w/o lazy laoding)
would be extremely fast.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message