lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Høydahl (JIRA) <j...@apache.org>
Subject [jira] [Commented] (SOLR-2493) SolrQueryParser constantly parse luceneMatchVersion in solrconfig. Large performance hit.
Date Thu, 05 May 2011 16:43:03 GMT

    [ https://issues.apache.org/jira/browse/SOLR-2493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13029426#comment-13029426
] 

Jan Høydahl commented on SOLR-2493:
-----------------------------------

+1

@Michael, agree on this. But instead of relying on a monolithic solrconfig.xml file or .yml
file, isn't it better to re-design configuration to fit a path/node concept more fine-grained
(like ZK nodes)? It doesn't feel quite right to store solrconfig.xml and schema.xml as a huge
string in the SolrCloud ZK schema. It would be better to have stuff like /solr/configs/configA/general/abortOnConfigurationError=false
as a separate config node. Likewise /solr/configs/configA/schema/types/text_en to define fieldType
text_en. The config concept won't need to be bound to ZK either. There could be pluggable
backend implementations, where one could read/write the existing XML formats.

> SolrQueryParser constantly parse luceneMatchVersion in solrconfig. Large performance
hit.
> -----------------------------------------------------------------------------------------
>
>                 Key: SOLR-2493
>                 URL: https://issues.apache.org/jira/browse/SOLR-2493
>             Project: Solr
>          Issue Type: Bug
>          Components: search
>    Affects Versions: 3.1
>            Reporter: Stephane Bailliez
>            Assignee: Uwe Schindler
>            Priority: Blocker
>              Labels: core, parser, performance, request, solr
>             Fix For: 3.1.1, 3.2, 4.0
>
>         Attachments: SOLR-2493-3.x.patch, SOLR-2493.patch
>
>
> I' m putting this as blocker as I think this is a serious issue that should be adressed
asap with a release. With the current code this is no way near suitable for production use.
> For each instance created SolrQueryParser calls
>  
> getSchema().getSolrConfig().getLuceneVersion("luceneMatchVersion", Version.LUCENE_24)
> instead of using
> getSchema().getSolrConfig().luceneMatchVersion
> This creates a massive performance hit. For each request, there is generally 3 query
parsers created and each of them will parse the xml node in config which involve creating
an instance of XPath and behind the scene the usual factory finder pattern quicks in within
the xml parser and does a loadClass.
> The stack is typically:
>    at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:363)
>        at com.sun.org.apache.xml.internal.dtm.ObjectFactory.findProviderClass(ObjectFactory.java:506)
>        at com.sun.org.apache.xml.internal.dtm.ObjectFactory.lookUpFactoryClass(ObjectFactory.java:217)
>        at com.sun.org.apache.xml.internal.dtm.ObjectFactory.createObject(ObjectFactory.java:131)
>        at com.sun.org.apache.xml.internal.dtm.ObjectFactory.createObject(ObjectFactory.java:101)
>        at com.sun.org.apache.xml.internal.dtm.DTMManager.newInstance(DTMManager.java:135)
>        at com.sun.org.apache.xpath.internal.XPathContext.<init>(XPathContext.java:100)
>        at com.sun.org.apache.xpath.internal.jaxp.XPathImpl.eval(XPathImpl.java:201)
>        at com.sun.org.apache.xpath.internal.jaxp.XPathImpl.evaluate(XPathImpl.java:275)
>        at org.apache.solr.core.Config.getNode(Config.java:230)
>        at org.apache.solr.core.Config.getVal(Config.java:256)
>        at org.apache.solr.core.Config.getLuceneVersion(Config.java:325)
>        at org.apache.solr.search.SolrQueryParser.<init>(SolrQueryParser.java:76)
>        at org.apache.solr.schema.IndexSchema.getSolrQueryParser(IndexSchema.java:277)
> With the current 3.1 code, I do barely 250 qps with 16 concurrent users with a near empty
index.
> Switching SolrQueryParser to use getSchema().getSolrConfig().luceneMatchVersion and doing
a quick bench test, performance become reasonable beyond 2000 qps.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message