lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ryan McKinley (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-2493) SolrQueryParser constantly parse luceneMatchVersion in solrconfig. Large performance hit.
Date Wed, 04 May 2011 23:02:03 GMT

    [ https://issues.apache.org/jira/browse/SOLR-2493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13029028#comment-13029028
] 

Ryan McKinley commented on SOLR-2493:
-------------------------------------

bq. I thought Solr's backwards policy is about public HTTP-REST-APIs, why care on implementation
details behind, why do we need to keep backwards? This is just a dumb question I never understood.
As long as Solr behaves identical to the outside who cares if we change method signatures/class
names?

Yes, the compatibility priorities are for HTTP-REST APIs, next is probably the external config
files -- since changing them may be a hassel.

In 4.0, we should be able to change the internal method signatures/class names pretty easily.
 If changing getDouble() to readDouble() makes things more clear, I'm +1  -- but it still
feels like a band-aid, one more incremental improvement.  

Long term, i would love to see the custom config system we have replaced with something standard...
like spring, or simly POJOs that are loaded (and saved!) via XStream.  This is the bigger
pile of work I was referring to.



> SolrQueryParser constantly parse luceneMatchVersion in solrconfig. Large performance
hit.
> -----------------------------------------------------------------------------------------
>
>                 Key: SOLR-2493
>                 URL: https://issues.apache.org/jira/browse/SOLR-2493
>             Project: Solr
>          Issue Type: Bug
>          Components: search
>    Affects Versions: 3.1
>            Reporter: Stephane Bailliez
>            Assignee: Uwe Schindler
>            Priority: Blocker
>              Labels: core, parser, performance, request, solr
>             Fix For: 3.1.1, 3.2, 4.0
>
>         Attachments: SOLR-2493-3.x.patch, SOLR-2493.patch
>
>
> I' m putting this as blocker as I think this is a serious issue that should be adressed
asap with a release. With the current code this is no way near suitable for production use.
> For each instance created SolrQueryParser calls
>  
> getSchema().getSolrConfig().getLuceneVersion("luceneMatchVersion", Version.LUCENE_24)
> instead of using
> getSchema().getSolrConfig().luceneMatchVersion
> This creates a massive performance hit. For each request, there is generally 3 query
parsers created and each of them will parse the xml node in config which involve creating
an instance of XPath and behind the scene the usual factory finder pattern quicks in within
the xml parser and does a loadClass.
> The stack is typically:
>    at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:363)
>        at com.sun.org.apache.xml.internal.dtm.ObjectFactory.findProviderClass(ObjectFactory.java:506)
>        at com.sun.org.apache.xml.internal.dtm.ObjectFactory.lookUpFactoryClass(ObjectFactory.java:217)
>        at com.sun.org.apache.xml.internal.dtm.ObjectFactory.createObject(ObjectFactory.java:131)
>        at com.sun.org.apache.xml.internal.dtm.ObjectFactory.createObject(ObjectFactory.java:101)
>        at com.sun.org.apache.xml.internal.dtm.DTMManager.newInstance(DTMManager.java:135)
>        at com.sun.org.apache.xpath.internal.XPathContext.<init>(XPathContext.java:100)
>        at com.sun.org.apache.xpath.internal.jaxp.XPathImpl.eval(XPathImpl.java:201)
>        at com.sun.org.apache.xpath.internal.jaxp.XPathImpl.evaluate(XPathImpl.java:275)
>        at org.apache.solr.core.Config.getNode(Config.java:230)
>        at org.apache.solr.core.Config.getVal(Config.java:256)
>        at org.apache.solr.core.Config.getLuceneVersion(Config.java:325)
>        at org.apache.solr.search.SolrQueryParser.<init>(SolrQueryParser.java:76)
>        at org.apache.solr.schema.IndexSchema.getSolrQueryParser(IndexSchema.java:277)
> With the current 3.1 code, I do barely 250 qps with 16 concurrent users with a near empty
index.
> Switching SolrQueryParser to use getSchema().getSolrConfig().luceneMatchVersion and doing
a quick bench test, performance become reasonable beyond 2000 qps.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message