lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hoss Man (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-2493) SolrQueryParser constantly parse luceneMatchVersion in solrconfig. Large performance hit.
Date Wed, 04 May 2011 20:36:03 GMT

    [ https://issues.apache.org/jira/browse/SOLR-2493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13028943#comment-13028943
] 

Hoss Man commented on SOLR-2493:
--------------------------------

bq. this is solr's fault by having a getter that does some heavy duty xml shit.

that sounds like some serious buck passing.

All of the "get" methods on the Config class take in xpath expressions -- it should be obvious
to any one who uses them that they are going to do xpath parsing.

By the looks of it, the SolrConfig constructor was already creating a public final "luceneMatchVersion"
variable in it's constructor (using the xml parsing based COnfig method) it just wasn't getting
used by the query parser.

bq. In my opinion, the correct way to solve this is to make all methods in o.a.solr.core.Config
protected as they should only be called by subclasses doing the actual parsing.

I don't see how that would inherently protect us from this kind of mistake.

The cause of the problem came from needing public access to a "getLuceneVersion" type method
on SolrConfig (which is a subclass of Config)

even if all the methods in COnfig were protected, that could have very easily wound up being
implemented like so ...

{code}
  public Version getLuceneVersion() { return super.inefficientProtectedMethod(...) }
{code}

...and we would have had the same problem.

Bottom line: we just need to be careful about how/when the Config XML parsing methods are
used (protected or otherwise)

> SolrQueryParser constantly parse luceneMatchVersion in solrconfig. Large performance
hit.
> -----------------------------------------------------------------------------------------
>
>                 Key: SOLR-2493
>                 URL: https://issues.apache.org/jira/browse/SOLR-2493
>             Project: Solr
>          Issue Type: Bug
>          Components: search
>    Affects Versions: 3.1
>            Reporter: Stephane Bailliez
>            Assignee: Uwe Schindler
>            Priority: Blocker
>              Labels: core, parser, performance, request, solr
>             Fix For: 3.1.1, 3.2, 4.0
>
>         Attachments: SOLR-2493-3.x.patch, SOLR-2493.patch
>
>
> I' m putting this as blocker as I think this is a serious issue that should be adressed
asap with a release. With the current code this is no way near suitable for production use.
> For each instance created SolrQueryParser calls
>  
> getSchema().getSolrConfig().getLuceneVersion("luceneMatchVersion", Version.LUCENE_24)
> instead of using
> getSchema().getSolrConfig().luceneMatchVersion
> This creates a massive performance hit. For each request, there is generally 3 query
parsers created and each of them will parse the xml node in config which involve creating
an instance of XPath and behind the scene the usual factory finder pattern quicks in within
the xml parser and does a loadClass.
> The stack is typically:
>    at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:363)
>        at com.sun.org.apache.xml.internal.dtm.ObjectFactory.findProviderClass(ObjectFactory.java:506)
>        at com.sun.org.apache.xml.internal.dtm.ObjectFactory.lookUpFactoryClass(ObjectFactory.java:217)
>        at com.sun.org.apache.xml.internal.dtm.ObjectFactory.createObject(ObjectFactory.java:131)
>        at com.sun.org.apache.xml.internal.dtm.ObjectFactory.createObject(ObjectFactory.java:101)
>        at com.sun.org.apache.xml.internal.dtm.DTMManager.newInstance(DTMManager.java:135)
>        at com.sun.org.apache.xpath.internal.XPathContext.<init>(XPathContext.java:100)
>        at com.sun.org.apache.xpath.internal.jaxp.XPathImpl.eval(XPathImpl.java:201)
>        at com.sun.org.apache.xpath.internal.jaxp.XPathImpl.evaluate(XPathImpl.java:275)
>        at org.apache.solr.core.Config.getNode(Config.java:230)
>        at org.apache.solr.core.Config.getVal(Config.java:256)
>        at org.apache.solr.core.Config.getLuceneVersion(Config.java:325)
>        at org.apache.solr.search.SolrQueryParser.<init>(SolrQueryParser.java:76)
>        at org.apache.solr.schema.IndexSchema.getSolrQueryParser(IndexSchema.java:277)
> With the current 3.1 code, I do barely 250 qps with 16 concurrent users with a near empty
index.
> Switching SolrQueryParser to use getSchema().getSolrConfig().luceneMatchVersion and doing
a quick bench test, performance become reasonable beyond 2000 qps.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message