lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gus Heck (JIRA)" <>
Subject [jira] [Commented] (SOLR-8291) NPE calling export handler when useFilterForSortedQuery=true
Date Wed, 01 May 2019 14:30:00 GMT


Gus Heck commented on SOLR-8291:

Just hit this recently. [~ronnnnnnn]'s explanation seems to jibe with what I was finding as
I prowled the code. He found the spot where the bitsets were left null which I hadn't found
yet, so kudos and thanks to him there :). It looks like there have been no major changes to
the logic of the code in sortDocSet since LUCENE-2831 in 2011, and the export handler code
seems to only date back to 2014, so the assumptions in ExportQParserPlugin don't seem to match
the previously established behavior of not calling getLeafCollector after the last matching
doc. Given the history, I'd say that the assumption by ExportQParserPlugin that it can create
the array of bit sets in advance based on the leaf count is the underlying flaw.
    public TopDocsCollector getTopDocsCollector(int len,
                                                QueryCommand cmd,
                                                IndexSearcher searcher) throws IOException
      int leafCount = searcher.getTopReaderContext().leaves().size();
      FixedBitSet[] sets = new FixedBitSet[leafCount];
      return new ExportCollector(sets);
    } {code}
This makes the problem specific to the Export handler code path, and general search path stuff
like sortDocSet() probably should be left alone. A quick browse into half a dozen other TopDocsCollectors
don't show any use of the leaf count similar to this, but I did not do an exhaustive search.

Two fixes come to mind within the export handler code, the patch supplied by Ron, tweaked
to add the null check to the loop condition, or an else block break to avoid useless iterations.
The second is for ExportQParserPlugin to populate with empty bitsets, which is unattractive
for memory reasons, but solves the problem at the source.

The first solution (a version of Ron's patch) amounts to ExportWriter being more tolerant
in what it accepts and short circuiting earlier, so I think that's the preferable path vs
creating more objects for GC. I'll be testing such a patch on the system that exposed this
for me soon.

> NPE calling export handler when useFilterForSortedQuery=true
> ------------------------------------------------------------
>                 Key: SOLR-8291
>                 URL:
>             Project: Solr
>          Issue Type: Bug
>          Components: SolrCloud
>    Affects Versions: 5.2.1
>            Reporter: Ray
>            Priority: Major
>         Attachments: SOLR-8291.patch, solr.log
> *Updated*: The stacktrace below was created when the solrconfig.xml has the following
> {code}
>  <useFilterForSortedQuery>true</useFilterForSortedQuery>
> {code}
> It was determined that useFilterForSortedQuery is incompatible with the /export handler.
> See the comments near the end of the ticket for a potential work around if this flag
needs to be set.
> Get NPE during calling export handler, here is the stack trace:
> 	at org.apache.lucene.util.BitSetIterator.<init>(
> 	at org.apache.solr.response.SortingResponseWriter.write(
> 	at org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(
> 	at org.apache.solr.servlet.HttpSolrCall.writeResponse(
> 	at
> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(
> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(
> 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
> 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(
> 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
> 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(
> 	at org.apache.catalina.core.StandardWrapperValve.invoke(
> 	at org.apache.catalina.core.StandardContextValve.invoke(
> 	at
> 	at org.jboss.modcluster.catalina.CatalinaContext$RequestListenerValve.event(
> 	at org.jboss.modcluster.catalina.CatalinaContext$RequestListenerValve.invoke(
> 	at
> 	at
> 	at org.apache.catalina.core.StandardHostValve.invoke(
> 	at org.apache.catalina.valves.ErrorReportValve.invoke(
> 	at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(
> 	at org.apache.catalina.valves.AccessLogValve.invoke(
> 	at org.apache.catalina.core.StandardEngineValve.invoke(
> 	at org.jboss.web.tomcat.service.request.ActiveRequestResponseCacheValve.invoke(
> 	at org.apache.catalina.connector.CoyoteAdapter.service(
> 	at org.apache.coyote.ajp.AjpAprProcessor.process(
> 	at org.apache.coyote.ajp.AjpAprProtocol$AjpConnectionHandler.process(
> 	at$
> 	at
> It seems there are some FixedBitSet was set to null

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message