lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefan Moises <moi...@shoptimax.de>
Subject Re: NullpointerException when combining spellcheck component and synonyms
Date Tue, 21 Sep 2010 13:49:55 GMT
  Hi again,

well it turns out that it still doesn't work ...
Sometimes it works (i.e. for some cores), sometimes I still get the 
nullpointer - e.g. if I create a new core and use the same settings as a 
working one, but index different data, then I add a synonym (e.g. "foo 
=> bar") and activate spellchecking, then search for "foo" -  boom :(
And I can't really tell where this error comes from... any idea where to 
start looking?

Thanks,
Stefan

Am 01.09.2010 17:20, schrieb Stefan Moises:
>  doh, looks like I only forgot to add the spellcheck component to my 
> edismax request handler... now it works with:
>
> ...
> <arr name="last-components">
> <str>spellcheck</str>
> <str>elevator</str>
> </arr>
>
> What's strange is that spellchecking seemed to work *without* that 
> entry, too....
>
> Cheers,
> Stefan
>
> Am 01.09.2010 13:33, schrieb Stefan Moises:
>>  Hi there,
>>
>> I am using Solr from SVN, 
>> https://svn.apache.org/repos/asf/lucene/dev/trunk (my last 
>> update/build on my dev server was in July I think)...
>>
>> I've encountered a strange problem when using the Spellcheck 
>> component in combination with the SynonymFilter...
>> My "text" field is pretty standard, using the default synonyms.txt file:
>> <fieldType name="text" class="solr.TextField" 
>> positionIncrementGap="100">
>> <analyzer type="index">
>> <tokenizer class="solr.WhitespaceTokenizerFactory"/>
>> <!-- in this example, we will only use synonyms at query time
>> <filter class="solr.SynonymFilterFactory" 
>> synonyms="index_synonyms.txt" ignoreCase="true" expand="false"/>
>>         -->
>> <filter class="solr.StopFilterFactory" ignoreCase="true" 
>> words="stopwords.txt"/>
>> <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" 
>> generateNumberParts="1" catenateWords="1" catenateNumbers="1" 
>> catenateAll="0" splitOnCaseChange="1"/>
>> <filter class="solr.LowerCaseFilterFactory"/>
>> <!-- <filter class="solr.EnglishPorterFilterFactory" 
>> protected="protwords.txt"/> -->
>> <filter class="solr.SnowballPorterFilterFactory" 
>> protected="protwords.txt"/>
>> <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
>> </analyzer>
>> <analyzer type="query">
>> <tokenizer class="solr.WhitespaceTokenizerFactory"/>
>> <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" 
>> ignoreCase="true" expand="true"/>
>> <filter class="solr.StopFilterFactory" ignoreCase="true" 
>> words="stopwords.txt"/>
>> <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" 
>> generateNumberParts="1" catenateWords="0" catenateNumbers="0" 
>> catenateAll="0" splitOnCaseChange="1"/>
>> <filter class="solr.LowerCaseFilterFactory"/>
>> <!-- <filter class="solr.EnglishPorterFilterFactory" 
>> protected="protwords.txt"/> -->
>> <filter class="solr.SnowballPorterFilterFactory" 
>> protected="protwords.txt"/>
>> <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
>> </analyzer>
>> </fieldType>
>>
>> I have only added some terms at the end of synonyms.txt:
>> ...
>> # Synonym mappings can be used for spelling correction too
>> pixima => pixma
>>
>> tekanne => teekanne
>> teekane => teekanne
>> flashen => flaschen
>> flasen => flaschen
>>
>> Here is my query and the exception... if I turn off spellcheck, 
>> everything works as expected and the synonyms are found...
>>
>> INFO: [db] webapp=/solr path=/select 
>> params={mlt.minwl=3&spellcheck=true&facet=true&mlt.fl=oxmanu_oxid,oxvendor_oxid,oxtags,oxsearchkeys&spellcheck.q=flasen&mlt.mintf=1&facet.limit=-1&mlt=true&json.nl=map&hl.fl=oxtitle&hl.fl=oxshortdesc&hl.fl=oxlongdesc&hl.fl=oxtags&hl.fl=seodesc&hl.fl=seokeywords&wt=json&hl=true&rows=10&version=1.2&mlt.mindf=1&debugQuery=true&facet.sort=lex&start=0&q=flasen&facet.field=oxcat_oxid&facet.field=oxcat_oxidtitle&facet.field=oxprice&facet.field=oxmanu_oxid&facet.field=oxmanu_oxidtitle&facet.field=oxvendor_oxid&facet.field=oxvendor_oxidtitle&facet.field=attrgroup_oxid&facet.field=attrgroup_oxidtitle&facet.field=attrgroup_oxidvalue&facet.field=attrvalue_oxid&facet.field=attrvalue_oxidtitle&facet.field=attr2attrgroup_oxidtitle&qt=dismax&spellcheck.build=false}

>> hits=2 status=500 QTime=14
>> 01.09.2010 12:54:47 org.apache.solr.common.SolrException log
>> SCHWERWIEGEND: java.lang.NullPointerException
>>         at 
>> org.apache.lucene.util.AttributeSource.cloneAttributes(AttributeSource.java:470)
>>         at 
>> org.apache.lucene.analysis.synonym.SynonymFilter.incrementToken(SynonymFilter.java:128)
>>         at 
>> org.apache.lucene.analysis.core.StopFilter.incrementToken(StopFilter.java:260)
>>         at 
>> org.apache.lucene.analysis.miscellaneous.WordDelimiterFilter.incrementToken(WordDelimiterFilter.java:336)
>>         at 
>> org.apache.lucene.analysis.core.LowerCaseFilter.incrementToken(LowerCaseFilter.java:62)
>>         at 
>> org.apache.solr.handler.component.SpellCheckComponent.getTokens(SpellCheckComponent.java:380)
>>         at 
>> org.apache.solr.handler.component.SpellCheckComponent.process(SpellCheckComponent.java:127)
>>         at 
>> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:203)
>>         at 
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
>>         at org.apache.solr.core.SolrCore.execute(SolrCore.java:1323)
>>         at 
>> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:337)
>>         at 
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:240)
>>         at 
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
>>         at 
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>>         at 
>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
>>         at 
>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
>>         at 
>> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:465)
>>         at 
>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
>>         at 
>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
>>         at 
>> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
>>         at 
>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
>>         at 
>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
>>         at 
>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852)
>>         at 
>> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
>>         at 
>> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
>>         at java.lang.Thread.run(Thread.java:619)
>>
>> Thanks for any hint what I may be doing wrong! :)
>> Stefan
>>
>>
>

-- 
*******************************************
Stefan Moises
Senior Softwareentwickler

shoptimax GmbH
Guntherstraße 45 a
90461 Nürnberg
Amtsgericht Nürnberg HRB 21703
GF Friedrich Schreieck

Tel.: 0911/25566-25
Fax:  0911/25566-29
moises@shoptimax.de
http://www.shoptimax.de
*******************************************


Mime
View raw message