lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abdelhamid ABID <aeh.a...@gmail.com>
Subject Re: Solr Spellcheck on Large index size
Date Tue, 27 Apr 2010 20:58:46 GMT
Hi,
With the spellcheck.build=true, IMO solr will build the spellcheck
disctionnary at each request, so with the 29m documents solr can popup from
the server with some error like  "I quit"  :)
I would build the dictionnary once after data index creation, you may set
this option to the spell request handler in the solrconfig

<str name="buildOnCommit">true</str>



On 4/27/10, Kyle J G <kyleg@naparts.com> wrote:
>
>
> I am trying to create a spell checker for my companies website.
>
> Currently there are approx 29million documents in the index.
>
> When trying to create the spelling index it just seems to skip over the
> command.
>
> My fields in schema.xml look like the following:
>
> <field name="ID" type="int" indexed="true" stored="true" required="true" />
>         <field name="LineCode" type="string" indexed="true" stored="true"
> required="true" />
>         <field name="PartNumber" type="string" indexed="true" stored="true"
> required="true" />
>         <field name="CategoryName" type="string" indexed="true"
> stored="true"
> required="true" />
>         <field name="PartTerminologyName" type="string" indexed="true"
> stored="true" required="true" />
>         <field name="Year" type="int" indexed="true" stored="true"
> required="true"
> />
>         <field name="Make" type="string" indexed="true" stored="true"
> required="true" />
>         <field name="Model" type="string" indexed="true" stored="true"
> required="true" />
>         <field name="Submodel" type="string" indexed="true" stored="true"
> />
>         <field name="EngType" type="string" indexed="true" stored="true"
> required="true" />
>         <field name="Liter" type="string" indexed="true" stored="true"
> required="true" />
>         <field name="CC" type="int" indexed="true" stored="true"
> required="true" />
>         <field name="CID" type="int" indexed="true" stored="true"
> required="true"
> />
>         <field name="Fuel" type="string" indexed="true" stored="true"
> required="true" />
>         <field name="FuelDel" type="string" indexed="true" stored="true"
> required="true" />
>         <field name="Asp" type="string" indexed="true" stored="true"
> required="true" />
>         <field name="EngVin" type="string" indexed="true" stored="true"
> required="true" />
>         <field name="EngDesg" type="string" indexed="true" stored="true"
> required="true" />
>
> And copying fields as such:
>    <copyField source="Year" dest="text"/>
>    <copyField source="Make" dest="text"/>
>    <copyField source="Model" dest="text"/>
>    <copyField source="Fuel" dest="text"/>
>    <copyField source="CategoryName" dest="text"/>
>         <copyField source="text" dest="spell"/>
>
> My spell checker config looks like the following:
>
> <searchComponent name="spellcheck" class="solr.SpellCheckComponent">
>
>     <!-- <str name="queryAnalyzerFieldType">textSpell</str> -->
>
>     <lst name="spellchecker">
>       <str name="name">default</str>
>       <str name="field">spell</str>
>           <str name="buildOnCommit">true</str>
>           <str name="buildOnOptimize">true</str>
>       <str
>
> name="spellcheckIndexDir">C:\Users\kyleg\apache-solr-1.4.0\productGroups\solr\data\spellchecker</str>
>     </lst>
>
>     <!-- a spellchecker that uses a different distance measure
>     <lst name="spellchecker">
>       <str name="name">jarowinkler</str>
>       <str name="field">spell</str>
>       <str
>
> name="distanceMeasure">org.apache.lucene.search.spell.JaroWinklerDistance</str>
>       <str name="spellcheckIndexDir">./spellchecker2</str>
>     </lst>
>      -->
>
>     <!-- a file based spell checker -->
>     <lst name="spellchecker">
>       <str name="classname">solr.FileBasedSpellChecker</str>
>       <str name="name">file</str>
>       <str name="sourceLocation">spellings.txt</str>
>       <str name="characterEncoding">UTF-8</str>
>       <str name="spellcheckIndexDir">./spellcheckerFile</str>
>     </lst>
>   </searchComponent>
>
>
> The command that I am sending to try to build looks like the following:
>
> http://localhost:8983/solr/spell/?q=ACORA&version=2.2&start=0&rows=10&indent=on&spellcheck=true&spellcheck.dictionary=default&spellcheck.build=true&spellcheck.collate=true&spellcheck.limit=5
>
>
> I have also tried to reduce the size of the index to around 10,000
> documents
> and still no luck.
>
> Any help would be appreciated.
>
> Thank you,
> Kyle
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Solr-Spellcheck-on-Large-index-size-tp760416p760416.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>



-- 
Abdelhamid ABID
Software Engineer- J2EE / WEB

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message