lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Derek Poh <d...@globalsources.com>
Subject Re: Split words with period in between ("Co.Ltd") into separate tokens
Date Wed, 12 Oct 2016 09:04:27 GMT
Thank you for pointing out the flags.
I set generateWordParts=1 and the term is split up.

On 10/12/2016 3:26 PM, Modassar Ather wrote:
> Hi,
>
> The flags set in your WordDelimiterFilterFactory definition is 0.
> You can try with generateWordParts=1 and splitOnCaseChange=1 and see if it
> breaks as per your requirement.
> You can also try with other available flags enabled.
>
> Best,
> Modassar
>
> On Wed, Oct 12, 2016 at 12:44 PM, Derek Poh <dpoh@globalsources.com> wrote:
>
>> I tried adding Word Delimiter Filter to the field but it does not process
>> or it truncate away the term "Co.Ltd".
>>
>> <filter class="solr.WordDelimiterFilterFactory" generateWordParts="0"
>> generateNumberParts="0" catenateWords="0" catenateNumbers="0"
>> catenateAll="0" splitOnCaseChange="0"/>
>>
>> On 10/12/2016 8:54 AM, Derek Poh wrote:
>>
>>> Hi
>>>
>>> How can I split words with period in between into separate tokens.
>>> Eg. "Co.Ltd" => "Co" "Ltd" .
>>>
>>> I am using StandardTokenizerFactory and it does notreplace periods (dots)
>>> that are not followed by whitespace are kept as part of the token,
>>> including Internet domain names.
>>>
>>> This is the field definition,
>>>
>>> <fieldType name="text_general" class="solr.TextField"
>>> positionIncrementGap="100">
>>>        <analyzer type="index">
>>>          <tokenizer class="solr.StandardTokenizerFactory"/>
>>>          <filter class="solr.StopFilterFactory" ignoreCase="true"
>>> words="stopwords.txt" />
>>>          <filter class="solr.LowerCaseFilterFactory"/>
>>>        </analyzer>
>>>        <analyzer type="query">
>>>          <tokenizer class="solr.StandardTokenizerFactory"/>
>>>          <filter class="solr.StopFilterFactory" ignoreCase="true"
>>> words="stopwords.txt" />
>>>          <filter class="solr.SynonymFilterFactory"
>>> synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
>>>          <filter class="solr.LowerCaseFilterFactory"/>
>>>        </analyzer>
>>> </fieldType>
>>>
>>> Solr versionis 10.4.10.
>>>
>>> Derek
>>>
>>> ----------------------
>>> CONFIDENTIALITY NOTICE
>>> This e-mail (including any attachments) may contain confidential and/or
>>> privileged information. If you are not the intended recipient or have
>>> received this e-mail in error, please inform the sender immediately and
>>> delete this e-mail (including any attachments) from your computer, and you
>>> must not use, disclose to anyone else or copy this e-mail (including any
>>> attachments), whether in whole or in part.
>>> This e-mail and any reply to it may be monitored for security, legal,
>>> regulatory compliance and/or other appropriate reasons.
>>>
>> ----------------------
>> CONFIDENTIALITY NOTICE
>> This e-mail (including any attachments) may contain confidential and/or
>> privileged information. If you are not the intended recipient or have
>> received this e-mail in error, please inform the sender immediately and
>> delete this e-mail (including any attachments) from your computer, and you
>> must not use, disclose to anyone else or copy this e-mail (including any
>> attachments), whether in whole or in part.
>> This e-mail and any reply to it may be monitored for security, legal,
>> regulatory compliance and/or other appropriate reasons.
>>
>>


----------------------
CONFIDENTIALITY NOTICE 

This e-mail (including any attachments) may contain confidential and/or privileged information.
If you are not the intended recipient or have received this e-mail in error, please inform
the sender immediately and delete this e-mail (including any attachments) from your computer,
and you must not use, disclose to anyone else or copy this e-mail (including any attachments),
whether in whole or in part. 

This e-mail and any reply to it may be monitored for security, legal, regulatory compliance
and/or other appropriate reasons.
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message