lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Toru Matsuzawa (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-902) Check on PositionIncrement with StopFilter.
Date Mon, 04 Jun 2007 03:13:15 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12501085
] 

Toru Matsuzawa commented on LUCENE-902:
---------------------------------------

for stop word only "is".
sample words "A is B".

For instance,When Tokenizer on StopFilter returns the following as a result.
 termText  positionIncrement
  "A"        1
  "is"       1
  "are"      0
  "be"       0
  "B"        1

The result of StopFilter.
 termText  positionIncrement
  "A"        1
  "are"      0
  "be"       0
  "B"        1

"A" and "are" and "be" become the same positions.

When thinking that it will process the result of a Japanese morphological analysis with StopFilter,
it becomes a problem.

> Check on PositionIncrement  with StopFilter.
> --------------------------------------------
>
>                 Key: LUCENE-902
>                 URL: https://issues.apache.org/jira/browse/LUCENE-902
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Analysis
>    Affects Versions: 2.2
>            Reporter: Toru Matsuzawa
>         Attachments: stopfilter.patch
>
>
> PositionIncrement set with Tokenizer is not considered with StopFilter. 
> When PositionIncrement of Token is 1, it is deleted by StopFilter. However, when PositionIncrement
of Token following afterwards is 0, it is not deleted. 
> I think that it is necessary to be deleted. Because it is thought same Token when PositionIncrement
is 0.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message