lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Olivier Favre (JIRA)" <>
Subject [jira] [Commented] (LUCENE-3071) PathHierarchyTokenizer adaptation for urls: splits reversed
Date Thu, 05 May 2011 16:25:03 GMT


Olivier Favre commented on LUCENE-3071:

{{ant clean test}} did it for me, thanks!

As for the failing tests, it is because of the {{finalOffset}} that I set to {{path.length()}}.
I'm not sure whether I should use {{path.length()}}, as my tokens don't go up to there when
using the reverse mode.
When I take a look at the the end() function, I think that I "should" set it to the end of
the string. But I can't see it on the javadoc.
If the purpose of the {{finalOffset}} parameter in {{assertTokenStreamContents()}} it to make
sure of the {{endOffset}} of the last term, then I should not use {{path.length()}} blindly
when using reverse and skip.

Can you help me with the purpose of {{finalOffset}}? Or can I simply skip it in my tests (they
are working if I skip it)?


> PathHierarchyTokenizer adaptation for urls: splits reversed
> -----------------------------------------------------------
>                 Key: LUCENE-3071
>                 URL:
>             Project: Lucene - Java
>          Issue Type: New Feature
>          Components: contrib/analyzers
>            Reporter: Olivier Favre
>            Priority: Minor
>         Attachments: LUCENE-3071.patch, ant.log.tar.bz2
>   Original Estimate: 2h
>  Remaining Estimate: 2h
> {{PathHierarchyTokenizer}} should be usable to split urls the a "reversed" way (useful
for faceted search against urls):
> {{}} -> {{,, com}}
> Moreover, it should be able to skip a given number of first (or last, if reversed) tokens:
> {{/usr/share/doc/somesoftware/INTERESTING/PART}}
> Should give with 4 tokens skipped:

This message is automatically generated by JIRA.
For more information on JIRA, see:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message