Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
To: java-user@lucene.apache.org
From: Parit Bansal <Parit.Bansal@sib.swiss>
Subject: WordDelimiterIterator word splitting usecase
Message-ID: <26d4094f-c0c1-c1ca-706f-f534288752c4@sib.swiss>
Date: Fri, 22 Dec 2017 12:51:10 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.5.0
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US
archived-at: Fri, 22 Dec 2017 11:51:24 -0000

Hi,

I have been migrating and maintaining lucene indexing code for our use 
case since 2.x version (now we are are 6.6.1 migrating to 7.x) .

One problem I am constantly facing is regarding 
org.apache.lucene.analysis.miscellaneous.WordDelimiterIterator class 
that is defined final in lucene codebase.  In this class, there is a 
isBreak() method that defines when to split a word into subwords. One of 
the cases is *ALPHA->NUMERIC, NUMERIC->ALPHA :Don't split* (in the same 
if condition) .

Unfortunately, in my use case we strictly want *NUMERIC->ALPHA :Don't 
split* and there is no way around to change this behavior using the 
configurationFlags.

Since this isBreak() method is private and WordDelimiterFilterIterator 
class final therefore there is no possibility for subclassing and 
overriding this method.

Also, WordDelimiterFilterIterator is tightly coupled with 
WordDelimiterFilter (WordDelimiterGraphFilter in 7.x) and both are 
final. So this leaves me with only one option to copy paste their code 
into custom classes and change the behaviour. Clearly this is not a 
maintainable solution.

So, I am looking for advise what else is possible? OR is there a 
possibility of a patch/refactoring to fix isBreak() to use some new 
configuration flags?

- Best

Parit Bansal

(Developer www.uniprot.org)


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org