lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mikhail Khludnev <m...@apache.org>
Subject Re: How search code files for words which contains a given substrings?
Date Tue, 26 Jun 2018 13:04:20 GMT
I mean, you'd rather need offsets not positions, but I don't have something
definite to suggest.

On Tue, Jun 26, 2018 at 1:29 PM Gordin, Ira <ira.gordin@sap.com> wrote:

> Hello Mikhail,
>
> I see in the link you sent that PositionIncrementAttribute determines the
> position of this token relative to the previous Token in a TokenStream,
> used in phrase searching.
> I am not in phrase searching.
> Would you mind to explain how it can help me?
>
> Thanks,
> Ira
>
> -----Original Message-----
> From: Mikhail Khludnev [mailto:mkhl@apache.org]
> Sent: Tuesday, June 26, 2018 12:33 PM
> To: java-user@lucene.apache.org
> Subject: Re: How search code files for words which contains a given
> substrings?
>
> Hello, Ira.
> Note the difference between offset
>
> https://lucene.apache.org/core/7_3_0/core/org/apache/lucene/analysis/tokenattributes/OffsetAttribute.html
> and
> position
>
> https://lucene.apache.org/core/7_3_0/core/org/apache/lucene/analysis/tokenattributes/PositionIncrementAttribute.html
> in Lucene terminology.
> Please make sure you don't rebuild existing functionality
>
> https://lucene.apache.org/core/7_3_1/highlighter/org/apache/lucene/search/highlight/package-summary.html#package.description
>
>
> On Tue, Jun 26, 2018 at 10:57 AM Gordin, Ira <ira.gordin@sap.com> wrote:
>
> > Hi all,
> > I started to work on project which currently search code files for words
> > which contains a given substrings.
> > Currently it uses WhitespaceTokenizerand use regex query which wraps the
> > searched substring with '.*'.
> > For example, if one search for 'a', the query will be '/.*a.*/'. In this
> > way in the 'Mama loves banana' text, it will find tokens 'Mama' and
> > 'banana'.
> > Currently I need to get the start and end positions of matched tokens in
> > the line and the line number.
> > With TokenStream I can get start and end positions of  'Mama' and
> 'banana'
> > in the full text. But I need the positions of 'a'.
> > I see 2 options.
> > Option 1: to perform additional search in returned token.
> > Option 2: to use NGramTokenizer or NGramTokenFilter (not sure which of
> > them) and in this way I hope I will get the 'a' positions in TokenStream.
> > Additional question how I can get the line numbers and the positions
> > inside the line.
> > Many thanks in advance for your help,
> > Ira
> >
> >
>
> --
> Sincerely yours
> Mikhail Khludnev
>


-- 
Sincerely yours
Mikhail Khludnev

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message