lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Li Li <fancye...@gmail.com>
Subject Re: Question about MaxFieldLength
Date Sun, 28 Aug 2011 02:28:09 GMT
   It will affect the entire index because it 's a parameter of IndexWriter.
but you can modify it anytime you like before IndexWriter.addDocument.
If you want to truncate different fields with different maxLength. you
should  avoid multithreads' race condition.
   maybe you can add a TokenFilter to the end of analyzer train

  AtomicInteger field1Counter;
  public final boolean incrementToken() throws IOException {
               if(field1Counter.get()>=maxLength) return false;
               if (input.incrementToken()) {
                   field1Counter.incrementAndGet();
                   return true;
                }else{
                   return false;
                }
        }
   }
On Sun, Aug 28, 2011 at 9:45 AM, roz dev <rozdev29@gmail.com> wrote:

> Hi All
>
> I have a question regarding MaxFieldLength. Is it a limit for number of
> tokens in 1 field per document or entire index?
>
> Example:
>
> If MaxFieldLength is set to 100 and I add a document which has 105 tokens
> for 1 field then I expect 5 tokens to be ignored.
> but, if i add another document which has 95 tokens for same field then
> these
> 95 tokens should be added.
>
> Please advise.
>
> Thanks
> Saroj
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message