lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amy Zhou <amy.z...@systemware.com>
Subject RE: index large size file
Date Tue, 10 Mar 2009 14:47:39 GMT
My issue here is that large file is truncated with default MaxFieldLength 10,000 during indexing.
The file size I index could be 10mb or larger. 

My questions are:

1) If I chose MaxFieldLength as UNLIMITED instead of 100,000, what the performance could be?
2) Any other options?


-----Original Message-----
From: Mark Miller [mailto:markrmiller@gmail.com] 
Sent: Tuesday, March 10, 2009 9:37 AM
To: java-user@lucene.apache.org
Subject: Re: index large size file

Amy Zhou wrote:
> Hi,
>
> I'm having a couple of questions about indexing large size file. As my understanding,
the default MaxFieldLength 100,000. In Lucene 2.4, we can set the MaxFieldLength during constructor.
My questions are:
>   
The default is 10,000.
> 1) How's the performance if MaxFieldLength is set to UNLIMITED?
>   
It depends on how long your documents are. Its simply a cutoff - 
documents longer than n (10,000 by default) terms will be truncated.
> 2) Any other options for indexing large size file?
>   
What is the problem you are trying to address? Are you having trouble 
indexing a very large file? Can you share more details?


-- 
- Mark

http://www.lucidimagination.com




---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message