lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Trejkaz <trej...@trypticon.org>
Subject Re: OutOfMemoryError indexing large documents
Date Thu, 27 Nov 2014 01:34:14 GMT
On Wed, Nov 26, 2014 at 2:09 PM, Erick Erickson <erickerickson@gmail.com> wrote:
> Well
> 2> seriously consider the utility of indexing a 100+M file. Assuming
> it's mostly text, lots and lots and lots of queries will match it, and
> it'll score pretty low due to length normalization. And you probably
> can't return it to the user. And highlighting it will be a performance
> problem. And may blow out memory too. And...

Meanwhile, some of our users have expressed concern that they can't
view a 2GB text file which was returned in a Lucene result. They even
want to see the term hits and expect that to somehow perform the same
as a small file. Totally unreasonable. :)

TX

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message