lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From findbestopensource <findbestopensou...@gmail.com>
Subject Re: Cache full text into memory
Date Wed, 14 Jul 2010 07:02:35 GMT
You have two options
1. Store the compressed text as part of stored field in Solr.
2. Using external caching.
http://www.findbestopensource.com/tagged/distributed-caching
    You could use ehcache / Memcache / Membase.

The problem with external caching is you need to synchronize the deletions
and modification. Fetching the stored field from Solr is also faster.

Regards
Aditya
www.findbestopensource.com


On Wed, Jul 14, 2010 at 12:08 PM, Li Li <fancyerii@gmail.com> wrote:

>     I want to cache full text into memory to improve performance.
> Full text is only used to highlight in my application(But it's very
> time consuming, My avg query time is about 250ms, I guess it will cost
> about 50ms if I just get top 10 full text. Things get worse when get
> more full text because in disk, it scatters erverywhere for a query.).
> My full text per machine is about 200GB. The memory available for
> store full text is about 10GB. So I want to compress it in memory.
> Suppose compression ratio is 1:5, then I can load 1/4 full text in
> memory. I need a Cache component for it. Has anyone faced the problem
> before? I need some advice. Is it possbile using external tools such
> as MemCached? Thank you.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message