lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Rutherglen <jason.rutherg...@gmail.com>
Subject Re: Storing and loading the FST directly from disk
Date Sat, 04 Jun 2011 00:14:41 GMT
> (hint: try to provide a representation
> that will share as many suffixes and prefixes as possible since these
> conflate into a single path, no matter how many sequences you have)

It's just be user created keys, which will be sorted at least, and
probably will be highly likely to share large portions of the prefix.
Eg, I think timestamp is a common key type.

On Fri, Jun 3, 2011 at 12:06 AM, Dawid Weiss
<dawid.weiss@cs.put.poznan.pl> wrote:
>> here you should rather store pointers to another file and mmap that
>> file. Keep your FST as lean and compact as possible and make sure its
>> in memory. The compression should do a good job for you here!
>
> Yes, this is a good idea. If you can share a sample of that data that
> you want to keep in an FST I may be able to recommend something to
> keep it smaller in the FST (hint: try to provide a representation
> that will share as many suffixes and prefixes as possible since these
> conflate into a single path, no matter how many sequences you have).
>
> Dawid
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message