lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dawid Weiss <dawid.we...@gmail.com>
Subject Re: RAMDirectory or Redis
Date Sun, 02 Dec 2018 15:03:51 GMT
bq. We switched to ByteBuffersDirectory with 7.5, but
I actually didn't see much performance improvements or savings in memory.

Once the indexes are built I don't think there will be much of a
difference. The core problem with RAMDirectory was related to
synchronizations during merges/ file manipulations.

D.
On Sun, Dec 2, 2018 at 1:18 PM Arjen van der Meijden
<acmmailing@tweakers.net> wrote:
>
> I doubt using Redis as directory-storage will be very good. I'd expect
> it to have much more latency for reads and writes compared to any of
> lucene's own directories. And Lucene probably won't like it if another
> Lucene-instance changes that database.
>
> It may be interesting as a result-level cache though, if you have a
> relatively large amount of repeated queries.
>
> With regards to RAMDirectory (and its successor the
> ByteBuffersDirectory), it works well enough for our setup: we run an
> in-memory copy of our regular sql database in an elastic search like
> setup to support very fast responses with a form of faceted search which
> would require relatively complex sql-statements (and many repeated
> ones). Lucene is only used for textual search in that system.
>
> We didn't want any trouble trying to sync whatever is on disk with the
> database and/or having to purchase relatively expensive storage in the
> servers. The total application uses about 6GB of RAM, of which somewhere
> between 1/2 and 1/3 is for the Lucene directories (about 20-30).
> We run an independent copy of that application on each of our 4
> applications servers, so we can afford a relatively long startup time of
> the application. In all it only takes about 4-5 minutes to load all data
> and build in-memory lucene databases.
>
> Our largest Lucene directory is about 400MB, and RAMDirectory worked
> pretty good for this. We switched to ByteBuffersDirectory with 7.5, but
> I actually didn't see much performance improvements or savings in memory.
>
> If you do care about the rebuild time of your search engine, than just
> using Lucene's recommended database and not trying to add additional
> complexity with those in-memory databases is probably a good idea :)
>
> Best regards,
>
> Arjen
>
> On 2-12-2018 10:22, Joe MA wrote:
> > Greetings,
> >
> > Has anyone looked into using Redis or some other in-memory cache with Lucene?
> > It seems that ElasticSearch may do this.  Are there advantages to doing this versus,
say, the RAMDirectory class?
> >
> > Thanks in advance,
> > J
> >
> >
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message