lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ryan McKinley" <ryan...@gmail.com>
Subject Re: Instructables on solr
Date Thu, 05 Apr 2007 06:08:54 GMT
>
> Can you elaborate on "running SOLR-20 with a hibernate-solr auto link"?  You mean you
listen to Hibernate events and use them to keep the index served by Solr in sync with the
DB?
>

I built a HibernateEventWatcher modeled after the compass framework
that automatically gets notified on insert/update/delete.  Anything
saved that the "SolrDocumentBuilder" knows what to do with gets sent
to solr automatically.  This way the solr index stays in sync with the
SQL index without any explict work.

A first pass at this is here:
http://solrstuff.org/svn/solrj-hibernate/src/org/apache/solr/client/solrj/hibernate/SolrSync.java

lots of that changed in the production code... when SOLR-20
stabalizes, I'll put the good bits back in and hopefully post it in a
'contrib' section.


> Also, "pooling for 30 seconds on the client side..." - are you referring to keeping data
cached in the Solr client for 30 seconds and every 30 second sending it to Solr for indexing?


We are currently running with a single (no replication or load
balancing) solr server.  With multiple webapps pointing to it.  Rather
then manage commit timing on the client-side, we have autoCommit set
to 1second.  The multiple webapps can't start overlapping commits.

Since commit flushes the caches and forces you to reopen the
searchers, we want to do it as little as possible.  This is required
for instant access to uploaded images, but not required for stuff that
can take a bit longer...  for the other stuff the client keeps a queue
(I called it pooling) of stuff to send.  Every 30 secs, it sends it to
solr in a bulk update.  That time could be longer, I found it was the
minimum time to avoid multiple unnecessary commits for our usage
patterns.


> If so, why not index continuously, either in real-time or in some background thread that
feeds off of a "to index" queue?
>

yes, we have a background thread that queues changes and sends them all at once.


ryan

Mime
View raw message