lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <>
Subject Re: Concurrent DB updates and delta import misses few records
Date Mon, 27 Sep 2010 12:32:16 GMT
  You could get it from Solr, yes.  That didn't even occur to me because 
when I was designing my scripts, I didn't yet have a fully integrated 
Solr index. :)  With hindsight, I still wouldn't get it from Solr.  I 
would lose some flexibility and ease of administration.

It's certainly possible to store all build-related tracking information 
in the database.  The build system for our old search product did it 
that way.  I decided to go with simple text files in an NFS-mounted 
directory for the rewrite.  It's easier for me to administer, just ssh 
to a server and examine or modify simple one-line text files.  On the 
script side, the files get read into a Perl hash.  With the old system, 
I found it cumbersome to go through the database interfaces.  The only 
thing that's still in the database is the delete table, because it is 
populated by triggers on the metadata table.

On 9/23/2010 12:48 AM, Shashikant Kore wrote:
> Thanks for the pointer, Shawn.  It, definitely, is useful.
> I am wondering if you could retrieve minDid from the solr rather than
> storing it externally. Max id from Solr index and max id from DB should
> define the lower and upper thresholds, respectively, of the delta range. Am
> I missing something?

View raw message