lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lance Norskog <goks...@gmail.com>
Subject Re: DIH dataimport.properties with
Date Sun, 18 Apr 2010 00:02:44 GMT
The SolrEntityProcessor allows you to query a Solr instance and use
the results as DIH properties. You would have to create your own
regular query to do the delta-import instead of using the delta-import
feature.

https://issues.apache.org/jira/browse/SOLR-1499

On 4/16/10, Otis Gospodnetic <otis_gospodnetic@yahoo.com> wrote:
> Hm, why not just go to the MySQL master then?
>
> Otis
> ----
> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
> Hadoop ecosystem search :: http://search-hadoop.com/
>
>
>
> ----- Original Message ----
>> From: Michael Tibben <michael.tibben@stomp.com.au>
>> To: solr-user@lucene.apache.org
>> Sent: Thu, April 15, 2010 10:15:14 PM
>> Subject: DIH dataimport.properties with
>>
>> Hi,
>
> I am using the DIH to import data from a mysql slave. However, the
>> slave sometimes runs behind the master. The delay is variable, most of the
>> time
>> it is in sync, but sometimes can run behind by a few minutes.
>
> This is a
>> problem, because DIH uses dataimport.properties to determine the
>> last_index_time
>> for delta updates. This last_index_time does not correspond to the
>> position of
>> the slave, and so documents are being missed.
>
> What I need to be able to
>> do is tell DIH what the last_index_time should be. Or alternatively, be
>> able to
>> specify another property in dataimport.properties, perhaps called
>> datasource_version or similar.
>
> Is this possible?
>
>
> I have
>> thought of a sneaky way to hack around the issue. Just before the delta
>> update
>> is run, I will switch the system time to the mysql slave's replication
>> time. The
>> system is used for nothing but solr master, so I think this should work
>> OK. Any
>> thoughts?
>
> Regards,
>
> Michael
>


-- 
Lance Norskog
goksron@gmail.com

Mime
View raw message