lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lance Norskog <goks...@gmail.com>
Subject Re: DIH: alternative approach to deltaQuery
Date Fri, 17 Sep 2010 03:40:43 GMT
Database optimization is not like program optimization- it is wildly 
unpredictable.

What bugs me about the delta approach is using the last time DIH ran, 
rather than a timestamp from the DB. Oh well. Also, with SOLR-1499 you 
can query Solr directly to see what it has.

Lukas Kahwe Smith wrote:
> Hi,
>
> I think i have mentioned this approach before on this list, but I really think that the
deltaQuery approach which is currently explained as the "way to do updates" is far from ideal.
It seems to add a lot of redundant queries.
>
> I therefore propose to merge the initial import and delta queries using the below approach:
>
>          <entity name="person" query="SELECT * FROM foo
>          WHERE '${dataimporter.request.clean}' != 'false' OR last_updated>  '${dataimporter.last_index_time}'">
>
> Using this approach when clean = true the "last_updated>  '${dataimporter.last_index_time}"
should be optimized out by any sane RDBMS. And if clean = false it basically triggers the
delta query part to be evaluated.
>
> Is there any downside to this approach? Should this be added to the wiki?
>
> regards.
> Lukas Kahwe Smith
> mls@pooteeweet.org
>
>
>
>    

Mime
View raw message