lucene-solr-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Solr Wiki] Update of "DataImportHandlerFaq" by NoblePaul
Date Mon, 29 Dec 2008 10:31:49 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Solr Wiki" for change notification.

The following page has been changed by NoblePaul:
http://wiki.apache.org/solr/DataImportHandlerFaq

------------------------------------------------------------------------------
   * A Transformer can act on multiple fields . Or it can even do nothing to the data. Say
a !LogTransformer just logs data to a file
   * Transfomers can be chained . If declared in a field the order of chaining becomes unclear
  
+ === My delta-import goes out of memory . Any workaround ? ===
+ 
+ It is possible to do delta import using full-import.
+ Taken from the delta-import example
+ {{{
+ <dataConfig>
+     <dataSource driver="org.hsqldb.jdbcDriver" url="jdbc:hsqldb:/temp/example/ex" user="sa"
/>
+     <document>
+           <entity name="findDelta" query="select id from item where id in
+                                 (select item_id as id from feature where last_modified >
'${dataimporter.last_index_time}')
+                                 or id in 
+                                 (select item_id as id from item_category where item_id in

+                                     (select id as item_id from category where last_modified
> '${dataimporter.last_index_time}')
+                                 or last_modified > '${dataimporter.last_index_time}')
+                                 or last_modified > '${dataimporter.last_index_time}'"
rootEntity="false">
+             <entity name="item" query="select * from item where id='${findDelta.id}'>
+             <entity name="feature" query="select description as features from feature
where item_id='${item.ID}'">                
+             </entity>
+             <entity name="item_category" query="select CATEGORY_ID from item_category
where ITEM_ID='${item.ID}'">
+                 <entity name="category" query="select description as cat from category
where id = '${item_category.CATEGORY_ID}'">                    
+                 </entity>
+             </entity>
+         </entity>
+       </entity>
+     </document>
+ </dataConfig>
+ }}}
+ 
+ notice that for the entity 'findDelta' rootEntity is set to false, so a document is created
for each row from 'item'.
+ The command has to be 'full-import' and set clean=false
+ 

Mime
View raw message