lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Noble Paul നോബിള്‍ नोब्ळ्" <noble.p...@gmail.com>
Subject Re: Delta DataImport is not picking the modified value in DB
Date Mon, 29 Dec 2008 09:31:09 GMT
This may happen if your deltaQuery has a problem

Try using the deltaImportQuery also.

--

On Mon, Dec 29, 2008 at 2:57 PM, Shalin Shekhar Mangar
<shalinmangar@gmail.com> wrote:
> What does your data-config look like? especially the delta query part.
>
> On Mon, Dec 29, 2008 at 2:35 PM, Manupriya <manupriya.sinha@gmail.com>wrote:
>
>>
>> Hi,
>>
>> I am using Delta DataImport feature to partially refresh the indexes.
>>
>> Note - full-import is working perfectly fine. But I need to do delta-import
>> as I do not want to rebuild all the indexes.
>>
>> My database structure is same as mentioned in the example at
>> http://wiki.apache.org/solr/DataImportHandler.
>>
>> http://www.nabble.com/file/p21200498/example-schema.png
>> ---------------------------------------------------------------------------
>> The dataimport.properties has the value as -
>>
>> #Mon Dec 29 14:08:12 IST 2008
>> last_index_time=2008-12-29 14\:08\:12
>> ---------------------------------------------------------------------------
>> I have changed the last_modified for the 'tem' table as current timestamp.
>> And when I query the DB, I get the following result -
>>
>> http://www.nabble.com/file/p21200498/data.jpg
>>
>> -------------------------------------------------------------------------------------------------
>>
>> But when I call delta data-import, it returns me the following response -
>>
>> Server Response:
>>
>> {responseHeader={status=0,QTime=16},initArgs={defaults={config=db-data-config.xml}},command=delta-import,status=idle,importResponse=,statusMessages={},WARNING=This
>> response format is experimental.  It is likely to change in the future.}
>>
>>
>> ---------------------------------------------------------------------------------------------------
>> And when I query for data for '*:*', I get the following -
>> Note - Data returned by query doesnt reflect the correct timestamp from
>> database.
>>
>> SolrDocument(1)
>> Field Name & Value: includes=12
>> Field Name & Value: features=[demo feature, demo feature 1]
>> Field Name & Value: price=12.0
>> Field Name & Value: timestamp=Mon Dec 29 13:57:54 IST 2008
>> Field Name & Value: manu=manu-12
>> Field Name & Value: sku=1
>> Field Name & Value: id=1
>> Field Name & Value: popularity=1
>> Field Name & Value: weight=12.0
>> SolrDocument(2)
>> Field Name & Value: includes=43
>> Field Name & Value: features=demo feature 2
>> Field Name & Value: price=10.0
>> Field Name & Value: timestamp=Mon Dec 29 13:57:54 IST 2008
>> Field Name & Value: manu=Demo - 12
>> Field Name & Value: sku=2
>> Field Name & Value: id=2
>> Field Name & Value: popularity=5
>> Field Name & Value: weight=12.0
>> SolrDocument(3)
>> Field Name & Value: includes=1
>> Field Name & Value: features=demo -3
>> Field Name & Value: price=1.0
>> Field Name & Value: timestamp=Mon Dec 29 13:57:54 IST 2008
>> Field Name & Value: manu=manu - 36
>> Field Name & Value: sku=3
>> Field Name & Value: id=3
>> Field Name & Value: popularity=1
>> Field Name & Value: weight=1.0
>> SolrDocument(4)
>> Field Name & Value: includes=2
>> Field Name & Value: features=demo - 4
>> Field Name & Value: price=2.0
>> Field Name & Value: timestamp=Mon Dec 29 13:57:55 IST 2008
>> Field Name & Value: manu=manu - 46
>> Field Name & Value: sku=4
>> Field Name & Value: id=4
>> Field Name & Value: popularity=2
>> Field Name & Value: weight=2.0
>> SolrDocument(5)
>> Field Name & Value: includes=3
>> Field Name & Value: features=demo - 5
>> Field Name & Value: price=3.0
>> Field Name & Value: timestamp=Mon Dec 29 13:57:55 IST 2008
>> Field Name & Value: manu=manu - 56
>> Field Name & Value: sku=5
>> Field Name & Value: id=5
>> Field Name & Value: popularity=3
>> Field Name & Value: weight=3.0
>> SolrDocument(6)
>> Field Name & Value: includes=4
>> Field Name & Value: features=demo - 66
>> Field Name & Value: price=4.0
>> Field Name & Value: timestamp=Mon Dec 29 13:57:55 IST 2008
>> Field Name & Value: manu=manu - 66
>> Field Name & Value: sku=6
>> Field Name & Value: id=6
>> Field Name & Value: popularity=4
>> Field Name & Value: weight=4.0
>>
>>
>> ------------------------------------------------------------------------------------------
>> My Java code that calls delta import is as follows -
>>
>> public static void main(String[] args) throws IOException,
>> SolrServerException {
>> CommonsHttpSolrServer server = new
>> CommonsHttpSolrServer("http://localhost:8983/solr");
>> BinaryResponseParser parser = new BinaryResponseParser();
>> server.setParser(parser);
>>
>> ModifiableSolrParams params = new ModifiableSolrParams();
>> params.set("qt", "/dataimport");
>> params.set("command", "delta-import");
>> params.set("commit", "true");
>> params.set("wt","json");
>>
>> try{
>>
>>    QueryResponse response = server.query(params);
>>    server.commit();
>>    System.out.println("Server Response: " + response);
>>
>>    SolrQuery query = new SolrQuery("*:*");
>>    QueryResponse results = server.query(query);
>>
>>    SolrDocumentList list = results.getResults();
>>
>>    int index = 1;
>>    for (SolrDocument solrDocument : list) {
>>
>>        System.out.println("SolrDocument("+index+")");
>>        Iterator<Entry<String, Object>> iterator = solrDocument.iterator();
>>
>>        while(iterator.hasNext()){
>>                System.out.println("Field Name & Value: "+iterator.next());
>>        }
>>
>>        index++;
>>        }
>>
>>    }catch(Exception ex){
>>
>>        System.out.println("Exception Occured:"+ex);
>>    }
>>
>>  }
>> ---------------------------------------------------------------------------
>>
>> Does SOLR do some kind of caching? I dont understand as why the updated
>> values are not queried correctly from DB?
>>
>> There are no errors in the logs. But I can see an error on the server
>> console -
>>
>> SEVERE: Delta Import Failed
>> java.lang.NullPointerException
>>        at
>>
>> org.apache.solr.handler.dataimport.SqlEntityProcessor.getDeltaImportQuery(SqlEntityProcessor.java:153)
>>        at
>>
>> org.apache.solr.handler.dataimport.SqlEntityProcessor.getQuery(SqlEntityProcessor.java:125)
>>        at
>>
>> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:73)
>>        at
>>
>> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:285)
>>        at
>> org.apache.solr.handler.dataimport.DocBuilder.doDelta(DocBuilder.java:211)
>>        at
>> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:133)
>>        at
>>
>> org.apache.solr.handler.dataimport.DataImporter.doDeltaImport(DataImporter.java:359)
>>        at
>>
>> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:388)
>>        at
>>
>> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:377)
>>
>>
>> What can be the reason for this error?
>>
>> Thanks,
>> Manu
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Delta-DataImport-is-not-picking-the-modified-value-in-DB-tp21200498p21200498.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
>>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>



-- 
--Noble Paul

Mime
View raw message