lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vatuska <vatu...@yandex.ru>
Subject DIH deletedPkQuery seems doesn't work
Date Wed, 18 Dec 2013 05:35:27 GMT
Hello all.
I use Solr 4.6 and I have some problems with delta import from MySQL.
There is an information about deleting documents in logs, but they don't
deleted actually.
Update of documents works fine, so I think commit works.

There is a data-config:
...
<document name="entities">
        <entity name="object"
                pk="id"
                query="SELECT * FROM objects WHERE NOT isDeleted AND NOT
isHidden"
                deltaImportQuery="SELECT * FROM objects WHERE id =
'${dih.delta.id}'"
                deltaQuery="SELECT id FROM objects WHERE time
>='${dataimporter.entity.last_index_time}' AND NOT isDeleted AND NOT
isHidden"
                deletedPkQuery="SELECT id FROM objects WHERE time >=
'${dataimporter.entity.last_index_time}' AND (isDeleted OR isHidden)">
            <field column="id" name="id" />
            <field column="title" name="title"/>
            <field column="description" name="description"/>
            <field column="isPrivate" name="isPrivate"/>

            <entity name="parent_object"
                    pk="parentid, childid"
                    transformer="script:extractParent"
                    query="SELECT or.parentid, or.childid FROM
objectsrelations r
                               JOIN objects o ON (o.id=r.parentid) WHERE
r.childid='${object.id}' AND NOT o.isDeleted"/>
        </entity>
</document>
...

And there is a log:
INFO  - 2013-12-17 12:48:44.447;
org.apache.solr.handler.dataimport.DataImporter; Loading DIH Configuration:
data-config.xml
INFO  - 2013-12-17 12:48:44.454;
*org.apache.solr.handler.dataimport.config.DIHConfiguration; _version_ is a
required field in SolrSchema . But not found in DataConfig*
INFO  - 2013-12-17 12:48:44.457;
org.apache.solr.handler.dataimport.DataImporter; Data Configuration loaded
successfully
INFO  - 2013-12-17 12:48:44.457;
org.apache.solr.handler.dataimport.DataImporter; Starting Delta Import
INFO  - 2013-12-17 12:48:44.457; org.apache.solr.core.SolrCore;
[collection1] webapp=/solr path=/dataimport
params={clean=true&commit=true&command=delta-import&wt=javabin&qt=/dataimport&version=2}
status=0 QTime=10 
INFO  - 2013-12-17 12:48:44.464;
org.apache.solr.handler.dataimport.SimplePropertiesWriter; Read
dataimport.properties
INFO  - 2013-12-17 12:48:44.467; org.apache.solr.search.SolrIndexSearcher;
Opening Searcher@7b07b74d realtime
INFO  - 2013-12-17 12:48:44.468;
org.apache.solr.handler.dataimport.DocBuilder; Starting delta collection.
INFO  - 2013-12-17 12:48:44.486;
org.apache.solr.handler.dataimport.DocBuilder; Running ModifiedRowKey() for
Entity: parent_object
INFO  - 2013-12-17 12:48:44.487;
org.apache.solr.handler.dataimport.DocBuilder; Completed ModifiedRowKey for
Entity: parent_object rows obtained : 0
INFO  - 2013-12-17 12:48:44.487;
org.apache.solr.handler.dataimport.DocBuilder; Completed DeletedRowKey for
Entity: parent_object rows obtained : 0
INFO  - 2013-12-17 12:48:44.487;
org.apache.solr.handler.dataimport.DocBuilder; Completed parentDeltaQuery
for Entity: parent_object
INFO  - 2013-12-17 12:48:44.487;
org.apache.solr.handler.dataimport.DocBuilder; Running ModifiedRowKey() for
Entity: object
INFO  - 2013-12-17 12:48:44.487;
org.apache.solr.handler.dataimport.JdbcDataSource$1; Creating a connection
for entity object with URL:
jdbc:mysql://localhost:3306/mydb?characterEncoding=utf8
INFO  - 2013-12-17 12:48:44.506;
org.apache.solr.handler.dataimport.JdbcDataSource$1; Time taken for
getConnection(): 18
INFO  - 2013-12-17 12:48:44.531;
org.apache.solr.handler.dataimport.DocBuilder; Completed ModifiedRowKey for
Entity: object rows obtained : 0
*INFO  - 2013-12-17 12:48:44.531;
org.apache.solr.handler.dataimport.DocBuilder; Completed DeletedRowKey for
Entity: object rows obtained : 1*
INFO  - 2013-12-17 12:48:44.532;
org.apache.solr.handler.dataimport.DocBuilder; Completed parentDeltaQuery
for Entity: object
INFO  - 2013-12-17 12:48:44.532;
org.apache.solr.handler.dataimport.DocBuilder; Deleting stale documents 
*INFO  - 2013-12-17 12:48:44.532;
org.apache.solr.handler.dataimport.SolrWriter; Deleting document: 2*
INFO  - 2013-12-17 12:48:44.533;
org.apache.solr.handler.dataimport.DocBuilder; Delta Import completed
successfully
INFO  - 2013-12-17 12:48:44.582;
org.apache.solr.handler.dataimport.DocBuilder; Import completed successfully
INFO  - 2013-12-17 12:48:44.582;
org.apache.solr.update.DirectUpdateHandler2; start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
INFO  - 2013-12-17 12:48:44.650; org.apache.solr.core.SolrDeletionPolicy;
SolrDeletionPolicy.onCommit: commits: num=2

commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/home/user/solr-4.6.0/example/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@312cfd62;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2cj,generation=3043}

commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/home/user/solr-4.6.0/example/solr/collection1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@312cfd62;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2ck,generation=3044}
INFO  - 2013-12-17 12:48:44.652; org.apache.solr.core.SolrDeletionPolicy;
newest commit generation = 3044
INFO  - 2013-12-17 12:48:44.654; org.apache.solr.search.SolrIndexSearcher;
Opening Searcher@2de1c7ac main
INFO  - 2013-12-17 12:48:44.655;
org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
INFO  - 2013-12-17 12:48:44.655; org.apache.solr.core.QuerySenderListener;
QuerySenderListener sending requests to Searcher@2de1c7ac
main{StandardDirectoryReader(segments_2cj:12432:nrt)}
INFO  - 2013-12-17 12:48:44.655; org.apache.solr.core.QuerySenderListener;
QuerySenderListener done.
INFO  - 2013-12-17 12:48:44.655; org.apache.solr.core.SolrCore;
[collection1] Registered new searcher Searcher@2de1c7ac
main{StandardDirectoryReader(segments_2cj:12432:nrt)}
INFO  - 2013-12-17 12:48:44.657;
org.apache.solr.handler.dataimport.SimplePropertiesWriter; Read
dataimport.properties
INFO  - 2013-12-17 12:48:44.658;
org.apache.solr.handler.dataimport.SimplePropertiesWriter; Wrote last
indexed time to dataimport.properties
INFO  - 2013-12-17 12:48:44.658;
org.apache.solr.handler.dataimport.DocBuilder; Time taken = 0:0:0.194

Probably I do something wrong, but it looks like deleting has been done.
But "deleted" document is still displayed in search



--
View this message in context: http://lucene.472066.n3.nabble.com/DIH-deletedPkQuery-seems-doesn-t-work-tp4107205.html
Sent from the Solr - User mailing list archive at Nabble.com.

Mime
View raw message