Return-Path: Delivered-To: apmail-lucene-commits-archive@www.apache.org Received: (qmail 1128 invoked from network); 2 Nov 2010 16:09:53 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 2 Nov 2010 16:09:53 -0000 Received: (qmail 31503 invoked by uid 500); 2 Nov 2010 16:10:24 -0000 Mailing-List: contact commits-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@lucene.apache.org Delivered-To: mailing list commits@lucene.apache.org Received: (qmail 31496 invoked by uid 99); 2 Nov 2010 16:10:24 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 Nov 2010 16:10:24 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 Nov 2010 16:10:20 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id 4EEBE2388903; Tue, 2 Nov 2010 16:09:06 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1030098 - in /lucene/dev/trunk/solr: ./ contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/ contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/ contrib/dataimporthandler/src/test/resources/so... Date: Tue, 02 Nov 2010 16:09:06 -0000 To: commits@lucene.apache.org From: koji@apache.org X-Mailer: svnmailer-1.0.8 Message-Id: <20101102160906.4EEBE2388903@eris.apache.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: koji Date: Tue Nov 2 16:09:05 2010 New Revision: 1030098 URL: http://svn.apache.org/viewvc?rev=1030098&view=rev Log: SOLR-2057: DataImportHandler never calls UpdateRequestProcessor.finish() Modified: lucene/dev/trunk/solr/CHANGES.txt lucene/dev/trunk/solr/contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/DocBuilder.java lucene/dev/trunk/solr/contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/SolrWriter.java lucene/dev/trunk/solr/contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/AbstractDataImportHandlerTestCase.java lucene/dev/trunk/solr/contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/TestDocBuilder.java lucene/dev/trunk/solr/contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/TestDocBuilder2.java lucene/dev/trunk/solr/contrib/dataimporthandler/src/test/resources/solr/conf/contentstream-solrconfig.xml lucene/dev/trunk/solr/contrib/dataimporthandler/src/test/resources/solr/conf/dataimport-solrconfig.xml Modified: lucene/dev/trunk/solr/CHANGES.txt URL: http://svn.apache.org/viewvc/lucene/dev/trunk/solr/CHANGES.txt?rev=1030098&r1=1030097&r2=1030098&view=diff ============================================================================== --- lucene/dev/trunk/solr/CHANGES.txt (original) +++ lucene/dev/trunk/solr/CHANGES.txt Tue Nov 2 16:09:05 2010 @@ -540,6 +540,9 @@ Bug Fixes * SOLR-2190: change xpath from RSS 0.9 to 1.0 in slashdot sample. (koji) * SOLR-1962: SolrCore#initIndex should not use a mix of indexPath and newIndexPath (Mark Miller) + +* SOLR-2057: DataImportHandler never calls UpdateRequestProcessor.finish() + (Drew Farris via koji) Other Changes ---------------------- Modified: lucene/dev/trunk/solr/contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/DocBuilder.java URL: http://svn.apache.org/viewvc/lucene/dev/trunk/solr/contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/DocBuilder.java?rev=1030098&r1=1030097&r2=1030098&view=diff ============================================================================== --- lucene/dev/trunk/solr/contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/DocBuilder.java (original) +++ lucene/dev/trunk/solr/contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/DocBuilder.java Tue Nov 2 16:09:05 2010 @@ -205,6 +205,11 @@ public class DocBuilder { // Finished operation normally, commit now finish(lastIndexTimeProps); } + + if (writer != null) { + writer.finish(); + } + if (document.onImportEnd != null) { invokeEventListener(document.onImportEnd); } Modified: lucene/dev/trunk/solr/contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/SolrWriter.java URL: http://svn.apache.org/viewvc/lucene/dev/trunk/solr/contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/SolrWriter.java?rev=1030098&r1=1030097&r2=1030098&view=diff ============================================================================== --- lucene/dev/trunk/solr/contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/SolrWriter.java (original) +++ lucene/dev/trunk/solr/contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/SolrWriter.java Tue Nov 2 16:09:05 2010 @@ -125,6 +125,15 @@ public class SolrWriter { } } + void finish() { + try { + processor.finish(); + } catch (IOException e) { + throw new DataImportHandlerException(DataImportHandlerException.SEVERE, + "Unable to call finish() on UpdateRequestProcessor", e); + } + } + Properties readIndexerProperties() { Properties props = new Properties(); InputStream propInput = null; Modified: lucene/dev/trunk/solr/contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/AbstractDataImportHandlerTestCase.java URL: http://svn.apache.org/viewvc/lucene/dev/trunk/solr/contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/AbstractDataImportHandlerTestCase.java?rev=1030098&r1=1030097&r2=1030098&view=diff ============================================================================== --- lucene/dev/trunk/solr/contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/AbstractDataImportHandlerTestCase.java (original) +++ lucene/dev/trunk/solr/contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/AbstractDataImportHandlerTestCase.java Tue Nov 2 16:09:05 2010 @@ -19,6 +19,15 @@ package org.apache.solr.handler.dataimpo import org.apache.solr.SolrTestCaseJ4; import org.apache.solr.core.SolrCore; import org.apache.solr.request.LocalSolrQueryRequest; +import org.apache.solr.request.SolrQueryRequest; +import org.apache.solr.response.SolrQueryResponse; +import org.apache.solr.update.AddUpdateCommand; +import org.apache.solr.update.CommitUpdateCommand; +import org.apache.solr.update.DeleteUpdateCommand; +import org.apache.solr.update.MergeIndexesCommand; +import org.apache.solr.update.RollbackUpdateCommand; +import org.apache.solr.update.processor.UpdateRequestProcessor; +import org.apache.solr.update.processor.UpdateRequestProcessorFactory; import org.apache.solr.common.util.NamedList; import org.junit.After; import org.junit.Before; @@ -270,4 +279,69 @@ public abstract class AbstractDataImport return delegate.replaceTokens(template); } } + + public static class TestUpdateRequestProcessorFactory extends UpdateRequestProcessorFactory { + + @Override + public UpdateRequestProcessor getInstance(SolrQueryRequest req, + SolrQueryResponse rsp, UpdateRequestProcessor next) { + return new TestUpdateRequestProcessor(next); + } + + } + + public static class TestUpdateRequestProcessor extends UpdateRequestProcessor { + + public static boolean finishCalled = false; + public static boolean processAddCalled = false; + public static boolean processCommitCalled = false; + public static boolean processDeleteCalled = false; + public static boolean mergeIndexesCalled = false; + public static boolean rollbackCalled = false; + + public static void reset() { + finishCalled = false; + processAddCalled = false; + processCommitCalled = false; + processDeleteCalled = false; + mergeIndexesCalled = false; + rollbackCalled = false; + } + + public TestUpdateRequestProcessor(UpdateRequestProcessor next) { + super(next); + reset(); + } + + public void finish() throws IOException { + finishCalled = true; + super.finish(); + } + + public void processAdd(AddUpdateCommand cmd) throws IOException { + processAddCalled = true; + super.processAdd(cmd); + } + + public void processCommit(CommitUpdateCommand cmd) throws IOException { + processCommitCalled = true; + super.processCommit(cmd); + } + + public void processDelete(DeleteUpdateCommand cmd) throws IOException { + processDeleteCalled = true; + super.processDelete(cmd); + } + + public void processMergeIndexes(MergeIndexesCommand cmd) throws IOException { + mergeIndexesCalled = true; + super.processMergeIndexes(cmd); + } + + public void processRollback(RollbackUpdateCommand cmd) throws IOException { + rollbackCalled = true; + super.processRollback(cmd); + } + + } } Modified: lucene/dev/trunk/solr/contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/TestDocBuilder.java URL: http://svn.apache.org/viewvc/lucene/dev/trunk/solr/contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/TestDocBuilder.java?rev=1030098&r1=1030097&r2=1030098&view=diff ============================================================================== --- lucene/dev/trunk/solr/contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/TestDocBuilder.java (original) +++ lucene/dev/trunk/solr/contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/TestDocBuilder.java Tue Nov 2 16:09:05 2010 @@ -60,6 +60,7 @@ public class TestDocBuilder extends Abst di.runCmd(rp, swi); assertEquals(Boolean.TRUE, swi.deleteAllCalled); assertEquals(Boolean.TRUE, swi.commitCalled); + assertEquals(Boolean.TRUE, swi.finishCalled); assertEquals(0, swi.docs.size()); assertEquals(1, di.getDocBuilder().importStatistics.queryCount.get()); assertEquals(0, di.getDocBuilder().importStatistics.docCount.get()); @@ -81,6 +82,7 @@ public class TestDocBuilder extends Abst di.runCmd(rp, swi); assertEquals(Boolean.FALSE, swi.deleteAllCalled); assertEquals(Boolean.FALSE, swi.commitCalled); + assertEquals(Boolean.TRUE, swi.finishCalled); assertEquals(0, swi.docs.size()); assertEquals(1, di.getDocBuilder().importStatistics.queryCount.get()); assertEquals(0, di.getDocBuilder().importStatistics.docCount.get()); @@ -104,6 +106,7 @@ public class TestDocBuilder extends Abst di.runCmd(rp, swi); assertEquals(Boolean.TRUE, swi.deleteAllCalled); assertEquals(Boolean.TRUE, swi.commitCalled); + assertEquals(Boolean.TRUE, swi.finishCalled); assertEquals(1, swi.docs.size()); assertEquals(1, di.getDocBuilder().importStatistics.queryCount.get()); assertEquals(1, di.getDocBuilder().importStatistics.docCount.get()); @@ -134,6 +137,7 @@ public class TestDocBuilder extends Abst di.runCmd(rp, swi); assertEquals(Boolean.FALSE, swi.deleteAllCalled); assertEquals(Boolean.TRUE, swi.commitCalled); + assertEquals(Boolean.TRUE, swi.finishCalled); assertEquals(1, swi.docs.size()); assertEquals(1, di.getDocBuilder().importStatistics.queryCount.get()); assertEquals(1, di.getDocBuilder().importStatistics.docCount.get()); @@ -168,6 +172,7 @@ public class TestDocBuilder extends Abst di.runCmd(rp, swi); assertEquals(Boolean.TRUE, swi.deleteAllCalled); assertEquals(Boolean.TRUE, swi.commitCalled); + assertEquals(Boolean.TRUE, swi.finishCalled); assertEquals(3, swi.docs.size()); for (int i = 0; i < l.size(); i++) { Map map = (Map) l.get(i); @@ -189,6 +194,8 @@ public class TestDocBuilder extends Abst Boolean commitCalled = Boolean.FALSE; + Boolean finishCalled = Boolean.FALSE; + public SolrWriterImpl() { super(null, "."); } @@ -208,6 +215,10 @@ public class TestDocBuilder extends Abst public void commit(boolean b) { commitCalled = Boolean.TRUE; } + + public void finish() { + finishCalled = Boolean.TRUE; + } } public static final String dc_singleEntity = "\n" Modified: lucene/dev/trunk/solr/contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/TestDocBuilder2.java URL: http://svn.apache.org/viewvc/lucene/dev/trunk/solr/contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/TestDocBuilder2.java?rev=1030098&r1=1030097&r2=1030098&view=diff ============================================================================== --- lucene/dev/trunk/solr/contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/TestDocBuilder2.java (original) +++ lucene/dev/trunk/solr/contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/TestDocBuilder2.java Tue Nov 2 16:09:05 2010 @@ -52,6 +52,10 @@ public class TestDocBuilder2 extends Abs runFullImport(loadDataConfig("single-entity-data-config.xml")); assertQ(req("id:1"), "//*[@numFound='1']"); + + assertTrue("Update request processor processAdd was not called", TestUpdateRequestProcessor.processAddCalled); + assertTrue("Update request processor processCommit was not callled", TestUpdateRequestProcessor.processCommitCalled); + assertTrue("Update request processor finish was not called", TestUpdateRequestProcessor.finishCalled); } @Test @@ -66,6 +70,8 @@ public class TestDocBuilder2 extends Abs assertQ(req("id:1"), "//*[@numFound='1']"); assertTrue("Start event listener was not called", StartEventListener.executed); assertTrue("End event listener was not called", EndEventListener.executed); + assertTrue("Update request processor processAdd was not called", TestUpdateRequestProcessor.processAddCalled); + assertTrue("Update request processor finish was not called", TestUpdateRequestProcessor.finishCalled); } @Test @@ -200,6 +206,9 @@ public class TestDocBuilder2 extends Abs assertQ(req("id:2"), "//*[@numFound='0']"); assertQ(req("id:3"), "//*[@numFound='1']"); + assertTrue("Update request processor processDelete was not called", TestUpdateRequestProcessor.processDeleteCalled); + assertTrue("Update request processor finish was not called", TestUpdateRequestProcessor.finishCalled); + MockDataSource.clearCache(); rows = new ArrayList(); rows.add(createMap("id", "1", "desc", "one")); @@ -212,6 +221,10 @@ public class TestDocBuilder2 extends Abs assertQ(req("id:1"), "//*[@numFound='0']"); assertQ(req("id:2"), "//*[@numFound='0']"); assertQ(req("id:3"), "//*[@numFound='1']"); + + assertTrue("Update request processor processDelete was not called", TestUpdateRequestProcessor.processDeleteCalled); + assertTrue("Update request processor finish was not called", TestUpdateRequestProcessor.finishCalled); + } @Test Modified: lucene/dev/trunk/solr/contrib/dataimporthandler/src/test/resources/solr/conf/contentstream-solrconfig.xml URL: http://svn.apache.org/viewvc/lucene/dev/trunk/solr/contrib/dataimporthandler/src/test/resources/solr/conf/contentstream-solrconfig.xml?rev=1030098&r1=1030097&r2=1030098&view=diff ============================================================================== --- lucene/dev/trunk/solr/contrib/dataimporthandler/src/test/resources/solr/conf/contentstream-solrconfig.xml (original) +++ lucene/dev/trunk/solr/contrib/dataimporthandler/src/test/resources/solr/conf/contentstream-solrconfig.xml Tue Nov 2 16:09:05 2010 @@ -385,7 +385,7 @@ org.apache.solr.handler.UpdateRequestProcessor --> - + *:* @@ -395,5 +395,11 @@ --> + + + + + + Modified: lucene/dev/trunk/solr/contrib/dataimporthandler/src/test/resources/solr/conf/dataimport-solrconfig.xml URL: http://svn.apache.org/viewvc/lucene/dev/trunk/solr/contrib/dataimporthandler/src/test/resources/solr/conf/dataimport-solrconfig.xml?rev=1030098&r1=1030097&r2=1030098&view=diff ============================================================================== --- lucene/dev/trunk/solr/contrib/dataimporthandler/src/test/resources/solr/conf/dataimport-solrconfig.xml (original) +++ lucene/dev/trunk/solr/contrib/dataimporthandler/src/test/resources/solr/conf/dataimport-solrconfig.xml Tue Nov 2 16:09:05 2010 @@ -391,5 +391,11 @@ --> + + + + + +