lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Moen Endre <endre.m...@imr.no>
Subject RE: testing with EmbeddedSolrServer
Date Mon, 31 Aug 2015 09:15:57 GMT
Hi Mikhail,

Im trying to read 7-8 xml files of data that contain realistic data from our production server.
Then I would like to read this data into EmbeddedSolrServer to test for edge cases for our
custom date search. The use of EmbeddedSolrServer is purely to separate the data testing from
any environment that might change over time.

I would also like to avoid writing plumbing-code to import each field from the xml since I
already have a working DIH. 

I tried adding synchronous=true but it doesn’t look like it makes solr complete the import
before doing a search. 

Looking at the log it doesn’t seem process the import request:
[searcherExecutor-6-thread-1-processing-{core=nmdc}] DEBUG o.apache.solr.core.SolrCore.Request
- [nmdc] webapp=null path=null params={q=static+firstSearcher+warming+in+solrconfig.xml&distrib=false&event=firstSearcher}

...
[TEST-TestSolrEmbeddedServer.testNodeConfigConstructor-seed#[41C3C11DE20DD5CE]] INFO  org.apache.solr.core.CoreContainer
- registering core: nmdc
10:48:31.613 [TEST-TestSolrEmbeddedServer.testNodeConfigConstructor-seed#[41C3C11DE20DD5CE]]
INFO  o.apache.solr.core.SolrCore.Request - [nmdc] webapp=null path=/dataimport2 params={qt=%2Fdataimport2&command=full-import%26clean%3Dtrue%26synchronous%3Dtrue}
status=0 QTime=1 
{responseHeader={status=0,QTime=1},initArgs={defaults={config=dih-config.xml}},command=full-import&clean=true&synchronous=true,status=idle,importResponse=,statusMessages={}}
[TEST-TestSolrEmbeddedServer.testNodeConfigConstructor-seed#[41C3C11DE20DD5CE]] DEBUG o.apache.solr.core.SolrCore.Request
- [nmdc] webapp=null path=/select params={q=*%3A*} 
[TEST-TestSolrEmbeddedServer.testNodeConfigConstructor-seed#[41C3C11DE20DD5CE]] DEBUG o.a.s.h.component.QueryComponent
- process: q=*:*&df=text&rows=10&echoParams=explicit
[searcherExecutor-6-thread-1-processing-{core=nmdc}] DEBUG o.a.s.h.component.QueryComponent
- process: q=static+firstSearcher+warming+in+solrconfig.xml&distrib=false&df=text&event=firstSearcher&rows=10&echoParams=explicit
[searcherExecutor-6-thread-1-processing-{core=nmdc}] DEBUG o.a.s.search.stats.LocalStatsCache
- ## GET {q=static+firstSearcher+warming+in+solrconfig.xml&distrib=false&df=text&event=firstSearcher&rows=10&echoParams=explicit}
[searcherExecutor-6-thread-1-processing-{core=nmdc}] INFO  o.apache.solr.core.SolrCore.Request
- [nmdc] webapp=null path=null params={q=static+firstSearcher+warming+in+solrconfig.xml&distrib=false&event=firstSearcher}
hits=0 status=0 QTime=36 
[searcherExecutor-6-thread-1-processing-{core=nmdc}] INFO  org.apache.solr.core.SolrCore -
QuerySenderListener done.
[searcherExecutor-6-thread-1-processing-{core=nmdc}] INFO  org.apache.solr.core.SolrCore -
[nmdc] Registered new searcher Searcher@28be2785[nmdc] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
...
[TEST-TestSolrEmbeddedServer.testNodeConfigConstructor-seed#[41C3C11DE20DD5CE]] INFO  org.apache.solr.update.SolrCoreState
- Closing SolrCoreState
[TEST-TestSolrEmbeddedServer.testNodeConfigConstructor-seed#[41C3C11DE20DD5CE]] INFO  o.a.solr.update.DefaultSolrCoreState
- SolrCoreState ref count has reached 0 - closing IndexWriter
[TEST-TestSolrEmbeddedServer.testNodeConfigConstructor-seed#[41C3C11DE20DD5CE]] INFO  o.a.solr.update.DefaultSolrCoreState
- closing IndexWriter with IndexWriterCloser
[TEST-TestSolrEmbeddedServer.testNodeConfigConstructor-seed#[41C3C11DE20DD5CE]] DEBUG o.apache.solr.update.SolrIndexWriter
- Closing Writer DirectUpdateHandler2

Cheers
Endre

-----Original Message-----
From: Mikhail Khludnev [mailto:mkhludnev@griddynamics.com] 
Sent: 25. august 2015 19:43
To: solr-user
Subject: Re: testing with EmbeddedSolrServer

Hello,

I'm trying to guess what are you doing. It's not clear so far.
I found http://stackoverflow.com/questions/11951695/embedded-solr-dih
My conclusion, if you play with DIH and EmbeddedSolrServer you'd better to avoid the third
beast, you don't need to bother with tests.
I guess that main() is over while DIH runs in background thread. You need to loop status command
until import is over. or add synchronous=true parameter to full-import command it should switch
to synchronous mode:
https://github.com/apache/lucene-solr/blob/trunk/solr/contrib/dataimporthandler/src/java/org/apache/solr/handler/dataimport/DataImportHandler.java#L199

Take care


On Tue, Aug 25, 2015 at 4:41 PM, Moen Endre <endre.moen@imr.no> wrote:

> Is there an example of integration-testing with EmbeddedSolrServer 
> that loads data from a data importhandler - then queries the data? Ive 
> tried doing this based on 
> org.apache.solr.client.solrj.embedded.TestEmbeddedSolrServerConstructors.
>
> But no data is being imported.  Here is the test-class ive tried:
> https://gist.github.com/emoen/5d0a28df91c4c1127238
>
> Ive also tried writing a test by extending AbstractSolrTestCase - but 
> havnt got this working. Ive documented some of the log output here:
> http://stackoverflow.com/questions/32052642/solrcorestate-already-clos
> ed-with-unit-test-using-embeddedsolrserver-v-5-2-1
>
> Should I extend AbstractSolrTestCase or SolrTestCaseJ4 when writing tests?
>
> Cheers
> Endre
>



--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics

<http://www.griddynamics.com>
<mkhludnev@griddynamics.com>
Mime
View raw message