lucene-solr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Luc Caprini <Luc.Capr...@Sun.COM>
Subject Re: Problem with DataImportHandler and JDBC
Date Tue, 15 Sep 2009 18:10:19 GMT
Hi

Thanks for your quick answer. it is ok now...

In fact, I try to read BLOB from my SQL Database.

I'm trying the simplest test i can by only trying to read a blob column 
in a table.

So my dataconfig is :

<dataConfig>
<dataSource driver="com.mysql.jdbc.Driver" 
url="jdbc:mysql://localhost/testa" user="root" password="password" 
batchSize="1"/>
<document>
<entity name="a"   transformer="ClobTransformer" query="select * from a">
<field column="a"  name="mycol" clob="true" />
</entity>
</document>
</dataConfig>

Result is :

<?xml version="1.0" encoding="UTF-8" ?>
- <#> <response>
- <#> <lst name="*responseHeader*">
<int name="*status*">0</int>
<int name="*QTime*">70</int>
</lst>
- <#> <lst name="*initArgs*">
- <#> <lst name="*defaults*">
<str name="*config*">data-conf.xml</str>
</lst>
</lst>
<str name="*command*">full-import</str>
<str name="*mode*">debug</str>
<null name="*documents*" />
- <#> <lst name="*verbose-output*">
- <#> <lst name="*entity:a*">
- <#> <lst name="*document#1*">
<str name="*query*">select * from a</str>
<str name="*time-taken*">0:0:0.40</str>
<str>----------- row #1-------------</str>
<str name="*a*">[B:[B@3f4ebd</str>
<str>---------------------------------------------</str>
- <#> <lst name="*transformer:ClobTransformer*">
<str>---------------------------------------------</str>
<str name="*a*">[B:[B@3f4ebd</str>
<str>---------------------------------------------</str>
</lst>
</lst>
<lst name="*document#1*" />
</lst>
</lst>
<str name="*status*">idle</str>
<str name="*importResponse*">Configuration Re-loaded sucessfully</str>
- <#> <lst name="*statusMessages*">
<str name="*Total Requests made to DataSource*">1</str>
<str name="*Total Rows Fetched*">1</str>
<str name="*Total Documents Skipped*">0</str>
<str name="*Full Dump Started*">2009-09-15 20:06:12</str>
<str name="*Time taken*">0:0:0.60</str>
</lst>
<str name="*WARNING*">This response format is experimental. It is likely 
to change in the future.</str>
</response>

As you can see, my column is not transformed.

I read the code for ClobTransformer, and it seems that this class is 
only able to read Clob and not Blob.

Am I wrong ?

How i can achieve my goal ?

Thanks in advance

Regards

Luc
Le 15/09/2009 16:42, Noble Paul ???????  ?????? a écrit :
> the parameter name is batchSize
>
> by default it is set as 500
>
> On Tue, Sep 15, 2009 at 7:27 PM, Luc Caprini<Luc.Caprini@sun.com>  wrote:
>    
>> Hi,
>>
>> I'm trying using SOLR 1.4. and in my first test, I've got an issue with the
>> DataImportHandler ...
>>
>>
>> Config : SOLR 1.4 on tomcat 5.5.27 with mysql driver 5.1.7 on mysql 5.1
>>
>> My dataconfig is
>>
>> <dataConfig>
>> <dataSource driver="com.mysql.jdbc.Driver"
>> url="jdbc:mysql://localhost/testa" user="root" password="password"/>
>>
>> <document>
>> <entity name="a" query="select * from a">
>> <field column="b" name="named"/>
>> </entity>
>>
>> </document>
>> </dataConfig>
>>
>> And it generates this error
>>
>> Unable to execute query: select * from a Processing Document # 1 at
>> org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:72)
>> at
>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:251)
>> at
>> org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:208)
>> at
>> org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:39)
>> at
>> org.apache.solr.handler.dataimport.DebugLogger$2.getData(DebugLogger.java:184)
>> at
>> org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:58)
>> at
>> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:71)
>> at
>> org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:237)
>> at
>> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:339)
>> at
>> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:225)
>> at
>> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:167)
>> at
>> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:333)
>> at
>> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:393)
>> at
>> org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:203)
>> at
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
>> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1301) at
>> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
>> at
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
>> at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
>> at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
>> at
>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
>> at
>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172)
>> at
>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
>> at
>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
>> at
>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
>> at
>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
>> at
>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:875)
>> at
>> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
>> at
>> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
>> at
>> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
>> at
>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
>> at java.lang.Thread.run(Unknown Source) Caused by: java.sql.SQLException: *
>> Illegal value for setFetchSize()*. at
>> com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1055) at
>> com.mysql.jdbc.SQLError.createSQLException(SQLError.java:956) at
>> com.mysql.jdbc.SQLError.createSQLException(SQLError.java:926) at
>> com.mysql.jdbc.StatementImpl.setFetchSize(StatementImpl.java:2404) at
>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:240)
>> ... 30 more</str>
>>
>> I tried to add fetchsize as a parameter :
>> <dataSource driver="com.mysql.jdbc.Driver"
>> url="jdbc:mysql://localhost/testa" user="root" password="password"
>> fetchSize="1"/>
>> but result is the same.
>>
>> Thanks in advance for your assistance
>>
>>
>>
>> Kind regards
>>
>>
>> Luc
>>
>> --
>> <http://www.sun.com>      * Luc Caprini *
>> Client Solutions Architect
>> *Sun Microsystems, Inc.*
>> 13 Avenue Morane Saulnier
>> Velizy 78140 France
>> Phone +33 (0) 1 34 03 00 20
>> Mobile +33 (0) 6 12 30 16 22
>> Fax +33 (0) 1 34 03 10 11
>> Email Luc.Caprini@Sun.COM
>>
>>
>>
>>      
>
>
>    


-- 
<http://www.sun.com> 	* Luc Caprini *
Client Solutions Architect
*Sun Microsystems, Inc.*
13 Avenue Morane Saulnier
Velizy 78140 France
Phone +33 (0) 1 34 03 00 20
Mobile +33 (0) 6 12 30 16 22
Fax +33 (0) 1 34 03 10 11
Email Luc.Caprini@Sun.COM
	


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message