lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dan Davis <dansm...@gmail.com>
Subject Re: Tika HTTP 400 Errors with DIH
Date Mon, 08 Dec 2014 21:39:48 GMT
I would say that you could determine a row that gives a bad URL, and then
run it in DIH admin interface (or the command-line) with "debug" enabled
The url parameter going into tika should be present in its transformed form
before the next entity gets going.   This works in a similar scenario for
me.

On Tue, Dec 2, 2014 at 1:19 PM, Teague James <teaguej@insystechinc.com>
wrote:

> Hi all,
>
> I am using Solr 4.9.0 to index a DB with DIH. In the DB there is a URL
> field. In the DIH Tika uses that field to fetch and parse the documents.
> The
> URL from the field is valid and will download the document in the browser
> just fine. But Tika is getting HTTP response code 400. Any ideas why?
>
> ERROR
> BinURLDataSource
> java.io.IOException: Server returned HTTP response code: 400 for URL:
>
> EntityProcessorWrapper
> Exception in entity :
> tika_content:org.apache.solr.handler.dataimport.DataImportHandlerException:
> Exception in invoking url
>
> DIH
> <dataConfig>
>         <dataSource type="JdbcDataSource"
>               name="ds-1"
>               driver="net.sourceforge.jtds.jdbc.Driver"
>
> url="jdbc:jtds:sqlserver://
> 1.2.3.4/database;instance=INSTANCE;user=USER;pass
> word=PASSWORD" />
>
>         <dataSource type="BinURLDataSource" name="ds-2" />
>
>         <document>
>         <entity name="db_content" dataSource="ds-1"
> transformer="ClobTransformer, RegexTransformer"
>                 query="SELECT ContentID,
>                         DownloadURL
>                         FROM DATABASE.VIEW
>                         <field column="ContentID" name="id" />
>                         <field column="DownloadURL" clob="true"
> name="DownloadURL" />
>
>                         <entity name="tika_content"
> processor="TikaEntityProcessor" url="${db_content.DownloadURL}"
> onError="continue" dataSource="ds-2">
>                                 <field column="TikaParsedContent" />
>                         </entity>
>
>         </entity>
>         </document>
> </dataConfig>
>
> SCHEMA - Fields
> <field name="DownloadURL" type="string" indexed="true" stored="true" />
> <field name="TikaParsedContent" type="text_general" indexed="true"
> stored="true" multiValued="true"/>
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message