Mailing-List: contact dev-help@manifoldcf.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@manifoldcf.apache.org
Date: Sat, 25 Oct 2014 17:04:34 +0000 (UTC)
From: "Mingchun Zhao (JIRA)" <jira@apache.org>
To: dev@manifoldcf.apache.org
Message-ID: <JIRA.12750008.1414052268000.338671.1414256674282@Atlassian.JIRA>
In-Reply-To: <JIRA.12750008.1414052268000@Atlassian.JIRA>
References: <JIRA.12750008.1414052268000@Atlassian.JIRA>
 <JIRA.12750008.1414052268186@arcas>
Subject: [jira] [Reopened] (CONNECTORS-1079) the parsing in TikaExtractor
 always return empty result
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/CONNECTORS-1079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mingchun Zhao reopened CONNECTORS-1079:
---------------------------------------

Hi Karl,

Thank you for your help, I've tried your fix.
Unfortunately, this symptom still occurs even we have two ika-core.jar in both of lib and connector-lib directory.
It looks like that the two same jars cause jar conflict.
I tried to use ClassLoader to fix it, but gave up eventually. because that makes things more confusing.

Could you please confirm my suggestion as below:

1. Get rid of the tika-core.jar from lib directory(need to modify build.xml?)

2. Directly call Tika().detect to get MimeType instead of calling ExtensionMimeMap.mapToMimeType.
The related connectors as below(4 files):
connectors/filesystem/connector/src/main/java/org/apache/manifoldcf/crawler/connectors/filesystem/FileConnector.java
connectors/hdfs/connector/src/main/java/org/apache/manifoldcf/crawler/connectors/hdfs/HDFSRepositoryConnector.java
connectors/jcifs/connector/src/main/java/org/apache/manifoldcf/crawler/connectors/sharedrive/SharedDriveConnector.java
connectors/sharepoint/connector/src/main/java/org/apache/manifoldcf/crawler/connectors/sharepoint/SharePointRepository.java

3.Delete unused ExtensionMimeMap class which just contains one method to call Tika().detect to get MimeType.
framework/core/src/main/java/org/apache/manifoldcf/core/extmimemap/ExtensionMimeMap.java

Thanks.

> the parsing in TikaExtractor always return empty result
> -------------------------------------------------------
>
>                 Key: CONNECTORS-1079
>                 URL: https://issues.apache.org/jira/browse/CONNECTORS-1079
>             Project: ManifoldCF
>          Issue Type: Bug
>          Components: Tika extractor
>    Affects Versions: ManifoldCF 2.0
>            Reporter: Mingchun Zhao
>            Assignee: Karl Wright
>             Fix For: ManifoldCF 1.8, ManifoldCF 2.0
>
>
> When I use latest trunk source(2.0) to try the Tika content extractor,It did not return any expected results.
> I looked at it using debugging tools, found that the parser of Tika content extractor does not return any data.
> I've tried to move lib/tika-core-1.6.jar into connector-lib/, 
> Then, the Tika content extractor returned data as expected.
> My configurations are as below:
> ==
> Transformation:
>  Type: Tika content extractor
> Output:
>  Type:Solr(Use extract update handler=false)
> Repository:
>  type: Web
> Job:
>  1.type: repository
>  2.type: transformation
>  3.type: output
> ==
> Maybe, it is related to CONNECTORS-1074(?), 
> It looks like that the place of tika-core-1.6.jar affects the result of TikaExtractor.
>  


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)