Return-Path: 
 <dev-return-6551-apmail-tika-dev-archive=tika.apache.org@tika.apache.org>
X-Original-To: apmail-tika-dev-archive@www.apache.org
Delivered-To: apmail-tika-dev-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 2CC597ED2
	for <apmail-tika-dev-archive@www.apache.org>;
 Sat,  5 Nov 2011 22:21:15 +0000 (UTC)
Received: (qmail 80221 invoked by uid 500); 5 Nov 2011 22:21:15 -0000
Delivered-To: apmail-tika-dev-archive@tika.apache.org
Received: (qmail 80081 invoked by uid 500); 5 Nov 2011 22:21:15 -0000
Mailing-List: contact dev-help@tika.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:dev-help@tika.apache.org>
List-Unsubscribe: <mailto:dev-unsubscribe@tika.apache.org>
List-Post: <mailto:dev@tika.apache.org>
List-Id: <dev.tika.apache.org>
Reply-To: dev@tika.apache.org
Delivered-To: mailing list dev@tika.apache.org
Received: (qmail 80072 invoked by uid 99); 5 Nov 2011 22:21:15 -0000
Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
    by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 05 Nov 2011 22:21:15 +0000
X-ASF-Spam-Status: No, hits=-2001.2 required=5.0
	tests=ALL_TRUSTED,RP_MATCHES_RCVD
X-Spam-Check-By: apache.org
Received: from [140.211.11.116] (HELO hel.zones.apache.org) (140.211.11.116)
    by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 05 Nov 2011 22:21:12 +0000
Received: from hel.zones.apache.org (hel.zones.apache.org [140.211.11.116])
	by hel.zones.apache.org (Postfix) with ESMTP id 9627737013
	for <dev@tika.apache.org>; Sat,  5 Nov 2011 22:20:51 +0000 (UTC)
Date: Sat, 5 Nov 2011 22:20:51 +0000 (UTC)
From: "Joseph Vychtrle (Commented) (JIRA)" <jira@apache.org>
To: dev@tika.apache.org
Message-ID: 
 <785630917.3806.1320531651616.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <152996492.57421.1320354932473.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (TIKA-772) media type detection fails for html
 documents, results in text/plain instead of text/html
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394
X-Virus-Checked: Checked by ClamAV on apache.org


    [ https://issues.apache.org/jira/browse/TIKA-772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144853#comment-13144853 ] 

Joseph Vychtrle commented on TIKA-772:
--------------------------------------

But to be honest, it makes sense. Tika doesn't have a detector that would detect html media type, does it ? It doesn't have magic prefix and it doesn't know file name, all other Detectors are irrelevant here. What am I missing ?
                
> media type detection fails for html documents, results in text/plain instead of text/html
> -----------------------------------------------------------------------------------------
>
>                 Key: TIKA-772
>                 URL: https://issues.apache.org/jira/browse/TIKA-772
>             Project: Tika
>          Issue Type: Bug
>          Components: mime
>    Affects Versions: 0.10
>            Reporter: Joseph Vychtrle
>            Assignee: Jukka Zitting
>              Labels: detection, media-type
>         Attachments: html.zip, tika.png
>
>
> Hey, I was testing media type detection on most of the major document types, but when testing html documents of cca 5000 words that starts with :
> <?xml version="1.0" encoding="UTF-8"?>
> composed of root "html" element and "p" elements only, it always results in text/plain instead of text/html ...
> {code:title=Bar.java|borderStyle=solid}
> @Test
> public void testMediaType() throws Exception {
>         List<Document> allDocs = DocumentProvider.docsAsList();
> 	Map<Document, String> failed = new HashMap<Document, String>();
> 	for (Document doc : allDocs) {
> 		Tika tika = new Tika();
> 		String type = tika.detect(TikaInputStream.get(doc.getFile()));
> 		if(!doc.getMediaType().toString().equals(type))
> 				failed.put(doc, type);	
> 	}
> 	
> 	for (Document doc : failed.keySet()) {
> 		log.error("expected: " + doc.getMediaTypeString() + "; actual: " + failed.get(doc) + ";  path to file: " + doc.getFile().getAbsolutePath());
> 	}
> 	assertTrue(failed.isEmpty(), "mime type was incorrectly detected for : " + failed.size() + " documents;");
> }
> {code}
> Am I doing anything wrong ?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira