lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-3937) Workaround the XERCES-J bug in Benchmark
Date Thu, 29 Mar 2012 18:00:24 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-3937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13241445#comment-13241445
] 

Michael McCandless commented on LUCENE-3937:
--------------------------------------------

Note: I just run benchmark's conf/extractWikipedia.alg task on the XML export... when XERCESJ-1257
strikes you get this:
{noformat}
     ...
     [java]  936.83 sec --> main Wrote 2801000 line docs
     [java]  937.04 sec --> main Wrote 2802000 line docs
     [java]  937.27 sec --> main Wrote 2803000 line docs
     [java]  937.53 sec --> main Wrote 2804000 line docs
     [java]  937.79 sec --> main Wrote 2805000 line docs
     [java]  938.04 sec --> main Wrote 2806000 line docs
     [java]  938.35 sec --> main Wrote 2807000 line docs
     [java]  938.65 sec --> main Wrote 2808000 line docs
     [java]  938.88 sec --> main Wrote 2809000 line docs
     [java]  939.09 sec --> main Wrote 2810000 line docs
     [java]  939.09 sec --> main Wrote 2810000 line docs
     [java] Exception in thread "Thread-0" java.lang.RuntimeException: org.apache.xerces.impl.io.MalformedByteSequenceException:
Invalid byte 2 of 4-byte UTF-8 sequence.
     [java] 	at org.apache.lucene.benchmark.byTask.feeds.EnwikiContentSource$Parser.run(EnwikiContentSource.java:198)
     [java] 	at java.lang.Thread.run(Thread.java:619)
     [java] ####################
     [java] ###  D O N E !!! ###
     [java] Caused by: org.apache.xerces.impl.io.MalformedByteSequenceException: Invalid byte
2 of 4-byte UTF-8 sequence.
     [java] ####################
     [java] 	at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown
Source)
     [java] 	at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source)
     [java] 	at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
     [java] 	at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
     [java] 	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown
Source)
     [java] 	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown
Source)
     [java] 	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
     [java] 	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
     [java] 	at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
     [java] 	at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
     [java] 	at org.apache.lucene.benchmark.byTask.feeds.EnwikiContentSource$Parser.run(EnwikiContentSource.java:175)
     [java] 	... 1 more
     [java] Caused by: org.apache.xerces.impl.io.MalformedByteSequenceException: Invalid byte
2 of 4-byte UTF-8 sequence.
     [java] 	at org.apache.xerces.impl.io.UTF8Reader.invalidByte(Unknown Source)
     [java] 	at org.apache.xerces.impl.io.UTF8Reader.read(Unknown Source)
     [java] 	at org.apache.xerces.impl.XMLEntityScanner.load(Unknown Source)
     [java] 	at org.apache.xerces.impl.XMLEntityScanner.scanContent(Unknown Source)
     [java] 	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanContent(Unknown
Source)
     [java] 	... 8 more
{noformat}

                
> Workaround the XERCES-J bug in Benchmark
> ----------------------------------------
>
>                 Key: LUCENE-3937
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3937
>             Project: Lucene - Java
>          Issue Type: Bug
>            Reporter: Uwe Schindler
>         Attachments: LUCENE-3937.patch
>
>
> In becnhmark we have a patched version of XERCES which is hard to compile from source.
When looking at the code part patched and the source of EnwikiContentSource, to simply provide
the XML parser a Reader instead of InputStream, so the broken code is not triggered. This
assumes, that the XML-file is always UTF-8.... If not it will no longer work (because the
XML parser cannot switch encoding, if it only has a Reader).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message