Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@lucene.apache.org
Date: Mon, 5 Aug 2013 21:06:48 +0000 (UTC)
From: "Federico Chiacchiaretta (JIRA)" <jira@apache.org>
To: dev@lucene.apache.org
Message-ID: <JIRA.12661073.1375345012994.3640.1375736808970@arcas>
In-Reply-To: <JIRA.12661073.1375345012994@arcas>
References: <JIRA.12661073.1375345012994@arcas>
Subject: [jira] [Commented] (SOLR-5101) Invalid UTF-8 character 0xfffe
 during shard update
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/SOLR-5101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13729937#comment-13729937 ] 

Federico Chiacchiaretta commented on SOLR-5101:
-----------------------------------------------

Hi Yonik,
here is the stack trace on the target node:

ERROR - 2013-08-05 11:57:48.739; org.apache.solr.common.SolrException; java.lang.RuntimeException: [was class java.io.CharConversionException] Invalid UTF-8 character 0xfffe at char #6755, byte #6143)
        at com.ctc.wstx.util.ExceptionUtil.throwRuntimeException(ExceptionUtil.java:18)
        at com.ctc.wstx.sr.StreamScanner.throwLazyError(StreamScanner.java:731)
        at com.ctc.wstx.sr.BasicStreamReader.safeFinishToken(BasicStreamReader.java:3657)
        at com.ctc.wstx.sr.BasicStreamReader.getText(BasicStreamReader.java:809)
        at org.apache.solr.handler.loader.XMLLoader.readDoc(XMLLoader.java:393)
        at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:245)
        at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:173)
        at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92)
        at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
        at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
        at org.apache.solr.core.SolrCore.execute(SolrCore.java:1904)
        at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:659)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:362)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:158)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
        at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
        at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
        at org.eclipse.jetty.server.Server.handle(Server.java:368)
        at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
        at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
        at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
        at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
        at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:953)
        at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
        at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
        at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:679)
Caused by: java.io.CharConversionException: Invalid UTF-8 character 0xfffe at char #6755, byte #6143)
        at com.ctc.wstx.io.UTF8Reader.reportInvalid(UTF8Reader.java:335)
        at com.ctc.wstx.io.UTF8Reader.read(UTF8Reader.java:249)
        at com.ctc.wstx.io.MergedReader.read(MergedReader.java:101)
        at com.ctc.wstx.io.ReaderSource.readInto(ReaderSource.java:84)
        at com.ctc.wstx.io.BranchingReaderSource.readInto(BranchingReaderSource.java:57)
        at com.ctc.wstx.sr.StreamScanner.loadMore(StreamScanner.java:992)
        at com.ctc.wstx.sr.BasicStreamReader.readTextSecondary(BasicStreamReader.java:4628)
        at com.ctc.wstx.sr.BasicStreamReader.readCoalescedText(BasicStreamReader.java:4126)
        at com.ctc.wstx.sr.BasicStreamReader.finishToken(BasicStreamReader.java:3701)
        at com.ctc.wstx.sr.BasicStreamReader.safeFinishToken(BasicStreamReader.java:3649)
        ... 36 more

There is also an ongoing thread on the users' list (link to my first message http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201308.mbox/%3CCAHF6Yy7o21anw2rtx_%3DHaR%3DyQG_AoS1bqJLqF_YK4Ns2%2BzWHLQ%40mail.gmail.com%3E ).

Hope this can help, I can reproduce the issue to provide further logs if necessary.

Regards,
Federico Chiacchiaretta
                
> Invalid UTF-8 character 0xfffe during shard update
> --------------------------------------------------
>
>                 Key: SOLR-5101
>                 URL: https://issues.apache.org/jira/browse/SOLR-5101
>             Project: Solr
>          Issue Type: Bug
>          Components: clients - java
>    Affects Versions: 4.3
>         Environment: Ubuntu 12.04.2
> java version "1.6.0_27"
> OpenJDK Runtime Environment (IcedTea6 1.12.5) (6b27-1.12.5-0ubuntu0.12.04.1)
> OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)
>            Reporter: Federico Chiacchiaretta
>
> On data import from a PostgreSQL db, I get the following error in solr.log:
> ERROR - 2013-08-01 09:51:00.217; org.apache.solr.common.SolrException; shard update error RetryNode: http://172.16.201.173:8983/solr/archive/:org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Invalid UTF-8 character 0xfffe at char #416, byte #127)
>    at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:402)
>    at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
>    at org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:332)
>    at org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:306)
>    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>    at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>    at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
>    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>    at java.lang.Thread.run(Thread.java:679)
> This prevents the document from being successfully added to the index, and a few documents targeting the same shard are also missing.
> This happens silently, because data import completes successfully, and the whole number of documents reported as Added includes those who failed (and are actually lost).
> Is there a known workaround for this issue?
> Regards,
> Federico Chiacchiaretta

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org