flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stephan Ewen (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (FLINK-256) Nephele "loses" data packets
Date Fri, 13 Jun 2014 00:07:02 GMT

     [ https://issues.apache.org/jira/browse/FLINK-256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Stephan Ewen resolved FLINK-256.
--------------------------------

    Resolution: Fixed
      Assignee: Ufuk Celebi

Fixed with the introduction of the Netty library to replace custom NIO code.

> Nephele "loses" data packets
> ----------------------------
>
>                 Key: FLINK-256
>                 URL: https://issues.apache.org/jira/browse/FLINK-256
>             Project: Flink
>          Issue Type: Bug
>            Reporter: GitHub Import
>            Assignee: Ufuk Celebi
>              Labels: github-import
>             Fix For: pre-apache
>
>
> A pact job dies with the following message: "An error occurred in the channel: Expected
data packet 25 but received 27"
> The error occurred both in version 0.21 and with the current 0.4-snapshot. Apparently
only with bigger data sizes. 15GB Input data worked fine, 22GB crashed. I sent the job, that
caused the error to Stephan Ewen. 
> Full stack trace: 
> 15:32:11,639 ERROR eu.stratosphere.pact.runtime.task.RegularPactTask             - Error
in PACT code: Join tweets and dates (2/4)
> 15:32:11,640 ERROR eu.stratosphere.pact.runtime.task.RegularPactTask             - java.io.IOException:
An error occurred in the channel: Expected data packet 25 but received 27
> java.io.IOException: An error occurred in the channel: Expected data packet 25 but received
27
>         at eu.stratosphere.nephele.io.channels.bytebuffered.AbstractByteBufferedInputChannel.isClosed(AbstractByteBufferedInputChannel.java:144)
>         at eu.stratosphere.nephele.io.RuntimeInputGate.isClosed(RuntimeInputGate.java:261)
>         at eu.stratosphere.nephele.io.RuntimeInputGate.readRecord(RuntimeInputGate.java:182)
>         at eu.stratosphere.nephele.io.MutableRecordReader.next(MutableRecordReader.java:80)
>         at eu.stratosphere.pact.runtime.task.util.PactRecordNepheleReaderIterator.next(PactRecordNepheleReaderIterator.java:62)
>         at eu.stratosphere.pact.runtime.task.util.PactRecordNepheleReaderIterator.next(PactRecordNepheleReaderIterator.java:27)
>         at eu.stratosphere.pact.runtime.hash.MutableHashTable$ProbeIterator.next(MutableHashTable.java:1525)
>         at eu.stratosphere.pact.runtime.hash.MutableHashTable.processProbeIter(MutableHashTable.java:450)
>         at eu.stratosphere.pact.runtime.hash.MutableHashTable.nextRecord(MutableHashTable.java:536)
>         at eu.stratosphere.pact.runtime.hash.BuildFirstHashMatchIterator.callWithNextKey(BuildFirstHashMatchIterator.java:116)
>         at eu.stratosphere.pact.runtime.task.MatchDriver.run(MatchDriver.java:164)
>         at eu.stratosphere.pact.runtime.task.RegularPactTask.run(RegularPactTask.java:372)
>         at eu.stratosphere.pact.runtime.task.RegularPactTask.invoke(RegularPactTask.java:291)
>         at eu.stratosphere.nephele.execution.RuntimeEnvironment.run(RuntimeEnvironment.java:344)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: Expected data packet 25 but received 27
>         at eu.stratosphere.nephele.taskmanager.runtime.RuntimeInputChannelContext.queueTransferEnvelope(RuntimeInputChannelContext.java:148)
>         at eu.stratosphere.nephele.taskmanager.bytebuffered.ByteBufferedChannelManager.processEnvelopeWithBuffer(ByteBufferedChannelManager.java:365)
>         at eu.stratosphere.nephele.taskmanager.bytebuffered.ByteBufferedChannelManager.processEnvelope(ByteBufferedChannelManager.java:331)
>         at eu.stratosphere.nephele.taskmanager.bytebuffered.ByteBufferedChannelManager.processEnvelopeFromNetwork(ByteBufferedChannelManager.java:644)
>         at eu.stratosphere.nephele.taskmanager.bytebuffered.IncomingConnection.read(IncomingConnection.java:100)
>         at eu.stratosphere.nephele.taskmanager.bytebuffered.IncomingConnectionThread.doRead(IncomingConnectionThread.java:187)
>         at eu.stratosphere.nephele.taskmanager.bytebuffered.IncomingConnectionThread.run(IncomingConnectionThread.java:126)
> 15:32:12,362 WARN eu.stratosphere.pact.runtime.task.RegularPactTask             - Cancelling
PACT code: Join tweets and dates (2/4)
> 15:32:12,362 INFO eu.stratosphere.nephele.execution.ExecutionStateTransition    - TM:
ExecutionState set from RUNNING to FAILED for task Join tweets and dates (2/4)
> 15:32:12,362 ERROR eu.stratosphere.nephele.taskmanager.runtime.RuntimeTask       - java.io.IOException:
An error occurred in the channel: Expected data packet 25 but received 27
>         at eu.stratosphere.nephele.io.channels.bytebuffered.AbstractByteBufferedInputChannel.isClosed(AbstractByteBufferedInputChannel.java:144)
>         at eu.stratosphere.nephele.io.RuntimeInputGate.isClosed(RuntimeInputGate.java:261)
>         at eu.stratosphere.nephele.io.RuntimeInputGate.readRecord(RuntimeInputGate.java:182)
>         at eu.stratosphere.nephele.io.MutableRecordReader.next(MutableRecordReader.java:80)
>         at eu.stratosphere.pact.runtime.task.util.PactRecordNepheleReaderIterator.next(PactRecordNepheleReaderIterator.java:62)
>         at eu.stratosphere.pact.runtime.task.util.PactRecordNepheleReaderIterator.next(PactRecordNepheleReaderIterator.java:27)
>         at eu.stratosphere.pact.runtime.hash.MutableHashTable$ProbeIterator.next(MutableHashTable.java:1525)
>         at eu.stratosphere.pact.runtime.hash.MutableHashTable.processProbeIter(MutableHashTable.java:450)
>         at eu.stratosphere.pact.runtime.hash.MutableHashTable.nextRecord(MutableHashTable.java:536)
>         at eu.stratosphere.pact.runtime.hash.BuildFirstHashMatchIterator.callWithNextKey(BuildFirstHashMatchIterator.java:116)
>         at eu.stratosphere.pact.runtime.task.MatchDriver.run(MatchDriver.java:164)
>         at eu.stratosphere.pact.runtime.task.RegularPactTask.run(RegularPactTask.java:372)
>         at eu.stratosphere.pact.runtime.task.RegularPactTask.invoke(RegularPactTask.java:291)
>         at eu.stratosphere.nephele.execution.RuntimeEnvironment.run(RuntimeEnvironment.java:344)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: Expected data packet 25 but received 27
>         at eu.stratosphere.nephele.taskmanager.runtime.RuntimeInputChannelContext.queueTransferEnvelope(RuntimeInputChannelContext.java:148)
>         at eu.stratosphere.nephele.taskmanager.bytebuffered.ByteBufferedChannelManager.processEnvelopeWithBuffer(ByteBufferedChannelManager.java:365)
>         at eu.stratosphere.nephele.taskmanager.bytebuffered.ByteBufferedChannelManager.processEnvelope(ByteBufferedChannelManager.java:331)
>         at eu.stratosphere.nephele.taskmanager.bytebuffered.ByteBufferedChannelManager.processEnvelopeFromNetwork(ByteBufferedChannelManager.java:644)
>         at eu.stratosphere.nephele.taskmanager.bytebuffered.IncomingConnection.read(IncomingConnection.java:100)
>         at eu.stratosphere.nephele.taskmanager.bytebuffered.IncomingConnectionThread.doRead(IncomingConnectionThread.java:187)
>         at eu.stratosphere.nephele.taskmanager.bytebuffered.IncomingConnectionThread.run(IncomingConnectionThread.java:126)
> 15:32:12,394 INFO eu.stratosphere.nephele.taskmanager.runtime.RuntimeTask       - Canceling
Hashtag Polarity Match (2/4)
> ---------------- Imported from GitHub ----------------
> Url: https://github.com/stratosphere/stratosphere/issues/256
> Created by: [matpeters|https://github.com/matpeters]
> Labels: bug, runtime, user satisfaction, 
> Created at: Fri Nov 08 13:57:57 CET 2013
> State: open



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message