kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yogesh BG (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-5998) /.checkpoint.tmp Not found exception
Date Sun, 01 Oct 2017 06:56:02 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16187301#comment-16187301
] 

Yogesh BG commented on KAFKA-5998:
----------------------------------

Patch v1 is one way to not leave source behind. Another option is to delete the source.

Do you have the patch can you share? I will update my setup... We can not delete the source
manually... What is the impact of this issue? Its gonna drop some messages or it violates
any consumer semantics like exactly once?
 

> /.checkpoint.tmp Not found exception
> ------------------------------------
>
>                 Key: KAFKA-5998
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5998
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>    Affects Versions: 0.11.0.0
>            Reporter: Yogesh BG
>         Attachments: 5998.v1.txt
>
>
> I have one kafka broker and one kafka stream running... I am running its since two days
under load of around 2500 msgs per second.. On third day am getting below exception for some
of the partitions, I have 16 partitions only 0_0 and 0_1 gives this error
> {{09:43:25.955 [ks_0_inst-StreamThread-6] WARN  o.a.k.s.p.i.ProcessorStateManager - Failed
to write checkpoint file to /data/kstreams/rtp-kafkastreams/0_1/.checkpoint:
> java.io.FileNotFoundException: /data/kstreams/rtp-kafkastreams/0_1/.checkpoint.tmp (No
such file or directory)
>         at java.io.FileOutputStream.open(Native Method) ~[na:1.7.0_111]
>         at java.io.FileOutputStream.<init>(FileOutputStream.java:221) ~[na:1.7.0_111]
>         at java.io.FileOutputStream.<init>(FileOutputStream.java:171) ~[na:1.7.0_111]
>         at org.apache.kafka.streams.state.internals.OffsetCheckpoint.write(OffsetCheckpoint.java:73)
~[rtp-kafkastreams-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
>         at org.apache.kafka.streams.processor.internals.ProcessorStateManager.checkpoint(ProcessorStateManager.java:324)
~[rtp-kafkastreams-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
>         at org.apache.kafka.streams.processor.internals.StreamTask$1.run(StreamTask.java:267)
[rtp-kafkastreams-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
>         at org.apache.kafka.streams.processor.internals.StreamsMetricsImpl.measureLatencyNs(StreamsMetricsImpl.java:201)
[rtp-kafkastreams-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
>         at org.apache.kafka.streams.processor.internals.StreamTask.commit(StreamTask.java:260)
[rtp-kafkastreams-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
>         at org.apache.kafka.streams.processor.internals.StreamTask.commit(StreamTask.java:254)
[rtp-kafkastreams-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
>         at org.apache.kafka.streams.processor.internals.AssignedTasks$1.apply(AssignedTasks.java:322)
[rtp-kafkastreams-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
>         at org.apache.kafka.streams.processor.internals.AssignedTasks.applyToRunningTasks(AssignedTasks.java:415)
[rtp-kafkastreams-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
>         at org.apache.kafka.streams.processor.internals.AssignedTasks.commit(AssignedTasks.java:314)
[rtp-kafkastreams-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
>         at org.apache.kafka.streams.processor.internals.StreamThread.commitAll(StreamThread.java:700)
[rtp-kafkastreams-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
>         at org.apache.kafka.streams.processor.internals.StreamThread.maybeCommit(StreamThread.java:683)
[rtp-kafkastreams-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
>         at org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:523)
[rtp-kafkastreams-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
>         at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:480)
[rtp-kafkastreams-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
>         at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:457)
[rtp-kafkastreams-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
> 09:43:25.974 [ks_0_inst-StreamThread-15] WARN  o.a.k.s.p.i.ProcessorStateManager - Failed
to write checkpoint file to /data/kstreams/rtp-kafkastreams/0_0/.checkpoint:
> java.io.FileNotFoundException: /data/kstreams/rtp-kafkastreams/0_0/.checkpoint.tmp (No
such file or directory)
>         at java.io.FileOutputStream.open(Native Method) ~[na:1.7.0_111]
>         at java.io.FileOutputStream.<init>(FileOutputStream.java:221) ~[na:1.7.0_111]
>         at java.io.FileOutputStream.<init>(FileOutputStream.java:171) ~[na:1.7.0_111]
>         at org.apache.kafka.streams.state.internals.OffsetCheckpoint.write(OffsetCheckpoint.java:73)
~[rtp-kafkastreams-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
>         at org.apache.kafka.streams.processor.internals.ProcessorStateManager.checkpoint(ProcessorStateManager.java:324)
~[rtp-kafkastreams-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
>         at org.apache.kafka.streams.processor.internals.StreamTask$1.run(StreamTask.java:267)
[rtp-kafkastreams-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
>         at org.apache.kafka.streams.processor.internals.StreamsMetricsImpl.measureLatencyNs(StreamsMetricsImpl.java:201)
[rtp-kafkastreams-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
>         at org.apache.kafka.streams.processor.internals.StreamTask.commit(StreamTask.java:260)
[rtp-kafkastreams-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
>         at org.apache.kafka.streams.processor.internals.StreamTask.commit(StreamTask.java:254)
[rtp-kafkastreams-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
>         at org.apache.kafka.streams.processor.internals.AssignedTasks$1.apply(AssignedTasks.java:322)
[rtp-kafkastreams-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
>         at org.apache.kafka.streams.processor.internals.AssignedTasks.applyToRunningTasks(AssignedTasks.java:415)
[rtp-kafkastreams-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
>         at org.apache.kafka.streams.processor.internals.AssignedTasks.commit(AssignedTasks.java:314)
[rtp-kafkastreams-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
>         at org.apache.kafka.streams.processor.internals.StreamThread.commitAll(StreamThread.java:700)
[rtp-kafkastreams-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
>         at org.apache.kafka.streams.processor.internals.StreamThread.maybeCommit(StreamThread.java:683)
[rtp-kafkastreams-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
>         at org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:523)
[rtp-kafkastreams-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
>         at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:480)
[rtp-kafkastreams-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
>         at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:457)
[rtp-kafkastreams-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
> }}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message