hudi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "lamber-ken (Jira)" <j...@apache.org>
Subject [jira] [Updated] (HUDI-663) Fix HoodieDeltaStreamer offset not handled correctly
Date Thu, 05 Mar 2020 18:57:00 GMT

     [ https://issues.apache.org/jira/browse/HUDI-663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

lamber-ken updated HUDI-663:
----------------------------
    Status: Open  (was: New)

> Fix HoodieDeltaStreamer offset not handled correctly
> ----------------------------------------------------
>
>                 Key: HUDI-663
>                 URL: https://issues.apache.org/jira/browse/HUDI-663
>             Project: Apache Hudi (incubating)
>          Issue Type: Bug
>          Components: DeltaStreamer
>            Reporter: lamber-ken
>            Assignee: lamber-ken
>            Priority: Major
>
> Go ahead with [https://github.com/apache/incubator-hudi/issues/1375]
>  
> {code:java}
> 20/03/04 10:21:02 WARN scheduler.TaskSetManager: Lost task 0.0 in stage 14.0 (TID 10502,
biedva-worker-6.bigdatapoc.local, executor 7): og.apache.kafka.clients.consumer.OffsetOutOfRangeException:
Offsets out of range with no configured reset policy for partitions: {fr-bru-0=0}
>         at org.apache.kafka.clients.consumer.internals.Fetcher.parseCompletedFetch(Fetcher.java:1002)
>         at org.apache.kafka.clients.consumer.internals.Fetcher.fetchedRecords(Fetcher.java:508)
>         at org.apache.kafka.clients.consumer.KafkaConsumer.pollForFetches(KafkaConsumer.java:1261)
>         at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1189)
>         at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1115)
>         at org.apache.spark.streaming.kafka010.InternalKafkaConsumer.poll(KafkaDataConsumer.scala:200)
>         at org.apache.spark.streaming.kafka010.InternalKafkaConsumer.get(KafkaDataConsumer.scala:129)
>         at org.apache.spark.streaming.kafka010.KafkaDataConsumer$class.get(KafkaDataConsumer.scala:36)
>         at org.apache.spark.streaming.kafka010.KafkaDataConsumer$CachedKafkaDataConsumer.get(KafkaDataConsumer.scala:212)
>         at org.apache.spark.streaming.kafka010.KafkaRDDIterator.next(KafkaRDD.scala:261)
>         at org.apache.spark.streaming.kafka010.KafkaRDDIterator.next(KafkaRDD.scala:229)
>         at scala.collection.Iterator$$anon$11.next(Iterator.scala:410)
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message