beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xu Mingmin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-1640) data file missing when submit a job on Flink
Date Thu, 09 Mar 2017 17:47:38 GMT

    [ https://issues.apache.org/jira/browse/BEAM-1640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15903472#comment-15903472
] 

Xu Mingmin commented on BEAM-1640:
----------------------------------

[~aljoscha], it fails with Flink 1.2.0, Beam 0.6.0-SNAPSHOT. The same code works with Flink
1.1.4, which is Beam 0.5.0

> data file missing when submit a job on Flink
> --------------------------------------------
>
>                 Key: BEAM-1640
>                 URL: https://issues.apache.org/jira/browse/BEAM-1640
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-flink
>    Affects Versions: 0.6.0
>            Reporter: Xu Mingmin
>            Assignee: Aljoscha Krettek
>
> I've one file with path 'META-INF/jaas/kafka_jaas.conf' in my jar package. it works with
Beam 0.5.0, when I re-package it with 0.6.0-SNAPSHOT, it fails to submit with bin/flink command.
--Both run on YARN.
> The error is show as below, I guess this file maybe lost in Flink-Runner. 
> {code}
> Caused by: org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
> 	at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:702)
> 	at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:557)
> 	at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:540)
> 	at org.apache.beam.sdk.io.kafka.KafkaIO$Read$2.apply(KafkaIO.java:503)
> 	at org.apache.beam.sdk.io.kafka.KafkaIO$Read$2.apply(KafkaIO.java:501)
> 	at org.apache.beam.sdk.io.kafka.KafkaIO$UnboundedKafkaSource.generateInitialSplits(KafkaIO.java:620)
> 	at org.apache.beam.runners.flink.translation.wrappers.streaming.io.UnboundedSourceWrapper.<init>(UnboundedSourceWrapper.java:159)
> 	at org.apache.beam.runners.flink.FlinkStreamingTransformTranslators$UnboundedReadSourceTranslator.translateNode(FlinkStreamingTransformTranslators.java:267)
> 	... 33 more
> Caused by: org.apache.kafka.common.KafkaException: java.lang.IllegalArgumentException:
Could not find a 'KafkaClient' entry in jaas config.
> 	at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:86)
> 	at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:70)
> 	at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:83)
> 	at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:623)
> 	... 40 more
> Caused by: java.lang.IllegalArgumentException: Could not find a 'KafkaClient' entry in
jaas config.
> 	at io.ebay.rheos.kafka.security.iaf.IAFLogin.login(IAFLogin.java:54)
> 	at org.apache.kafka.common.security.authenticator.LoginManager.<init>(LoginManager.java:53)
> 	at org.apache.kafka.common.security.authenticator.LoginManager.acquireLoginManager(LoginManager.java:75)
> 	at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:78)
> 	... 43 more
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message