flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tzu-Li (Gordon) Tai" <tzuli...@apache.org>
Subject Re: KafkaProducer can not be instantiated
Date Wed, 05 Oct 2016 15:17:30 GMT
Hi Timo,

I haven’t had the chance to look at the producer side too much yet, but after a look in
the code,
I think it’s reasonable to remove the instantiation from the producer constructor.
The instantiation in the constructor is only used for partition list fetching & eager
properties validation
before running up the job. With an alternative to do the eager properties validation in the
constructor without relying on KafkaProducer,
it should be safe to remove it from the constructor.

The consumer side actually has the same problem right now too. I was hoping to bundle the
fix with a bigger task,
but would probably consider moving it up TODO list so it can be resolved sooner as a standalone
fix.

Cheers,
Gordon


On October 5, 2016 at 10:51:05 PM, Timo Walther (twalthr@apache.org) wrote:

Hey everyone,  

I'm currently rewriting the KafkaTabeSinkTest and discovered something  
that doesn't seem to be intended: Is it intended that  
FlinkKafkaProducer08 cannot be instantiated without a running Kafka  
instance?  

The constructor of FlinkKafkaProducerBase calls getKafkaProducer() which  
actually should be called in the open() method first. What happens if  
the Client has no access to the Kafka properties (e.g. using an remote  
execution environment)? Then it is impossible to create a KafkaProducer?  

Thanks.  

Timo  

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message