apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Weise (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (APEXMALHAR-2283) Issues with Kafka output operator
Date Thu, 13 Oct 2016 18:57:20 GMT

    [ https://issues.apache.org/jira/browse/APEXMALHAR-2283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572849#comment-15572849
] 

Thomas Weise commented on APEXMALHAR-2283:
------------------------------------------

It also appears that the user has to specify the key serializer, which makes no sense since
it is internal to the operator:
{code}
org.apache.kafka.common.config.ConfigException: Missing required configuration "key.serializer"
which has no default value.
        at org.apache.kafka.common.config.ConfigDef.parse(ConfigDef.java:148)
        at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:49)
        at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:56)
        at org.apache.kafka.clients.producer.ProducerConfig.<init>(ProducerConfig.java:315)
        at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:181)
        at org.apache.apex.malhar.kafka.AbstractKafkaOutputOperator.setup(AbstractKafkaOutputOperator.java:51)
        at org.apache.apex.malhar.kafka.KafkaSinglePortExactlyOnceOutputOperator.setup(KafkaSinglePortExactlyOnceOutputOperator.java:124)
        at org.apache.apex.malhar.kafka.KafkaSinglePortExactlyOnceOutputOperator.setup(KafkaSinglePortExactlyOnceOutputOperator.java:94)
{code}


> Issues with Kafka output operator
> ---------------------------------
>
>                 Key: APEXMALHAR-2283
>                 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2283
>             Project: Apache Apex Malhar
>          Issue Type: Improvement
>            Reporter: Siyuan Hua
>            Assignee: Siyuan Hua
>
> The abstract kafka output operator needs to be refactored
> 1. Needs to set some mandatory properties on operator level instead of kafka property
level.
> 2. More document and examples
> 3. Find a standard way to achieve exactly once in both 0.8 and 0.9
> More will be added when working on the ticket



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message