kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-5535) Transformations - tranformations for value broken on tombstone events
Date Fri, 21 Jul 2017 03:57:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-5535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16095737#comment-16095737
] 

ASF GitHub Bot commented on KAFKA-5535:
---------------------------------------

GitHub user ewencp opened a pull request:

    https://github.com/apache/kafka/pull/3559

    KAFKA-5535: Handle null values in ExtractField

    

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/ewencp/kafka kafka-5535-extract-field-null

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/kafka/pull/3559.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #3559
    
----
commit d8ffc8b4300cc2a0ee047af941dd4eb3bfcfd58f
Author: Ewen Cheslack-Postava <me@ewencp.org>
Date:   2017-07-21T03:55:26Z

    KAFKA-5535: Handle null values in ExtractField

----


> Transformations - tranformations for value broken on tombstone events
> ---------------------------------------------------------------------
>
>                 Key: KAFKA-5535
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5535
>             Project: Kafka
>          Issue Type: Bug
>          Components: KafkaConnect
>    Affects Versions: 0.10.2.1
>         Environment: Unbuntu 14.04
> Java 8
>            Reporter: Yelei Wu
>              Labels: newbie
>
> I'm trying to use the transformation for Kafka Connect and running into issues.
> The tranformation configuration is:
> -----------------
> "transforms": "GetAfter",
> "transforms.GetAfter.type": "org.apache.kafka.connect.transforms.ExtractField$Value",
> "transforms.GetAfter.field": "after",
> -----------------
> And I got the following errors occasionally´╝Ü
> -----------------
> org.apache.kafka.connect.errors.DataException: Only Map objects supported in absence
of schema for [field extraction], found: null
> 	at org.apache.kafka.connect.transforms.util.Requirements.requireMap(Requirements.java:38)
> 	at org.apache.kafka.connect.transforms.ExtractField.apply(ExtractField.java:57)
> 	at org.apache.kafka.connect.runtime.TransformationChain.apply(TransformationChain.java:39)
> 	at org.apache.kafka.connect.runtime.WorkerSinkTask.convertMessages(WorkerSinkTask.java:408)
> 	at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:249)
> 	at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:179)
> 	at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:148)
> 	at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:139)
> 	at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:182)
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:745)
> ----------------- 
> Seems that tombstone events break the transformation, but after checking the source code
for Transformations on Value (ExtractField$Value, ValueToKey, MaskField$Value, ReplaceField$Value),
none of them handles tombstone events explicitly, none of them work through Tombstone events
neither.
> Null check in those transformations may be nessesary.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message