beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Halperin (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (BEAM-223) KafkaIO: don't use SerializableCoder
Date Fri, 02 Dec 2016 20:05:58 GMT

     [ https://issues.apache.org/jira/browse/BEAM-223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Daniel Halperin resolved BEAM-223.
----------------------------------
       Resolution: Fixed
    Fix Version/s: 0.4.0-incubating

> KafkaIO: don't use SerializableCoder
> ------------------------------------
>
>                 Key: BEAM-223
>                 URL: https://issues.apache.org/jira/browse/BEAM-223
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-java-extensions
>            Reporter: Daniel Halperin
>            Assignee: Raghu Angadi
>             Fix For: 0.4.0-incubating
>
>
> Reuven says:
> {quote}
> I noticed that we're using SerializableCoder for the checkpoint mark in KafkaIO. This
is generally highly discouraged in streaming pipelines. Partially because it's inefficient,
but more importantly because Java serialization is not guaranteed to be stable. If a user
updates their pipeline, the new pipeline may not be able to decode the existing checkpoint
marks; this will either cause exceptions to be thrown, or data loss.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message