kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Guozhang Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-4322) StateRestoreCallback begin and end indication
Date Tue, 22 Nov 2016 18:08:58 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-4322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15687453#comment-15687453

Guozhang Wang commented on KAFKA-4322:

I see. Just for clarification for "own logging" are you just adding logging statement in your
code and are only looking at those or you are using a different logging framework than log4j?
For the latter case I think it should be fine as Streams is the same as other clients that
it simply hook sl4j interface which could use any compatible logging frameworks, not only

Streams is designed to try to abstract the operational legwork from users and let them focus
on the computational logic only, so we would really like to, if possible, support such information
to users since as you mentioned, whether / what has been restored upon initialization should
be a common feature which we 'd better support out-of-the-box rather than enforcing users
to always specify such logic in a callback.

> StateRestoreCallback begin and end indication
> ---------------------------------------------
>                 Key: KAFKA-4322
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4322
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>    Affects Versions:
>            Reporter: Mark Shelton
>            Assignee: Mark Shelton
>            Priority: Minor
> In Kafka Streams, the StateRestoreCallback interface provides only a single method "restore(byte[]
key, byte[] value)" that is called for every key-value pair to be restored. 
> It would be nice to have "beginRestore" and "endRestore" methods as part of StateRestoreCallback.
> Kafka Streams would call "beginRestore" before restoring any keys, and would call "endRestore"
when it determines that it is done. This allows an implementation, for example, to report
on the number of keys restored and perform a commit after the last key was restored. Other
uses are conceivable.

This message was sent by Atlassian JIRA

View raw message