kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward Capriolo (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-232) ConsumerConnector has no access to "getOffsetsBefore"
Date Thu, 05 Jan 2012 21:28:39 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180851#comment-13180851

Edward Capriolo commented on KAFKA-232:

Interesting idea. I am not sure that will work well, many producers will be logging and I
would expect that some messages will arrive out of order. Especially as the minute turns over.
In the worst case scenario a producer might have an unexpected shutdown and wake up at some
time later. Ideally I would like the producers to not have to write special headers etc. I
thought of recording the last message seen assuming that each record is more or less unique
but there are edge cases here as well.

I opened this issue up because I am wondering why the two consumers have different functionality.
Which one should be used for which cases? It is not entirely clear to a new user like myself.

>From a mile-high view it seems like if the ConsumerConnection has access to commitOffset()
it should be able to call getOffset(). Is there some technical complexity in implementing

> ConsumerConnector has no access to "getOffsetsBefore" 
> ------------------------------------------------------
>                 Key: KAFKA-232
>                 URL: https://issues.apache.org/jira/browse/KAFKA-232
>             Project: Kafka
>          Issue Type: New Feature
>            Reporter: Edward Capriolo
>            Priority: Minor
> kafka.javaapi.SimpleConsumer has "getOffsetsBefore". I would like this ability in KafkaMessageStream
or in ConsumerConnector. In this way clients can access their current position.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message