kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeetu Bethina (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (KAFKA-5466) Unable to fetch elements from WindowStore when timestamp is before the first inserted Elem timestamp
Date Sun, 18 Jun 2017 15:20:00 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-5466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jeetu Bethina updated KAFKA-5466:
---------------------------------
    Description: 
I am unable to fetch elements that are inserted with a timestamp before the timestamp of the
first element inserted into a WindowStore.

This affects newly created WindowStores or one that is restored from a changelog topic.

The method I use to insert into the Window Store is 

http://docs.confluent.io/3.2.1/streams/javadocs/org/apache/kafka/streams/state/WindowStore.html#put-K-V-long-

Looking into the code, the issue might be with how the minSegmentId is being calculated here.

https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/state/internals/Segments.java#L88

Something like the below fixes the issue with regards to the WindowStore but I am not sure
what the implications would be for Streams as a whole.

if (minSegmentId == Long.MAX_VALUE) {
    minSegmentId = maxSegmentId;
} else {
    minSegmentId = segmentId < minSegmentId ? segmentId : minSegmentId;
}

  was:
I am unable to fetch elements that are inserted with a timestamp before the timestamp of the
first element inserted into the WindowStore.

This affects a newly created WindowStore or one that is restored from a changelog topic.

The method I use to insert into the Window Store is 

http://docs.confluent.io/3.2.1/streams/javadocs/org/apache/kafka/streams/state/WindowStore.html#put-K-V-long-

Looking into the code, the issue might be with how the minSegmentId is being calculated here.

https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/state/internals/Segments.java#L88

Something like the below fixes the issue with regards to the WindowStore but I am not sure
what the implications would be for Streams as a whole.

if (minSegmentId == Long.MAX_VALUE) {
    minSegmentId = maxSegmentId;
} else {
    minSegmentId = segmentId < minSegmentId ? segmentId : minSegmentId;
}


> Unable to fetch elements from WindowStore when timestamp is before the first inserted
Elem timestamp
> ----------------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-5466
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5466
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>    Affects Versions: 0.10.2.1
>            Reporter: Jeetu Bethina
>
> I am unable to fetch elements that are inserted with a timestamp before the timestamp
of the first element inserted into a WindowStore.
> This affects newly created WindowStores or one that is restored from a changelog topic.
> The method I use to insert into the Window Store is 
> http://docs.confluent.io/3.2.1/streams/javadocs/org/apache/kafka/streams/state/WindowStore.html#put-K-V-long-
> Looking into the code, the issue might be with how the minSegmentId is being calculated
here.
> https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/state/internals/Segments.java#L88
> Something like the below fixes the issue with regards to the WindowStore but I am not
sure what the implications would be for Streams as a whole.
> if (minSegmentId == Long.MAX_VALUE) {
>     minSegmentId = maxSegmentId;
> } else {
>     minSegmentId = segmentId < minSegmentId ? segmentId : minSegmentId;
> }



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message