kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias J. Sax (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-4750) KeyValueIterator returns null values
Date Sat, 24 Jun 2017 08:56:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-4750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16061866#comment-16061866

Matthias J. Sax commented on KAFKA-4750:

I just double check {{RocksDBStore}} and the call chain is {{delete(key) -> put(key, null)
-> putInternal(key, null)}} with:
private void putInternal(byte[] rawKey, byte[] rawValue) {
        if (rawValue == null) {
            try {
                db.delete(wOptions, rawKey);
            } catch (RocksDBException e) {
                throw new ProcessorStateException("Error while removing key " + serdes.keyFrom(rawKey)
                        " from store " + this.name, e);
        } else {
 // ...

Also not, that {{ChangeLoggingKeyValueBytesStore}} has similar behavior: {{delete(key) ->
put(key, null) -> inner.put(key, null) / changeLogger.put(key,null)}} -- as {{ChangeLoggingKeyValueBytesStore}}
wraps the actual store to just make sure the update is written to the changelog, the inner
implementation {{inner.put(key,null)}} should handle the {{null}} value as a delete. Thus
I am not sure what the actual root cause for this problem is.

As you mentioned that it worked in 0.10.0, it might be related to the KTable cache. [~mihbor]
Can you try to disable caching to see if the issues goes away? This would support my theory
and explain why it broke in 0.10.1. Was is the actual operation you are doing? Do you change
any store default configs?

> KeyValueIterator returns null values
> ------------------------------------
>                 Key: KAFKA-4750
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4750
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>    Affects Versions:,,
>            Reporter: Michal Borowiecki
>            Assignee: Evgeny Veretennikov
>              Labels: newbie
> The API for ReadOnlyKeyValueStore.range method promises the returned iterator will not
return null values. However, after upgrading from to we found null values
are returned causing NPEs on our side.
> I found this happens after removing entries from the store and I found resemblance to
SAMZA-94 defect. The problem seems to be as it was there, when deleting entries and having
a serializer that does not return null when null is passed in, the state store doesn't actually
delete that key/value pair but the iterator will return null value for that key.
> When I modified our serilizer to return null when null is passed in, the problem went
away. However, I believe this should be fixed in kafka streams, perhaps with a similar approach
as SAMZA-94.

This message was sent by Atlassian JIRA

View raw message