kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-5972) Flatten SMT does not work with null values
Date Wed, 04 Oct 2017 23:04:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-5972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16192178#comment-16192178
] 

ASF GitHub Bot commented on KAFKA-5972:
---------------------------------------

GitHub user shivsantham opened a pull request:

    https://github.com/apache/kafka/pull/4021

    KAFKA-5972 Flatten SMT does not work with null values

    A bug in Flatten SMT while doing tests with different SMTs that are provided out-of-box.
Flatten SMT does not work as expected with schemaless JSON that has properties with null values.
    
    Example json:
      {A={D=dValue, B=null, C=cValue}}
    The issue is in if statement that checks for null value.
    
    CURRENT VERSION:
      for (Map.Entry<String, Object> entry : originalRecord.entrySet()) {
                final String fieldName = fieldName(fieldNamePrefix, entry.getKey());
                Object value = entry.getValue();
                if (value == null) {
                    newRecord.put(fieldName(fieldNamePrefix, entry.getKey()), null);
                    return;
                }
    
    PROPOSED VERSION:
      for (Map.Entry<String, Object> entry : originalRecord.entrySet()) {
                final String fieldName = fieldName(fieldNamePrefix, entry.getKey());
                Object value = entry.getValue();
                if (value == null) {
                    newRecord.put(fieldName(fieldNamePrefix, entry.getKey()), null);
                    continue;
                }

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/shivsantham/kafka trunk

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/kafka/pull/4021.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #4021
    
----
commit ff377759a943c7bfb89a56ad721e7ba1b3b0b24c
Author: siva santhalingam <ssanthalingam@netskope.com>
Date:   2017-09-28T23:37:47Z

    KAFKA-5967 Ineffective check of negative value in CompositeReadOnlyKeyValueStore#approximateNumEntries()
    
    long total = 0;
       for (ReadOnlyKeyValueStore<K, V> store : stores) {
              total += store.approximateNumEntries();
       }
    
    return total < 0 ? Long.MAX_VALUE : total;
    
    The check for negative value seems to account for wrapping. However, wrapping can happen
within the for loop. So the check should be performed inside the loop.

commit 3ea736ac17a4a8ce799b1214f6c0b167b44ee977
Author: siva santhalingam <ssanthalingam@netskope.com>
Date:   2017-09-29T03:16:29Z

     KAFKA-5967 Ineffective check of negative value in CompositeReadOnlyKeyValueStore
    
    Adding a test for #KAFKA-5967

commit 921664384a7d6f53e2cc76cf5699021cdca73893
Author: siva santhalingam <ssanthalingam@netskope.com>
Date:   2017-09-30T08:22:41Z

     KAFKA-5967 Ineffective check of negative value in CompositeReadOnlyKeyValueStore
    
    -Fixing test

commit 48e50992139c03099b3249195efc316a84d6bba1
Author: siva santhalingam <ssanthalingam@netskope.com>
Date:   2017-10-03T17:41:18Z

    Flatten SMT does not work with null values
    
    A bug in Flatten SMT while doing tests with different SMTs that are provided out-of-box.
Flatten SMT does not work as expected with schemaless JSON that has properties with null values.
    
    Example json:
      {A={D=dValue, B=null, C=cValue}}
    The issue is in if statement that checks for null value.
    
    CURRENT VERSION:
      for (Map.Entry<String, Object> entry : originalRecord.entrySet()) {
                final String fieldName = fieldName(fieldNamePrefix, entry.getKey());
                Object value = entry.getValue();
                if (value == null) {
                    newRecord.put(fieldName(fieldNamePrefix, entry.getKey()), null);
                    return;
                }
    
    PROPOSED VERSION:
      for (Map.Entry<String, Object> entry : originalRecord.entrySet()) {
                final String fieldName = fieldName(fieldNamePrefix, entry.getKey());
                Object value = entry.getValue();
                if (value == null) {
                    newRecord.put(fieldName(fieldNamePrefix, entry.getKey()), null);
                    continue;
                }

commit 7abd3d5c76febc3e3f90b140aaf4c8bec7e08e8a
Author: siva santhalingam <ssanthalingam@netskope.com>
Date:   2017-10-03T18:07:07Z

    Revert "Flatten SMT does not work with null values"
    
    This reverts commit 48e50992139c03099b3249195efc316a84d6bba1.

commit 6f9c11726d1baf9eb2446867899218a4f46a77a4
Author: shivsantham <siva.santhalingam@gmail.com>
Date:   2017-10-04T22:53:49Z

    Merge branch 'trunk' of https://github.com/shivsantham/kafka into trunk

commit 084466bfd1f8bb9c9a07ec4f8255a42dfc6b8768
Author: shivsantham <siva.santhalingam@gmail.com>
Date:   2017-10-04T22:59:27Z

    KAFKA-5972 Flatten SMT does not work with null values
    
    A bug in Flatten SMT while doing tests with different SMTs that are provided out-of-box.
Flatten SMT does not work as expected with schemaless JSON that has properties with null values.
    
    Example json:
      {A={D=dValue, B=null, C=cValue}}
    The issue is in if statement that checks for null value.
    
    CURRENT VERSION:
      for (Map.Entry<String, Object> entry : originalRecord.entrySet()) {
                final String fieldName = fieldName(fieldNamePrefix, entry.getKey());
                Object value = entry.getValue();
                if (value == null) {
                    newRecord.put(fieldName(fieldNamePrefix, entry.getKey()), null);
                    return;
                }
    
    PROPOSED VERSION:
      for (Map.Entry<String, Object> entry : originalRecord.entrySet()) {
                final String fieldName = fieldName(fieldNamePrefix, entry.getKey());
                Object value = entry.getValue();
                if (value == null) {
                    newRecord.put(fieldName(fieldNamePrefix, entry.getKey()), null);
                    continue;
                }

----


> Flatten SMT does not work with null values
> ------------------------------------------
>
>                 Key: KAFKA-5972
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5972
>             Project: Kafka
>          Issue Type: Bug
>          Components: KafkaConnect
>    Affects Versions: 0.11.0.0, 0.11.0.1
>            Reporter: Tomas Zuklys
>            Assignee: siva santhalingam
>            Priority: Minor
>              Labels: easyfix, patch
>         Attachments: kafka-transforms.patch
>
>
> Hi,
> I noticed a bug in Flatten SMT while doing tests with different SMTs that are provided
out-of-box.
> Flatten SMT does not work as expected with schemaless JSON that has properties with null
values. 
> Example json:
> {code}
>   {A={D=dValue, B=null, C=cValue}}
> {code}
> The issue is in if statement that checks for null value.
> Current version:
> {code}
>   for (Map.Entry<String, Object> entry : originalRecord.entrySet()) {
>             final String fieldName = fieldName(fieldNamePrefix, entry.getKey());
>             Object value = entry.getValue();
>             if (value == null) {
>                 newRecord.put(fieldName(fieldNamePrefix, entry.getKey()), null);
>                 return;
>             }
> ...
> {code}
> should be
> {code}
>   for (Map.Entry<String, Object> entry : originalRecord.entrySet()) {
>             final String fieldName = fieldName(fieldNamePrefix, entry.getKey());
>             Object value = entry.getValue();
>             if (value == null) {
>                 newRecord.put(fieldName(fieldNamePrefix, entry.getKey()), null);
>                 continue;
>             }
> {code}
> I have attached a patch containing the fix for this issue.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message