nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Juan Pablo Gardella <gardellajuanpa...@gmail.com>
Subject Re: Cannot convert to Record a valid Avro schema
Date Tue, 27 Feb 2018 14:38:10 GMT
Mark,

I've attached a simple project to reproduce the bugs, and it fails either
default to 0 or [0]:

Tests in error:
  testIssue1(com.foo.TestIssueDefaultValues): Cannot set the default value
for field [listOfInt] to [0] because that is not a valid value for Data
Type [ARRAY[INT]]
  testIssue2(com.foo.TestIssueDefaultValues): Cannot set the default value
for field [listOfInt] to [[0]] because that is not a valid value for Data
Type [ARRAY[INT]]

So when you said: *Then it works properly, giving us the default value of a
1-element array with the value 0 as the only element*, at least using Nifi
1.5.0 does not work.

Regarding to:
*But what you expect to happenis for this case to be treated the same as if
the schema had said:*

*"type": {"type": "array", "items": "int" }, "default": []*
>>
>>
*So that if the field is not specified, you get an empty array for the
value.*

*Is that accurate? -> **YES*. Basically I guess Nifi should mimic Avro
(when it creates a RecordSchema from Avro schema) in this case in order to
support that scenarios.

Thanks a lot,
Juan

On Tue, 27 Feb 2018 at 11:25 Mark Payne <markap14@hotmail.com> wrote:

> Juan,
>
> OK, thanks. I was trying to understand what the intent was with that
> schema. So to make sure that everyone
> is on the same page:
>
> If I use the following field in the schema:
>
> "type": {"type": "array", "items": "int" }, "default": [ 0 ]
>>>
>>>
> Then it works properly, giving us the default value of a 1-element array
> with the value 0 as the only element.
>
> But if I use:
>
> "type": {"type": "array", "items": "int" }, "default": 0
>>>
>>>
> Then currently it throws an Exception because the default value is not an
> array. But what you expect to happen
> is for this case to be treated the same as if the schema had said:
>
> "type": {"type": "array", "items": "int" }, "default": []
>>>
>>>
> So that if the field is not specified, you get an empty array for the
> value.
>
> Is that accurate?
>
> Thanks
> -Mark
>
>
> On Feb 27, 2018, at 9:18 AM, Juan Pablo Gardella <
> gardellajuanpablo@gmail.com> wrote:
>
> Thanks Mike, agree but Avro does not complain and it allows using it. The
> schema is used in production and I cannot change it for now.
>
> On Tue, 27 Feb 2018 at 11:17 Mike Thomsen <mikerthomsen@gmail.com> wrote:
>
>> That doesn't look like the right way to specify an empty array. This SO
>> example fits about what I'd expect:
>>
>> https://stackoverflow.com/a/42140165/284538
>>
>> So it should be default:[0]
>>
>> On Tue, Feb 27, 2018 at 8:56 AM, Mark Payne <markap14@hotmail.com> wrote:
>>
>>> Juan,
>>>
>>> So the scenario that you laid out in the NIFI-4893 is not one that I've
>>> personally
>>> encountered. What does it mean exactly to have an Avro schema with an
>>> "array" type
>>> that has a value? In the example that you laid out, the field has:
>>>
>>> "type": {"type": "array", "items": "int" }, "default": 0
>>>
>>>
>>> In this case, what should be the value of this field if it's not
>>> specified? A single-element array with the value of 0?
>>> From looking at the PR that was submitted, it appears to set the
>>> defaultValue to a new (empty) ArrayList.
>>> I would think that maybe it should set defaultValue to
>>>
>>> new Integer[] {0};
>>>
>>> But I am not certain of the semantics here.
>>>
>>> Thanks
>>> -Mark
>>>
>>>
>>> On Feb 27, 2018, at 7:30 AM, Juan Pablo Gardella <
>>> gardellajuanpablo@gmail.com> wrote:
>>>
>>> Hello team,
>>>
>>> I could not fix the issue. I did a patch but it's solves the issue, but
>>> I believe is not the correct solution. Anyone that knows Record Framework
>>> can help me on that?
>>>
>>> Thanks in advance,
>>> Juan
>>>
>>> On Mon, 19 Feb 2018 at 22:33 Juan Pablo Gardella <
>>> gardellajuanpablo@gmail.com> wrote:
>>>
>>>> I saw an issue in a test :(. I will continue looking into current
>>>> approach.
>>>>
>>>> On Mon, 19 Feb 2018 at 22:23 Juan Pablo Gardella <
>>>> gardellajuanpablo@gmail.com> wrote:
>>>>
>>>>> Hello team,
>>>>>
>>>>> I filed an issue at https://issues.apache.org/jira/browse/NIFI-4893.
>>>>> I discovered using a complex Avro schema. I've isolated the issue and
also
>>>>> did a patch. At least, this solve the issue but actually I don't know
well
>>>>> the implications on that solution.
>>>>>
>>>>> Please let me know what do you think. I have another issue related to
>>>>> Avro and Record, I will file the issue tomorrow.
>>>>>
>>>>> Thanks,
>>>>> Juan
>>>>>
>>>>
>>>
>>
>

Mime
View raw message