avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alparslan Avcı <alparslan.a...@agmlab.com>
Subject Re: Nullable Fields
Date Wed, 22 Jan 2014 08:08:57 GMT
Hi Doug,

Thanks for the detailed explanation first. The reason to ask this question is to
learn the details of design decisions. And now, I understand the design better.
Since I'm developing other projects using Avro, these details help me to build better solutions.

> Null types in unions will always cause an exceptional situation since they
> are not 'types' at all.

And about this statement, I wanted to state that we always have to consider null type when
we
develop an algorithm about union types. For example; when deciding the possible type of a
field's
value, we check for union type and dig into recursively if so. While we are doing this, we
also
have to check for null type at any level as an exceptional situation. In concept, possibility
to be 'null' or not is a property for a field. A field can be any type and can have null
values or not besides. This aspect would be more appropriate in terms of OOD, I think.

Anyway, as you have said that design decision is now past. And it costs so much to make a
change
in design like this.

Thanks,
Alparslan.


On 21-01-2014 22:19, Doug Cutting wrote:
> On Mon, Jan 20, 2014 at 3:50 AM, Alparslan Avcı
> <alparslan.avci@agmlab.com> wrote:
>> {"name": "field1", "type": ["null", {"type":"map", "values":["null",
>> "string"]}],"default":null}
>>
>> can be represented as like
>>
>> {"name": "field1", "type": {"type":"map", "values":"string",
>> "nullable":true}, "nullable":true, "default":null}
> You'd like an alternate syntax for expressing nullability, is that
> right?  The semantics would be the same, a value could either be of
> the named type or it could be null, right?
>
> The original design was done this way because, once we had unions and
> a null type, it seemed better to compose these two concepts than to
> invent another equivalent concept.  (Would the above two schemas be
> equal?  Probably not.  Would they express exactly the same datatypes?
> Probably.)
>
> See also this talk (made after Avro was designed):
>
>    http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare
>
> But, whether we like it or not, that design decision is now past.  We
> cannot easily make such a change in the JSON schema language since it
> would create compatibility problems.  For example, a data file
> generated using the new syntax could not be read by code that had not
> yet been upgraded to process this syntax.  Existing code would not
> know that "nullable" is a special property and would fail in strange
> ways, since it would expect to find items of the specified type, not
> union values.
>
> We can devise higher-level schema languages that are translated into
> Avro schemas and that handle nullability differently.  So you could
> define a pre-processor for JSON schemas that converted them from the
> above syntax into Avro's standard syntax, and use this when
> interacting with Avro.
>
>> Null types in unions will always cause an exceptional situation since they
>> are not 'types' at all.
> I don't understand this statement.
>
> Doug


Mime
View raw message