avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Victor Mota (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (AVRO-1335) C++ should support field default values
Date Fri, 20 Jan 2017 22:39:26 GMT

    [ https://issues.apache.org/jira/browse/AVRO-1335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15832518#comment-15832518
] 

Victor Mota edited comment on AVRO-1335 at 1/20/17 10:38 PM:
-------------------------------------------------------------

The patch fixes this issue, there's just a one line change that causes the issue Peter documented
^. 

If you change the call below in NodeRecord::printJson:
leafAttributes_.get(i)->printDefaultToJson(defaultValues[i], os, depth);
to 
leafAttributes_.get(i)->printDefaultToJson(defaultValues[i], os, depth, false);

Then Pierre's patch works. Would be great to have this fixed in the latest release, field
default values is a very crucial feature that is missing in the C++ library. The issue should
also be updated to remove the Affected Version (since it affects every version) and arguably
it should be a bug instead of an improvement.


was (Author: vimota):
The patch fixes this issue, there's just a one line change that causes the issue Peter documented
^. 

If you change the call below in NodeRecord::printJson:
leafAttributes_.get(i)->printDefaultToJson(defaultValues[i], os, depth);
to 
leafAttributes_.get(i)->printDefaultToJson(defaultValues[i], os, depth, false);

Then Pierre's patch works. Would be great to have this fixed in the latest release, field
default values is a very crucial feature that is missing in the C++ library.

> C++ should support field default values
> ---------------------------------------
>
>                 Key: AVRO-1335
>                 URL: https://issues.apache.org/jira/browse/AVRO-1335
>             Project: Avro
>          Issue Type: Improvement
>          Components: c++
>    Affects Versions: 1.7.4
>            Reporter: Bin Guo
>         Attachments: AVRO-1335.patch
>
>
> We found that resolvingDecoder could not provide bidirectional compatibility between
different version of schemas.
> Especially for records, for example:
> {code:title=First schema}
> {
>     "type": "record",
>     "name": "TestRecord",
>     "fields": [
>         {
>             "name": "MyData",
> 			"type": {
> 				"type": "record",
> 				"name": "SubData",
> 				"fields": [
> 					{
> 						"name": "Version1",
> 						"type": "string"
> 					}
> 				]
> 			}
>         },
> 	{
>             "name": "OtherData",
>             "type": "string"
>         }
>     ]
> }
> {code}
> {code:title=Second schema}
> {
>     "type": "record",
>     "name": "TestRecord",
>     "fields": [
>         {
>             "name": "MyData",
> 			"type": {
> 				"type": "record",
> 				"name": "SubData",
> 				"fields": [
> 					{
> 						"name": "Version1",
> 						"type": "string"
> 					},
> 					{
> 						"name": "Version2",
> 						"type": "string"
> 					}
> 				]
> 			}
>         },
> 	{
>             "name": "OtherData",
>             "type": "string"
>         }
>     ]
> }
> {code}
> Say, node A knows only the first schema and node B knows the second schema, and the second
schema has more fields. 
> Any data generated by node B can be resolved by first schema 'cause the additional field
is marked as skipped.
> But data generated by node A can not be resolved by second schema and throws an exception
*"Don't know how to handle excess fields for reader."*
> This is because data is resolved exactly according to the auto-generated codec_traits
which trying to read the excess field.
> The problem is we just can not only ignore the excess field in record, since the data
after the troublesome record also needs to be resolved.
> Actually this problem stucked us for a very long time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message