avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zoltan Farkas (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (AVRO-1340) use default to allow old readers to specify default enum value when encountering new enum symbols
Date Tue, 22 Aug 2017 19:30:00 GMT

    [ https://issues.apache.org/jira/browse/AVRO-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16137220#comment-16137220
] 

Zoltan Farkas edited comment on AVRO-1340 at 8/22/17 7:29 PM:
--------------------------------------------------------------

[~cutting] what do you thing about the following:

#  Symbol aliases example to correct misspellings. (similar to field aliasses):  

 v1:

 {code}
 { "type": "enum",
   "name": "Suit",
  "symbols" : ["SPADES", "HEARTS", "DIAMONDS", "CLUS"]
 }
 {code}

 v2 (correcting CLUS):

 {code}
 { "type": "enum",
   "name": "Suit",
   "symbols" : ["SPADES", "HEARTS", "DIAMONDS", "CLUB”]
   “symbolAliasses” : {
       “CLUB” : [“CLUS”]
   }
 }
 {code}

 v3 (correcting CLUB):

 {code}
 { "type": "enum",
   "name": "Suit",
   "symbols" : ["SPADES", "HEARTS", "DIAMONDS", "CLUBS”]
   “symbolAliasses” : {
       “CLUBS” : [“CLUS”, “CLUB”]
   }
 }
 {code}

 sample conversion:

 v1: { “val” : “CLUS”} <-> v3: { “val” : “CLUBS” }

 the following enum schema would be illegal:

 {code}
 { "type": "enum",
   "name": "Suit", 
   "symbols" : ["SPADES", "HEARTS", "DIAMONDS", "CLUBS”]
   “symbolAliasses” : {
       “CLUBS” : [“CLUS”, “CLUB”, “SPADES”]
   }
 }
 {code}

 since it contains a duplicate definition. (“SPADES”, “CLUBS”)

# Evolution example with a fallback symbol:
 
 V1:

 {code}
 { "type": "enum",
   "name": "Suit",
   "symbols" : [“UNKNOWN”, ”SPADES", "HEARTS", "DIAMONDS", "CLUBS”],
   “symbolAliasses” : {
       “CLUBS” : [“CLUS”, “CLUB”]
   },
   “fallbackSymbol” : “UNKNOWN”
 }
 {code}

 and a record using the above enum:

 {code}
 {
   "type": "record",
   "name": “Example”,
   "fields" : [
     {"name": “enumValue”,
       "type": “Suit”,
       “default” : “SPADES”},
   ]
 }
 {code}

 V2 (adding "NEWVAL" symbol):

 {code}
 { "type": "enum",
   "name": "Suit",
   "symbols" : [“UNKNOWN”, ”SPADES", "HEARTS", "DIAMONDS", "CLUBS”, “NEWVAL”]
   “symbolAliasses” : {
       “CLUBS” : [“CLUS”, “CLUB”],
   },
   “fallbackSymbol” : “UNKNOWN”
 }
 {code}
 
 V3 (removing "SPADES" symbol):

 {code}
 { "type": "enum",
   "name": "Suit",
   "symbols" : [“UNKNOWN”,  "HEARTS", "DIAMONDS", "CLUBS”, “NEWVAL”]
   “symbolAliasses” : {
       “CLUBS” : [“CLUS”, “CLUB”],
   },
   “fallbackSymbol” : “UNKNOWN”
 }
 {code}


 sample conversions:

 v2 : { “enumValue” : “NEWVAL” } -> v1 : {“enumValue” : “UNKNOWN”}

 v1: {“enumValue” : “UNKNOWN”} -> v2  : {“enumValue” : “UNKNOWN”}

 v1: {“enumValue” : “SPADES”} -> v3  : {“enumValue” : “UNKNOWN”}

 would this be acceptable?




was (Author: zolyfarkas):
[~cutting] what do you thing about the following:

#  Symbol aliases example to correct misspellings. (similar to field aliasses):  

v1:

{code}
{ "type": "enum",
  "name": "Suit",
  "symbols" : ["SPADES", "HEARTS", "DIAMONDS", "CLUS"]
}
{code}

v2 (correcting CLUS):

{code}
{ "type": "enum",
  "name": "Suit",
  "symbols" : ["SPADES", "HEARTS", "DIAMONDS", "CLUB”]
  “symbolAliasses” : {
      “CLUB” : [“CLUS”]
  }
}
{code}

v3 (correcting CLUB):

{code}
{ "type": "enum",
  "name": "Suit",
  "symbols" : ["SPADES", "HEARTS", "DIAMONDS", "CLUBS”]
  “symbolAliasses” : {
      “CLUBS” : [“CLUS”, “CLUB”]
  }
}
{code}

sample conversion:

v1: { “val” : “CLUS”} <-> v3: { “val” : “CLUBS” }

the following enum schema would be illegal:

{code}
{ "type": "enum",
  "name": "Suit",
  "symbols" : ["SPADES", "HEARTS", "DIAMONDS", "CLUBS”]
  “symbolAliasses” : {
      “CLUBS” : [“CLUS”, “CLUB”, “SPADES”]
  }
}
{code}

since it contains a duplicate definition. (“SPADES”, “CLUBS”)


# Evolution example with a fallback symbol:

V1:

{code}
{ "type": "enum",
  "name": "Suit",
  "symbols" : [“UNKNOWN”, ”SPADES", "HEARTS", "DIAMONDS", "CLUBS”],
  “symbolAliasses” : {
      “CLUBS” : [“CLUS”, “CLUB”]
  },
  “fallbackSymbol” : “UNKNOWN”
}
{code}

and a record using the above enum:

{code}
{
  "type": "record",
  "name": “Example”,
  "fields" : [
    {"name": “enumValue”,
      "type": “Suit”,
      “default” : “SPADES”},
  ]
}
{code}

V2 (adding "NEWVAL" symbol):

{code}
{ "type": "enum",
  "name": "Suit",
  "symbols" : [“UNKNOWN”, ”SPADES", "HEARTS", "DIAMONDS", "CLUBS”, “NEWVAL”]
  “symbolAliasses” : {
      “CLUBS” : [“CLUS”, “CLUB”],
  },
  “fallbackSymbol” : “UNKNOWN”
}
{code}

V3 (removing "SPADES" symbol):

{code}
{ "type": "enum",
  "name": "Suit",
  "symbols" : [“UNKNOWN”,  "HEARTS", "DIAMONDS", "CLUBS”, “NEWVAL”]
  “symbolAliasses” : {
      “CLUBS” : [“CLUS”, “CLUB”],
  },
  “fallbackSymbol” : “UNKNOWN”
}
{code}


sample conversion:

v2 : { “enumValue” : “NEWVAL” } -> v1 : {“enumValue” : “UNKNOWN”}

v1: {“enumValue” : “UNKNOWN”} -> v2  : {“enumValue” : “UNKNOWN”}

v1: {“enumValue” : “SPADES”} -> v3  : {“enumValue” : “UNKNOWN”}

would this be acceptable?



> use default to allow old readers to specify default enum value when encountering new
enum symbols
> -------------------------------------------------------------------------------------------------
>
>                 Key: AVRO-1340
>                 URL: https://issues.apache.org/jira/browse/AVRO-1340
>             Project: Avro
>          Issue Type: Improvement
>          Components: spec
>         Environment: N/A
>            Reporter: Jim Donofrio
>            Priority: Minor
>
> The schema resolution page says:
> > if both are enums:
> > if the writer's symbol is not present in the reader's enum, then an
> error is signalled.
> This makes it difficult to use enum's because you can never add a enum value and keep
old reader's compatible. Why not use the default option to refer to one of enum values so
that when a old reader encounters a enum ordinal it does not recognize, it can default to
the optional schema provided one. If the old schema does not provide a default then the older
reader can continue to fail as it does today.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message