avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From HILEM Youcef <youcef.hi...@laposte.fr>
Subject add a type to a union
Date Mon, 07 Dec 2015 22:15:06 GMT
Hi,

At La Poste Pôle Colis we use Avro in our new reactive architecture (kafka, spark streaming,
Cassandra, elasticsearch, play framework).

In our modeling we used the type union to bring together in one schema all trace events of
a package (arrival, departure, transportation, ...) at the body attribute.

Example :
{
"namespace" : "fr.laposte.colis.schema.pivot.message",
"name" : "Message",
"type" : "record",
"doc" : "Cette structure défini les caractéristiques de base d'un message. Peut(doit) être
spécialisée pour un usage particulier",
                                "fields" : [
                                               {
                                                               "name" : "header",
                                                               "type" : "fr.laposte.colis.schema.pivot.common.message.MessageHeader",
                                                               "doc" :  "Entête du message"
                                               },{
                                                               "name" : "body",
                                                               "type" : ["fr.laposte.colis.schema.pivot.announcement.AnnouncementEventMessageBody",
                                                                              "fr.laposte.colis.schema.pivot.delivery.DeliveryEventMessageBody",
                                                                              "fr.laposte.colis.schema.pivot.handling.HandlingEventMessageBody",
                                                                              "fr.laposte.colis.schema.pivot.crm.CrmEventMessageBody",
                                                                              "fr.laposte.colis.schema.pivot.customs.transport.CustomsTransportMessageBody",
                                                                              "fr.laposte.colis.schema.pivot.customs.consignment.CustomsContainerEventMessageBody",
                                                                              "fr.laposte.colis.schema.pivot.customs.consignment.CustomsParcelEventMessageBody",
                                                                              "fr.laposte.colis.schema.pivot.rest.common.Rest",
                                                                              "fr.laposte.colis.schema.pivot.reject.RejectMessageBody",
                                                                              "fr.laposte.colis.schema.pivot.dpmo.defectrequest.DefectRequestEventMessageBody",
                                                                              "fr.laposte.colis.schema.pivot.dpmo.defectresult.DefectResultEventMessageBody",
                                                                               "fr.laposte.colis.schema.timeout.TimeoutMessageBody",
                                                                               "fr.laposte.colis.schema.notification.Notification"
                                                                              ],
                                                               "doc" :  "Abstraction du corps
de message. Peut-être substitué par tout type dérivé du type MessageBody"
                                               }
                                ]
}

However, as well explained at (https://martin.kleppmann.com/2012/12/05/schema-evolution-in-avro-protocol-buffers-thrift.html)
: "Union types are powerful, but you must take care when changing them. If you want to add
a type to a union, you first need to update all readers with the new schema, so that they
know what to expect. Only once all readers are updated, the writers may start putting this
new type in the records they generate"

My question : is a default value for field "body" is sufficient so that if the reader encounters
a union branch it does not know about, it can substitute the default value (see http://grokbase.com/t/avro/user/11b3bn6r6z/does-extending-union-break-compatibility)
?

Thank you in advance for your help.

Mime
View raw message