avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bart Verwilst <li...@verwilst.be>
Subject Changing Avro schemas for daily imports
Date Wed, 19 Dec 2012 10:01:17 GMT
Hello!

Every night, we fetch mysql rows with a timestamp of the day before, 
and store them into avro, creating a Y-M-d.avro file on HDFS daily.

This is the schema:

{
   "namespace": "asp",
   "type": "record",
   "name": "trace",
   "fields": [
     {
       "type": "long",
       "name": "id"
     },
     {
       "type": "long",
       "name": "timestamp"
     },
     {
       "type": [
         "int",
         "null"
       ],
       "name": "latitude"
     },
     {
       "type": [
         "int",
         "null"
       ],
       "name": "longitude"
     }
   ]
}

Now i would like to change timestamp so it can be null as well. The 
plan is to just change the timestamp type in the schema starting from 
the next day. I'm pretty sure it won't affect lookups in any way ( like 
when using Pig ), but I thought I would ask to be sure ( since the 
structure itself doesn't change, only the type ). I wouldn't want to run 
into a gotcha after months of importing with my adjusted schema. :)

Thanks in advance!

Kind regards,

Bart

Mime
View raw message