avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alan Miller <alan.mill...@gmail.com>
Subject Re: C-API schema
Date Tue, 28 Apr 2015 00:45:45 GMT
The avro-tools compiler encodes the doc & custom field fine:

[alan@localhost]$ java -jar avro-tools-1.7.7.jar compile schema test.avsc .
Input files to compile:
  test.avsc

[alan@localhost]$ grep "public static final"   org/me/MyRecord.java
  public static final org.apache.avro.Schema SCHEMA$ = new
org.apache.avro.Schema.Parser().parse("{\"type\":\"record\",\"name\":\"MyRecord\",\"namespace\":\"org.me\",\"doc\":\"Version_0.1\",\"fields\":[{\"name\":\"myname\",\"type\":\"string\"}],\"rev\":\"0.1\"}");

On Mon, Apr 27, 2015 at 5:40 PM, Alan Miller <alan.miller3@gmail.com> wrote:
> Here's a more concise example of what I'm doing.
>
> BTW: I also don't see my "doc" object in the data file ( I also tried:
>    "aliases": ["MyAlias"], )
> so I'm at a loss as to how to write custom meta-data via the C-API
>
> [alan@localhost]$ cat writer.c
> #include "avro.h"
> #include <stdio.h>
> #include <stdlib.h>
>
> int main(int argc, char *argv[])
> {
> int rval;
> int written = 0;
> avro_file_writer_t file_writer;
> char avro_file[128];
> char json_schema[16 * 1024];
> avro_schema_t schema = NULL;
> avro_schema_error_t schema_error = NULL;
>
> FILE *fp = fopen("test.avsc", "r");
> rval = fread(json_schema, 1, sizeof(json_schema) - 1, fp);
> json_schema[rval] = '\0';
> rval = avro_schema_from_json(json_schema, strlen(json_schema),
> &schema, &schema_error);
> fprintf(stderr, "Loaded avro schema. rval %d\n", rval);
> fclose(fp);
>
> snprintf(avro_file, sizeof(avro_file), "%s", "test.avro");
> rval = avro_file_writer_create("test.avro", schema, &file_writer);
> fprintf(stderr, "Created writer. rval %d\n", rval);
>
> avro_datum_t mydatum = avro_record(schema);
> avro_record_set(mydatum, "myname", avro_givestring("alan", NULL));
> rval = avro_file_writer_append(file_writer, mydatum);
> if (rval == 0) {
>  fprintf(stderr, "Successfully appended datum to file. rval %d\n", rval);
>  written ++;
> } else {
> fprintf(stderr, "Unable to append data to file!\n");
> exit(EXIT_FAILURE);
> }
> avro_datum_decref(mydatum);
>
> rval = avro_file_writer_close(file_writer);
> fprintf(stderr, "Closed writer. rval %d\n", rval);
> fprintf(stderr, "Wrote %d records to %s\n", written, "test.avro");
>
> return 0;
> }
>
>
> [alan@localhost]$ make writer
> building writer
> gcc -Wall -g -I/opt/avro/include -L/opt/avro/lib -l avro -o writer writer.c
>
> [alan@localhost]$ ./writer
> Loaded avro schema. rval 0
> Created writer. rval 0
> Successfully appended datum to file. rval 0
> Closed writer. rval 0
> Wrote 1 records to test.avro
>
> [alan@localhost]$ java -jar avro-tools-1.7.7.jar getmeta test.avro
> avro.codec null
> avro.schema {"type":"record","name":"MyRecord","namespace":"org.me","fields":[{"name":"myname","type":{"type":"string"}}]}
>
> [alan@localhost]$ cat test.avsc
> {
>   "type" : "record",
>   "name" : "MyRecord",
>   "namespace": "org.me",
>   "doc": "Version_0.1",
>   "rev": "0.1",
>   "fields" : [
>     { "name": "myname",  "type": "string" }
>   ]
> }
>
> [alan@localhost]$ avro cat test.avro
> {"myname": "alan"}
>
> [alan@localhost]$ avro cat --print-schema test.avro
> {
>     "namespace": "org.me",
>     "type": "record",
>     "name": "MyRecord",
>     "fields": [
>         {
>             "type": {
>                 "type": "string"
>             },
>             "name": "myname"
>         }
>     ]
> }
>
> [alan@localhost]$ java -jar avro-tools-1.7.7.jar getmeta test.avro
> avro.codec null
> avro.schema {"type":"record","name":"MyRecord","namespace":"org.me","fields":[{"name":"myname","type":{"type":"string"}}]}
>
> [alan@localhost]$ java -jar avro-tools-1.7.7.jar tojson  test.avro
> {"myname":"alan"}
>
>
> On Mon, Apr 27, 2015 at 3:10 PM, Alan Miller <alan.miller3@gmail.com> wrote:
>> Is it possible to write custom meta-field with the C-API?
>>
>> I tried this:
>> {
>>   "type" : "record",
>>   "name" : "MySchema",
>>   "namespace": "org.my",
>>   "aliases": ["Cerberus_address"],
>>   "doc": "MYSCHEMA_V1",
>>   "my.version": "v1",
>>   "fields" : [
>>     { "name": "name",  "type": "string"    }
>> }
>>
>> And write a record with the C-API (v1.7.7) I only get this:
>>
>> java -jar /opt/avro/avro-tools-1.7.7.jar getmeta test.avro
>> avro.codec null
>> avro.schema {"type":"record","name":"MySchema","namespace":"org.my","fields":[{"name":"name","type":{"type":"string"}]}]}
>>
>> Alan

Mime
View raw message