avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AVRO-672) Convert JSON Text Input to Avro Tool
Date Fri, 17 Feb 2012 21:36:57 GMT

    [ https://issues.apache.org/jira/browse/AVRO-672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210586#comment-13210586
] 

Doug Cutting commented on AVRO-672:
-----------------------------------

Leith, is the tool that Ron provided here the one you need?  If so, then we can probably resuscitate
this patch and get it committed.  If not, is there a specific tool you need (e.g., CSV or
TSV)?  Thanks!
                
> Convert JSON Text Input to Avro Tool
> ------------------------------------
>
>                 Key: AVRO-672
>                 URL: https://issues.apache.org/jira/browse/AVRO-672
>             Project: Avro
>          Issue Type: New Feature
>          Components: java
>            Reporter: Ron Bodkin
>         Attachments: AVRO-672.patch, AVRO-672.patch
>
>
> The attached patch allows reading a JSON-formatted text file in, converting to a conforming
Avro text file, emitting one record per line, e.g., it can read this input file:
> {"intval":12}
> {"intval":-73,"strval":"hello, there!!"}
> with this schema:
> { "type":"record", "name":"TestRecord", "fields": [ {"name":"intval","type":"int"}, {"name":"strval","type":["string",
"null"]}]}
> returning valid Avro. This is different than the DataFileWriteTool, which would read
in the following internal encoding:
> {"intval":12,"strval":null}
> {"intval":-73,"strval":{"string":"hello, there!!"}}
> In general, the internal encodings used by Avro aren't natural when reading in JSON text
that appears in the wild. Likewise, this utility allows changing invalid Avro identifier characters
into an underscore, again to tolerate JSON that wasn't designed to be readable by Avro.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message