avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From cutt...@apache.org
Subject Re: My mainly idea about implement data communicate tool between json/xml/csv and avro data files
Date Thu, 25 Mar 2010 16:36:13 GMT
> On Thu, Mar 25, 2010 at 12:52 AM, Scott Carey <scott@richrelevance.com>wrote:
>>  I'm not sure it makes sense to map Avro data into CSV.

I agree that mapping arbitrary Avro data into CSV is difficult.  But, 
for some cases it might be sensible, for example, when the top-level 
schema is a record whose fields are primitive types.  In general, one 
could simply flatten the schema to primitive types, and escape values 
which contain commas.  This will not work well with recursive schemas or 
unions, and one can only restore such a format to Avro if one has the 
identical schema, but I think these might be acceptable, necessary 
limitations.  Errors can be generated if these conditions are not met.

Peng Cui wrote:
> But i think, why we do not generate a schema for
> each CSV data file?

Yes, I think such an approach could be practical and useful.

We should consider uses cases.  One use case is exporting Avro data to 
tools that accept CSV, e.g., a spreadsheet.  A spreadsheet will never 
represent the full structure of Avro data, but, when possible, it might 
still be useful to be able to export Avro data to a spreadsheet.

Doug

Mime
View raw message