avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ryan Blue (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AVRO-1744) New AvroValueInputFormat class to process Avro messages without a key.
Date Tue, 06 Oct 2015 17:59:27 GMT

    [ https://issues.apache.org/jira/browse/AVRO-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14945467#comment-14945467
] 

Ryan Blue commented on AVRO-1744:
---------------------------------

This seems reasonable to me since some Avro values (maps) that can't be used in keys. Could
you open a pull request or submit a patch?

> New AvroValueInputFormat class to process Avro messages without a key.
> ----------------------------------------------------------------------
>
>                 Key: AVRO-1744
>                 URL: https://issues.apache.org/jira/browse/AVRO-1744
>             Project: Avro
>          Issue Type: New Feature
>          Components: java
>    Affects Versions: 1.7.7
>            Reporter: Mark S
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> My Avro messages do not contain a Key value and as such, there are no Avro supporting
classes to deal with this.
> As I was unable to use the following classes,
> * org.apache.avro.mapreduce.AvroKeyInputFormat
> * org.apache.avro.mapreduce.AvroKeyValueInputFormat
> I created the following supporting classes:
> * [AvroValueInputFormat|https://github.com/mark1900/druid-sandbox/blob/master/spark-hdfs-avro-test/src/main/java/test/avro/AvroValueInputFormat.java]
> * [AvroValueRecordReader|https://github.com/mark1900/druid-sandbox/blob/master/spark-hdfs-avro-test/src/main/java/test/avro/AvroValueRecordReader.java]
> * [AvroValueOutputFormat|https://github.com/mark1900/druid-sandbox/blob/master/spark-hdfs-avro-test/src/main/java/test/avro/AvroValueOutputFormat.java]
> * [AvroValueRecordWriter|https://github.com/mark1900/druid-sandbox/blob/master/spark-hdfs-avro-test/src/main/java/test/avro/AvroValueRecordWriter.java]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message