apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Saumya Mohan (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (APEXMALHAR-2034) Avro File Input Operator
Date Tue, 15 Aug 2017 20:48:00 GMT

     [ https://issues.apache.org/jira/browse/APEXMALHAR-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Saumya Mohan updated APEXMALHAR-2034:
-------------------------------------
    Description: 

This JIRA is used to create a Module on top of AvroFileInputOperator and AvroToPojo operator.
The stream between the two operators will be set to CONTAINER_LOCAL which is required as Avro
objects are not serialized by Kryo. This will help users to directly use the module which
has the locality set to CONTAINER_LOCAL.

  was:
This operator would extend the AbstractFileInputOperator to read Avro Container files.

Input would be an Avro Container File.
Output would be a GenericRecord.

There would be 2 additional optional ports,
1.FilePort - for completed files.
2.FailedRecordsPort - this will capture fileName,Offset & error message as a string for
handling by a downstream operator.

This operator can be used in isolation or with the AvroToPojo operator to read an Avro record
and convert it to a POJO.

---------------------------------------------------------------------------------------------------------------------
This JIRA is used to create a Module on top of AvroFileInputOperator and AvroToPojo operator.
The stream between the two operators will be set to CONTAINER_LOCAL which is required as Avro
objects are not serialized by Kryo. This will help users to directly use the module which
has the locality set to CONTAINER_LOCAL.


> Avro File Input Operator
> ------------------------
>
>                 Key: APEXMALHAR-2034
>                 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2034
>             Project: Apache Apex Malhar
>          Issue Type: New Feature
>            Reporter: devendra tagare
>            Assignee: Saumya Mohan
>
> This JIRA is used to create a Module on top of AvroFileInputOperator and AvroToPojo operator.
The stream between the two operators will be set to CONTAINER_LOCAL which is required as Avro
objects are not serialized by Kryo. This will help users to directly use the module which
has the locality set to CONTAINER_LOCAL.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message