apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (APEXMALHAR-2034) Avro File Input Operator
Date Tue, 15 Aug 2017 18:34:00 GMT

    [ https://issues.apache.org/jira/browse/APEXMALHAR-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16127684#comment-16127684

ASF GitHub Bot commented on APEXMALHAR-2034:

SaumyaMohan opened a new pull request #665: APEXMALHAR-2034 Adding new Avro Module to encapsulate
Container File to GenericRecord to POJO transformation
URL: https://github.com/apache/apex-malhar/pull/665
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:

> Avro File Input Operator
> ------------------------
>                 Key: APEXMALHAR-2034
>                 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2034
>             Project: Apache Apex Malhar
>          Issue Type: New Feature
>            Reporter: devendra tagare
>            Assignee: Saumya Mohan
> This operator would extend the AbstractFileInputOperator to read Avro Container files.
> Input would be an Avro Container File.
> Output would be a GenericRecord.
> There would be 2 additional optional ports,
> 1.FilePort - for completed files.
> 2.FailedRecordsPort - this will capture fileName,Offset & error message as a string
for handling by a downstream operator.
> This operator can be used in isolation or with the AvroToPojo operator to read an Avro
record and convert it to a POJO.
> ---------------------------------------------------------------------------------------------------------------------
> This JIRA is used to create a Module on top of AvroFileInputOperator and AvroToPojo operator.
The stream between the two operators will be set to CONTAINER_LOCAL which is required as Avro
objects are not serialized by Kryo. This will help users to directly use the module which
has the locality set to CONTAINER_LOCAL.

This message was sent by Atlassian JIRA

View raw message