Mailing-List: contact dev-help@apex.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@apex.apache.org
Date: Fri, 11 Aug 2017 22:21:00 +0000 (UTC)
From: "Saumya Mohan (JIRA)" <jira@apache.org>
To: dev@apex.incubator.apache.org
Message-ID: <JIRA.12954020.1459187831000.15864.1502490060262@Atlassian.JIRA>
In-Reply-To: <JIRA.12954020.1459187831000@Atlassian.JIRA>
References: <JIRA.12954020.1459187831000@Atlassian.JIRA> <JIRA.12954020.1459187831339@jira-lw-us.apache.org>
Subject: [jira] [Commented] (APEXMALHAR-2034) Avro File Input Operator
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Fri, 11 Aug 2017 22:21:10 -0000


    [ https://issues.apache.org/jira/browse/APEXMALHAR-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16124191#comment-16124191 ] 

Saumya Mohan commented on APEXMALHAR-2034:
------------------------------------------

Hi Thomas, this JIRA is just the first step toward improving the Avro input functionality. As part of this JIRA we're creating a module to encapsulate Avro Container File -> Generic Record -> POJO transformation which user can use directly. Further enhancements will take place as part of separate JIRAs.

> Avro File Input Operator
> ------------------------
>
>                 Key: APEXMALHAR-2034
>                 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2034
>             Project: Apache Apex Malhar
>          Issue Type: New Feature
>            Reporter: devendra tagare
>            Assignee: Saumya Mohan
>
> This operator would extend the AbstractFileInputOperator to read Avro Container files.
> Input would be an Avro Container File.
> Output would be a GenericRecord.
> There would be 2 additional optional ports,
> 1.FilePort - for completed files.
> 2.FailedRecordsPort - this will capture fileName,Offset & error message as a string for handling by a downstream operator.
> This operator can be used in isolation or with the AvroToPojo operator to read an Avro record and convert it to a POJO.
> ---------------------------------------------------------------------------------------------------------------------
> This JIRA is used to create a Module on top of AvroFileInputOperator and AvroToPojo operator. The stream between the two operators will be set to CONTAINER_LOCAL which is required as Avro objects are not serialized by Kryo. This will help users to directly use the module which has the locality set to CONTAINER_LOCAL.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)