Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id E62CC200CFA for ; Tue, 15 Aug 2017 20:34:09 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id E33321672BE; Tue, 15 Aug 2017 18:34:09 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 3C5901672BF for ; Tue, 15 Aug 2017 20:34:09 +0200 (CEST) Received: (qmail 46041 invoked by uid 500); 15 Aug 2017 18:34:07 -0000 Mailing-List: contact dev-help@apex.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@apex.apache.org Delivered-To: mailing list dev@apex.apache.org Received: (qmail 46030 invoked by uid 99); 15 Aug 2017 18:34:07 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 15 Aug 2017 18:34:07 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 8B647180687 for ; Tue, 15 Aug 2017 18:34:06 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -99.202 X-Spam-Level: X-Spam-Status: No, score=-99.202 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id kzUgMquSD05B for ; Tue, 15 Aug 2017 18:34:05 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 8D0AC5F659 for ; Tue, 15 Aug 2017 18:34:05 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 3219AE0E1A for ; Tue, 15 Aug 2017 18:34:03 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 46B06218FA for ; Tue, 15 Aug 2017 18:34:00 +0000 (UTC) Date: Tue, 15 Aug 2017 18:34:00 +0000 (UTC) From: "ASF GitHub Bot (JIRA)" To: dev@apex.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (APEXMALHAR-2034) Avro File Input Operator MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Tue, 15 Aug 2017 18:34:10 -0000 [ https://issues.apache.org/jira/browse/APEXMALHAR-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16127684#comment-16127684 ] ASF GitHub Bot commented on APEXMALHAR-2034: -------------------------------------------- SaumyaMohan opened a new pull request #665: APEXMALHAR-2034 Adding new Avro Module to encapsulate Container File to GenericRecord to POJO transformation URL: https://github.com/apache/apex-malhar/pull/665 ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org > Avro File Input Operator > ------------------------ > > Key: APEXMALHAR-2034 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2034 > Project: Apache Apex Malhar > Issue Type: New Feature > Reporter: devendra tagare > Assignee: Saumya Mohan > > This operator would extend the AbstractFileInputOperator to read Avro Container files. > Input would be an Avro Container File. > Output would be a GenericRecord. > There would be 2 additional optional ports, > 1.FilePort - for completed files. > 2.FailedRecordsPort - this will capture fileName,Offset & error message as a string for handling by a downstream operator. > This operator can be used in isolation or with the AvroToPojo operator to read an Avro record and convert it to a POJO. > --------------------------------------------------------------------------------------------------------------------- > This JIRA is used to create a Module on top of AvroFileInputOperator and AvroToPojo operator. The stream between the two operators will be set to CONTAINER_LOCAL which is required as Avro objects are not serialized by Kryo. This will help users to directly use the module which has the locality set to CONTAINER_LOCAL. -- This message was sent by Atlassian JIRA (v6.4.14#64029)