Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 6CAD4200B49 for ; Mon, 4 Jul 2016 17:27:13 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 6B862160A34; Mon, 4 Jul 2016 15:27:13 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id AD3E1160A76 for ; Mon, 4 Jul 2016 17:27:12 +0200 (CEST) Received: (qmail 21123 invoked by uid 500); 4 Jul 2016 15:27:11 -0000 Mailing-List: contact dev-help@pig.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@pig.apache.org Delivered-To: mailing list dev@pig.apache.org Received: (qmail 20988 invoked by uid 500); 4 Jul 2016 15:27:11 -0000 Delivered-To: apmail-hadoop-pig-dev@hadoop.apache.org Received: (qmail 20924 invoked by uid 99); 4 Jul 2016 15:27:11 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 04 Jul 2016 15:27:11 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 1569D2C02A5 for ; Mon, 4 Jul 2016 15:27:11 +0000 (UTC) Date: Mon, 4 Jul 2016 15:27:11 +0000 (UTC) From: "sarath (JIRA)" To: pig-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (PIG-4943) Schema issue while storing multiple pig outputs using CSVExcelStorage MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Mon, 04 Jul 2016 15:27:13 -0000 [ https://issues.apache.org/jira/browse/PIG-4943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sarath updated PIG-4943: ------------------------ Description: I have a script which stores 2 relations with different schema using CSVExcelStorage. The issue which i see is that the script picks up the last store function and takes the schema in that and puts it for all store functions , overriding the previous store schemas. My Sample Script Looks like this :-- ============================================================= masterInput = load 'hbase://xyz' using org.apache.pig.backend.hadoop.hbase.HBaseStorage( 'f:a,f:b,f:c,f:d') as (a,b,c,d); input2 = foreach masterInput generate a,b; input3 = foreach masterInput generate c,d; store input2 into '/dir/ab' using org.apache.pig.piggybank.storage.CSVExcelStorage('\t','YES_MULTILINE', 'UNIX', 'WRITE_OUTPUT_HEADER'); store input3 into '/dir/cd' using org.apache.pig.piggybank.storage.CSVExcelStorage('\t','YES_MULTILINE', 'UNIX', 'WRITE_OUTPUT_HEADER'); ============================================================= Where *a,b,c,d* are my headers in my source file Expected Output : ||file 1|| |a|b |10|20 ||file 2|| |c|d |30|40 Actual Output : ||file 1|| |c|d |10|20 ||file 2|| |c|d |30|40 was: I have a script which stores 2 relations with different schema using CSVExcelStorage. The issue which i see is that the script picks up the last store function and takes the schema in that and puts it for all store functions , overriding the previous store schemas.Is this a known issue and is there a fix for this ? My Sample Script Looks like this :-- ============================================================= masterInput = load 'hbase://xyz' using org.apache.pig.backend.hadoop.hbase.HBaseStorage( 'f:a,f:b,f:c,f:d') as (a,b,c,d); input2 = foreach masterInput generate a,b; input3 = foreach masterInput generate c,d; store input2 into '/dir/ab' using org.apache.pig.piggybank.storage.CSVExcelStorage('\t','YES_MULTILINE', 'UNIX', 'WRITE_OUTPUT_HEADER'); store input3 into '/dir/cd' using org.apache.pig.piggybank.storage.CSVExcelStorage('\t','YES_MULTILINE', 'UNIX', 'WRITE_OUTPUT_HEADER'); ============================================================= Where *a,b,c,d* are my headers in my source file Expected Output : ||file 1|| |a|b |10|20 ||file 2|| |c|d |30|40 Actual Output : ||file 1|| |c|d |10|20 ||file 2|| |c|d |30|40 > Schema issue while storing multiple pig outputs using CSVExcelStorage > --------------------------------------------------------------------- > > Key: PIG-4943 > URL: https://issues.apache.org/jira/browse/PIG-4943 > Project: Pig > Issue Type: Bug > Components: piggybank > Affects Versions: 0.14.0 > Reporter: sarath > Priority: Minor > > I have a script which stores 2 relations with different schema using CSVExcelStorage. > The issue which i see is that the script picks up the last store function and takes the schema in that and puts it for all store functions , overriding the previous store schemas. > My Sample Script Looks like this :-- > ============================================================= > masterInput = load 'hbase://xyz' using org.apache.pig.backend.hadoop.hbase.HBaseStorage( > 'f:a,f:b,f:c,f:d') > as (a,b,c,d); > input2 = foreach masterInput > generate > a,b; > input3 = foreach masterInput > generate > c,d; > store input2 into '/dir/ab' > using org.apache.pig.piggybank.storage.CSVExcelStorage('\t','YES_MULTILINE', 'UNIX', 'WRITE_OUTPUT_HEADER'); > store input3 into '/dir/cd' > using org.apache.pig.piggybank.storage.CSVExcelStorage('\t','YES_MULTILINE', 'UNIX', 'WRITE_OUTPUT_HEADER'); > ============================================================= > Where *a,b,c,d* are my headers in my source file > Expected Output : > ||file 1|| > |a|b > |10|20 > ||file 2|| > |c|d > |30|40 > Actual Output : > ||file 1|| > |c|d > |10|20 > ||file 2|| > |c|d > |30|40 -- This message was sent by Atlassian JIRA (v6.3.4#6332)