hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Viraj Bhat (JIRA)" <j...@apache.org>
Subject [jira] Created: (PIG-1243) Passing Complex map types to and from streaming causes a problem
Date Thu, 18 Feb 2010 23:27:28 GMT
Passing Complex map types to and from streaming causes a problem

                 Key: PIG-1243
                 URL: https://issues.apache.org/jira/browse/PIG-1243
             Project: Pig
          Issue Type: Bug
    Affects Versions: 0.6.0
            Reporter: Viraj Bhat
             Fix For: 0.7.0

I have a program which generates different types of Maps fields and stores it into PigStorage.
A = load '/user/viraj/three.txt' using PigStorage();

B = foreach A generate ['a'#'12'] as b:map[], ['b'#['c'#'12']] as c, ['c'#{(['d'#'15']),(['e'#'16'])}]
as d;

store B into '/user/viraj/pigtest' using PigStorage();

Now I test the previous output in the below script to make sure I have the right results.
I also pass this data to a Perl script and I observe that the complex Map types I have generated,
are lost when I get the result back.

DEFINE CMD `simple.pl` SHIP('simple.pl');

A = load '/user/viraj/pigtest' using PigStorage() as (simpleFields, mapFields, mapListFields);

B = foreach A generate $0, $1, $2;

dump B;

C = foreach A generate  (chararray)simpleFields#'a' as value, $0,$1,$2;

D = stream C through CMD as (a0:map[], a1:map[], a2:map[]);

dump D;

dumping B results in:


dumping D results in:


The Perl script used here is:

use warnings;

use strict;

while(<>) {




Is there an issue with handling of complex Map fields within streaming? How can I fix this
to obtain the right result?


This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message