Mailing-List: contact pig-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: pig-dev@hadoop.apache.org
Message-ID: <939756797.1239857115657.JavaMail.jira@brutus>
Date: Wed, 15 Apr 2009 21:45:15 -0700 (PDT)
From: "George Mavromatis (JIRA)" <jira@apache.org>
To: pig-dev@hadoop.apache.org
Subject: [jira] Created: (PIG-767) Schema reported from DESCRIBE and actual
 schema of inner bags are different.
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Schema reported from DESCRIBE and actual schema of inner bags are different.
----------------------------------------------------------------------------

                 Key: PIG-767
                 URL: https://issues.apache.org/jira/browse/PIG-767
             Project: Pig
          Issue Type: Bug
            Reporter: George Mavromatis
             Fix For: 0.2.0


The following script:

urlContents = LOAD 'inputdir' USING BinStorage() AS (url:bytearray, pg:bytearray);
-- describe and dump are in-sync
DESCRIBE urlContents;
DUMP urlContents;

urlContentsG = GROUP urlContents BY url;
DESCRIBE urlContentsG;

urlContentsF = FOREACH urlContentsG GENERATE group,urlContents.pg;

DESCRIBE urlContentsF;
DUMP urlContentsF;


Prints for the DESCRIBE commands:

urlContents: {url: chararray,pg: chararray}
urlContentsG: {group: chararray,urlContents: {url: chararray,pg: chararray}}
urlContentsF: {group: chararray,pg: {pg: chararray}}

The reported schemas for urlContentsG and urlContentsF are wrong. They are also against the section "Schemas for Complex Data Types" in http://wiki.apache.org/pig-data/attachments/FrontPage/attachments/plrm.htm#_Schemas.

As expected, actual data observed from DUMP urlContentsG and DUMP urlContentsF do contain the tuple inside the inner bags.

The correct schema for urlContentsG is:  {group: chararray,urlContents: {t1:(url: chararray,pg: chararray)}}

This may sound like a technicality, but it isn't. For instance, a UDF that assumes an inner bag of {chararray} will not work with {(chararray)}. 


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.