Mailing-List: contact dev-help@pig.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@pig.apache.org
Date: Fri, 15 Apr 2011 19:02:05 +0000 (UTC)
From: "Olga Natkovich (JIRA)" <jira@apache.org>
To: pig-dev@hadoop.apache.org
Message-ID: 
 <1093991774.61005.1302894125752.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: <3354267.205291296003644275.JavaMail.jira@thor>
Subject: [jira] [Commented] (PIG-1822) Need to document how types work in
 Pig
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/PIG-1822?page=3Dcom.atlassian.j=
ira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D130203=
98#comment-13020398 ]=20

Olga Natkovich commented on PIG-1822:
-------------------------------------

Please, append the following subsection to the end of Schemas section:

How Pig Handles Schema

As you can see from above,  with a few exceptions, Pig can infer the schema=
 of a relationship upfront. You can see the schema of particular relation v=
ia describe. Pig enforces this computed schema during the actually computat=
ion by casting the input data to the expected data type. If the process is =
successful, the results are returned to the user; otherwise, a warning will=
 be generated for each record that failed to convert. Note that Pig does no=
t know upfront the type of the actually data and will determine this and pe=
rform the right conversion on the fly.

Having a deterministic schema is very powerful; however, sometimes it comes=
 at the cost of performance. Consider the following example:

A =3D load =E2=80=98input=E2=80=99 as (x, y, z);
B =3D foreach A generate x+y;

If you do describe on B, you will see a single column of type double. This =
is because Pig makes the safest choice and takes the largest numeric type w=
hen the schema is not know. In practice, the input data can be containing i=
nteger values; however, Pig will cast the data to double and make sure that=
 a double result is returned.

If the schema of a relationship can=E2=80=99t be inferred, Pig will just us=
e the runtime data as is and propagate it through the pipeline.


> Need to document how types work in Pig
> --------------------------------------
>
>                 Key: PIG-1822
>                 URL: https://issues.apache.org/jira/browse/PIG-1822
>             Project: Pig
>          Issue Type: Improvement
>          Components: documentation
>            Reporter: Olga Natkovich
>            Assignee: Olga Natkovich
>             Fix For: 0.9.0
>
>
> What is static and what is dynamic.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira