hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shubham Chopra (JIRA)" <j...@apache.org>
Subject [jira] Updated: (PIG-443) Illustrate for the Types branch
Date Wed, 24 Sep 2008 05:07:44 GMT

     [ https://issues.apache.org/jira/browse/PIG-443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Shubham Chopra updated PIG-443:

    Attachment: pig_types_24_09.patch

* I have fixed the connection issue. I think I had misunderstood the way PigContext works.
Using a proper pigContext fixed the connection problem.

* Deducing a schema from sample data is a bit of a problem. Consider the following case:
a = load 'somefile' as (x, y, z);
b = filter a by some-condition;
c = foreach b generate x;
illustrate c;

If 'b' is an empty set, 'c' would also be an empty set. I wouldn't be able to depend on sample
data alone to deduce the schemas. I might end up using the same logic as the getSchema methods
to deduce the schema for 'c' in this case.

Also, schemas are used only while generating synthetic data and displaying the final set of

* That issue with cogroup I guess is again because of a no-schema load. Like pointed out earlier,
the augmentation phase where I try to generate synthetic data used schemas.

I have done a modification in the code. I now check for schema while reading the data and
throw an exception if I don't find one asking users to provide a schema.

> Illustrate for the Types branch
> -------------------------------
>                 Key: PIG-443
>                 URL: https://issues.apache.org/jira/browse/PIG-443
>             Project: Pig
>          Issue Type: New Feature
>    Affects Versions: types_branch
>            Reporter: Shubham Chopra
>            Assignee: Shubham Chopra
>             Fix For: types_branch
>         Attachments: pig_types_22_09.patch, pig_types_23_09.patch, pig_types_24_09.patch
> This implementation of Illustrate uses visitors to traverse the plans.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message