pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Coveney (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-2641) Create toJSON function for all complex types: tuples, bags and maps
Date Sun, 17 Feb 2013 12:41:13 GMT

    [ https://issues.apache.org/jira/browse/PIG-2641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13580178#comment-13580178
] 

Jonathan Coveney commented on PIG-2641:
---------------------------------------

Gunther's points are good ones, Russell. Should definitely be taken into account.

I was thinking about easier ways to deal with vararg issues, and I think I thought of one.
Instead of using getArgToFunc, on the front end in outputSchema, you can just verify that
the schema is homogenous and concat-able (i.e. all strings, all longs, whatever). If you want
to be really fancy, you can find the code that defines whether or not a type is coercible
and base it on the first argument (i.e. string long long could be concat-able as all strings).

I've realized now that the existence of getInputSchema largely makes the getArgToFunc construct
irrelevant except in the case where you need frontend specific information passed to the backend
via the constructor based on the type of the input (which is not the case here).

Then on the back end, you lazy initialize based on the getInputSchema schema. You can constructor
a concatenator based on the Schema which actually does the concatenating, and then just run
every argument against it (i.e. if you detect String, String, String you initialize a StringConcatenator).
There are a couple of ways you could design this, and this is where you could take into account
coercion rules.

Bam, no deep changes necessary to support varargs.
                
> Create toJSON function for all complex types: tuples, bags and maps
> -------------------------------------------------------------------
>
>                 Key: PIG-2641
>                 URL: https://issues.apache.org/jira/browse/PIG-2641
>             Project: Pig
>          Issue Type: New Feature
>          Components: piggybank
>    Affects Versions: 0.12
>         Environment: Foggy. Damn foggy.
>            Reporter: Russell Jurney
>            Assignee: Russell Jurney
>              Labels: chararray, fun, happy, input, json, output, pants, pig, piggybank,
string, wonderdog
>             Fix For: 0.12
>
>         Attachments: PIG-2641.patch
>
>   Original Estimate: 96h
>  Remaining Estimate: 96h
>
> It is a travesty that there are no UDFs in Piggybanks that, given an arbitrary Pig datatype,
return a JSON string of same. I intend to fix this problem.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message