pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Ciemiewicz (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-1824) Support import modules in Jython UDF
Date Tue, 01 Mar 2011 17:48:37 GMT

    [ https://issues.apache.org/jira/browse/PIG-1824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13000982#comment-13000982

David Ciemiewicz commented on PIG-1824:

I don't think it is appropriate to just leave this up to the end user to figure this stuff

Especially when the errors won't be discovered until the user attempts to run the code on
the grid
then must decipher the errors
then must track down the individual dependency files
then must try to figure out how to ship the necessary files
then must try to track down why it still doesn't work because the import files contained dependencies
on imported files
then must track down the subsequent dependencies
then ...

If jython itself does not provide hooks to enumerate all dependencies after parsing, would
it be possible to build a tool which recurses the imports and then provides information to
the end user on how to package all the dependencies for ship (or better just does it).

Couldn't this be a requirement for all language bindings to provide a method or script for
enumerating all dependent files, even if the interpreter implementation in Java doesn't provide
this functionality natively?

> Support import modules in Jython UDF
> ------------------------------------
>                 Key: PIG-1824
>                 URL: https://issues.apache.org/jira/browse/PIG-1824
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Richard Ding
>            Assignee: Richard Ding
> Currently, Jython UDF script doesn't support Jython import statement as in the following
> {code}
> #!/usr/bin/python
> import re
> @outputSchema("y:bag{t:tuple(word:chararray)}")
> def strsplittobag(content,regex):
>         return re.compile(regex).split(content)
> {code}
> Can Pig automatically locate the Jython module file and ship it to the backend? Or should
we add a ship clause to let user explicitly specify the module to ship? 

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message