pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xuefu Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PIG-1622) DEFINE streaming options are ill defined and not properly documented
Date Mon, 25 Apr 2011 16:31:03 GMT

     [ https://issues.apache.org/jira/browse/PIG-1622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Xuefu Zhang updated PIG-1622:
-----------------------------

    Attachment: PIG-1622-1.patch

Added a minor change for a test case.

> DEFINE streaming options are ill defined and not properly documented
> --------------------------------------------------------------------
>
>                 Key: PIG-1622
>                 URL: https://issues.apache.org/jira/browse/PIG-1622
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.7.0
>            Reporter: Alan Gates
>            Assignee: Corinne Chandel
>            Priority: Minor
>             Fix For: 0.9.0
>
>         Attachments: PIG-1622-1.patch, PIG-1622.patch
>
>
> According to the documentation (http://hadoop.apache.org/pig/docs/r0.7.0/piglatin_ref2.html#DEFINE)
the syntax for DEFINE when used to define a streaming command is:
> DEFINE cmd INPUT(stdin|path) OUTPUT(stdout|stderr|path) SHIP(path [, path, ...]) CACHE
(path [, path, ...])
> However, the actual parser accepts something pretty different.  Consider the following
script:
> {code}
> define strm `wc -l` INPUT(stdin) 
>                     CACHE('/Users/gates/.vimrc#myvim') 
>                     OUTPUT(stdin)
>                     INPUT('/tmp/fred') 
>                     OUTPUT('/tmp/bob')
>                     SHIP('/Users/gates/.bashrc') 
>                     SHIP('/Users/gates/.vimrc') 
>                     CACHE('/Users/gates/.bashrc#mybash')
>                     stderr('/tmp/errors' limit 10);
> A = load '/Users/gates/test/data/studenttab10';
> B = stream A through strm;
> dump B;
> {code}
> The above actually parsers.  I see several issues here:
> # What do multiple INPUT and OUTPUT statements mean in the context of streaming?  These
should not be allowed.
> # The documentation implies an order (INPUT, OUTPUT, SHIP, CACHE) that is not enforced
by the parser.  We should either enforce the order in the parser or update the documentation.
 Most likely the latter to avoid breaking existing scripts.
> # Why are multiple SHIP and CACHE clauses allowed when each can take multiple paths?
 It seems we should only allow one of each.
> # The error clause is completely different that what is given in the documentation. 
I suspect this is a documentation error and the grammar supported by the parser here is what
we want.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message