hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-94) Pig Streaming functional spec proposal
Date Wed, 19 Mar 2008 20:48:24 GMT

    [ https://issues.apache.org/jira/browse/PIG-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12580554#action_12580554
] 

Arun C Murthy commented on PIG-94:
----------------------------------

Thanks for the review Pi... comments inline:

bq. In the parser, EXECCOMMAND doesn't seem to accept backtick escape characters. Do plan
to have it or just say backticks aren't allowed?
Yes, the plan to say that backticks aren't allowed.

bq. Agree with Alan to have a timeout mechanism. You already have got the outputBytes variable
in PigExecutableManager. Possibly just use it to monitor the movement.
Yep, as I noted earlier it's the same problem as with Hadoop Streaming. There are applications
which just collect _all_ of input data before processing it and in those cases outputBytes
isn't useful at all... lets discuss.

bq. The error handling logic by just printing out on work load nodes will not be visible to
Pig user.
The error is also collected on HDFS in the _logs sub-dir of the final output directory...
does that work?

bq. This following line in HadoopExecutableManager will not work on Windows:-
It does work on Cygwin on Windows (which is a pre-req for Hadoop).


> Pig Streaming functional spec proposal
> --------------------------------------
>
>                 Key: PIG-94
>                 URL: https://issues.apache.org/jira/browse/PIG-94
>             Project: Pig
>          Issue Type: New Feature
>            Reporter: Olga Natkovich
>            Assignee: Arun C Murthy
>         Attachments: patch.txt, PIG-94_1_1_20080304.patch, PIG-94_1_20080303.patch, PIG-94_2_0_20080317.patch,
PigStreamingTestPlan.htm
>
>
> This issue is for discussion about Pig streaming functional spec.
> http://wiki.apache.org/pig/PigStreamingFunctionalSpec

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message