pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cheolsoo Park (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (PIG-3515) Shell commands are limited from OS buffer
Date Fri, 18 Oct 2013 16:40:43 GMT

     [ https://issues.apache.org/jira/browse/PIG-3515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Cheolsoo Park reassigned PIG-3515:
----------------------------------

    Assignee: Anastasis Andronidis

Assigning to Anastasis.

> Shell commands are limited from OS buffer
> -----------------------------------------
>
>                 Key: PIG-3515
>                 URL: https://issues.apache.org/jira/browse/PIG-3515
>             Project: Pig
>          Issue Type: Bug
>          Components: parser
>    Affects Versions: 0.12.0, 0.11.1
>         Environment: Centos 6.5, Mac OS X 10.8.5
>            Reporter: Anastasis Andronidis
>            Assignee: Anastasis Andronidis
>              Labels: command-line, parser
>         Attachments: PIG-3515.patch, PIG-3515-test.patch
>
>
> Executing shell commands in pig scripts, may stuck due to Java buffer limitations.
> Example:
> %declare VAR   `cat 100kbytes.txt`
> Produce:
> 2013-10-09 15:25:56,825 [main] INFO  org.apache.pig.tools.parameters.PreprocessorContext
- Executing command : cat 100kbytes.txt
> The execution hungs, so you have to interrupt the program.
> ^C
> 2013-10-09 15:26:10,066 [main] ERROR org.apache.pig.Main - ERROR 2999: Unexpected internal
error. Error executing shell command: cat 100kbytes.txt. Command exit with exit code of 130
> Explanation:
> The problem lies in org.apache.pig.tools.parameters.PreprocessorContext#executeShellCommand
method, line: 191 (trunk, at revision 1531874).
> We wait for the process to complete before we get all the output, but the process waits
for an indefinite amount of time, due to the fact that there is no space left in the buffer
and the output can't be delivered.
> References:
> [1] http://www.javaworld.com/javaworld/jw-12-2000/jw-1229-traps.html?page=2
> [2] http://vyvaks.wordpress.com/2006/05/27/does-runtimeexec-hangs-in-java/
> Solution:
> I attach a test case that illustrates the problem and a patch as a solution. I would
really like some comments.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message