pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cheolsoo Park (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PIG-3515) Shell commands are limited from OS buffer
Date Sat, 19 Oct 2013 03:59:47 GMT

     [ https://issues.apache.org/jira/browse/PIG-3515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Cheolsoo Park updated PIG-3515:

       Resolution: Fixed
    Fix Version/s: 0.13.0
           Status: Resolved  (was: Patch Available)

Hi Aniket, sorry for the late reply. It looks good to me too. [~andronat], thank you for the
good catch! I didn't know this until now. lol

Since I was already running the test, I went ahead and committed the patch to trunk. I didn't
commit the test case because I thought this bug doesn't need a unit test. Please let me know
if you want me to commit the test case.

> Shell commands are limited from OS buffer
> -----------------------------------------
>                 Key: PIG-3515
>                 URL: https://issues.apache.org/jira/browse/PIG-3515
>             Project: Pig
>          Issue Type: Bug
>          Components: parser
>    Affects Versions: 0.12.0, 0.11.1
>         Environment: Centos 6.5, Mac OS X 10.8.5
>            Reporter: Anastasis Andronidis
>            Assignee: Anastasis Andronidis
>              Labels: command-line, parser
>             Fix For: 0.13.0
>         Attachments: PIG-3515.patch, PIG-3515-test.patch
> Executing shell commands in pig scripts, may stuck due to Java buffer limitations.
> Example:
> %declare VAR   `cat 100kbytes.txt`
> Produce:
> 2013-10-09 15:25:56,825 [main] INFO  org.apache.pig.tools.parameters.PreprocessorContext
- Executing command : cat 100kbytes.txt
> The execution hungs, so you have to interrupt the program.
> ^C
> 2013-10-09 15:26:10,066 [main] ERROR org.apache.pig.Main - ERROR 2999: Unexpected internal
error. Error executing shell command: cat 100kbytes.txt. Command exit with exit code of 130
> Explanation:
> The problem lies in org.apache.pig.tools.parameters.PreprocessorContext#executeShellCommand
method, line: 191 (trunk, at revision 1531874).
> We wait for the process to complete before we get all the output, but the process waits
for an indefinite amount of time, due to the fact that there is no space left in the buffer
and the output can't be delivered.
> References:
> [1] http://www.javaworld.com/javaworld/jw-12-2000/jw-1229-traps.html?page=2
> [2] http://vyvaks.wordpress.com/2006/05/27/does-runtimeexec-hangs-in-java/
> Solution:
> I attach a test case that illustrates the problem and a patch as a solution. I would
really like some comments.

This message was sent by Atlassian JIRA

View raw message