hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (JIRA)" <j...@apache.org>
Subject [jira] Updated: (PIG-188) There seems to be some mismatches between the actual stderr log and what I expected
Date Tue, 08 Apr 2008 01:47:24 GMT

     [ https://issues.apache.org/jira/browse/PIG-188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Arun C Murthy updated PIG-188:
------------------------------

    Attachment: PIG-188_0_20080407.patch

Looks like PIG-55 broke the feature where the InputSplit was displayed correctly in the logs...
fixed now.

> There seems to be some mismatches between the actual stderr log and what I expected
> -----------------------------------------------------------------------------------
>
>                 Key: PIG-188
>                 URL: https://issues.apache.org/jira/browse/PIG-188
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Xu Zhang
>            Assignee: Arun C Murthy
>         Attachments: PIG-188_0_20080407.patch, PigLoggingTest.cpp
>
>
> With the following Pig script, I got streaming logs as shown below.  The job for running
this script is job_200804041056_0182.  What PigLoggingTest does in this case is simply take
tab delimited lines from STDIN and then output them to SDTOUT as tab delimited lines (so the
same line comes in and out of PigLogginTest) after spitting out 10 STDERR messages.  Also
as shown in the UI of job_200804041056_0181, there were a total of 21 tasks (1 map and 20
reduces).
> From all these, I would expect the number of input records and output records to match
in the log.  Also, I would expect there to be 26 logs.  In addition, since there was no error
when running the script, all exit code should 0.
> However, there are actually only  6 logs.  The number of input records and output records
does not match.  The logs show that some of the tasks exit with -127.
> In addition, the Input-split *** values in the logs do not make much sense to me:
> {quote}
> Input-split file: null
> Input-split start-offset: -1
> Input-split length: -1
> {quote}
> Here is Pig script:
> {code}
> define X `PigLoggingTest 10 t` ship('./cplusplus/PigLoggingTest') stderr('logging_test_1');
> A = load '/user/pig/tests/data/singlefile/studenttab10k' as (name, age, gpa);
> B = stream A through X;
> store B into 'logging_test_1';
> C = load 'logging_test_1/_logs/logging_test_1';
> store C into 'results_26';
> {code}
> Here are the logs:
> {noformat}
> ===== Task Information Header =====
> Command: PigLoggingTest 10 t 
> Start time: Fri Apr 04 19:18:44 PDT 2008
> Input-split file: null
> Input-split start-offset: -1
> Input-split length: -1
> =====          * * *          =====
> This is stderr message number 1
> This is stderr message number 2
> This is stderr message number 3
> This is stderr message number 4
> This is stderr message number 5
> This is stderr message number 6
> This is stderr message number 7
> This is stderr message number 8
> This is stderr message number 9
> This is stderr message number 10
> ===== Task Information Footer =====
> End time: Fri Apr 04 19:18:45 PDT 2008
> Exit code: 0
> Input records: 10000
> Input bytes: 1898380 bytes 
> Output records: 4
> Output bytes: 219446 bytes (stdout using org.apache.pig.builtin.BinaryStorage)
> =====          * * *          =====
> ===== Task Information Header =====
> Command: PigLoggingTest 10 t 
> Start time: Fri Apr 04 19:31:34 PDT 2008
> Input-split file: null
> Input-split start-offset: -1
> Input-split length: -1
> =====          * * *          =====
> This is stderr message number 1
> This is stderr message number 2
> This is stderr message number 3
> This is stderr message number 4
> This is stderr message number 5
> This is stderr message number 6
> This is stderr message number 7
> This is stderr message number 8
> This is stderr message number 9
> This is stderr message number 10
> ===== Task Information Footer =====
> End time: Fri Apr 04 19:31:36 PDT 2008
> Exit code: 0
> Input records: 10000
> Input bytes: 1898380 bytes 
> Output records: 4
> Output bytes: 219446 bytes (stdout using org.apache.pig.builtin.BinaryStorage)
> =====          * * *          =====
> ===== Task Information Header =====
> Command: ./cplusplus/PigLoggingTest 10 t 
> Start time: Fri Apr 04 10:11:22 PDT 2008
> Input-split file: null
> Input-split start-offset: -1
> Input-split length: -1
> =====          * * *          =====
> ===== Task Information Footer =====
> End time: Fri Apr 04 10:11:22 PDT 2008
> Exit code: -127
> Input records: 747
> Input bytes: 141796 bytes 
> Output records: 0
> Output bytes: 0 bytes (stdout using org.apache.pig.builtin.BinaryStorage)
> =====          * * *          =====
> ===== Task Information Header =====
> Command: ./cplusplus/PigLoggingTest 10 t 
> Start time: Fri Apr 04 10:11:28 PDT 2008
> Input-split file: null
> Input-split start-offset: -1
> Input-split length: -1
> =====          * * *          =====
> ===== Task Information Footer =====
> End time: Fri Apr 04 10:11:28 PDT 2008
> Exit code: -127
> Input records: 747
> Input bytes: 141796 bytes 
> Output records: 0
> Output bytes: 0 bytes (stdout using org.apache.pig.builtin.BinaryStorage)
> =====          * * *          =====
> ===== Task Information Header =====
> Command: ./cplusplus/PigLoggingTest 10 t 
> Start time: Fri Apr 04 10:11:32 PDT 2008
> Input-split file: null
> Input-split start-offset: -1
> Input-split length: -1
> =====          * * *          =====
> ===== Task Information Footer =====
> End time: Fri Apr 04 10:11:33 PDT 2008
> Exit code: -127
> Input records: 747
> Input bytes: 141796 bytes 
> Output records: 0
> Output bytes: 0 bytes (stdout using org.apache.pig.builtin.BinaryStorage)
> =====          * * *          =====
> ===== Task Information Header =====
> Command: ./cplusplus/PigLoggingTest 10 t 
> Start time: Fri Apr 04 10:11:37 PDT 2008
> Input-split file: null
> Input-split start-offset: -1
> Input-split length: -1
> =====          * * *          =====
> ===== Task Information Footer =====
> End time: Fri Apr 04 10:11:37 PDT 2008
> Exit code: -127
> Input records: 747
> Input bytes: 141796 bytes 
> Output records: 0
> Output bytes: 0 bytes (stdout using org.apache.pig.builtin.BinaryStorage)
> =====          * * *          =====
> {noformat}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message