hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Viraj Bhat (JIRA)" <j...@apache.org>
Subject [jira] Created: (PIG-1377) Pig/Zebra fails without proper error message when the mapred.jobtracker.maxtasks.per.job exceeds threshold
Date Tue, 13 Apr 2010 21:01:50 GMT
Pig/Zebra fails without proper error message when the mapred.jobtracker.maxtasks.per.job exceeds
threshold
----------------------------------------------------------------------------------------------------------

                 Key: PIG-1377
                 URL: https://issues.apache.org/jira/browse/PIG-1377
             Project: Pig
          Issue Type: Bug
          Components: impl
    Affects Versions: 0.6.0, 0.7.0
            Reporter: Viraj Bhat


I have a Zebra script which generates huge amount of mappers around 400K. The mapred.jobtracker.maxtasks.per.job
is currently set at 200k. The job fails at the initialization phase. It is very hard to find
out the cause.

We need a way to report the right error message to users. Unfortunately for Pig to get this
error in the backend, Map Reduce Jira: https://issues.apache.org/jira/browse/MAPREDUCE-1049
needs to be fixed.
{code}

-- Sorted format
%set default_parallel 100;
raw = load '/user/viraj/generated/raw/zebra-sorted/20100203'
                USING org.apache.hadoop.zebra.pig.TableLoader('', 'sorted')
                as (id,
                        timestamp,
                        code,
                        ip,
                        host,
                        reference,
                        type,
                        flag,
                        params : map[]
                );
describe raw;
user_events = filter raw by id == 'viraj';
describe user_events;
dump user_events;
sorted_events = order user_events by id, timestamp;
dump sorted_events;
store sorted_events into 'finalresult';
{code}

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message