hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-12199) LLAP: unnecessary preemption of the same stage
Date Thu, 15 Oct 2015 22:25:05 GMT
Sergey Shelukhin created HIVE-12199:
---------------------------------------

             Summary: LLAP: unnecessary preemption of the same stage
                 Key: HIVE-12199
                 URL: https://issues.apache.org/jira/browse/HIVE-12199
             Project: Hive
          Issue Type: Bug
            Reporter: Sergey Shelukhin


6-node cluster x 16 executors, nothing else running on it
{code}
hive> select sum(l_extendedprice * l_discount) as revenue from testing.lineitem where l_shipdate
>= '1993-01-01' and l_shipdate < '1994-01-01' ;
...
----------------------------------------------------------------------------------------------
        VERTICES      MODE        STATUS  TOTAL  COMPLETED  RUNNING  PENDING  FAILED  KILLED
 
----------------------------------------------------------------------------------------------
Map 1 ..........      llap     SUCCEEDED    176        176        0        0       0     
 8  
Reducer 2 ......      llap     SUCCEEDED      1          1        0        0       0     
 1 
{code}

those killed mappers are preempted:
{code}
015-10-15 18:07:07,154 INFO [Dispatcher thread: Central] history.HistoryEventHandler: [HISTORY][DAG:dag_1442254312093_2158_1][Event:TASK_ATTEMPT_FINISHED]:
vertexName=Map 1, taskAttemptId=attempt_1442254312093_2158_1_00_000103_0, startTime=1444946823795,
finishTime=1444946827152, timeTaken=3357, status=KILLED, errorEnum=EXTERNAL_PREEMPTION, diagnostics=Attempt
preempted, lastDataEventSourceTA=null, lastDataEventTime=0, counters=Counters: 1, org.apache.tez.common.counters.DAGCounter,
RACK_LOCAL_TASKS=1
{code}

There's no reason for mappers of the same stage to preempt each other in this case
 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message