hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "XIAOBING ZHOU" <xzho...@gmail.com>
Subject Re: Review Request 31333: Killed job is not display correctly by ATS hook failure
Date Tue, 24 Feb 2015 01:19:13 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31333/
-----------------------------------------------------------

(Updated Feb. 24, 2015, 1:19 a.m.)


Review request for hive.


Summary (updated)
-----------------

 Killed job is not display correctly by ATS hook failure


Repository: hive-git


Description
-------

PROBLEM: After killing the job throug CLI by crtl+c, we are seeing the following exception.
Also the job is shown as running with timer still counting on the UI

Interrupting... Be patient, this might take some time. 
Press Ctrl+C again to kill JVM 
Map 1: 20/40 Reducer 2: 0/3 Reducer 3: 0/1 
Map 1: 22/40 Reducer 2: 0/3 Reducer 3: 0/1 
Map 1: 25/40 Reducer 2: 0/3 Reducer 3: 0/1 
Map 1: 26/40 Reducer 2: 0/3 Reducer 3: 0/1 
Exiting the JVM 
Trying to shutdown DAG 
Shutting down tez session. 
Status: Killed 
Job received Kill while in RUNNING state. 
Vertex killed, vertexName=Reducer 3, vertexId=vertex_1421602925091_3868_2_00, diagnostics=Vertex
received Kill while in RUNNING state., Vertex killed due to user-initiated job kill. failedTasks:0

Vertex killed, vertexName=Map 1, vertexId=vertex_1421602925091_3868_2_02, diagnostics=Vertex
received Kill while in RUNNING state., Vertex killed due to user-initiated job kill. failedTasks:0

Vertex killed, vertexName=Reducer 2, vertexId=vertex_1421602925091_3868_2_01, diagnostics=Vertex
received Kill while in RUNNING state., Vertex killed due to user-initiated job kill. failedTasks:0

DAG killed due to user-initiated kill. failedVertices:0 killedVertices:3 
FAILED: Hive Internal Error: java.lang.IllegalStateException(Shutdown in progress) 
java.lang.IllegalStateException: Shutdown in progress 
at java.lang.ApplicationShutdownHooks.add(ApplicationShutdownHooks.java:66) 
at java.lang.Runtime.addShutdownHook(Runtime.java:211) 
at org.apache.hadoop.hive.ql.hooks.ATSHook.<init>(ATSHook.java:71) 
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)

at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

at java.lang.reflect.Constructor.newInstance(Constructor.java:526) 
at java.lang.Class.newInstance(Class.java:374) 
at org.apache.hadoop.hive.ql.hooks.HookUtils.getHooks(HookUtils.java:59) 
at org.apache.hadoop.hive.ql.Driver.getHooks(Driver.java:1177) 
at org.apache.hadoop.hive.ql.Driver.getHooks(Driver.java:1161) 
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1316) 
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1093) 
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:916) 
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:906) 
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268) 
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220) 
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423) 
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:793) 
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686) 
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625) 
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
at java.lang.reflect.Method.invoke(Method.java:606) 
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
hive> EL Info: 2015-01-21 16:20:51.483-ServerSession(1006268460)-EclipseLink, version:
Eclipse Persistence Services - 2.5.1.v20130918-f2b9fc5 
EL Info: connection: 2015-01-21 16:20:52.025-ServerSession(1006268460)-file:/usr/lib/hive/lib/xasecure-audit.jar_xa_server_url=jdbc:mysql://bkcttplpd036.verizon.com/xasecure_user=xalogger
login successful 
EL Info: connection: 2015-01-21 16:20:52.082-ServerSession(1006268460)-file:/usr/lib/hive/lib/xasecure-audit.jar_xa_server_url=jdbc:mysql://bkcttplpd036.verizon.com/xasecure_user=xalogger
logout successful


Diffs
-----

  ql/src/java/org/apache/hadoop/hive/ql/hooks/ATSHook.java 513a2fa 

Diff: https://reviews.apache.org/r/31333/diff/


Testing
-------


Thanks,

XIAOBING ZHOU


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message