hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Szehon Ho (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-7668) Improve error logging for spark test crashes
Date Sat, 09 Aug 2014 00:34:12 GMT

     [ https://issues.apache.org/jira/browse/HIVE-7668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Szehon Ho updated HIVE-7668:
----------------------------

    Description: 
While working on HIVE-7665, I noticed that spark local cluster will crash entire JVM on any
dependency version mismatch, either due to native code linkage errors if you are unlucky,
or gracefully (see spark's ExecutorUncaughtExceptionHandler.scala) most of the time.

To debug those issues, it would be nice to log a thread-dump upon normal JVM exits by spark
by ExecutorUncaughtExceptionHandler, as its unexpected by Hive's test framework.  One way
is to add a shutdownHook that does this.

  was:
While doing HIVE-7665, I noticed that spark local cluster will crash on any dependency version
mismatch, either due to native code link errors (like in jetty) if unlucky, or gracefully
(see spark's ExecutorUncaughtExceptionHandler.scala).

To debug those issues, it would be nice to log a thread-dump upon normal JVM exits by spark,
as its unexpected by Hive's test framework.  One way is to add a shutdownHook that does this.


> Improve error logging for spark test crashes
> --------------------------------------------
>
>                 Key: HIVE-7668
>                 URL: https://issues.apache.org/jira/browse/HIVE-7668
>             Project: Hive
>          Issue Type: Bug
>          Components: Spark, Testing Infrastructure
>            Reporter: Szehon Ho
>
> While working on HIVE-7665, I noticed that spark local cluster will crash entire JVM
on any dependency version mismatch, either due to native code linkage errors if you are unlucky,
or gracefully (see spark's ExecutorUncaughtExceptionHandler.scala) most of the time.
> To debug those issues, it would be nice to log a thread-dump upon normal JVM exits by
spark by ExecutorUncaughtExceptionHandler, as its unexpected by Hive's test framework.  One
way is to add a shutdownHook that does this.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message