spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From l...@apache.org
Subject spark git commit: [SQL] In InsertIntoFSBasedRelation.insert, log cause before abort job/task.
Date Wed, 13 May 2015 15:36:13 GMT
Repository: spark
Updated Branches:
  refs/heads/master 10c546e9d -> b061bd517


[SQL] In InsertIntoFSBasedRelation.insert, log cause before abort job/task.

We need to add a log entry before calling `abortTask`/`abortJob`. Otherwise, an exception
from `abortTask`/`abortJob` will shadow the real cause.

cc liancheng

Author: Yin Huai <yhuai@databricks.com>

Closes #6105 from yhuai/logCause and squashes the following commits:

8dfe0d8 [Yin Huai] Log cause.


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/b061bd51
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/b061bd51
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/b061bd51

Branch: refs/heads/master
Commit: b061bd517a3dc26e7f37a334f49c3465d98334c6
Parents: 10c546e
Author: Yin Huai <yhuai@databricks.com>
Authored: Wed May 13 23:36:19 2015 +0800
Committer: Cheng Lian <lian@databricks.com>
Committed: Wed May 13 23:36:19 2015 +0800

----------------------------------------------------------------------
 .../src/main/scala/org/apache/spark/sql/sources/commands.scala     | 2 ++
 1 file changed, 2 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/b061bd51/sql/core/src/main/scala/org/apache/spark/sql/sources/commands.scala
----------------------------------------------------------------------
diff --git a/sql/core/src/main/scala/org/apache/spark/sql/sources/commands.scala b/sql/core/src/main/scala/org/apache/spark/sql/sources/commands.scala
index fe8be5b..a294297 100644
--- a/sql/core/src/main/scala/org/apache/spark/sql/sources/commands.scala
+++ b/sql/core/src/main/scala/org/apache/spark/sql/sources/commands.scala
@@ -121,6 +121,7 @@ private[sql] case class InsertIntoFSBasedRelation(
       writerContainer.commitJob()
       relation.refresh()
     } catch { case cause: Throwable =>
+      logError("Aborting job.", cause)
       writerContainer.abortJob()
       throw new SparkException("Job aborted.", cause)
     }
@@ -143,6 +144,7 @@ private[sql] case class InsertIntoFSBasedRelation(
         }
         writerContainer.commitTask()
       } catch { case cause: Throwable =>
+        logError("Aborting task.", cause)
         writerContainer.abortTask()
         throw new SparkException("Task failed while writing rows.", cause)
       }


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message