hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sahil Takiar (JIRA)" <>
Subject [jira] [Updated] (HIVE-13517) Hive logs in Spark Executor and Driver should show thread-id.
Date Thu, 23 Mar 2017 17:17:41 GMT


Sahil Takiar updated HIVE-13517:
    Attachment: HIVE-13517.2.patch

Attaching updated patch that adds some testing. Instead of explicitly copying the {{}}
file from {{data/conf/spark/}} into the Spark distribution, the new config {{hive.spark.log4j.file}}
is now used.

I'm not 100% sure this was the original intention of this JIRA, but it is a useful feature.
Users can now add custom log4j files without modifying the Spark installation.

In a future patch, we could add a better default log4j properties file, which has the thread-id.

[~kellyzly] what do you think of this patch?

> Hive logs in Spark Executor and Driver should show thread-id.
> -------------------------------------------------------------
>                 Key: HIVE-13517
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>          Components: Spark
>    Affects Versions: 1.2.1, 2.0.0
>            Reporter: Szehon Ho
>            Assignee: Sahil Takiar
>         Attachments: executor-driver-log.PNG, HIVE-13517.1.patch, HIVE-13517.2.patch
> In Spark, there might be more than one task running in one executor. Similarly, there
may be more than one thread running in Driver.
> This makes debugging through the logs a nightmare. It would be great if there could be
thread-ids in the logs.

This message was sent by Atlassian JIRA

View raw message