hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sahil Takiar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-13517) Hive logs in Spark Executor and Driver should show thread-id.
Date Tue, 21 Mar 2017 02:20:41 GMT

    [ https://issues.apache.org/jira/browse/HIVE-13517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15933988#comment-15933988
] 

Sahil Takiar commented on HIVE-13517:
-------------------------------------

[~kellyzly] yes thats correct, but I think the default {{log4j.properties}} file in Spark
is https://github.com/apache/spark/blob/master/conf/log4j.properties.template - which doesn't
contain %t (thread name) in the layout pattern.

I think the goal of this JIRA is to specify a different {{log4j.properties}} for Spark executors
such that the thread name is in the layout pattern by default. This should make debugging
HoS jobs much easier.

[~szehon] is my understanding correct?

> Hive logs in Spark Executor and Driver should show thread-id.
> -------------------------------------------------------------
>
>                 Key: HIVE-13517
>                 URL: https://issues.apache.org/jira/browse/HIVE-13517
>             Project: Hive
>          Issue Type: Bug
>          Components: Spark
>    Affects Versions: 1.2.1, 2.0.0
>            Reporter: Szehon Ho
>            Assignee: liyunzhang_intel
>         Attachments: executor-driver-log.PNG
>
>
> In Spark, there might be more than one task running in one executor. Similarly, there
may be more than one thread running in Driver.
> This makes debugging through the logs a nightmare. It would be great if there could be
thread-ids in the logs.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message