spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Liang-Chi Hsieh (JIRA)" <>
Subject [jira] [Created] (SPARK-4913) Fix incorrect event log path
Date Sun, 21 Dec 2014 16:15:13 GMT
Liang-Chi Hsieh created SPARK-4913:

             Summary: Fix incorrect event log path
                 Key: SPARK-4913
             Project: Spark
          Issue Type: Bug
            Reporter: Liang-Chi Hsieh

SPARK-2261 uses a single file to log events for an app. `eventLogDir` in `ApplicationDescription`
is replaced with `eventLogFile`. However, `ApplicationDescription` in `SparkDeploySchedulerBackend`
is initialized with `SparkContext`'s `eventLogDir`. It is just the log directory, not the
actual log file path. `Master.rebuildSparkUI` can not correctly rebuild a new SparkUI for
the app.

Because the `ApplicationDescription` is remotely registered with `Master` and the app's id
is then generated in `Master`, we can not get the app id in advance before registration. So
the received description needs to be modified with correct `eventLogFile` value.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message