spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "SK (JIRA)" <>
Subject [jira] [Commented] (SPARK-3610) History server log name should not be based on user input
Date Wed, 24 Sep 2014 00:03:33 GMT


SK commented on SPARK-3610:

I made the following change to EventLoggingListener.scala, L57 and this took care of the problem
(basically just removed parentheses and commas). So I am now able to access the app details
on the history server. Thanks.

private val name = appName.replaceAll("[ :/]", "-").replaceAll("[${}'\"]", "_")
    .toLowerCase + "-" + System.currentTimeMillis
  val logDir = Utils.resolveURI(logBaseDir) + "/" + name.stripSuffix("/")

> History server log name should not be based on user input
> ---------------------------------------------------------
>                 Key: SPARK-3610
>                 URL:
>             Project: Spark
>          Issue Type: Bug
>    Affects Versions: 1.1.0
>            Reporter: SK
>            Priority: Critical
> Right now we use the user-defined application name when creating the logging file for
the history server. We should use some type of GUID generated from inside of Spark instead
of allowing user input here. It can cause errors if users provide characters that are not
valid in filesystem paths.
> Original bug report:
> {quote}
> The default log files for the Mllib examples use a rather long naming convention that
includes special characters like parentheses and comma.For e.g. one of my log files is named
> When I click on the program on the history server page (at port 18080), to view the detailed
application logs, the history server crashes and I need to restart it. I am using Spark 1.1
on a mesos cluster.
> I renamed the  log file by removing the special characters and  then it loads up correctly.
I am not sure which program is creating the log files. Can it be changed so that the default
log file naming convention does not include  special characters? 
> {quote}

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message