spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Owen (JIRA)" <>
Subject [jira] [Commented] (SPARK-18933) Different log output between Terminal screen and stderr file
Date Fri, 30 Dec 2016 09:42:58 GMT


Sean Owen commented on SPARK-18933:

stdout and stderr exist for all processes. You can redirect them as you like.

> Different log output between Terminal screen and stderr file
> ------------------------------------------------------------
>                 Key: SPARK-18933
>                 URL:
>             Project: Spark
>          Issue Type: Bug
>          Components: Deploy, Documentation, Web UI
>    Affects Versions: 1.6.3
>         Environment: Yarn mode and standalone mode
>            Reporter: Sean Wong
>   Original Estimate: 612h
>  Remaining Estimate: 612h
> First of all, I use the default in the Spark conf/
> But I found that the log output(e.g., INFO) is different between Terminal screen and
stderr File. Some INFO logs exist in both of them. Some INFO logs exist in either of them.
Why this happens? Is it supposed that the output logs are same between the terminal screen
and stderr file? 
> Then I did a Test. I modified the source code in SparkContext.scala and add one line
log code "logInfo("This is textFile")" in the textFile function. However, after running an
application, I found the log "This is textFile" shown in the terminal screen. no such log
in the stderr file. I am not sure if this is a bug. So, hope you can solve this question.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message