spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Martin (JIRA)" <>
Subject [jira] [Commented] (SPARK-6305) Add support for log4j 2.x to Spark
Date Tue, 21 Aug 2018 18:06:00 GMT


Chris Martin commented on SPARK-6305:


I've taken a look at this and I'd like to sync up to see your perspective (given that you
already looked at this).  From what I can see:

1) As you say there's a load of pom manipulation to do in order to exclude log4j1 and related
components from dependencies. 

2) There are a bunch of unit tests that rely on hooking into log4j internals in order to capture
log output and make assertions against them.

3) The org.apache.spark.internal.Logging trait has some fairly low level log4j logic.

4) There are a few file that would have to be converted to the log4j2 format.

Of these 1) is a pain but it's fairly mechanical and I would hope we could write some sort
of automated check to tell us if log4j1 is still being brought in.  2) is fairly easily solvable;
I have some code to do this.  3) worries me as this class is doing some fairly hairy stuff
and I'm not sure of the use cases- it would be good to have a chat about this. 4) is simple
enough as far as spark goes, but the fly in the ointment is that all existing user log configuration
would need to be changed.













> Add support for log4j 2.x to Spark
> ----------------------------------
>                 Key: SPARK-6305
>                 URL:
>             Project: Spark
>          Issue Type: Improvement
>          Components: Build
>            Reporter: Tal Sliwowicz
>            Priority: Minor
> log4j 2 requires replacing the slf4j binding and adding the log4j jars in the classpath.
Since there are shaded jars, it must be done during the build.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message