spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From steveloughran <>
Subject [GitHub] spark pull request #21146: [SPARK-23654][BUILD][WiP] remove jets3t as a depe...
Date Tue, 24 Apr 2018 20:02:57 GMT
GitHub user steveloughran opened a pull request:

    [SPARK-23654][BUILD][WiP] remove jets3t as a dependency of spark

    ## What changes were proposed in this pull request?
    With the update of bouncy-castle JAR in Spark 2.3; jets3t doesn't work any more; hence
the s3n 
    connector to S3 is dead. Only one person has noticed so far. The hadoop s3n connector
is never going to be updated to work with a later version of jets3t, instead the code has
just been [cut from hadoop 3.x entirely.](
    This patch remove the declarations of jets3t from the POMs which include it, so it is
not being packaged up, as it has no purpose any more.
    This also updates the dev/deps dependency files so it is not expected
    It doesn't update the transitive dependencies, which highlights what transitively changes
    * javax.activation goes down from 1.1.1. to 1.1
    * bcprov-jdk15on-1.58.jar, base64-2.3.8.jar, java-xmlbuilder-1.1.jar are all missing
    That's an interesting question and why I've tagged this as WiP: what to do here? 
    * It makes sense to keep activation at the 1.1.1 level
    * The others? if they aren't being used, then removal is the cleanest choice, as it avoids
imposing JARs downstream. But their removal may cause surprises
    ## How was this patch tested?
    Seeing what jenkins has to say about the missing transitive dependencies. 

You can merge this pull request into a Git repository by running:

    $ git pull cloud/SPARK-23654-jets3t

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #21146
commit c011acc110e5de49a1215ad7bdb6054ba326013a
Author: Steve Loughran <stevel@...>
Date:   2018-03-28T09:34:11Z

    SPARK-23654 remove jets3t as a dependency of spark
    This also updates the dev/deps dependency files so it is not expected
    Change-Id: I15798710925d378de97523f7f89dbe5bd1cc8582



To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message