predictionio-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Donald Szeto <don...@apache.org>
Subject Re: Issue with loading dependencies and jars
Date Tue, 06 Mar 2018 23:08:27 GMT
Hi Shane,

Do you know the version of hadoop-aws.jar and aws-java-sdk.jar that you are
using?

You are also right that you can modify the class path in
bin/compute-classpath.sh as a short term fix. The current order is
following the output of your target system's `ls`, so the order is not
guaranteed like you speak. Right before the last line (echo $CLASSPATH),
you can add a line to make sure that the JAR you want to be loaded first is
at the very beginning.

The long term fix would be matching the hadoop-aws.jar, aws-java-sdk.jar,
and PIO's S3 storage JAR version to have a binary compatible transitive
dependency tree. This could be quite time consuming though.

Regards,
Donald

On Mon, Mar 5, 2018 at 6:57 AM, Shane Johnson <shanewaldenjohnson@gmail.com>
wrote:

> Hi team,
>
> I have been working with Mars to get a little more insight to an error I
> am seeing. After narrowing down the issue, I am trying to find a way to
> force the order of loading the jars. I am a novice in Scala and am looking
> for a way to continue leveraging PredictionIO on Heroku but have run into a
> roadblock as I believe ordering the jars would be a change to PredictionIO
> itself. I would greatly appreciate your expertise and guidance.
>
> Error:
>
> Exception
>
>  in thread "main" java.lang.NoSuchMethodError: com.amazonaws.services.s3.transfer.TransferManager.<init>(Lcom/amazonaws/services/s3/AmazonS3;Ljava/util/concurrent/ThreadPoolExecutor;)V
>
> 2018-03-02T02:49:42.245083+00:00 app[release.7253]: 	at org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:287)
>
> 2018-03-02T02:49:42.245086+00:00 app[release.7253]: 	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2669)
>
> 2018-03-02T02:49:42.245087+00:00 app[release.7253]: 	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94)
>
> 2018-03-02T02:49:42.245088+00:00 app[release.7253]: 	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2703)
>
> 2018-03-02T02:49:42.245090+00:00 app[release.7253]: 	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2685)
>
>
> Context:
>
>    - I am using the predictionIO buildpack on heroku and using version 12.
>    - I am using S3 to Persist the model objects as I am using the
>    PAlgorithm vs L2LAlgorithm.
>    - I see an intermittent error when the Algorithm.scala file tries to
>    connect to S3. Sometimes it works and sometime is does not.
>    - When it fails I get the message above - I think I have narrowed this
>    down to the ordering of loading the jars.
>
> *I have done a number of tests and can see that the problem always occurs
> with a certain order of jars. This issue appears to be intermittent and
> could be solved by forcing the order of the jars. I am assuming this could
> be a straightforward fix, *
> *I believe it is in the compute-classpath.sh. Perhaps it's something I can
> even do on my end as an end user but I don't believe that to be the case.*
>
> *Please let me know if this is something that can be changed and committed
> or if I can contribute in someway. Again, I am fairly new to Scala and big
> data infrastructure so I am intimidated by the thought of making the change
> myself but we are at a roadblock:)*
>
> *When aws-java-sdk.jar is loaded before
> (hadoop-aws.jar and pio-data-s3-assembly-0.12.0-incubating.jar) it is
> always successful. If it is loaded after it fails with the message above.*
>
> *Success Examples:*
>
> --jars
>   file:/app/PredictionIO-dist/lib/spark/aws-java-sdk.jar,
>   file:/app/PredictionIO-dist/lib/spark/hadoop-aws.jar,
>   file:/app/PredictionIO-dist/lib/spark/pio-data-s3-assembly-
> 0.12.0-incubating.jar,
>   ...
>
> --jars
>   file:/app/PredictionIO-dist/lib/spark/aws-java-sdk.jar,
>   file:/app/PredictionIO-dist/lib/spark/pio-data-s3-assembly-
> 0.12.0-incubating.jar,
>   file:/app/PredictionIO-dist/lib/spark/hadoop-aws.jar,
>   ...
>
> *Failure Examples:*
>
> --jars
>   file:/app/PredictionIO-dist/lib/spark/hadoop-aws.jar,
>   file:/app/PredictionIO-dist/lib/spark/pio-data-s3-assembly-
> 0.12.0-incubating.jar,
>   file:/app/PredictionIO-dist/lib/spark/aws-java-sdk.jar,
>   ...
>
> --jars
>   file:/app/PredictionIO-dist/lib/spark/pio-data-s3-assembly-
> 0.12.0-incubating.jar,
>   file:/app/PredictionIO-dist/lib/spark/hadoop-aws.jar,
>   file:/app/PredictionIO-dist/lib/spark/aws-java-sdk.jar,
>   ...
>
>
> Thank you again for your help!
>
> *Shane Johnson | 801.360.3350 <(801)%20360-3350>*
> LinkedIn <https://www.linkedin.com/in/shanewjohnson> | Facebook
> <https://www.facebook.com/shane.johnson.71653>
>

Mime
View raw message