predictionio-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mars Hall <mars.h...@salesforce.com>
Subject Re: Issue with loading dependencies and jars
Date Fri, 09 Mar 2018 19:28:09 GMT
At this point, I'm just searching the internet for "Not a valid command:
assemblyPackageDependency" errors, which I image you are too.

Does this error occur locally when you `pio build` this engine?

Are there any diffs in your local code?

How are you running locally? Are you using the buildpack's "local dev"
setup?

On Fri, Mar 9, 2018 at 6:48 PM, Shane Johnson <shane@liftiq.com> wrote:

> Mars, to test what may be happening I just reverted to the original
> buildpack (https://github.com/heroku/predictionio-buildpack.git) and
> removed the variable PREDICTIONIO_DIST_URL and I am still getting the
> same error. I don't know where I would have introduced this. Have you seen
> it before, perhaps this is what is causing the custom dist to not be used?
> I'm guessing the build failed before and I was overlooking that. Now I need
> to find out why the build is failing:)
>
>  Quietly logging. (Set `PIO_VERBOSE=true` for detailed build log.)
>
>        [INFO] [Engine$] Using command '/tmp/build_28c7923a4f807cd0eaa37b606989fe11/lift-iq-score-f7694693324fc7acebdc6effb0134c218eb7434e/PredictionIO-dist/sbt/sbt'
at /tmp/build_28c7923a4f807cd0eaa37b606989fe11/lift-iq-score-f7694693324fc7acebdc6effb0134c218eb7434e
to build.
>
>        [INFO] [Engine$] If the path above is incorrect, this process will fail.
>
>        [INFO] [Engine$] Uber JAR disabled. Making sure lib/pio-assembly-0.12.0-incubating.jar
is absent.
>
>        [INFO] [Engine$] Going to run: /tmp/build_28c7923a4f807cd0eaa37b606989fe11/lift-iq-score-f7694693324fc7acebdc6effb0134c218eb7434e/PredictionIO-dist/sbt/sbt
 package assemblyPackageDependency in /tmp/build_28c7923a4f807cd0eaa37b606989fe11/lift-iq-score-f7694693324fc7acebdc6effb0134c218eb7434e
>
>        [ERROR] [Engine$] [error] Not a valid command: assemblyPackageDependency
>
>        [ERROR] [Engine$] [error] Not a valid key: assemblyPackageDependency (similar:
sbtDependency)
>
>        [ERROR] [Engine$] [error] assemblyPackageDependency
>
>        [ERROR] [Engine$] [error]                          ^
>
>        [ERROR] [Engine$] Return code of build command: /tmp/build_28c7923a4f807cd0eaa37b606989fe11/lift-iq-score-f7694693324fc7acebdc6effb0134c218eb7434e/PredictionIO-dist/sbt/sbt
 package assemblyPackageDependency is 1. Aborting.
>
>  !     Push rejected, failed to compile PredictionIO app.
>
>  !     Push failed
>
>
>
>
> *Shane Johnson | LIFT IQ*
> *Founder | CEO*
>
> *www.liftiq.com <http://www.liftiq.com/>* or *shane@liftiq.com
> <shane@liftiq.com>*
> mobile: (801) 360-3350
> LinkedIn <https://www.linkedin.com/in/shanewjohnson/>  |  Twitter
> <https://twitter.com/SWaldenJ> |  Facebook
> <https://www.facebook.com/shane.johnson.71653>
>
>
>
> On Fri, Mar 9, 2018 at 11:35 AM, Shane Johnson <shane@liftiq.com> wrote:
>
>> It looks like the URL is the correct URL of the custom PredictionIO dist.
>> It looks like there is another error that might be occurring here when I
>> went to deploy.
>>
>> -----> JVM Common app detected
>>
>> -----> Installing JDK 1.8... done
>>
>> -----> PredictionIO app detected
>>
>> -----> Install core components
>>
>>        *+ PredictionIO (https://s3-us-west-1.amazonaws.com/predictionio/0.12.0-incubating/apache-predictionio-0.12.0-incubating-bin.tar.gz
<https://s3-us-west-1.amazonaws.com/predictionio/0.12.0-incubating/apache-predictionio-0.12.0-incubating-bin.tar.gz>)*
>>
>>        + Spark (spark-2.1.1-bin-hadoop2.7)
>>
>> -----> Install supplemental components
>>
>>        + PostgreSQL (JDBC)
>>
>>        + S3 HDFS (AWS SDK)
>>
>>        + S3 HDFS (Hadoop-AWS)
>>
>>          Writing default 'core-site.xml.erb'
>>
>>        + local Maven repo from buildpack (contents)
>>
>> -----> Configure PredictionIO
>>
>>        Writing default 'pio-env.sh'
>>
>>        Writing default 'spark-defaults.conf.erb'
>>
>>        + Maven repo from buildpack (build.sbt entry)
>>
>>        Set-up environment via '.profile.d/' scripts
>>
>> -----> Install JVM (heroku/jvm-common)
>>
>> -----> PredictionIO engine
>>
>>        Quietly logging. (Set `PIO_VERBOSE=true` for detailed build log.)
>>
>>        [INFO] [Engine$] Using command '/tmp/build_a27b71b6803e5ee2db3367fa69981626/lift-iq-score-f7694693324fc7acebdc6effb0134c218eb7434e/PredictionIO-dist/sbt/sbt'
at /tmp/build_a27b71b6803e5ee2db3367fa69981626/lift-iq-score-f7694693324fc7acebdc6effb0134c218eb7434e
to build.
>>
>>        [INFO] [Engine$] If the path above is incorrect, this process will fail.
>>
>>        [INFO] [Engine$] Uber JAR disabled. Making sure lib/pio-assembly-0.12.0-incubating.jar
is absent.
>>
>>        [INFO] [Engine$] Going to run: /tmp/build_a27b71b6803e5ee2db3367fa69981626/lift-iq-score-f7694693324fc7acebdc6effb0134c218eb7434e/PredictionIO-dist/sbt/sbt
 package assemblyPackageDependency in /tmp/build_a27b71b6803e5ee2db3367fa69981626/lift-iq-score-f7694693324fc7acebdc6effb0134c218eb7434e
>>
>>        [ERROR] [Engine$] [error] Not a valid command: assemblyPackageDependency
>>
>>        [ERROR] [Engine$] [error] Not a valid key: assemblyPackageDependency (similar:
sbtDependency)
>>
>>        [ERROR] [Engine$] [error] assemblyPackageDependency
>>
>>        [ERROR] [Engine$] [error]                          ^
>>
>>        [ERROR] [Engine$] Return code of build command: /tmp/build_a27b71b6803e5ee2db3367fa69981626/lift-iq-score-f7694693324fc7acebdc6effb0134c218eb7434e/PredictionIO-dist/sbt/sbt
 package assemblyPackageDependency is 1. Aborting.
>>
>>  !     Push rejected, failed to compile PredictionIO app.
>>
>>  !     Push failed
>>
>>
>>
>>
>> *Shane Johnson | LIFT IQ*
>> *Founder | CEO*
>>
>> *www.liftiq.com <http://www.liftiq.com/>* or *shane@liftiq.com
>> <shane@liftiq.com>*
>> mobile: (801) 360-3350
>> LinkedIn <https://www.linkedin.com/in/shanewjohnson/>  |  Twitter
>> <https://twitter.com/SWaldenJ> |  Facebook
>> <https://www.facebook.com/shane.johnson.71653>
>>
>>
>>
>> On Fri, Mar 9, 2018 at 11:17 AM, Mars Hall <mars.hall@salesforce.com>
>> wrote:
>>
>>> I'm lost as to how such direct manipulation of CLASSPATH is not
>>> appearing in the logged spark-submit command.
>>>
>>> What could cause this!?
>>>
>>> I just pushed a version of the buildpack which should help debug.
>>> Assuming only a single buildpack is assigned to the app, here's how to set
>>> it:
>>>
>>>   heroku buildpacks:set https://github.com/heroku/pred
>>> ictionio-buildpack#debug-custom-dist
>>>
>>> Then redeploy the engine an check the build log for the line:
>>>
>>>       + PredictionIO ($URL)
>>>
>>> Please confirm that it is the URL of your custom PredictionIO dist.
>>>
>>> On Fri, Mar 9, 2018 at 2:47 PM, Shane Johnson <shane@liftiq.com> wrote:
>>>
>>>> Thanks Donald and Mars,
>>>>
>>>> I created a new distribution (
>>>> <https://s3-us-west-1.amazonaws.com/predictionio/0.12.0-incubating/apache-predictionio-0.12.0-incubating-bin.tar.gz>
>>>> https://s3-us-west-1.amazonaws.com/predictionio/0.12.0-incu
>>>> bating/apache-predictionio-0.12.0-incubating-bin.tar.gz) with the
>>>> added CLASSPATH code and pointed to the distribution with
>>>> the PREDICTIONIO_DIST_URL variable within the engine app in Heroku.
>>>>
>>>> CLASSPATH="/app/PredictionIO-dist/lib/spark/aws-java-sdk.jar
>>>> :$CLASSPATH"
>>>> echo "$CLASSPATH"
>>>>
>>>> It didn't seem to force the aws-java-sdk to load first as I reviewed
>>>> the release logs. Should the aws-java-sdk.jar show up as the first file
>>>> within the --jars section when this is added CLASSPATH="/app/Predicti
>>>> onIO-dist/lib/spark/aws-java-sdk.jar:$CLASSPATH".
>>>>
>>>> I'm still getting the NoSuchMethodError when the *aws-java-sdk.jar* loads
>>>> after the *pio-data-s3-assembly-0.12.0-incubating.jar**. *Do you have
>>>> other suggestions to try? I was also testing locally to change the order
of
>>>> the --jars but changes to the compute-classpath.sh didn't seem to change
>>>> the order of the jars in the logs.
>>>>
>>>> Running train on releaseā€¦
>>>>
>>>> Picked up JAVA_TOOL_OPTIONS: -Xmx12g -Dfile.encoding=UTF-8
>>>>
>>>> [INFO] [Runner$] Submission command: /app/PredictionIO-dist/vendors/spark-hadoop/bin/spark-submit
--driver-memory 13g --class org.apache.predictionio.workflow.CreateWorkflow --jars file:/app/PredictionIO-dist/lib/postgresql_jdbc.jar,file:/app/target/scala-2.11/template-scala-parallel-liftscoring-assembly-0.1-SNAPSHOT-deps.jar,file:/app/target/scala-2.11/template-scala-parallel-liftscoring_2.11-0.1-SNAPSHOT.jar,file:/app/PredictionIO-dist/lib/spark/pio-data-hdfs-assembly-0.12.0-incubating.jar,file:/app/PredictionIO-dist/lib/spark/pio-data-localfs-assembly-0.12.0-incubating.jar,file:/app/PredictionIO-dist/lib/spark/pio-data-elasticsearch-assembly-0.12.0-incubating.jar,file:/app/PredictionIO-dist/lib/spark/hadoop-aws.jar,file:/app/PredictionIO-dist/lib/spark/pio-data-hbase-assembly-0.12.0-incubating.jar,*file:/app/PredictionIO-dist/lib/spark/pio-data-s3-assembly-0.12.0-incubating.jar*,file:/app/PredictionIO-dist/lib/spark/pio-data-jdbc-assembly-0.12.0-incubating.jar,*file:/app/PredictionIO-dist/lib/spark/aws-java-sdk.jar*
--files file:/app/PredictionIO-dist/conf/log4j.properties,file:/app/PredictionIO-dist/conf/core-site.xml
--driver-class-path /app/PredictionIO-dist/conf:/app/PredictionIO-dist/conf:/app/PredictionIO-dist/lib/postgresql_jdbc.jar:/app/PredictionIO-dist/conf
--driver-java-options -Dpio.log.dir=/app file:/app/PredictionIO-dist/lib/pio-assembly-0.12.0-incubating.jar
--engine-id org.template.liftscoring.LiftScoringEngine --engine-version 0c35eebf403cf91fe77a64921d76aa1ca6411d20
--engine-variant file:/app/engine.json --verbosity 0 --json-extractor Both --env
>>>>
>>>>
>>>> Error:
>>>>
>>>> Exception in thread "main" java.lang.NoSuchMethodError: com.amazonaws.services.s3.transfer.TransferManager.<init>(Lcom/amazonaws/services/s3/AmazonS3;Ljava/util/concurrent/ThreadPoolExecutor;)V
>>>>
>>>> 	at org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:287)
>>>>
>>>> 	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2669)
>>>>
>>>> 	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94)
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> *Shane Johnson | LIFT IQ*
>>>> *Founder | CEO*
>>>>
>>>> *www.liftiq.com <http://www.liftiq.com/>* or *shane@liftiq.com
>>>> <shane@liftiq.com>*
>>>> mobile: (801) 360-3350
>>>> LinkedIn <https://www.linkedin.com/in/shanewjohnson/>  |  Twitter
>>>> <https://twitter.com/SWaldenJ> |  Facebook
>>>> <https://www.facebook.com/shane.johnson.71653>
>>>>
>>>>
>>>>
>>>> On Wed, Mar 7, 2018 at 1:01 PM, Mars Hall <mars.hall@salesforce.com>
>>>> wrote:
>>>>
>>>>> Shane,
>>>>>
>>>>> On Wed, Mar 7, 2018 at 4:49 AM, Shane Johnson <shane@liftiq.com>
>>>>> wrote:
>>>>>
>>>>>>
>>>>>> Re: adding a line to ensure a jar is loaded first. Is this what you
>>>>>> are referring to...(line at the bottom in red)?
>>>>>>
>>>>>
>>>>>
>>>>> I believe the code would need to look like this to effect the output
>>>>> classpath as intended:
>>>>>
>>>>>
>>>>>> CLASSPATH="/app/PredictionIO-dist/lib/spark/aws-java-sdk.jar
>>>>>> :$CLASSPATH"
>>>>>> echo "$CLASSPATH"
>>>>>>
>>>>>
>>>>>
>>>>> aws-java-sdk.jar is already in the CLASSPATH though, So, the script
>>>>> will need to be skip or remove it first.
>>>>>
>>>>> --
>>>>> *Mars Hall
>>>>> 415-818-7039 <(415)%20818-7039>
>>>>> Customer Facing Architect
>>>>> Salesforce Platform / Heroku
>>>>> San Francisco, California
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> *Mars Hall
>>> 415-818-7039 <(415)%20818-7039>
>>> Customer Facing Architect
>>> Salesforce Platform / Heroku
>>> San Francisco, California
>>>
>>
>>
>


-- 
*Mars Hall
415-818-7039
Customer Facing Architect
Salesforce Platform / Heroku
San Francisco, California

Mime
View raw message