predictionio-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Donald Szeto <don...@apache.org>
Subject Re: Issue with loading dependencies and jars
Date Wed, 07 Mar 2018 01:58:26 GMT
Even easier: skip cloning, and just edit the shell script directly in the
binary distribution. Hope that works.

Regards,
Donald

On Tue, Mar 6, 2018 at 5:41 PM Shane Johnson <shane@liftiq.com> wrote:

> Thanks Mars and Donald. I think this gets me to next steps:
>
>    - Clone PredictionIO 0.12 and adjust the bin/compute-classpath.sh to
>    have aws-java-sdk-1.7.4 loaded first.
>    - Create custom binary distribution of PredicionIO 0.12.
>    - Add config var to point to custom binary distribution.
>
> This is very helpful. Thank you!
>
> *Shane Johnson | LIFT IQ*
> *Founder | CEO*
>
> *www.liftiq.com <http://www.liftiq.com/>* or *shane@liftiq.com
> <shane@liftiq.com>*
> mobile: (801) 360-3350
> LinkedIn <https://www.linkedin.com/in/shanewjohnson/>  |  Twitter
> <https://twitter.com/SWaldenJ> |  Facebook
> <https://www.facebook.com/shane.johnson.71653>
>
>
>
> On Tue, Mar 6, 2018 at 4:52 PM, Mars Hall <mars.hall@salesforce.com>
> wrote:
>
>>
>>
>> On Tue, Mar 6, 2018 at 11:39 PM, Shane Johnson <shane@liftiq.com> wrote:
>>
>>>
>>> Do you know the version of hadoop-aws.jar and aws-java-sdk.jar that you
>>> are using?
>>>
>>> I do not know what version is being used. Is this something that I can
>>> specify or control? I am using the PredictionIO buildpack
>>> https://github.com/heroku/predictionio-buildpack. I am *not* specifying
>>> these in my build.sbt currently.
>>>
>>
>> The versions are specified in the "bin/common/setup-runtime" script of
>> the buildpack:
>>
>> https://github.com/heroku/predictionio-buildpack/blob/master/bin/common/setup-runtime#L50
>>
>> Currently they are:
>>
>>    - hadoop-aws-2.7.3
>>    - aws-java-sdk-1.7.4
>>
>> If you fork the buildpack, those download URLs (currently hosted in my S3
>> bucket) can be changed.
>>
>>
>>
>>> You are also right that you can modify the class path in
>>> bin/compute-classpath.sh as a short term fix. The current order is
>>> following the output of your target system's `ls`, so the order is not
>>> guaranteed like you speak. Right before the last line (echo $CLASSPATH),
>>> you can add a line to make sure that the JAR you want to be loaded first is
>>> at the very beginning.
>>>
>>> I believe this would need to be something that is edited on master (
>>> https://github.com/apache/predictionio) as the buildpack leverages it
>>> vs. a cloned version of the code that I can edit, am I thinking through
>>> that correctly? I may need to circle back with Mars to see if there are any
>>> other options to get this to work with Heroku. Is this something that can
>>> be committed to master? The aws-java-sdk.jar is the one that we need to
>>> load first.
>>>
>>
>> If you build your own custom binary distribution of PredictionIO 0.12 and
>> upload it to a publicly accessible URL, then the buildpack can be
>> configured to use it by setting config var `PREDICTIONIO_DIST_URL` which is
>> used by the same setup script as above:
>>
>> https://github.com/heroku/predictionio-buildpack/blob/master/bin/common/setup-runtime#L82
>>
>> This is how I tested my pre-release builds on Heroku.
>>
>> --
>> *Mars Hall
>> 415-818-7039 <(415)%20818-7039>
>> Customer Facing Architect
>> Salesforce Platform / Heroku
>> San Francisco, California
>>
>
>

Mime
View raw message