predictionio-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Donald Szeto <don...@apache.org>
Subject Re: Use of common argument scratch-uri and remote deployment
Date Tue, 04 Oct 2016 20:00:35 GMT
Hi Bansari,

The --scratch-uri switch is only useful with "pio train/deploy" using YARN
cluster mode, which is your case. It tells PredictionIO where to copy
PredictionIO JARs and engine.json for YARN cluster mode to work properly.

1. Make sure HADOOP_CONF_DIR is set properly in conf/pio-env.sh.
2. Provide an HDFS URL to --scratch-uri. You need to have write access to
this location.

Regards,
Donald

On Tue, Oct 4, 2016 at 11:21 AM, Pat Ferrel <pat@occamsmachete.com> wrote:

> No idea about 'scratch-uri’ but once you build a model if you have
> specified (in pio-env.sh) that pio use hdfs for the model storage it will
> already be available to any machine that has access to hdfs. It somewhat
> depends on the template, the Universal Recommender uses Elasticsearch for
> model storage so any machine with access to ES will have the model.
>
>
> On Oct 4, 2016, at 10:09 AM, Bansari Shah <bansari.jan93@gmail.com> wrote:
>
> Hi Donald,
>
> I am running my spark cluster of 3 node with YARN and spark driver is
> within cluster.
>
> Thanks
> Regards,
> Bansari
>
> On Tue, Oct 4, 2016 at 9:59 PM, Donald Szeto <donald@apache.org> wrote:
>
>> Hi Bansari,
>>
>> How are you running your Spark cluster? Standalone, YARN, or Mesos? Are
>> you running the Spark driver on the client or within the cluster?
>>
>> Regards,
>> Donald
>>
>> On Tue, Oct 4, 2016 at 5:55 AM, Bansari Shah <bansari.jan93@gmail.com>
>> wrote:
>>
>>> Hi,
>>> Can you please guide me how to use 'scratch-uri' argument in case of
>>>  transferring all necessary files to remote location.
>>>
>>> And can you please suggest me any way for deploying model on remote
>>> location which is trained and build on other machine.
>>>
>>> Please consider it.
>>>
>>> Thank you,
>>>
>>> Regards,
>>> Bansari
>>>
>>
>>
>
>

Mime
View raw message