Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 67126200C43 for ; Sun, 26 Mar 2017 20:35:15 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 65A89160B87; Sun, 26 Mar 2017 18:35:15 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 379F1160B82 for ; Sun, 26 Mar 2017 20:35:14 +0200 (CEST) Received: (qmail 48084 invoked by uid 500); 26 Mar 2017 18:35:13 -0000 Mailing-List: contact user-help@predictionio.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@predictionio.incubator.apache.org Delivered-To: mailing list user@predictionio.incubator.apache.org Received: (qmail 48074 invoked by uid 99); 26 Mar 2017 18:35:12 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 26 Mar 2017 18:35:12 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 91F6E185CAB for ; Sun, 26 Mar 2017 18:35:12 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3.13 X-Spam-Level: *** X-Spam-Status: No, score=3.13 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, HTML_MESSAGE=2, HTML_OBFUSCATE_05_10=0.001, JMQ_TRACKER=0.5, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id sE4OOmNVb8AD for ; Sun, 26 Mar 2017 18:35:10 +0000 (UTC) Received: from mail-it0-f43.google.com (mail-it0-f43.google.com [209.85.214.43]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id C5E7D5F3FE for ; Sun, 26 Mar 2017 18:35:09 +0000 (UTC) Received: by mail-it0-f43.google.com with SMTP id 190so32422651itm.0 for ; Sun, 26 Mar 2017 11:35:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=Ky9Nhk9HGeBFSXGYSKBFNxWLRJXt5h8MVwMulc8UG9M=; b=OzkZIHqFBaoAJajKvsju5S1J6EFbK20uHLVyPVBakLgNyfaIbyo+8K7H/SL3abwO5I A0Arhe9BjLZH0o3p1WC8BxNHWp1K68gVNM5r2Tmbj3iqy5iWidMEgIA7ymf5PG/Bwuw3 P+2lsC3gKY1qBjGU7r9dxYMH3XWHR/TwagjOC8UjIjTzKBXt9USdhkx4uStHog7nH3p+ DEwHpNBLtZfQU3owU/2sI94ALAdAOzXwY5V7aOgqT/g5ZVbFuRSTwP50SABKZVN0XQvv 7hsgqpt3JTveexd/kilrh0yPU2HB6MAUqpk2RCNPYuxfog5R4qitvDEV4PYVtDf7Oyeg ZGjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=Ky9Nhk9HGeBFSXGYSKBFNxWLRJXt5h8MVwMulc8UG9M=; b=VAJQB2chNJmoBkO1iCU5MG5swiAtGtVkzbhKDI9kfODd0COUo8hyv2lU7gUdQWS5Lf OnZqDzwvjIik4JllcLzMwkUlIzUjXGOIIGVCJ/R5F2fFhNHuxyDEZzY9lBZxfvyw8RBE b9yygjRFTHuIxEL0/V3wzxdcIRKEIq8RQcr2Ec9Cpguom8/gES3fa/WBhNaV4Bhp2ix/ ZIRCJVEESV2Z5pPwmhaJRtUBoCZzyTptgE/9scNZOzWY9NS53YY6jMwCJywNfp5/SRZb mvKKC68piursJVzRlEF7vp38hU8mzdtQ8BUUfaqoHNKzEiHk93hK4sdrzvbWfpxQP3ky YiNw== X-Gm-Message-State: AFeK/H0TMFxQsX3HKWn409UXAMlu2cugxSD/AtgNC0Ez2s5yLilxqCMtCuEo/HNY4IJ1/4nijSxRRu4GuLZcew== X-Received: by 10.107.46.163 with SMTP id u35mr19445542iou.196.1490552889753; Sun, 26 Mar 2017 11:28:09 -0700 (PDT) MIME-Version: 1.0 Received: by 10.107.6.219 with HTTP; Sun, 26 Mar 2017 11:28:09 -0700 (PDT) In-Reply-To: References: From: Vaghawan Ojha Date: Mon, 27 Mar 2017 00:13:09 +0545 Message-ID: Subject: Re: Need Help Building To: user@predictionio.incubator.apache.org Content-Type: multipart/alternative; boundary=001a113ab588835c89054ba663d0 archived-at: Sun, 26 Mar 2017 18:35:15 -0000 --001a113ab588835c89054ba663d0 Content-Type: text/plain; charset=UTF-8 Yes, you were right! It's because of the owner of the dir isn't the same as pio command. Can you tell me, do I have to do pio build, again whenever I updated my events data for new train? Thank you Vaghawan On Mon, Mar 27, 2017 at 12:11 AM, Marius Rabenarivo < mariusrabenarivo@gmail.com> wrote: > You can set JAVA_OPTS inside .profile in your home directory > > Add > > export JAVA_OPTS="Xmx4g" > > inside your .profile or .bashrc > > Access to LOCALFS is due to OS privilege I think. > > The owner of the directory should be the same as the runner of the pio > command > > 2017-03-26 22:13 GMT+04:00 Vaghawan Ojha : > >> the plain error is like this: >> >> [ERROR] [Storage$] Error initializing storage client for source LOCALFS >> Exception in thread "main" org.apache.predictionio.data.storage.StorageClientException: >> Data source LOCALFS was not properly initialized. >> at org.apache.predictionio.data.storage.Storage$$anonfun$10.app >> ly(Storage.scala:282) >> at org.apache.predictionio.data.storage.Storage$$anonfun$10.app >> ly(Storage.scala:282) >> at scala.Option.getOrElse(Option.scala:120) >> at org.apache.predictionio.data.storage.Storage$.getDataObject( >> Storage.scala:281) >> at org.apache.predictionio.data.storage.Storage$.getDataObjectF >> romRepo(Storage.scala:266) >> at org.apache.predictionio.data.storage.Storage$.getModelDataMo >> dels(Storage.scala:382) >> at org.apache.predictionio.workflow.CoreWorkflow$.runTrain( >> CoreWorkflow.scala:79) >> at org.apache.predictionio.workflow.CreateWorkflow$.main(Create >> Workflow.scala:250) >> at org.apache.predictionio.workflow.CreateWorkflow.main(CreateW >> orkflow.scala) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce >> ssorImpl.java:62) >> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe >> thodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:498) >> at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy >> $SparkSubmit$$runMain(SparkSubmit.scala:672) >> at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit >> .scala:180) >> at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205) >> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:120) >> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) >> >> Because without doing sudo that LOCALFS doesn't execute correctly. I >> don't know how else could I run it. Thanks >> >> On Sun, Mar 26, 2017 at 11:56 PM, Vaghawan Ojha >> wrote: >> >>> Where would I set JAVA_OPTS is it in pio-env.sh? >>> >>> On Sun, Mar 26, 2017 at 11:49 PM, Marius Rabenarivo < >>> mariusrabenarivo@gmail.com> wrote: >>> >>>> You have do add pass-through parameters to the pio train command >>>> >>>> pio train -- --executor-memory 4g --driver-memory 4g >>>> >>>> and set JAVA_OPTS="Xmx4g" environment variable >>>> >>>> 2017-03-26 21:37 GMT+04:00 Vaghawan Ojha : >>>> >>>>> Hi, >>>>> Thanks but the error was because I was not inside the template dir >>>>> while running pio build. It builded now successfully, but it seems in every >>>>> step there is some crazy errors awaiting for me. Now it actually fails at >>>>> training. Can you suggest me anything from the train log? >>>>> I'm sorry but they are really hard to grab unless I ask for help. >>>>> >>>>> Thank you very much >>>>> >>>>> On Sun, Mar 26, 2017 at 10:00 PM, Marius Rabenarivo < >>>>> mariusrabenarivo@gmail.com> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> The error is : >>>>>> >>>>>> [ERROR] [Storage$] Error initializing storage client for source PGSQL >>>>>> >>>>>> I think you need to change it to HBASE if you want to use HBase >>>>>> >>>>>> PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=PGSQL >>>>>> -> >>>>>> PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE >>>>>> >>>>>> in your pio-env.sh >>>>>> >>>>>> And start HBase before if not using the pio-start-all script. >>>>>> >>>>>> If you want to use PostreSQL pio-start-all attempt to start it too. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> 2017-03-26 19:29 GMT+04:00 Vaghawan Ojha : >>>>>> >>>>>>> I followed the procedure of manual install, everything was fine >>>>>>> until I stumbled into the pio build. >>>>>>> >>>>>>> I've a directory something like this /abc/pio0.0.10/pio and inside >>>>>>> that another dir pio, in total it would be like : >>>>>>> /abc/pio0.0.10/pio / >>>>>>> >>>>>>> where do I actually run build? inside /abc/pio0.0.10 or >>>>>>> /abc/pio0.0.10/pio / ? >>>>>>> >>>>>>> I don't know but I get some weird errors which I can't properly >>>>>>> diagnose. I"ve attached my log file here. I've followed to load the engine >>>>>>> template. here http://predictionio.incub >>>>>>> ator.apache.org/templates/recommendation/quickstart/ >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > --001a113ab588835c89054ba663d0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Yes, you were right! It's because of the owner of the = dir isn't the same as pio command. Can you tell me, do I have to do pio= build, again whenever I updated my events data for new train?=C2=A0
Thank you
Vaghawan

On M= on, Mar 27, 2017 at 12:11 AM, Marius Rabenarivo <mariusrabenarivo= @gmail.com> wrote:
You can set JAVA_OPTS inside .profile in your home direc= tory

Add

export JAVA_OPTS=3D"Xmx4g"
<= br>
inside your .profile or .bashrc

Acces= s to LOCALFS is due to OS privilege I think.

The owner of the = directory should be the same as the runner of the pio command

2017-03-26 22:13 GMT+04:00 Vaghawan Ojha <vaghawan7= 81@gmail.com>:
the plain error is like this:=C2=A0

[ERROR] [S= torage$] Error initializing storage client for source LOCALFS
Exc= eption in thread "main" org.apache.predictionio.data.storage= .StorageClientException: Data source LOCALFS was not properly initialized.<= /div>
at org.apache.predict= ionio.data.storage.Storage$$anonfun$10.apply(Storage.scala:282)
at org.apache.predicti= onio.data.storage.Storage$$anonfun$10.apply(Storage.scala:282)
at scala.Option.getOrEl= se(Option.scala:120)
at org.apache.predictionio.data.storage.Storage$.getDataObject(Storage.scala:281)
at org.apache.predictionio.data.storage.Storage$.getDataObjectFr= omRepo(Storage.scala:266)
at org.apache.predictionio.data.storage.Storage$.getModelDataModels(Storage.scala:382)
<= /span>at org.apache.predictionio.workflow.CoreWorkflow$.runTrain(= CoreWorkflow.scala:79)
at org.apache.predictionio.workflow.CreateWorkflow$.main(Create= Workflow.scala:250)
a= t org.apache.predictionio.workflow.CreateWorkflow.main(CreateWork= flow.scala)
at sun.re= flect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAcces= sorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invok= e(Method.java:498)
at= org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$Spa= rkSubmit$$runMain(SparkSubmit.scala:672)
at org.apache.spark.deploy.SparkSubmit$.doRun= Main$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubm= it.scala:205)
at= org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:120)=
at org.apache.spark.= deploy.SparkSubmit.main(SparkSubmit.scala)

<= div>Because without doing sudo that LOCALFS doesn't execute correctly. = I don't know how else could I run it. Thanks

On Sun, Mar 26, 2017 at 11:56 PM, Vaghawan Ojha &= lt;vaghawan781@g= mail.com> wrote:
Where would I set JAVA_OPTS is it in pio-env.sh?

On Sun, Mar 26, 2017= at 11:49 PM, Marius Rabenarivo <mariusrabenarivo@gmail.com&g= t; wrote:
You have do add pass-through parameters to the pio train command

=
pio train -- --executor-memory 4g --driver-memory 4g

and= set JAVA_OPTS=3D"Xmx4g" environment variable

2017-03= -26 21:37 GMT+04:00 Vaghawan Ojha <vaghawan781@gmail.com>:
Hi,
Thanks but th= e error was because I was not inside the template dir while running pio bui= ld. It builded now successfully, but it seems in every step there is some c= razy errors awaiting for me. Now it actually fails at training. Can you sug= gest me anything from the train log?
I'm sorry but they are r= eally hard to grab unless I ask for help.

Thank yo= u very much
On Sun, Mar 26, 2017 at 10:00 PM, Marius Rabena= rivo <mariusrabenarivo@gmail.com> wrote:
Hi,

The error is= : 
[ERROR] [Storage$] Error initializing storag=
e client for source PGSQL

I think you =
need to change it to HBASE if you want to use HBase

PIO_STORAGE_REPO=
SITORIES_EVENTDATA_SOURCE=3DPGSQL
->
PIO_STORAGE_REPOSITORIES=
_EVENTDATA_SOURCE=3DHBASE

in your=
 pio-env.sh
And start HBase before if not =
using the pio-start-all script.

If you=
 want to use PostreSQL pio-start-all attempt to start it too.







201= 7-03-26 19:29 GMT+04:00 Vaghawan Ojha <vaghawan781@gmail.com>:
I followed the pr= ocedure of manual install, everything was fine until I stumbled into the pi= o build.=C2=A0

I've a directory something like this = /abc/pio0.0.10/pio and inside that another dir pio, in total it would be li= ke :
/abc/pio0.0.10/pio /

where do I act= ually run build? inside /abc/pio0.0.10 or /abc/pio0.0.10/pio / ?
=
I don't know but I get some weird errors which I can'= ;t properly diagnose. I"ve attached my log file here. I've followe= d to load the engine template. here=C2=A0h= ttp://predictionio.incubator.apache.org/templates/recommendation/= quickstart/









--001a113ab588835c89054ba663d0--