predictionio-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kenneth Chan <kenn...@apache.org>
Subject Re: Exception: Job aborted due to stage failure: Task 0 in stage 3.0 failed 1 times,
Date Mon, 10 Jul 2017 02:27:34 GMT
based you log, the exception was triggered by line 77 and 73 of
Datasource.scala of the engine. What is the code?

btw, look like you were using the simple recommendation templat given your
class path `org.example.recommendation`,  not the E-Commerce reomm one ?


```
2017-07-09 11:21:16,693 ERROR org.apache.spark.executor.Executor [Executor
task launch worker-2] -

Exception in task 0.0 in stage 3.0 (TID 9)

java.lang.NullPointerException

at
com.google.common.collect.ImmutableSet.copyFromCollection(ImmutableSet.java:350)

at com.google.common.collect.ImmutableSet.copyOf(ImmutableSet.java:345)

at org.example.recommendation.DataSource$2.call(DataSource.java:77)

at org.example.recommendation.DataSource$2.call(DataSource.java:73)
````

On Sun, Jul 9, 2017 at 2:31 AM, Sebastian Fix <sebastian.fix@smartgate.ch>
wrote:

> Hi Kenneth & Vaghawan
>
> Thanks for your answer. You will find the error attached.
>
> @Vaghawan
>
> At the Documentation the Event Data Requirements are set to:
>
>    - Users' *view* events
>    - Users' *buy* events
>    - Items' with *categories* properties
>    - Constraint *unavailableItems* set events
>
>
> The Documentation set the following parameter. Are all of these parameter
> required?
>
> I uploaded with a .json import:
>
>    - Product-Items
>    - User
>    - User Buy Events.
>
>
>
>
>
> In the pio.log file i also found this exception:
>
> 2017-07-08 10:50:15,153 ERROR org.apache.spark.scheduler.TaskSetManager
> [task-result-getter-1] - Task 0 in stage 3.0 failed 1 times; aborting job
> 2017-07-09 11:21:10,476 WARN  org.apache.predictionio.workflow.WorkflowUtils$
> [main] - Environment variable MYSQL_JDBC_DRIVER is pointing to a
> nonexistent file /users/sebastian/Development/
> Recommendation/PredictionIO-0.11.0-incubating/lib/mysql-connector-ja$
> 2017-07-09 11:21:10,487 INFO  org.apache.predictionio.tools.Runner$
> [main] - Submission command: /users/sebastian/Development/
> Recommendation/PredictionIO-0.11.0-incubating/vendors/spark-
> 1.6.3-bin-hadoop2.6/bin/spark-submit --class org.apache.predictionio.wo$
> 2017-07-09 11:21:12,665 INFO  org.apache.predictionio.controller.Engine
> [main] - Extracting datasource params...
> 2017-07-09 11:21:12,754 INFO  org.apache.predictionio.workflow.WorkflowUtils$
> [main] - No 'name' is found. Default empty String will be used.
> 2017-07-09 11:21:12,805 INFO  org.apache.predictionio.controller.Engine
> [main] - Datasource params: (,org.example.recommendation.
> DataSourceParams@2c715e84)
> 2017-07-09 11:21:12,805 INFO  org.apache.predictionio.controller.Engine
> [main] - Extracting preparator params...
> 2017-07-09 11:21:12,806 INFO  org.apache.predictionio.controller.Engine
> [main] - Preparator params: (,Empty)
> 2017-07-09 11:21:12,819 INFO  org.apache.predictionio.controller.Engine
> [main] - Extracting serving params...
> 2017-07-09 11:21:12,819 INFO  org.apache.predictionio.controller.Engine
> [main] - Serving params: (,Empty)
> 2017-07-09 11:21:14,148 INFO  Remoting [sparkDriverActorSystem-akka.actor.default-dispatcher-2]
> - Starting remoting
> 2017-07-09 11:21:14,301 INFO  Remoting [sparkDriverActorSystem-akka.actor.default-dispatcher-2]
> - Remoting started; listening on addresses :[akka.tcp://
> sparkDriverActorSystem@192.168.1.35:59998]
> 2017-07-09 11:21:15,256 INFO  org.apache.predictionio.controller.Engine$
> [main] - EngineWorkflow.train
> 2017-07-09 11:21:15,257 INFO  org.apache.predictionio.controller.Engine$
> [main] - DataSource: org.example.recommendation.DataSource@1b13467c
> 2017-07-09 11:21:15,257 INFO  org.apache.predictionio.controller.Engine$
> [main] - Preparator: org.example.recommendation.Preparator@1fedf0a4
> 2017-07-09 11:21:15,257 INFO  org.apache.predictionio.controller.Engine$
> [main] - AlgorithmList: List(org.example.recommendation.Algorithm@
> 1d901f20)
> 2017-07-09 11:21:15,258 INFO  org.apache.predictionio.controller.Engine$
> [main] - Data sanity check is on.
> 2017-07-09 11:21:15,857 INFO  org.apache.predictionio.controller.Engine$
> [main] - org.example.recommendation.TrainingData supports data sanity
> check. Performing check.
> 2017-07-09 11:21:16,693 ERROR org.apache.spark.executor.Executor
> [Executor task launch worker-2] - Exception in task 0.0 in stage 3.0 (TID 9)
> java.lang.NullPointerException
>
>
>
> I’ll have a look later this day at the link you send me. I’ll also try to
> check the database what data are in there.
>
>
> Thanks for your help
> Sebastian
>
>
>
>
> Am 09.07.2017 um 05:01 schrieb Kenneth Chan <kenneth@apache.org>:
>
> Hi Sebastian,
>
> would you post the full stack trace?
> would you go to Spark master and look into more detailed Spark log of the
> failure? it may give more hint where in engine code trigger the  null
> pointer exception
>
> You could also follow here to debug and see if the data is imported as
> expected, or add some debug message in your engine to get more info
> https://predictionio.incubator.apache.org/customize/troubleshooting/
>
>
>
> On Sat, Jul 8, 2017 at 5:12 AM, Vaghawan Ojha <vaghawan781@gmail.com>
> wrote:
>
> Hi,
>
> As much as I can understand from the error message, the Null Exception
> happens when you have wrong data or no data at all. Please make sure
> everything is fine in your data.
>
> Sometimes, the event store can store the data you provide, but the
> template you might be using may need other kind of data, so please make
> sure you're following the right doc and providing the right kind of data.
>
> Thanks
>
> On Sat, Jul 8, 2017 at 2:39 PM, Sebastian Fix <sebastian.fix@smartgate.ch>
> wrote:
>
> Hey Keneth
>
> Thank for the answer.
>
> I’managed to import the following data:
>
>
> After running „pio build“ i get the following error during the „pio train“
> process:
>
> [ERROR] [Executor] Exception in task 0.0 in stage 3.0 (TID 9)
> [WARN] [TaskSetManager] Lost task 0.0 in stage 3.0 (TID 9, localhost):
> java.lang.NullPointerException
>
> [ERROR] [TaskSetManager] Task 0 in stage 3.0 failed 1 times; aborting job
> Exception in thread "main" org.apache.spark.SparkException: Job aborted
> due to stage failure: Task 0 in stage 3.0 failed 1 times, most recent
> failure: Lost task 0.0 in stage 3.0 (TID 9, localhost):
> java.lang.NullPointerException
>
> Driver stacktrace:
> at org.apache.spark.scheduler.DAGScheduler.org
> <http://org.apache.spark.scheduler.dagscheduler.org/>$apache$spark$sch
> eduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1431)
> Caused by: java.lang.NullPointerException
>
>
> Best regards
> Sebastian
>
>
> Am 08.07.2017 um 09:52 schrieb Kenneth Chan <kenneth@apache.org>:
>
> re: " i’m always getting an exception with the initial load from User &
> Product"
> what's the error you got?
>
> On Fri, Jul 7, 2017 at 12:46 PM, Sebastian Fix <sebastian.fix@smartgate.ch
> > wrote:
>
> Hello together,
>
> I finally got the System running and would like to import some initial
> data now.
>
> I’m using the E-Commerce-Template: http://predictionio.incubator.apache.or
> g/templates/ecommercerecommendation/quickstart/
>
> I've the following data’s i would like to import:
>
>    - User data
>
>
>    - Products data
>
>
>    - Transaction data
>
>
> When i try to use the *pio import --appid 1 --input import/data-user.json* command
> i’m always getting an exception with the initial load from User & Product.
> My .json file is following the structure mentioned on the tutorial.
>
>
>    - For User Data: {"event":"$set","entityType":"
>    user","entityId":"user001","eventTime":““}
>    - For Products  {"event":"$set","entityType“:
>    "item","entityId“:"product001","eventTime":““}
>
>
>    - For Transactions: {"entityId“:"user001","entityType":"user","eve
>    nt":"buy","targetEntityType":"item","targetEntityId“:"product001“}
>
>
> Only the Import for Transactions is working I found this template here:
> https://gist.github.com/vaghawan/0a5fb8ddb85e03631dd50
> 0d7c8f0677d#file-data-sample-json
>
> So my quesiton, how can I import Data’s like
>
>
>    - User Data
>    - User Categories
>    - Product Data
>    - Product Categories
>
>
> with a .json file? What is the file structure?
>
> Thanks for any help,
> Sebastian
>
>
>
>
>
>
>
>

Mime
View raw message