gearpump-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GEARPUMP-147) java.lang.ClassNotFoundException when running on worker with worker.executor-share-same-jvm-as-worker = true
Date Tue, 31 May 2016 11:48:12 GMT

    [ https://issues.apache.org/jira/browse/GEARPUMP-147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15307610#comment-15307610
] 

ASF GitHub Bot commented on GEARPUMP-147:
-----------------------------------------

GitHub user PipsqueakH opened a pull request:

    https://github.com/apache/incubator-gearpump/pull/30

    Gearpump 143

    add refinement as to GEARPUMP-147. 

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/PipsqueakH/incubator-gearpump GEARPUMP-143

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-gearpump/pull/30.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #30
    
----
commit a063d5f04182ae43c20e6ecd1a5a7d0fe89380da
Author: Pipsqueak <snipeinmaze@gmail.com>
Date:   2016-05-29T13:09:58Z

    fix GEARPUMP-143 add doc worker.executor-share-same-jvm-as-worker

commit 26e5e09691110f84fcd3420eab0a4cfd203ce851
Author: Pipsqueak <snipeinmaze@gmail.com>
Date:   2016-05-30T11:42:48Z

    fix GEARPUMP-143 refine deploy config doc

commit c7aa7d76b655dd176ad8b1fee28da64498369aab
Author: yghan <yghan@cupdata.com>
Date:   2016-05-31T05:46:13Z

    fix GEARPUMP-143 refine the deploy config doc

----


> java.lang.ClassNotFoundException when running on worker with worker.executor-share-same-jvm-as-worker
= true
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: GEARPUMP-147
>                 URL: https://issues.apache.org/jira/browse/GEARPUMP-147
>             Project: Apache Gearpump
>          Issue Type: Task
>            Reporter: Karol Brejna
>            Assignee: Sean Zhong
>         Attachments: gear.pc.conf, gear.pc.slimworker.conf, logs.slimworker.zip, prepare.sh
>
>
> Doing GEARPUMP-146 I hit a problem: I was able to run a master on pc and a worker on
raspi. After deploying wordcountjava-2.11-0.8.0-assembly.jar the example didn't start.
> I reproduced this on "pure pc" cluster.
> I've prepared dedicated config files:
> * gear.pc.conf			   - normal (default) configuration for master and workers ran on a
PC
> * gear.pc.slimworker.conf  - intends to have lower requirements for memory, cpu - to
be used on a device (raspi, edison)
> *Obtaining binary and creating config files*
> {code}
> wget https://github.com/gearpump/gearpump/releases/download/0.8.0/gearpump-2.11-0.8.0.zip
> unzip gearpump-2.11-0.8.0.zip
> cd gearpump-2.11-0.8.0
> cp conf/gear.conf conf/gear.original.conf 
> cp conf/gear.conf conf/gear.pc.conf 
> cp conf/gear.conf conf/gear.pc.slimworker.conf 
> {code}
> *Preparing "PC" conf*
> The only real change here is putting the IP of the machine that will run the master:
> {code}
>   cluster {
>     masters = ["192.168.1.115:3000"]
>   }
>   
>   hostname = "192.168.1.115"
> {code}
>   
> *Preparing device conf*
> Compared to previous config, this config should set up less worker slots, smaller -Xmx
jvm switch for worker, and executor-share-same-jvm-as-worker set to true:  
> {code}
>   worker.slots = 100
>   worker.executor-share-same-jvm-as-worker = true
>   executor {
>     vmargs = "-server -Xms512M -Xmx512M -Xss1M -XX:+HeapDumpOnOutOfMemoryError -XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=80 -XX:+UseParNewGC -XX:NewRatio=3  -Djava.rmi.server.hostname=localhost"
>     extraClasspath = ""
>   }
> {code}
> *Running the cluster:*
> {code}
> echo "Starting the master"
> cp conf/gear.pc.conf conf/gear.conf
> bin/master -ip 192.168.1.115 -port 3000 &
> echo "Starting services"
> bin/services &
> echo "Starting a worker (configured as on the device)"
> cp conf/gear.pc.slimworker.conf conf/gear.conf
> bin/worker &
> {code}
> *Deploing the DAG*
> I've deployed wordcountjava-2.11-0.8.0-assembly.jar.
> The app deploys, but doesn't run.
> (After changing   worker.executor-share-same-jvm-as-worker  to false and restarting the
cluster, example dag works.)
> See the attached logs:
> {code}
> 2016-05-18 08:47:31,453 ERROR OneForOneStrategy: io.gearpump.streaming.examples.wordcountjava.Split
> java.lang.ClassNotFoundException: io.gearpump.streaming.examples.wordcountjava.Split
> 	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> 	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> 	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> 	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> 	at io.gearpump.streaming.task.TaskUtil$.loadClass(TaskUtil.scala:30)
> 	at io.gearpump.streaming.executor.TaskLauncher.launch(TaskLauncher.scala:55)
> 	at io.gearpump.streaming.executor.Executor$$anonfun$dynamicDagPhase1$1.applyOrElse(Executor.scala:147)
> 	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170)
> 	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
> 	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
> 	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
> 	at akka.actor.Actor$class.aroundReceive(Actor.scala:480)
> 	at io.gearpump.streaming.executor.Executor.aroundReceive(Executor.scala:55)
> 	at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
> 	at akka.actor.ActorCell.invoke(ActorCell.scala:495)
> 	at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
> 	at akka.dispatch.Mailbox.run(Mailbox.scala:224)
> 	at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
> 	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> 	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> 	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> 	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> {code}
>   
> *Update*
> bq. Putting DAG jar in worker classpath fixes the error. *Now I am able to run Gearpump
app on Raspberry PI!*



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message