gearpump-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karol Brejna (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GEARPUMP-147) java.lang.ClassNotFoundException when running on worker with worker.executor-share-same-jvm-as-worker = true
Date Fri, 20 May 2016 10:23:12 GMT

    [ https://issues.apache.org/jira/browse/GEARPUMP-147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15293123#comment-15293123
] 

Karol Brejna commented on GEARPUMP-147:
---------------------------------------

[~HuafengWang] Adding the DAG jar to worker's classpath (by phisically placing it there) did
the trick. Now the application is running fine! 
If the behavior is intentional (it kind of make sense), we should document it, I've opened
ticket for this (https://issues.apache.org/jira/browse/GEARPUMP-143). Are there any other
assumptions, consequences? (I assume, for example, that gearpump.executor.vmargs is then ignored...)


I could take the info together and put it http://gearpump.apache.org/releases/latest/deployment-configuration.html.





> java.lang.ClassNotFoundException when running on worker with worker.executor-share-same-jvm-as-worker
= true
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: GEARPUMP-147
>                 URL: https://issues.apache.org/jira/browse/GEARPUMP-147
>             Project: Apache Gearpump
>          Issue Type: Task
>            Reporter: Karol Brejna
>            Assignee: Huafeng Wang
>         Attachments: gear.pc.conf, gear.pc.slimworker.conf, logs.slimworker.zip, prepare.sh
>
>
> Doing GEARPUMP-146 I hit a problem: I was able to run a master on pc and a worker on
raspi. After deploying wordcountjava-2.11-0.8.0-assembly.jar the example didn't start.
> I reproduced this on "pure pc" cluster.
> I've prepared dedicated config files:
> * gear.pc.conf			   - normal (default) configuration for master and workers ran on a
PC
> * gear.pc.slimworker.conf  - intends to have lower requirements for memory, cpu - to
be used on a device (raspi, edison)
> *Obtaining binary and creating config files*
> {code}
> wget https://github.com/gearpump/gearpump/releases/download/0.8.0/gearpump-2.11-0.8.0.zip
> unzip gearpump-2.11-0.8.0.zip
> cd gearpump-2.11-0.8.0
> cp conf/gear.conf conf/gear.original.conf 
> cp conf/gear.conf conf/gear.pc.conf 
> cp conf/gear.conf conf/gear.pc.slimworker.conf 
> {code}
> *Preparing "PC" conf*
> The only real change here is putting the IP of the machine that will run the master:
> {code}
>   cluster {
>     masters = ["192.168.1.115:3000"]
>   }
>   
>   hostname = "192.168.1.115"
> {code}
>   
> *Preparing device conf*
> Compared to previous config, this config should set up less worker slots, smaller -Xmx
jvm switch for worker, and executor-share-same-jvm-as-worker set to true:  
> {code}
>   worker.slots = 100
>   worker.executor-share-same-jvm-as-worker = true
>   executor {
>     vmargs = "-server -Xms512M -Xmx512M -Xss1M -XX:+HeapDumpOnOutOfMemoryError -XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=80 -XX:+UseParNewGC -XX:NewRatio=3  -Djava.rmi.server.hostname=localhost"
>     extraClasspath = ""
>   }
> {code}
> *Running the cluster:*
> {code}
> echo "Starting the master"
> cp conf/gear.pc.conf conf/gear.conf
> bin/master -ip 192.168.1.115 -port 3000 &
> echo "Starting services"
> bin/services &
> echo "Starting a worker (configured as on the device)"
> cp conf/gear.pc.slimworker.conf conf/gear.conf
> bin/worker &
> {code}
> *Deploing the DAG*
> I've deployed wordcountjava-2.11-0.8.0-assembly.jar.
> The app deploys, but doesn't run.
> (After changing   worker.executor-share-same-jvm-as-worker  to false and restarting the
cluster, example dag works.)
> See the attached logs:
> {code}
> 2016-05-18 08:47:31,453 ERROR OneForOneStrategy: io.gearpump.streaming.examples.wordcountjava.Split
> java.lang.ClassNotFoundException: io.gearpump.streaming.examples.wordcountjava.Split
> 	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> 	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> 	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> 	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> 	at io.gearpump.streaming.task.TaskUtil$.loadClass(TaskUtil.scala:30)
> 	at io.gearpump.streaming.executor.TaskLauncher.launch(TaskLauncher.scala:55)
> 	at io.gearpump.streaming.executor.Executor$$anonfun$dynamicDagPhase1$1.applyOrElse(Executor.scala:147)
> 	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170)
> 	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
> 	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
> 	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
> 	at akka.actor.Actor$class.aroundReceive(Actor.scala:480)
> 	at io.gearpump.streaming.executor.Executor.aroundReceive(Executor.scala:55)
> 	at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
> 	at akka.actor.ActorCell.invoke(ActorCell.scala:495)
> 	at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
> 	at akka.dispatch.Mailbox.run(Mailbox.scala:224)
> 	at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
> 	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> 	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> 	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> 	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> {code}
>   



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message