gearpump-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Huafeng Wang (JIRA)" <j...@apache.org>
Subject [jira] [Reopened] (GEARPUMP-124) SinkTask fails to start after change parallelism
Date Mon, 16 May 2016 02:30:12 GMT

     [ https://issues.apache.org/jira/browse/GEARPUMP-124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Huafeng Wang reopened GEARPUMP-124:
-----------------------------------

The root cause of this issue is that when changing the parallelism of DataSource/DataSink
processor, the original task's configuration is not inherit to the new processor.

Though the fix merge the configuration to the new processor in the frontend, it will make
the rest call very long and may fail due to URI length exceeds the limitation. 

> SinkTask fails to start after change parallelism
> ------------------------------------------------
>
>                 Key: GEARPUMP-124
>                 URL: https://issues.apache.org/jira/browse/GEARPUMP-124
>             Project: Apache Gearpump
>          Issue Type: Bug
>    Affects Versions: 0.8.0
>            Reporter: Qi Shu
>            Assignee: Huafeng Wang
>             Fix For: 0.8.1
>
>
> App started, run for a while, everything is ok, change the parallelism of sink(our own
hdfs sink), then the sink got following exceptions:
> [INFO] [05/11/2016 17:22:30.865] [Executor@app2exec23] Transit to state Application Ready.
This transition takes 99671 milliseconds
> [ERROR] [05/11/2016 17:22:30.865] [Executor@app2exec23] We got java.lang.reflect.InvocationTargetException
from Some(TaskId(6,0)), we will treat it as MessageLoss, so that the system will replay all
lost message
> java.lang.reflect.InvocationTargetException
> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> 	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> 	at io.gearpump.streaming.task.TaskWrapper.onStart(TaskWrapper.scala:89)
> 	at io.gearpump.streaming.task.TaskActor.onStart(TaskActor.scala:102)
> 	at io.gearpump.streaming.task.TaskActor.io$gearpump$streaming$task$TaskActor$$onStartClock(TaskActor.scala:194)
> 	at io.gearpump.streaming.task.TaskActor$$anonfun$waitForStartClock$1.applyOrElse(TaskActor.scala:211)
> 	at akka.actor.Actor$class.aroundReceive(Actor.scala:480)
> 	at io.gearpump.streaming.task.TaskActor.aroundReceive(TaskActor.scala:41)
> 	at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
> 	at akka.actor.ActorCell.invoke(ActorCell.scala:495)
> 	at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
> 	at akka.dispatch.Mailbox.run(Mailbox.scala:224)
> 	at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
> 	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> 	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> 	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> 	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> Caused by: java.util.NoSuchElementException: None.get
> 	at scala.None$.get(Option.scala:347)
> 	at scala.None$.get(Option.scala:345)
> 	at io.gearpump.streaming.sink.DataSinkTask.<init>(DataSinkTask.scala:35)
> 	... 19 more
> [WARN] [05/11/2016 17:22:30.867] [OneForOneStrategy] null
> [INFO] [05/11/2016 17:22:30.877] [Executor@app2exec23] Executor received restart tasks
> [INFO] [05/11/2016 17:22:30.878] [Express$] RegisterLocalActor: 25769803776, actor: processor_6_task_0



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message