gearpump-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Manu Zhang (JIRA)" <>
Subject [jira] [Commented] (GEARPUMP-67) Support IoT temp isolation model
Date Thu, 21 Apr 2016 07:56:25 GMT


Manu Zhang commented on GEARPUMP-67:

Comments history,


according to Gearpump Actor Hierarchy,

Worker and Executors have been closely monitored/supervised by Worker Manager and AppMaster.
Thus, any network isolation of a worker will be regarded as a worker failure. This makes the
IoT case with network isolation impossible.

I think we have several solutions here:

    relax this monitor/supervision chain. That is, for certain worker/executor, admin or user
can specify that this is remote on edge device. So when network isolation detected, Worker
manager and AppMaster can handle it with different strategy. However below are the uncertainties
(need further investigation):
        what will worker/executor behave when it find himself isolated? Is there any policy
to configure? Or it will die immediately?
        how to route executor output to a temp storage transparently? I assume transparency
is a desired feature.
    keep edge device as a whole actor hierarchy. That is, edge device runs its own gearpump
service instance. So basically, the data center gearpump cluster (DC) and edge device gearpump
cluster/service (EDGE) are separate. But, add some facility to allow DC to control EDGE (upload
app, kill app, etc). This is easier. The main drawback here is we lose the task move transparency.
In no. 1 solution, we actually can move task between edge and data center transparently as
they are the same gearpump. But, in this solution this is no longer possible.

Solution 2 sounds reasonable. Data transparency is IMO required for computing intensive devices.
For IoT devices e.g. temperature sensors, CCTVs, they are location based. Why they need a
task move transparency?

> Support IoT temp isolation model 
> ---------------------------------
>                 Key: GEARPUMP-67
>                 URL:
>             Project: Apache Gearpump
>          Issue Type: New Feature
>            Reporter: Manu Zhang
>            Assignee: Weihua Jiang
> imported from [] on behalf of [~whjiang].
Here's his proposal. 
> One important characteristics of IoT deployment is that: the edge device can be temp
out of reach from data center. This out-reach can last days, even weeks.
> And the typical deployment requires that, when such a dis-connection happens, the edge
device shall be able to run the deployed task continuously and flush its output to local disk.
And when the connection reestablished, the stored events will be resend to data center.
> I am not sure how difficult for Gearpump to support such a model. But, if we want to
target IoT, this is a MUST HAVE, I assume.

This message was sent by Atlassian JIRA

View raw message