gearpump-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Weihua Jiang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GEARPUMP-19) JarStoreService is not on the client classpath
Date Thu, 07 Apr 2016 03:24:25 GMT

    [ https://issues.apache.org/jira/browse/GEARPUMP-19?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15229599#comment-15229599
] 

Weihua Jiang commented on GEARPUMP-19:
--------------------------------------

Maybe we shall use following solution mentioned at #1346 (https://github.com/gearpump/gearpump/issues/1346):
A solution to this problem is change the flow how a JAR is uploaded.

Currently, jar is uploaded via this flow:

For local jar store
```
1. client ask master for the file server address
2. client upload file to file server, and file server persist to master local directory.
3. worker download file from file server.
```

For DFS jar store
```
1. client upload file directly to DFS directory
2. worker download file from DFS directory
```
The solution to avoid current problem and also eliminate the hadoop dependency at client side
#1244 is using following flow (considering worker can be at IoT device, so it is also better
if HDFS dependency can be avoided there):
```
1. client ask master for the file server address
2. client upload file to file server, and file server persist to configured master local or
HDFS directory.
3. worker download file from master local directory via file server.
```
So, now Jar Store is totally behind Master and file server.


> JarStoreService is not on the client classpath
> ----------------------------------------------
>
>                 Key: GEARPUMP-19
>                 URL: https://issues.apache.org/jira/browse/GEARPUMP-19
>             Project: Apache Gearpump
>          Issue Type: Bug
>          Components: core, daemon
>    Affects Versions: 0.8.0
>            Reporter: Manu Zhang
>            Priority: Blocker
>
> To submit an application jar, client creates JarStore and requires JarStoreService (daemon
module) to be on the classpath but daemon is not published. This breaks any application that
is not submit by Gearpump. E.g. 
> {code}
> Exception in thread "main" java.util.NoSuchElementException: None.get
> 	at scala.None$.get(Option.scala:347)
> 	at scala.None$.get(Option.scala:345)
> 	at io.gearpump.jarstore.JarStoreService$.get(JarStoreService.scala:81)
> 	at io.gearpump.jarstore.JarStoreService$.get(JarStoreService.scala:72)
> 	at io.gearpump.cluster.client.ClientContext.<init>(ClientContext.scala:61)
> 	at io.gearpump.cluster.client.ClientContext$.apply(ClientContext.scala:164)
> 	at io.gearpump.cluster.client.ClientContext.apply(ClientContext.scala)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message