incubator-s4-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From JiHyoun Park <apr...@gmail.com>
Subject Re: To deploy an app on multiple machines
Date Tue, 28 Aug 2012 08:33:26 GMT
Dear Matthieu

Thank you for your comment.
I copied the s4r to the other machine and made the zk server refer to the
local instance....
This is my current temporary solution.
I will try your method.

Best Regards
Jihyoun Park


On Tue, Aug 28, 2012 at 3:57 PM, Matthieu Morel <mmorel@apache.org> wrote:

> Hi,
>
> the /tmp path is probably local to the hosts on which you created the s4r
> file. Therefore, files under that path are not accessible from the other
> host.
>
> I suggest you copy/move the generated s4r to a shared directory, and
> update the s4r location in Zookeeper. You may also host as a static file in
> a web server.
>
> See the -s4r option in the "s4 deploy" command.
>
>
> Hope this helps,
>
> Matthieu
>
>
>
> On 8/28/12 4:16 AM, JiHyoun Park wrote:
>
>> Hi,
>>
>> I tried to run 'myApp (the basic testing sample)' on 2 machines (s4test4
>> and s4test5).
>> s4test5 is where the zkServer resides.
>> I created 2 clusters (cluster1 with 2 nodes for app, cluster2 with 1
>> node for adapter).
>> Then I started 1 app node on s4test5 and another app node and an adapter
>> node on s4test4.
>> The remote nodes on s4test4 were successfully attached to the zkServer
>> on s4test5 (pls refer to the s4 status attached at the bottom).
>> However, when I deployed the app and the adapter on s4test5, I got this
>> error message at the app node and the adater node in s4test4.
>>
>> org.apache.s4.deploy.**DeploymentFailedException: Cannot retrieve file
>> from uri [file:/tmp/1346061569272-0/**myApp.s4r]
>>
>> Is it a zookeeper problem or what should I do?
>> Below is the procedure that I followed to run the app using 2 machines.
>>
>> [@s4test5]
>> ./s4 zkServer
>>
>> ./s4 newCluster -c=cluster1 -nbTasks=2 -flp=12000
>> ./s4 newCluster -c=cluster2 -nbTasks=1 -flp=13000
>>
>> ./s4 node -c=cluster1
>>
>> [@s4test4]
>> ./s4 node -c=cluster1 -zk=s4test5
>> ./s4 node -c=cluster2 -p=s4.adapter.output.stream=**names  -zk=s4test5
>>
>> [@s4test5]
>> ./s4 deploy -appName=myApp -c=cluster1 -b=`pwd`/build.gradle
>> -a=hello.HelloApp
>> ./s4 deploy -appName=s4App -c=cluster2 -b=`pwd`/build.gradle
>> -a=hello.HelloInputAdapter
>>
>> echo "Bob" | nc localhost 15000
>>
>>
>> [root@s4test4 apache-s4-0.5.0-incubating-**src]# ./s4 node -c=cluster1
>> -zk=s4test5
>> 18:00:52.655 [main] INFO  org.apache.s4.core.Main - Initializing S4 node
>> with :
>> - comm module class [org.apache.s4.comm.**DefaultCommModule]
>> - comm configuration file [default.s4.comm.properties from classpath]
>> - core module class [org.apache.s4.core.**DefaultCoreModule]
>> - core configuration file[default.s4.core.**properties from classpath]
>> - extra modules: []
>> - inline parameters: []
>> 18:00:52.664 [main] DEBUG org.apache.s4.core.Main - Adding named
>> parameters for injection : [s4.cluster.zk_address=**s4test5]
>> 18:00:52.953 [main] INFO  org.apache.s4.core.Main - Starting S4 node.
>> This node will automatically download applications published for the
>> cluster it belongs to
>> 18:00:53.025 [main] INFO  o.a.s.comm.topology.**AssignmentFromZK - New
>> session:88215482878918664; state is : SyncConnected
>> 18:00:53.092 [main] INFO  o.a.s.comm.topology.**AssignmentFromZK -
>> Successfully acquired task:Task-1 by s4test4
>> 0 [ZkClient-EventThread-13-**s4test5] ERROR
>> org.I0Itec.zkclient.**ZkEventThread  - Error handling event ZkEvent[Data
>> of /s4/clusters/cluster1/app/**s4App changed sent to
>> org.apache.s4.deploy.**DistributedDeploymentManager$**
>> AppChangeListener@10ba97c3]
>> org.apache.s4.deploy.**DeploymentFailedException: Cannot retrieve file
>> from uri [file:/tmp/1346061569272-0/**myApp.s4r]
>>      at
>> org.apache.s4.deploy.**FileSystemS4RFetcher.fetch(**
>> FileSystemS4RFetcher.java:38)
>>      at
>> org.apache.s4.deploy.**DistributedDeploymentManager.**fetchS4App(**
>> DistributedDeploymentManager.**java:151)
>>      at
>> org.apache.s4.deploy.**DistributedDeploymentManager.**deployApplication(*
>> *DistributedDeploymentManager.**java:119)
>>      at
>> org.apache.s4.deploy.**DistributedDeploymentManager$**AppChangeListener.*
>> *handleDataChange(**DistributedDeploymentManager.**java:168)
>>      at org.I0Itec.zkclient.ZkClient$**6.run(ZkClient.java:547)
>>      at org.I0Itec.zkclient.**ZkEventThread.run(**ZkEventThread.java:71)
>>
>>
>> [root@s4test5 apache-s4-0.5.0-incubating-**src]# ./s4 status
>>
>> App Status
>> ------------------------------**------------------------------**
>> ------------------------------**------------------------------**
>> ----------
>>          Name
>> Cluster                                                  URI
>> ------------------------------**------------------------------**
>> ------------------------------**------------------------------**
>> ----------
>>         s4App              cluster2
>> file:/tmp/1346061580311-0/**s4App.s4r
>>         myApp             cluster1      file:/tmp/1346061569272-0/**
>> myApp.s4r
>> ------------------------------**------------------------------**
>> ------------------------------**------------------------------**
>> ----------
>>
>> Cluster Status
>> ------------------------------**------------------------------**
>> ------------------------------**------------------------------**
>> ----------
>>
>>            Active nodes
>>          Name                App           Tasks
>> ------------------------------**------------------------------**
>> --------------------
>>                                                     Number    Task
>> id                         Host                         Port
>> ------------------------------**------------------------------**
>> ------------------------------**------------------------------**
>> ----------
>>        cluster2             s4App           1         1
>> Task-0                    s4test4                     13000
>>        cluster1             myApp          2         2
>> Task-0                    s4test5                     12000
>>
>> Task-1                    s4test4                     12001
>> ------------------------------**------------------------------**
>> ------------------------------**------------------------------**
>> ----------
>>
>> Stream Status
>> ------------------------------**------------------------------**
>> ------------------------------**------------------------------**
>> ----------
>>          Name
>> Producers                                              Consumers
>> ------------------------------**------------------------------**
>> ------------------------------**------------------------------**
>> ----------
>>         names
>> --                                              cluster1(myApp)
>> ------------------------------**------------------------------**
>> ------------------------------**------------------------------**
>> ----------
>>
>>
>> Best Regards
>> Jihyoun Park
>>
>
>

Mime
View raw message