incubator-s4-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kishore g <g.kish...@gmail.com>
Subject Re: Information about Apache S4 and Helix
Date Tue, 14 Jan 2014 16:28:10 GMT
Hi Sergio,

It did work for me, there were some changes we did to keep it backward
compatible with existing S4.

Here are the steps
BUILD
./gradlew eclipse
./gradlew clean install -DskipTests
./gradlew s4-tools:installApp

SETUP CLUSTER

./s4 zkServer -clean
&
//start zookeeper
./s4 addNodes   -helix -c=cluster1 -nbNodes=2 -flp=12000
//create cluster for app with 2 nodes
./s4 addNodes   -helix -c=cluster2 -nbNodes=1 -flp=13000
//create cluster for adapter with 1 node
./s4 createTask -helix -c=cluster1 -id=AggregatedTopicSeen -p=6 -r=1
-s=AggregatedTopicSeen     //create the task, this will partition the
stream and not PE
./s4 createTask -helix -c=cluster1 -id=RawStatus -p=6 -r=1
-s=RawStatus                                         //create the task,
this will partition the stream and not PE
./s4 createTask -helix -c=cluster1 -id=TopicSeen -p=6 -r=1
-s=TopicSeen                                          //create the task,
this will partition the stream and not PE

BUILD APP & DEPLOY
./s4 s4r -a=org.apache.s4.example.twitter.TwitterCounterApp
-b=`pwd`/test-apps/twitter-counter/build.gradle twitter-counter  //build
twitter-counter app
./s4 deploy -helix -c=cluster1
-s4r=file:`pwd`/test-apps/twitter-counter/build/libs/twitter-counter.s4r
-appName=twitter-counter //deploy twitter-counter app
./s4 deploy -appName=twitter-adapter -c=cluster2
-b=`pwd`/test-apps/twitter-adapter/build.gradle
-p=s4.adapter.output.stream=RawStatus  //build twitter-adapter, deploy does
not work here, use next command to deploy.
./s4 deploy -helix -c=cluster2
-s4r=file:`pwd`/test-apps/twitter-adapter/build/libs/twitter-adapter.s4r
-appName=twitter-adapter -p=s4.adapter.output.stream=RawStatus

START S4 Nodes
./s4 node -helix -cluster=cluster1 -zk=localhost:2181 &
./s4 node -helix -cluster=cluster1 -zk=localhost:2181 &

START ADAPTER Node
./s4 node -helix -cluster=cluster2 -zk=localhost:2181 &

tail -f TopNTopics.txt //shows the top trending topics

You need to setup twitter4j.properties in your home directory to access
twitter feed.



thanks,
Kishore G



On Mon, Jan 13, 2014 at 7:13 AM, Sergio Vavassori <svavassori@conwet.com>wrote:

> On 9 January 2014 20:29, kishore g <g.kishore@gmail.com> wrote:
>
> > I will take a stab at making S4-110-new branch working.
> >
>
> OK, thanks.
>
> Let me know if you progress on that.
>
> Best Regards,
> Sergio Vavassori
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message