helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kishore g <g.kish...@gmail.com>
Subject Re: question about helix-agent
Date Fri, 31 Jan 2014 22:55:45 GMT
By the way, if you prefer Python there is a Helix python agent. It should
not hard to add the ability to run a command on every transition using
python agent.

https://pypi.python.org/pypi/pyhelix



On Mon, Jan 27, 2014 at 5:29 AM, Benoit Vanderbeck <
benoit.vanderbeck@vaph.be> wrote:

> Thanks,
>
> your explanation did clarify.
> Will keep on playing with helix.
>
> regards,
>
> B
>
>
> On 23 January 2014 19:15, kishore g <g.kishore@gmail.com> wrote:
>
>> Not sure if subscribed to the mailing list. You can subscribe by sending
>> email to user-subscribe@helix.apache.org
>>
>> ---------- Forwarded message ----------
>> From: kishore g <g.kishore@gmail.com>
>> Date: Thu, Jan 23, 2014 at 9:30 AM
>> Subject: Re: question about helix-agent
>> To: user@helix.apache.org
>>
>>
>> Hi,
>>
>> The command needs to set on a per resource basis or at a cluster level.
>> The per resource config is specified under
>> /CONFIGS/RESOURCE/<resource_name> or /CONFIGS/CLUSTER/<cluster_name>.
>>
>> You did set it in the right place
>> /VAPH_DEV/CONFIGS/RESOURCE/simpleHttpServer
>>
>> But the configuration keys need to be slightly different.
>>
>> Take a look at the integration test TestHelixAgent
>>
>> The configuration looks like this
>> TestHelixAgent_test/CONFIGS/CLUSTER/TestHelixAgent_test
>>
>> {
>>   "id" : "TestHelixAgent_test",
>>   "listFields" : {
>>   },
>>   "mapFields" : {
>>   },
>>   "simpleFields" : {
>>     "SLAVE-MASTER.command" : "simpleHttpClient.py SLAVE-MASTER",
>>     "SLAVE-MASTER.command.timeout" : "0",
>>     "SLAVE-MASTER.command.workingDir" :
>> "/Users/kgopalak/Documents/projects/incubator-helix/helix-agent/src/main/scripts/integration-test/script",
>>     "OFFLINE-SLAVE.command" : "simpleHttpClient.py OFFLINE-SLAVE",
>>     "OFFLINE-SLAVE.command.workingDir" :
>> "/Users/kgopalak/Documents/projects/incubator-helix/helix-agent/src/main/scripts/integration-test/script",
>>     "MASTER-SLAVE.command" : "simpleHttpClient.py MASTER-SLAVE",
>>     "MASTER-SLAVE.command.workingDir" :
>> "/Users/kgopalak/Documents/projects/incubator-helix/helix-agent/src/main/scripts/integration-test/script",
>>     "SLAVE-OFFLINE.command" : "simpleHttpClient.py SLAVE-OFFLINE",
>>     "SLAVE-OFFLINE.command.workingDir" :
>> "/Users/kgopalak/Documents/projects/incubator-helix/helix-agent/src/main/scripts/integration-test/script",
>>     "OFFLINE-DROPPED.command" : "nop"
>>   }
>> }
>>
>> You can set these configs using helix admin api.
>>
>> Via java api
>>     HelixConfigScope scope =
>>         new
>> HelixConfigScopeBuilder(ConfigScopeProperty.CLUSTER).forCluster(clusterName).build();
>>     CommandConfig.Builder builder = new CommandConfig.Builder();
>>     CommandConfig cmdConfig =
>>         builder.setTransition("SLAVE", "MASTER")
>>                   .setCommand("simpleHttpClient.py SLAVE-MASTER")
>>                   .setCommandWorkingDir(workingDir).setCommandTimeout("0")
>>                   .setPidFile(pidFile)
>>                   .build();
>>     configAccessor.set(scope, cmdConfig.toKeyValueMap());
>>
>> Using Helix Admin
>> USAGE: --setConfig <ConfigScope(e.g. RESOURCE) ConfigScopeArgs(e.g.
>> myCluster,testDB) KeyValueMap(e.g. k1=v1,k2=v2)>      Set configs
>> example:
>>       ./helix-admin.sh --zkSvr localhost:2181 --setConfig CLUSTER
>> VAPH_DEV SLAVE-MASTER.command=<cmd>,SLAVE-MASTER.workingDir=<dir>,
>> SLAVE-MASTER.command.pidfile=<pid_file>
>>
>>
>> Let us know if this does not work.
>>
>> thanks,
>> Kishore G
>>
>>
>>
>>
>>
>>
>>
>>
>> On Thu, Jan 23, 2014 at 4:58 AM, Benoit Vanderbeck <
>> benoit.vanderbeck@vaph.be> wrote:
>>
>>>  Hi,
>>>
>>> i'm trying to run the helix-agent. But I can't seem to figure out how to
>>> configure the command that should be run when the statetransition changes
>>> from offline to online.
>>> I have tried with a simple setup : One instance and one resource
>>>
>>> IdealState for simpleHttpServer:
>>> {
>>>   "id" : "simpleHttpServer",
>>>   "mapFields" : {
>>>     "simpleHttpServer_0" : {
>>>       "vbxl003_80" : "ONLINE"
>>>     },
>>>     "simpleHttpServer_1" : {
>>>       "vbxl003_80" : "ONLINE"
>>>     },
>>>     "simpleHttpServer_2" : {
>>>       "vbxl003_80" : "ONLINE"
>>>     }
>>>   },
>>>   "listFields" : {
>>>     "simpleHttpServer_0" : [ "vbxl003_80" ],
>>>     "simpleHttpServer_1" : [ "vbxl003_80" ],
>>>     "simpleHttpServer_2" : [ "vbxl003_80" ]
>>>   },
>>>    "simpleFields" : {
>>>     "COMMAND" : "/tmp/simpleHttpServer.py",
>>>     "IDEAL_STATE_MODE" : "AUTO",
>>>     "NUM_PARTITIONS" : "3",
>>>     "OFFLINE-ONLINE.COMMAND" : "/tmp/simpleHttpServer.py",
>>>     "REBALANCE_MODE" : "SEMI_AUTO",
>>>     "REPLICAS" : "1",
>>>     "STATE_MODEL_DEF_REF" : "OnlineOffline",
>>>     "STATE_MODEL_FACTORY_NAME" : "DEFAULT"
>>>   }
>>> }
>>>
>>> ExternalView for simpleHttpServer:
>>> {
>>>    "id" : "simpleHttpServer",
>>>   "mapFields" : {
>>>   },
>>>   "listFields" : {
>>>   },
>>>   "simpleFields" : {
>>>     "BUCKET_SIZE" : "0"
>>>   }
>>>
>>>
>>> But when I start the agent I get following error :
>>>
>>> 677  [pool-2-thread-1] ERROR
>>> org.apache.helix.messaging.handling.HelixStateTransitionHandler  -
>>> Exception while executing a state transition task simpleHttpServer_0
>>> java.lang.reflect.InvocationTargetException
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>  at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>  at
>>> org.apache.helix.messaging.handling.HelixStateTransitionHandler.invoke(HelixStateTransitionHandler.java:299)
>>> at
>>> org.apache.helix.messaging.handling.HelixStateTransitionHandler.handleMessage(HelixStateTransitionHandler.java:253)
>>>  at
>>> org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:85)
>>> at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:42)
>>>  at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>  at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>  at java.lang.Thread.run(Thread.java:662)
>>> Caused by: java.lang.Exception: Unable to find command for transition
>>> from:OFFLINE to:ONLINE
>>>
>>>
>>> So the agent can not find what command should be executed when going
>>> from OFFLINE to ONLINE.
>>>
>>> I also tried to edit the configs tree in the znode
>>>
>>> get /VAPH_DEV/CONFIGS/RESOURCE/simpleHttpServer
>>> {
>>>   "id" : "simpleHttpServer",
>>>   "simpleFields" : {
>>>     "COMMAND" : "/tmp/simpleHttpServer.py",
>>>     "WORKING_DIR" : "/tmp",
>>>     "PID" : "/tmp/mypid",
>>>     "OFFLINE-ONLINE.COMMAND" : "/tmp/simpleHttpServer.py"
>>>   },
>>>   "listFields" : {
>>>    },
>>>   "mapFields" : {
>>>   }
>>> }
>>>
>>>
>>> But that didn't work either.
>>>
>>> Can someone point me in the right direction ? Where en how do define the
>>> command that the agent should execute.
>>>
>>> Regards,
>>>
>>> Benoît
>>> --
>>>  Benoît Vanderbeck
>>> Systeembeheerder VAPH
>>> tel: +32 2 225 84 80
>>>
>>>
>>> Raadpleeg uw VAPH-dossier online via http://mijn.vaph.be.
>>> ------------------------------------------------------------
>>> ---------------------
>>> http://www.vaph.be/disclaimer
>>
>>
>>
>>
>
>
> --
> Benoît Vanderbeck
> Systeembeheerder VAPH
> tel: +32 2 225 84 80
>
>
> Raadpleeg uw VAPH-dossier online via http://mijn.vaph.be.
> ------------------------------------------------------------
> ---------------------
> http://www.vaph.be/disclaimer
>

Mime
View raw message