Return-Path: X-Original-To: apmail-helix-user-archive@minotaur.apache.org Delivered-To: apmail-helix-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A61A3C71F for ; Fri, 14 Jun 2013 11:29:21 +0000 (UTC) Received: (qmail 78971 invoked by uid 500); 14 Jun 2013 11:29:21 -0000 Delivered-To: apmail-helix-user-archive@helix.apache.org Received: (qmail 78947 invoked by uid 500); 14 Jun 2013 11:29:21 -0000 Mailing-List: contact user-help@helix.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@helix.incubator.apache.org Delivered-To: mailing list user@helix.incubator.apache.org Received: (qmail 78940 invoked by uid 99); 14 Jun 2013 11:29:21 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 14 Jun 2013 11:29:21 +0000 X-ASF-Spam-Status: No, hits=-0.1 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of mingfang@mac.com designates 17.172.204.236 as permitted sender) Received: from [17.172.204.236] (HELO st11p01mm-asmtp001.mac.com) (17.172.204.236) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 14 Jun 2013 11:29:16 +0000 Received: from [192.168.1.101] (pool-108-46-156-132.nycmny.fios.verizon.net [108.46.156.132]) by st11p01mm-asmtp001.mac.com (Oracle Communications Messaging Server 7u4-27.05(7.0.4.27.4) 64bit (built Apr 23 2013)) with ESMTPSA id <0MOD00G2PRW6H760@st11p01mm-asmtp001.mac.com> for user@helix.incubator.apache.org; Fri, 14 Jun 2013 11:28:56 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.10.8794,1.0.431,0.0.0000 definitions=2013-06-14_03:2013-06-14,2013-06-14,1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 ipscore=0 suspectscore=0 phishscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=6.0.2-1305010000 definitions=main-1306140065 Subject: Re: Is zookeeper for Helix has standalone JAVA process? References: <9B1649B4-6309-4D37-A167-D4FE67EC25DA@mac.com> From: Ming Fang Content-type: multipart/alternative; boundary=Apple-Mail-539A3970-55E1-4D3C-9BC7-9911E9FFACC7 X-Mailer: iPad Mail (10B329) In-reply-to: Message-id: Date: Fri, 14 Jun 2013 07:28:54 -0400 To: "user@helix.incubator.apache.org" Content-transfer-encoding: 7bit MIME-version: 1.0 (1.0) X-Virus-Checked: Checked by ClamAV on apache.org --Apple-Mail-539A3970-55E1-4D3C-9BC7-9911E9FFACC7 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable ZAC currently runs Zookeeper in standalone mode. =20 Therefore you can only run one instance.=20 If you also run the participant in it then that limits you to one instance o= f the participant also.=20 Sent from my iPad On Jun 14, 2013, at 7:23 AM, Erwin Karbasi wrote: > Hi Ming, >=20 > Thank you for fast turnaround response. >=20 > Could I run my app that uses Helix (participant), controller and zookeeper= on the same process? >=20 > Cheers, > Erwin >=20 > On Jun 14, 2013 2:16 PM, "Ming Fang" wrote: >> Erwin >>=20 >> Here is an example of running Zookeeper, and Controller in the same proce= ss, >> https://github.com/mingfang/apache-helix/blob/master/helix-example/src/ma= in/java/org/apache/helix/examples/ZAC.java >> We have this in production.=20 >>=20 >> Sent from my iPad >>=20 >> On Jun 14, 2013, at 3:34 AM, Erwin Karbasi wrote: >>=20 >>> I know that it's better that participants, controller and zookeeper will= have separate java processes but we have more then 40 boxes with app server= that we'd like to manage them with Helix Leader election and cannot add mor= e processes because each additional java process requires management (monito= ring, alarms, KPI, etc...), so we prefer to avoid the additional java proces= ses. We'll not manage to approve additional processes >>>=20 >>> Your insight would appreciated. >>>=20 >>> Thanks, >>> Erwin >>>=20 >>> On Jun 14, 2013 9:58 AM, "kishore g" wrote: >>>> Hi Erwin, >>>>=20 >>>> This is how I would setup your cluster. Allocate 3 boxes that only run z= ookeeper and Helix controller. Its better that they are two separate process= . These boxes need not be powerful but isolating zookeeper and helix control= ler from actual server is a good deployment practice. See http://zookeeper.a= pache.org/doc/trunk/zookeeperAdmin.html#sc_zkMulitServerSetup on how to use z= ookeeper clustering ensemble. >>>>=20 >>>>=20 >>>> Run the real servers on the rest of the boxes. Do not start zookeeper a= s part of these processes, instead simply provide the zookeeper address (zk1= :port1,zk2:port2,zk3:port3) when you invoke Helix Api. >>>>=20 >>>>=20 >>>> Thanks, >>>> Kishore G >>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>> On Thu, Jun 13, 2013 at 10:05 PM, Erwin Karbasi wr= ote: >>>>> Hi Kishore, >>>>>=20 >>>>> We have more then 50 nodes. >>>>>=20 >>>>> Cheers, >>>>> Erwin >>>>>=20 >>>>> On Jun 14, 2013 1:51 AM, "kishore g" wrote: >>>>>> Hi Erwin, >>>>>>=20 >>>>>> How many nodes do you have in your cluster ? >>>>>>=20 >>>>>> thanks, >>>>>> Kishore G >>>>>>=20 >>>>>>=20 >>>>>> On Thu, Jun 13, 2013 at 1:43 PM, Erwin Karbasi w= rote: >>>>>>> Hello Maters, >>>>>>>=20 >>>>>>> We don't want to have separate/specific JAVA process for Zookeeper. >>>>>>> As I figured out from the Master-Slave sample we can start the Zooke= eper programmatically using the Zookeeper API. >>>>>>>=20 >>>>>>> I have several questions: >>>>>>>=20 >>>>>>> 1. If we are using the Zookeeper API for initializing the Zookeeper f= rom the Helix app (out app that uses Helix) would we have specific process f= or Zookeeper? Would we have one process for our app and another for the Zook= eeper?=20 >>>>>>> What happens if we'll not initialize the Zookeeper from the Helix ap= p programmatically but using the commands, whether then would we have specif= ic process for Zookeeper? >>>>>>>=20 >>>>>>> 2. If we'll decide to initialize Zookeeper from Helix app, programma= tically, How can we use Zookeeper clustering (ensemble) , is there any API f= or Zookeeper clustering, any direction would highly appreciated. >>>>>>>=20 >>>>>>> 3. Is it possible to start the Zookeeper by using commands and not i= nitializing specific process for it but using the Helix app (our app that us= es Helix API)? >>>>>>>=20 >>>>>>> I hope my questions are enough clear. >>>>>>>=20 >>>>>>> Thanks in advance,=20 >>>>>>> Erwin Karbasi >>>>>>> AT&T, Senior Software Architect >>>>>>=20 >>>>=20 --Apple-Mail-539A3970-55E1-4D3C-9BC7-9911E9FFACC7 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 7bit
ZAC currently runs Zookeeper in standalone mode.  
Therefore you can only run one instance. 
If you also run the participant in it then that limits you to one instance of the participant also. 


Sent from my iPad

On Jun 14, 2013, at 7:23 AM, Erwin Karbasi <erwin@optinity.com> wrote:

Hi Ming,

Thank you for fast turnaround response.

Could I run my app that uses Helix (participant), controller and zookeeper on the same process?

Cheers,
Erwin

On Jun 14, 2013 2:16 PM, "Ming Fang" <mingfang@mac.com> wrote:
Erwin

Here is an example of running Zookeeper, and Controller in the same process,
We have this in production. 

Sent from my iPad

On Jun 14, 2013, at 3:34 AM, Erwin Karbasi <erwin@optinity.com> wrote:

I know that it's better that participants, controller and zookeeper will have separate java processes but we have more then 40 boxes with app server that we'd like to manage them with Helix Leader election and cannot add more processes because each additional java process requires management (monitoring, alarms, KPI, etc...), so we prefer to avoid the additional java processes. We'll not manage to approve additional processes

Your insight would appreciated.

Thanks,
Erwin

On Jun 14, 2013 9:58 AM, "kishore g" <g.kishore@gmail.com> wrote:
Hi Erwin,

This is how I would setup your cluster. Allocate 3 boxes that only run zookeeper and Helix controller. Its better that they are two separate process. These boxes need not be powerful but isolating zookeeper and helix controller from actual server is a good deployment practice. See http://zookeeper.apache.org/doc/trunk/zookeeperAdmin.html#sc_zkMulitServerSetup on how to use zookeeper clustering ensemble.


Run the real servers on the rest of the boxes. Do not start zookeeper as part of these processes, instead simply provide the zookeeper address (zk1:port1,zk2:port2,zk3:port3) when you invoke Helix Api.


Thanks,
Kishore G








On Thu, Jun 13, 2013 at 10:05 PM, Erwin Karbasi <erwin@optinity.com> wrote:

Hi Kishore,

We have  more then 50 nodes.

Cheers,
Erwin

On Jun 14, 2013 1:51 AM, "kishore g" <g.kishore@gmail.com> wrote:
Hi Erwin,

How many nodes do you have in your cluster ?

thanks,
Kishore G


On Thu, Jun 13, 2013 at 1:43 PM, Erwin Karbasi <erwin@optinity.com> wrote:
Hello Maters,

We don't want to have separate/specific JAVA process for Zookeeper.
As I figured out from the Master-Slave sample we can start the Zookeeper programmatically using the Zookeeper API.

I have several questions:

1. If we are using the Zookeeper API for initializing the Zookeeper from the Helix app (out app that uses Helix) would we have specific process for Zookeeper? Would we have one process for our app and another for the Zookeeper?
What happens if we'll not initialize the Zookeeper from the Helix app programmatically but using the commands, whether then would we have specific process for Zookeeper?

2. If we'll decide to initialize Zookeeper from Helix app, programmatically, How can we use Zookeeper clustering (ensemble) , is there any API for Zookeeper clustering, any direction would highly appreciated.

3. Is it possible to start the Zookeeper by using commands and not initializing specific process for it but using the Helix app (our app that uses Helix API)?

I hope my questions are enough clear.

Thanks in advance,
Erwin Karbasi
AT&T, Senior Software Architect


--Apple-Mail-539A3970-55E1-4D3C-9BC7-9911E9FFACC7--