kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Rosenberg <...@squareup.com>
Subject Re: How to create the initial zookeeper chroot path for zk.connect?
Date Wed, 08 May 2013 23:37:42 GMT
It works if I manually create the chroot first.  But this is a bit
cumbersome if I want to do an automated roll out to multiple deployments,
etc....but workable....

Should I file a jira?


On Wed, May 8, 2013 at 4:31 PM, Jason Rosenberg <jbr@squareup.com> wrote:

> I'm seeing this issue with a single node zk instance, on my localhost.  If
> my zkconnect is "localhost:12345", it works...
>
> but if I add a chroot, e.g.: "localhost:12345/xyz", I get the same error:
> java.lang.IllegalArgumentException: Path length must be > 0
>
> I also get the error if I do: "locahost:12345,localhost:67890/xyz"
>
> Do I actually have to create the chroot manually first?  Or shouldn't it
> get created automatically?
>
> This using 0.8.0 latest.....
>
> Is this the same issue others are seeing (I realize previously the issue
> was with multiple zk hosts).  Does this need a jira?
>
> Also, I think the config page for zookeeper.connect really needs to be
> updated to make explicit that the 'chroot' part only gets added at the very
> end, because it's not clear at all....
>
> Jason
>
>
> On Mon, Apr 22, 2013 at 6:46 AM, Ryan Chan <ryanchan404@gmail.com> wrote:
>
>> It would be better if there is another configuration directive, e.g.
>> zk.chroot for the chroot path, currently it is not consistent as we also
>> need to specify the port for each zookeeper, isn't?
>>
>> Anyway, the doc can better explained this situation..
>>
>> Thanks anyway!
>>
>>
>> On Sun, Apr 21, 2013 at 11:10 PM, Scott Clasen <scott@heroku.com> wrote:
>>
>> > Since There is only 1 chroot for a zk cluster, if you specified for each
>> > server there would be a potential for error/mismatch
>> >
>> > Things would probably go really bad if you had mismatched chroots :)
>> >
>> > Sent from my iPhone
>> >
>> > On Apr 21, 2013, at 1:34 AM, Ryan Chan <ryanchan404@gmail.com> wrote:
>> >
>> > > Thanks, this solved the problem.
>> > >
>> > > But the connection string as "Zk1:2181,zk2:2181,zk3;2181/Kafka", seems
>> > > unintuitive?
>> > >
>> > >
>> > > On Sun, Apr 21, 2013 at 2:29 AM, Scott Clasen <scott@heroku.com>
>> wrote:
>> > >
>> > >> Afaik you only put the chroot on the end of the zk conn str...
>> > >>
>> > >> Zk1:2181,zk2:2181,zk3;2181/Kafka
>> > >>
>> > >> Not
>> > >>
>> > >> Zk1:2181/kafka,zk2:2181/Kafka,zk3:2181/Kafka
>> > >>
>> > >>
>> > >> Sent from my iPhone
>> > >>
>> > >> On Apr 20, 2013, at 9:03 AM, Neha Narkhede <neha.narkhede@gmail.com>
>> > >> wrote:
>> > >>
>> > >>> Please file a bug and mention the Kafka and zookeeper versions
used
>> for
>> > >> the
>> > >>> test.
>> > >>>
>> > >>> Thanks,
>> > >>> Neha
>> > >>>
>> > >>> On Saturday, April 20, 2013, Ryan Chan wrote:
>> > >>>
>> > >>>> Hello,
>> > >>>>
>> > >>>> Tried, still the same...
>> > >>>>
>> > >>>>
>> > >>>> bin/zkCli.sh -server
>> zookeeper1:2181,zookeeper2:2181,zookeeper3:2181
>> > >>>> [zk: zookeeper1,zookeeper2,zookeeper3(CONNECTED) 0]  ls /
>> > >>>> [testkafka, consumers, brokers, zookeeper]
>> > >>>> [zk: zookeeper1,zookeeper2,zookeeper3(CONNECTED) 1] rmr /testkafka
>> > >>>> [zk: zookeeper1,zookeeper2,zookeeper3(CONNECTED) 2] create
>> /testkafka
>> > ''
>> > >>>> [zk: zookeeper1,zookeeper2,zookeeper3(CONNECTED) 3] ls /
>> > >>>> [testkafka, consumers, brokers, zookeeper]
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>> And restart Kafka
>> > >>>>
>> > >>>> [2013-04-20 09:20:58,336] FATAL Fatal error during
>> KafkaServerStable
>> > >>>> startup. Prepare to shutdown (kafka.server.KafkaServerStartable)
>> > >>>> java.lang.IllegalArgumentException: Path length must be >
0
>> > >>>> at
>> > org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:48)
>> > >>>> at
>> > org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:35)
>> > >>>> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:626)
>> > >>>> at org.I0Itec.zkclient.ZkConnection.create(ZkConnection.java:87)
>> > >>>> at org.I0Itec.zkclient.ZkClient$1.call(ZkClient.java:308)
>> > >>>> at org.I0Itec.zkclient.ZkClient$1.call(ZkClient.java:304)
>> > >>>> at
>> org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675)
>> > >>>> at org.I0Itec.zkclient.ZkClient.create(ZkClient.java:304)
>> > >>>> at org.I0Itec.zkclient.ZkClient.createPersistent(ZkClient.java:213)
>> > >>>> at org.I0Itec.zkclient.ZkClient.createPersistent(ZkClient.java:223)
>> > >>>> at org.I0Itec.zkclient.ZkClient.createPersistent(ZkClient.java:223)
>> > >>>> at kafka.utils.ZkUtils$.createParentPath(ZkUtils.scala:47)
>> > >>>> at kafka.utils.ZkUtils$.createEphemeralPath(ZkUtils.scala:59)
>> > >>>> at
>> > >>
>> kafka.utils.ZkUtils$.createEphemeralPathExpectConflict(ZkUtils.scala:71)
>> > >>>> at
>> > >>
>> kafka.server.KafkaZooKeeper.registerBrokerInZk(KafkaZooKeeper.scala:54)
>> > >>>> at kafka.log.LogManager.startup(LogManager.scala:130)
>> > >>>> at kafka.server.KafkaServer.startup(KafkaServer.scala:81)
>> > >>>> at
>> > >>
>> kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:34)
>> > >>>> at kafka.Kafka$.main(Kafka.scala:47)
>> > >>>> at kafka.Kafka.main(Kafka.scala)
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>> Maybe I should report a bug?
>> > >>>> (I posted here first just to know if I have done sth stupid)
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>> On Sat, Apr 20, 2013 at 1:02 PM, Neha Narkhede <
>> > neha.narkhede@gmail.com
>> > >> <javascript:;>
>> > >>>>> wrote:
>> > >>>>
>> > >>>>> Hmm, so if you use all 3 zookeeper servers will creating
and
>> reading
>> > >>>>> the node, do you still see the problem ?
>> > >>>>>
>> > >>>>> zkCli.sh -server zookeeper1:2181,zookeeper2:2181,zookeeper3:2181
>> > >>>>> create /testkafka
>> > >>>>> ls /
>> > >>>>>
>> > >>>>> Thanks
>> > >>>>> Neha
>> > >>>>>
>> > >>>>> On Fri, Apr 19, 2013 at 8:55 PM, Ryan Chan <ryanchan404@gmail.com
>> >
>> > >>>> wrote:
>> > >>>>>> Hi,
>> > >>>>>>
>> > >>>>>> Actually I followed the above link to setup my zookeeper1
to
>> > >>>> zookeeper3.
>> > >>>>>>
>> > >>>>>> They are in the same quorum, as you can see in my above
example
>> that
>> > >>>>> when I
>> > >>>>>> created the /testkafka path in zookeeper1, I can see
list it in
>> > >>>>> zookeeper2
>> > >>>>>>
>> > >>>>>> Thanks
>> > >>>>>>
>> > >>>>>>
>> > >>>>>> On Sat, Apr 20, 2013 at 12:08 AM, Neha Narkhede <
>> > >>>> neha.narkhede@gmail.com
>> > >>>>>> wrote:
>> > >>>>>>
>> > >>>>>>> I'm pretty sure the issue is that those 3 zookeepers
are not
>> part
>> > of
>> > >>>>>>> the same quorum. Try following the quickstart for
starting up a
>> > local
>> > >>>>>>> zookeeper instance or follow
>> > >>
>> >
>> http://zookeeper.apache.org/doc/r3.3.3/zookeeperAdmin.html#sc_zkMulitServerSetup
>> > >>>>>>> for a zookeeper cluster setup.
>> > >>>>>>>
>> > >>>>>>> Thanks,
>> > >>>>>>> Neha
>> > >>>>>>>
>> > >>>>>>> On Thu, Apr 18, 2013 at 10:50 PM, Jonathan Creasy
<jc@box.com>
>> > >> wrote:
>> > >>>>>>>> I made the patch to create the chroot and it
doesn't handle
>> > multiple
>> > >>>>> zk
>> > >>>>>>>> addresses.
>> > >>>>>>>>
>> > >>>>>>>> We fixed it but I guess that patch didn't get
submitted. I will
>> > >>>> make a
>> > >>>>>>>> ticket here to get that done.
>> > >>>>>>>> On Apr 18, 2013 10:47 PM, "Ryan Chan" <ryanchan404@gmail.com>
>> > >>>> wrote:
>> > >>>>>>>>
>> > >>>>>>>>> Yes, using the latest Kafka 0.7.2, just
tried to reproduce
>> again
>> > >>>>>>>>>
>> > >>>>>>>>> 1. Install a single node Kafka, three nodes
zookeeper
>> instances
>> > >>>>>>>>>
>> > >>>>>>>>>   kafka1
>> > >>>>>>>>>   zookeeper1
>> > >>>>>>>>>   zookeeper2
>> > >>>>>>>>>   zookeeper3
>> > >>>>>>>>>
>> > >>>>>>>>> 2. Using a simple Kafka config, able to
start without error in
>> > the
>> > >>>>> log
>> > >>>>>>>>>
>> > >>>>>>>>>   brokerid=1
>> > >>>>>>>>>   log.dir=/data/kafka
>> > >>>>>>>>>   zk.connect=zookeeper1:2181,zookeeper2:2181,zookeeper3:2181
>> > >>>>>>>>>
>> > >>>>>>>>> 3. Now, create a path in zookeeper1
>> > >>>>>>>>>
>> > >>>>>>>>>   zkCli.sh -server zookeeper1:2181
>> > >>>>>>>>>
>> > >>>>>>>>>       ls /
>> > >>>>>>>>>       [consumers, brokers, zookeeper]
>> > >>>>>>>>>       create /testkafka ''
>> > >>>>>>>>>       ls /
>> > >>>>>>>>>       [testkafka, consumers, brokers, zookeeper]
>> > >>>>>>>>>
>> > >>>>>>>>> Quit & Done.
>> > >>>>>>>>>
>> > >>>>>>>>> 4. Verify from zookeeper2
>> > >>>>>>>>>
>> > >>>>>>>>>   zkCli.sh -server zookeeper2:2181
>> > >>>>>>>>>       ls /
>> > >>>>>>>>>       [testkafka, consumers, brokers, zookeeper]
>> > >>>>>>>>>
>> > >>>>>>>>> Seems ok
>> > >>>>>>>>>
>> > >>>>>>>>> 4. Update kafka config and restart
>> > >>
>> >
>> zk.connect=zookeeper1:2181/testkafka,zookeeper2:2181/testkafka,zookeeper3:2181/testkafka
>> > >>>>>>>>>
>> > >>>>>>>>> Restart and have the error
>> > >>>>>>>>>
>> > >>>>>>>>> [2013-04-19 05:35:14,846] FATAL Fatal error
during
>> > >>>> KafkaServerStable
>> > >>>>>>>>> startup. Prepare to shutdown
>> (kafka.server.KafkaServerStartable)
>> > >>
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message