kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jun Rao <jun...@gmail.com>
Subject Re: How to create the initial zookeeper chroot path for zk.connect?
Date Thu, 09 May 2013 04:28:30 GMT
Actually, we already have a jira: KAFKA-294.

Thanks,

Jun


On Wed, May 8, 2013 at 4:37 PM, Jason Rosenberg <jbr@squareup.com> wrote:

> It works if I manually create the chroot first.  But this is a bit
> cumbersome if I want to do an automated roll out to multiple deployments,
> etc....but workable....
>
> Should I file a jira?
>
>
> On Wed, May 8, 2013 at 4:31 PM, Jason Rosenberg <jbr@squareup.com> wrote:
>
> > I'm seeing this issue with a single node zk instance, on my localhost.
>  If
> > my zkconnect is "localhost:12345", it works...
> >
> > but if I add a chroot, e.g.: "localhost:12345/xyz", I get the same error:
> > java.lang.IllegalArgumentException: Path length must be > 0
> >
> > I also get the error if I do: "locahost:12345,localhost:67890/xyz"
> >
> > Do I actually have to create the chroot manually first?  Or shouldn't it
> > get created automatically?
> >
> > This using 0.8.0 latest.....
> >
> > Is this the same issue others are seeing (I realize previously the issue
> > was with multiple zk hosts).  Does this need a jira?
> >
> > Also, I think the config page for zookeeper.connect really needs to be
> > updated to make explicit that the 'chroot' part only gets added at the
> very
> > end, because it's not clear at all....
> >
> > Jason
> >
> >
> > On Mon, Apr 22, 2013 at 6:46 AM, Ryan Chan <ryanchan404@gmail.com>
> wrote:
> >
> >> It would be better if there is another configuration directive, e.g.
> >> zk.chroot for the chroot path, currently it is not consistent as we also
> >> need to specify the port for each zookeeper, isn't?
> >>
> >> Anyway, the doc can better explained this situation..
> >>
> >> Thanks anyway!
> >>
> >>
> >> On Sun, Apr 21, 2013 at 11:10 PM, Scott Clasen <scott@heroku.com>
> wrote:
> >>
> >> > Since There is only 1 chroot for a zk cluster, if you specified for
> each
> >> > server there would be a potential for error/mismatch
> >> >
> >> > Things would probably go really bad if you had mismatched chroots :)
> >> >
> >> > Sent from my iPhone
> >> >
> >> > On Apr 21, 2013, at 1:34 AM, Ryan Chan <ryanchan404@gmail.com> wrote:
> >> >
> >> > > Thanks, this solved the problem.
> >> > >
> >> > > But the connection string as "Zk1:2181,zk2:2181,zk3;2181/Kafka",
> seems
> >> > > unintuitive?
> >> > >
> >> > >
> >> > > On Sun, Apr 21, 2013 at 2:29 AM, Scott Clasen <scott@heroku.com>
> >> wrote:
> >> > >
> >> > >> Afaik you only put the chroot on the end of the zk conn str...
> >> > >>
> >> > >> Zk1:2181,zk2:2181,zk3;2181/Kafka
> >> > >>
> >> > >> Not
> >> > >>
> >> > >> Zk1:2181/kafka,zk2:2181/Kafka,zk3:2181/Kafka
> >> > >>
> >> > >>
> >> > >> Sent from my iPhone
> >> > >>
> >> > >> On Apr 20, 2013, at 9:03 AM, Neha Narkhede <
> neha.narkhede@gmail.com>
> >> > >> wrote:
> >> > >>
> >> > >>> Please file a bug and mention the Kafka and zookeeper versions
> used
> >> for
> >> > >> the
> >> > >>> test.
> >> > >>>
> >> > >>> Thanks,
> >> > >>> Neha
> >> > >>>
> >> > >>> On Saturday, April 20, 2013, Ryan Chan wrote:
> >> > >>>
> >> > >>>> Hello,
> >> > >>>>
> >> > >>>> Tried, still the same...
> >> > >>>>
> >> > >>>>
> >> > >>>> bin/zkCli.sh -server
> >> zookeeper1:2181,zookeeper2:2181,zookeeper3:2181
> >> > >>>> [zk: zookeeper1,zookeeper2,zookeeper3(CONNECTED) 0]  ls
/
> >> > >>>> [testkafka, consumers, brokers, zookeeper]
> >> > >>>> [zk: zookeeper1,zookeeper2,zookeeper3(CONNECTED) 1] rmr
> /testkafka
> >> > >>>> [zk: zookeeper1,zookeeper2,zookeeper3(CONNECTED) 2] create
> >> /testkafka
> >> > ''
> >> > >>>> [zk: zookeeper1,zookeeper2,zookeeper3(CONNECTED) 3] ls
/
> >> > >>>> [testkafka, consumers, brokers, zookeeper]
> >> > >>>>
> >> > >>>>
> >> > >>>>
> >> > >>>> And restart Kafka
> >> > >>>>
> >> > >>>> [2013-04-20 09:20:58,336] FATAL Fatal error during
> >> KafkaServerStable
> >> > >>>> startup. Prepare to shutdown (kafka.server.KafkaServerStartable)
> >> > >>>> java.lang.IllegalArgumentException: Path length must be
> 0
> >> > >>>> at
> >> > org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:48)
> >> > >>>> at
> >> > org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:35)
> >> > >>>> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:626)
> >> > >>>> at org.I0Itec.zkclient.ZkConnection.create(ZkConnection.java:87)
> >> > >>>> at org.I0Itec.zkclient.ZkClient$1.call(ZkClient.java:308)
> >> > >>>> at org.I0Itec.zkclient.ZkClient$1.call(ZkClient.java:304)
> >> > >>>> at
> >> org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675)
> >> > >>>> at org.I0Itec.zkclient.ZkClient.create(ZkClient.java:304)
> >> > >>>> at
> org.I0Itec.zkclient.ZkClient.createPersistent(ZkClient.java:213)
> >> > >>>> at
> org.I0Itec.zkclient.ZkClient.createPersistent(ZkClient.java:223)
> >> > >>>> at
> org.I0Itec.zkclient.ZkClient.createPersistent(ZkClient.java:223)
> >> > >>>> at kafka.utils.ZkUtils$.createParentPath(ZkUtils.scala:47)
> >> > >>>> at kafka.utils.ZkUtils$.createEphemeralPath(ZkUtils.scala:59)
> >> > >>>> at
> >> > >>
> >> kafka.utils.ZkUtils$.createEphemeralPathExpectConflict(ZkUtils.scala:71)
> >> > >>>> at
> >> > >>
> >> kafka.server.KafkaZooKeeper.registerBrokerInZk(KafkaZooKeeper.scala:54)
> >> > >>>> at kafka.log.LogManager.startup(LogManager.scala:130)
> >> > >>>> at kafka.server.KafkaServer.startup(KafkaServer.scala:81)
> >> > >>>> at
> >> > >>
> >> kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:34)
> >> > >>>> at kafka.Kafka$.main(Kafka.scala:47)
> >> > >>>> at kafka.Kafka.main(Kafka.scala)
> >> > >>>>
> >> > >>>>
> >> > >>>>
> >> > >>>> Maybe I should report a bug?
> >> > >>>> (I posted here first just to know if I have done sth stupid)
> >> > >>>>
> >> > >>>>
> >> > >>>>
> >> > >>>>
> >> > >>>> On Sat, Apr 20, 2013 at 1:02 PM, Neha Narkhede <
> >> > neha.narkhede@gmail.com
> >> > >> <javascript:;>
> >> > >>>>> wrote:
> >> > >>>>
> >> > >>>>> Hmm, so if you use all 3 zookeeper servers will creating
and
> >> reading
> >> > >>>>> the node, do you still see the problem ?
> >> > >>>>>
> >> > >>>>> zkCli.sh -server zookeeper1:2181,zookeeper2:2181,zookeeper3:2181
> >> > >>>>> create /testkafka
> >> > >>>>> ls /
> >> > >>>>>
> >> > >>>>> Thanks
> >> > >>>>> Neha
> >> > >>>>>
> >> > >>>>> On Fri, Apr 19, 2013 at 8:55 PM, Ryan Chan <
> ryanchan404@gmail.com
> >> >
> >> > >>>> wrote:
> >> > >>>>>> Hi,
> >> > >>>>>>
> >> > >>>>>> Actually I followed the above link to setup my
zookeeper1 to
> >> > >>>> zookeeper3.
> >> > >>>>>>
> >> > >>>>>> They are in the same quorum, as you can see in
my above example
> >> that
> >> > >>>>> when I
> >> > >>>>>> created the /testkafka path in zookeeper1, I can
see list it in
> >> > >>>>> zookeeper2
> >> > >>>>>>
> >> > >>>>>> Thanks
> >> > >>>>>>
> >> > >>>>>>
> >> > >>>>>> On Sat, Apr 20, 2013 at 12:08 AM, Neha Narkhede
<
> >> > >>>> neha.narkhede@gmail.com
> >> > >>>>>> wrote:
> >> > >>>>>>
> >> > >>>>>>> I'm pretty sure the issue is that those 3
zookeepers are not
> >> part
> >> > of
> >> > >>>>>>> the same quorum. Try following the quickstart
for starting up
> a
> >> > local
> >> > >>>>>>> zookeeper instance or follow
> >> > >>
> >> >
> >>
> http://zookeeper.apache.org/doc/r3.3.3/zookeeperAdmin.html#sc_zkMulitServerSetup
> >> > >>>>>>> for a zookeeper cluster setup.
> >> > >>>>>>>
> >> > >>>>>>> Thanks,
> >> > >>>>>>> Neha
> >> > >>>>>>>
> >> > >>>>>>> On Thu, Apr 18, 2013 at 10:50 PM, Jonathan
Creasy <jc@box.com
> >
> >> > >> wrote:
> >> > >>>>>>>> I made the patch to create the chroot
and it doesn't handle
> >> > multiple
> >> > >>>>> zk
> >> > >>>>>>>> addresses.
> >> > >>>>>>>>
> >> > >>>>>>>> We fixed it but I guess that patch didn't
get submitted. I
> will
> >> > >>>> make a
> >> > >>>>>>>> ticket here to get that done.
> >> > >>>>>>>> On Apr 18, 2013 10:47 PM, "Ryan Chan"
<ryanchan404@gmail.com
> >
> >> > >>>> wrote:
> >> > >>>>>>>>
> >> > >>>>>>>>> Yes, using the latest Kafka 0.7.2,
just tried to reproduce
> >> again
> >> > >>>>>>>>>
> >> > >>>>>>>>> 1. Install a single node Kafka, three
nodes zookeeper
> >> instances
> >> > >>>>>>>>>
> >> > >>>>>>>>>   kafka1
> >> > >>>>>>>>>   zookeeper1
> >> > >>>>>>>>>   zookeeper2
> >> > >>>>>>>>>   zookeeper3
> >> > >>>>>>>>>
> >> > >>>>>>>>> 2. Using a simple Kafka config, able
to start without error
> in
> >> > the
> >> > >>>>> log
> >> > >>>>>>>>>
> >> > >>>>>>>>>   brokerid=1
> >> > >>>>>>>>>   log.dir=/data/kafka
> >> > >>>>>>>>>   zk.connect=zookeeper1:2181,zookeeper2:2181,zookeeper3:2181
> >> > >>>>>>>>>
> >> > >>>>>>>>> 3. Now, create a path in zookeeper1
> >> > >>>>>>>>>
> >> > >>>>>>>>>   zkCli.sh -server zookeeper1:2181
> >> > >>>>>>>>>
> >> > >>>>>>>>>       ls /
> >> > >>>>>>>>>       [consumers, brokers, zookeeper]
> >> > >>>>>>>>>       create /testkafka ''
> >> > >>>>>>>>>       ls /
> >> > >>>>>>>>>       [testkafka, consumers, brokers,
zookeeper]
> >> > >>>>>>>>>
> >> > >>>>>>>>> Quit & Done.
> >> > >>>>>>>>>
> >> > >>>>>>>>> 4. Verify from zookeeper2
> >> > >>>>>>>>>
> >> > >>>>>>>>>   zkCli.sh -server zookeeper2:2181
> >> > >>>>>>>>>       ls /
> >> > >>>>>>>>>       [testkafka, consumers, brokers,
zookeeper]
> >> > >>>>>>>>>
> >> > >>>>>>>>> Seems ok
> >> > >>>>>>>>>
> >> > >>>>>>>>> 4. Update kafka config and restart
> >> > >>
> >> >
> >>
> zk.connect=zookeeper1:2181/testkafka,zookeeper2:2181/testkafka,zookeeper3:2181/testkafka
> >> > >>>>>>>>>
> >> > >>>>>>>>> Restart and have the error
> >> > >>>>>>>>>
> >> > >>>>>>>>> [2013-04-19 05:35:14,846] FATAL Fatal
error during
> >> > >>>> KafkaServerStable
> >> > >>>>>>>>> startup. Prepare to shutdown
> >> (kafka.server.KafkaServerStartable)
> >> > >>
> >> >
> >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message