kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jamie Wang <jamie.w...@actuate.com>
Subject RE: Loading kafka server within a Java process
Date Thu, 14 Feb 2013 22:51:45 GMT
Thank you both for the info. I'll keep my eye open since I do have a need to embed a consumer
in the same process.

Jamie

-----Original Message-----
From: Eric Tschetter [mailto:cheddar@metamarkets.com] 
Sent: Thursday, February 14, 2013 2:44 PM
To: users@kafka.apache.org
Subject: Re: Loading kafka server within a Java process

Jamie,

As Jay says, this is definitely possible.  We also do it at Metamarkets.

The one thing that we found odd when doing it though is that you cannot (or
couldn't with the version we are using, I'm not sure about 0.8) embed a
producer that produces for an embedded broker.  You instead have to have
your producer talk over the "network" to the broker even though it is
running inside the same process.  We haven't run into any issues doing this
yet, but it's something to be aware of.

--Eric


On Thu, Feb 14, 2013 at 4:39 PM, Jay Kreps <jay.kreps@gmail.com> wrote:

> Yes, it will work. We do this at LinkedIn, actually.
>
> -Jay
>
> On Thu, Feb 14, 2013 at 12:15 PM, Jamie Wang <jamie.wang@actuate.com>
> wrote:
> > I have a situation where we have constraint on the number of processes
> we can use and hence, is it possible to load kafka server within another
> Java process? Looking at the kafka.kafka it seems that I can create a Java
> object that does most of what Kafka.kafka.main() do.  Any thoughts if this
> will work? What are some of the things I need to watch out? And is there
> another better way to do this?  Thank you for your time.
> >
> > Jamie
>
Mime
View raw message