hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <tdunn...@maprtech.com>
Subject Re: Hadoop RPC call response post processing
Date Wed, 29 Dec 2010 06:48:08 GMT
Nah... this is hadoop RPC

On Tue, Dec 28, 2010 at 6:42 PM, Lance Norskog <goksron@gmail.com> wrote:

> Are you connecting to this JVM with RMI? RMI does a very nasty thing
> with garbage collection: it forces a blocking collection every 60
> seconds. Really. You have to change this with a system property.
>
> On Tue, Dec 28, 2010 at 5:56 PM, Todd Lipcon <todd@cloudera.com> wrote:
> > On Tue, Dec 28, 2010 at 1:00 PM, Stefan Groschupf <sg@101tec.com> wrote:
> >
> >> Hi Todd,
> >> Right, that is the code I'm looking into. Though Responder is inner
> private
> >> class and is created  "responder = new Responder();"
> >> It would be great if the Responder implementation could be configured.
> >> Do you have any idea how to overwrite the Responder?
> >>
> >
> > Nope, it's not currently pluggable, nor do I think there's any compelling
> > reason to make it pluggable. It's coupled quite tightly to the
> > implementation right now.
> >
> > Perhaps you can hack something in a git branch, and if it has good
> results
> > on something like NNBench it could be a general contribution?
> >
> > -Todd
> >
> >
> >> On Dec 27, 2010, at 8:21 PM, Todd Lipcon wrote:
> >>
> >> > Hi Stefan,
> >> >
> >> > Sounds interesting.
> >> >
> >> > Maybe you're looking for o.a.h.ipc.Server$Responder?
> >> >
> >> > -Todd
> >> >
> >> > On Mon, Dec 27, 2010 at 8:07 PM, Stefan Groschupf <sg@101tec.com>
> wrote:
> >> >
> >> >> Hi All,
> >> >> I'm browsing the RPC code since quite a while now trying to find any
> >> entry
> >> >> point / interceptor slot that allows me to handle a RPC call response
> >> >> writable after it was send over the wire.
> >> >> Does anybody has an idea how break into the RPC code from outside.
> All
> >> the
> >> >> interesting methods are private. :(
> >> >>
> >> >> Background:
> >> >> Heavy use of the RPC allocates hugh amount of Writable objects. We
> saw
> >> in
> >> >> multiple systems  that the garbage collect can get so busy that the
> jvm
> >> >> almost freezes for seconds. Things like zookeeper sessions time out
> in
> >> that
> >> >> cases.
> >> >> My idea is to create an object pool for writables. Borrowing an
> object
> >> from
> >> >> the pool is simple since this happen in our custom code, though we
do
> >> know
> >> >> when the writable return was send over the wire and can be returned
> into
> >> the
> >> >> pool.
> >> >> A dirty hack would be to overwrite the write(out) method in the
> >> writable,
> >> >> assuming that is the last thing done with the writable, though turns
> out
> >> >> that this method is called in other cases too, e.g. to measure
> >> throughput.
> >> >>
> >> >> Any ideas?
> >> >>
> >> >> Thanks,
> >> >> Stefan
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > Todd Lipcon
> >> > Software Engineer, Cloudera
> >>
> >>
> >
> >
> > --
> > Todd Lipcon
> > Software Engineer, Cloudera
> >
>
>
>
> --
> Lance Norskog
> goksron@gmail.com
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message