incubator-kato-spec mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Poole <spoole...@googlemail.com>
Subject Re: Kato & Native Memory
Date Thu, 25 Jun 2009 16:16:40 GMT
On Thu, Jun 25, 2009 at 4:54 PM, Andrew Hall <ahall451@googlemail.com>wrote:

> Hi Steve,
>
> A few answers inline below:
>
> On Thu, Jun 25, 2009 at 3:07 PM, Steve Poole <spoole167@googlemail.com>
> wrote:
> >
> <snip>
> >
> > If we gave users this information what would they do with it?   I assume
> > that they would want to use it to influence their applications and
> > configurations.   Do you have anything to share about why users want this
> > information?
>
> Many users who hit a native OOM have no idea what to do next. There's
> very little visibility into the native heap and often the only thing
> they can do is raise a support request on one of the software vendors
> in the stack.


Um,  are you saying that you want to expose this information just so
customers can beat you up about it  ? :-)

>
> These stats provide the first layer of detail when looking at the
> native heap. If the data shows large amount of memory associated with
> threads or classloaders, they can use a tool such as MAT
> (http://www.eclipse.org/mat/) to understand more. If the stats show
> the JVM is using vast quantities of native memory, they can raise a
> bug but with some confidence they're looking in the right place.
>

Lets assume then that the stats data was made available - from a customer
perspective it would look something like this?

threads   2000 entries,  100MB storage
classloaders  50000 entries 2500MB storage
other stuff  1000000 entries  1000MB storage


I'm really interested in the end game -  once the data is made available how
will some use it.
 Since you raised this I assume you have direct knowledge of some examples.
Can you perhaps share some sanitised war stories?

 I really want to understand what happens when a native memory problem is
encountered - from how a customer first realises that its happened right
though to how a JVM/JIT developer figures out the underlying bug.


> >
> <snip>
> >
> > I agree - its sort of the least we could do in this area :-)      How
> would
> > you want to see the data -  as a list  , a map (eyecatcher -> list of
> hits
> > )   or would this best be done by providing a visitor like mechanism?
> >
>
> If we put eyecatchers around every native heap allocation, for
> example, there could be several million data blocks to extract, so I
> don't think you'd want to access a list or iterator of particular
> blocks. I think you're more likely to visit every block of interest,
> extract some useful information and move on.
>
> To do that efficiently I think you'd want something like a Visitor -
> but possibly have an method on the Visitor object to do the pattern
> matching of the byte stream (i.e. looking for the eyecatcher). An
> eyecatcher may not be a simple binary string - and allowing the user
> to supply logic to match it makes it more flexible. I don't know
> whether that makes more sense as a separate Strategy object or as part
> of the Visitor interface.
>
> <snip>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message