From kato-spec-return-187-apmail-incubator-kato-spec-archive=incubator.apache.org@incubator.apache.org Thu Jun 25 16:17:03 2009 Return-Path: Delivered-To: apmail-incubator-kato-spec-archive@minotaur.apache.org Received: (qmail 87978 invoked from network); 25 Jun 2009 16:17:02 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 25 Jun 2009 16:17:02 -0000 Received: (qmail 93997 invoked by uid 500); 25 Jun 2009 16:17:13 -0000 Delivered-To: apmail-incubator-kato-spec-archive@incubator.apache.org Received: (qmail 93969 invoked by uid 500); 25 Jun 2009 16:17:13 -0000 Mailing-List: contact kato-spec-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: kato-spec@incubator.apache.org Delivered-To: mailing list kato-spec@incubator.apache.org Received: (qmail 93959 invoked by uid 99); 25 Jun 2009 16:17:13 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 25 Jun 2009 16:17:13 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of spoole167@googlemail.com designates 209.85.219.221 as permitted sender) Received: from [209.85.219.221] (HELO mail-ew0-f221.google.com) (209.85.219.221) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 25 Jun 2009 16:17:01 +0000 Received: by ewy21 with SMTP id 21so283975ewy.12 for ; Thu, 25 Jun 2009 09:16:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=VJEV7Oxs/wOZY721z25isvC/+VUtaHzsyU1a5Xllxf0=; b=ub2CIGxaPTtCnUjIUFGmBsRZzNGNpwSY2jmlyxdNeBlg5fwie01aes7uZq1lLoR6Jo NgB6WCBDMMTPD1NBdk4QM2tFMSiiGEZCrP5RfKx7WqkbUyuXywa54ST//9xxvHPKoFS7 fTs8y2EYyg3jHIphGb67pN88qGcaKCVX8ic+M= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=Uv2i/hGZWyodhNOct7KrZMfwZZ7jrNWgQvfX2aQCNTwW1PbUzT+2LPhh2F5Clt66yV kYoI0YvAhFL6Pdht676aOCmgj+C/puyBCZgthlHZsQ7Dc8wKMq+cyfoOxnDK/xJlQJx7 AOuHJtc1v+BP6WTnxUoVXP2Yc01HiEdtdVcoM= MIME-Version: 1.0 Received: by 10.211.195.15 with SMTP id x15mr2360060ebp.18.1245946600969; Thu, 25 Jun 2009 09:16:40 -0700 (PDT) In-Reply-To: <6032540d0906250854h59ab765dldff8668b48d435ec@mail.gmail.com> References: <6032540d0906250430o1df9e725j8f5cf9a54f236750@mail.gmail.com> <6032540d0906250854h59ab765dldff8668b48d435ec@mail.gmail.com> Date: Thu, 25 Jun 2009 17:16:40 +0100 Message-ID: Subject: Re: Kato & Native Memory From: Steve Poole To: kato-spec@incubator.apache.org Content-Type: multipart/alternative; boundary=0015174beb1c8f5786046d2e8c57 X-Virus-Checked: Checked by ClamAV on apache.org --0015174beb1c8f5786046d2e8c57 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit On Thu, Jun 25, 2009 at 4:54 PM, Andrew Hall wrote: > Hi Steve, > > A few answers inline below: > > On Thu, Jun 25, 2009 at 3:07 PM, Steve Poole > wrote: > > > > > > > If we gave users this information what would they do with it? I assume > > that they would want to use it to influence their applications and > > configurations. Do you have anything to share about why users want this > > information? > > Many users who hit a native OOM have no idea what to do next. There's > very little visibility into the native heap and often the only thing > they can do is raise a support request on one of the software vendors > in the stack. Um, are you saying that you want to expose this information just so customers can beat you up about it ? :-) > > These stats provide the first layer of detail when looking at the > native heap. If the data shows large amount of memory associated with > threads or classloaders, they can use a tool such as MAT > (http://www.eclipse.org/mat/) to understand more. If the stats show > the JVM is using vast quantities of native memory, they can raise a > bug but with some confidence they're looking in the right place. > Lets assume then that the stats data was made available - from a customer perspective it would look something like this? threads 2000 entries, 100MB storage classloaders 50000 entries 2500MB storage other stuff 1000000 entries 1000MB storage I'm really interested in the end game - once the data is made available how will some use it. Since you raised this I assume you have direct knowledge of some examples. Can you perhaps share some sanitised war stories? I really want to understand what happens when a native memory problem is encountered - from how a customer first realises that its happened right though to how a JVM/JIT developer figures out the underlying bug. > > > > > > > I agree - its sort of the least we could do in this area :-) How > would > > you want to see the data - as a list , a map (eyecatcher -> list of > hits > > ) or would this best be done by providing a visitor like mechanism? > > > > If we put eyecatchers around every native heap allocation, for > example, there could be several million data blocks to extract, so I > don't think you'd want to access a list or iterator of particular > blocks. I think you're more likely to visit every block of interest, > extract some useful information and move on. > > To do that efficiently I think you'd want something like a Visitor - > but possibly have an method on the Visitor object to do the pattern > matching of the byte stream (i.e. looking for the eyecatcher). An > eyecatcher may not be a simple binary string - and allowing the user > to supply logic to match it makes it more flexible. I don't know > whether that makes more sense as a separate Strategy object or as part > of the Visitor interface. > > > --0015174beb1c8f5786046d2e8c57--