Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id B9BC9200AE1 for ; Mon, 6 Jun 2016 20:08:21 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id B7FDB160A24; Mon, 6 Jun 2016 18:08:21 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 0BF11160A1E for ; Mon, 6 Jun 2016 20:08:20 +0200 (CEST) Received: (qmail 73323 invoked by uid 500); 6 Jun 2016 18:08:20 -0000 Mailing-List: contact dev-help@kafka.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@kafka.apache.org Delivered-To: mailing list dev@kafka.apache.org Received: (qmail 73310 invoked by uid 99); 6 Jun 2016 18:08:19 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 06 Jun 2016 18:08:19 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 67C0AC05B3 for ; Mon, 6 Jun 2016 18:08:19 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.03 X-Spam-Level: ** X-Spam-Status: No, score=2.03 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.001, HTML_MESSAGE=2, KAM_INFOUSMEBIZ=0.75, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx2-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id aEHCFTeALpGa for ; Mon, 6 Jun 2016 18:08:16 +0000 (UTC) Received: from mail-wm0-f44.google.com (mail-wm0-f44.google.com [74.125.82.44]) by mx2-lw-eu.apache.org (ASF Mail Server at mx2-lw-eu.apache.org) with ESMTPS id 62E985F3DA for ; Mon, 6 Jun 2016 18:08:16 +0000 (UTC) Received: by mail-wm0-f44.google.com with SMTP id n184so104235078wmn.1 for ; Mon, 06 Jun 2016 11:08:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to; bh=WaSpcGv9PPWL+oFxLlBX5aPxZcvilAvsrXiWYH3bcR4=; b=P7h5TOsKzEYytPfrrOd3W14xVe8zIJMCG40FbvA5GYfCJOjSoRohCO16UamLzf1uW0 //I+sxo/XnML9JhB9NC8P3CtrqSmoufjw0bMTEeE5a4DFSgC7qjWdog6YoE2mMxz89FV 6KCYmWah2zwSuF5RU9DnOWA2WDxi830QRQdUo1k8qtIPwAL35jBGq6DWLfcsMmRZ1BXm C4sOFusZBAdhMQYwyIOfQGO+8efcD8GuTOUl79T85S7v84K0MflisP34DhoNYp7LDoJY p1II+lJ8Ii0e882K/Mk43nVtzO02fkHDpSKqz5KwgBCV/5Gze3Iso4QeXqrscB0mEwGg +qzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to; bh=WaSpcGv9PPWL+oFxLlBX5aPxZcvilAvsrXiWYH3bcR4=; b=HlHzcfQ4116IIDym4b6yFAxdGyKLhjgrOfLMQ/cf2/TStO+0LA7OWG8q2V49yR3Se1 kLc7REyWUcI6+6gzFNzPguHpRknKVZzzPbciajF5PssSqW1hicI8+r9QuMrlkD0qKE8K uhR9iMUrmMVZHRCiwgFaX9cMZeFfyKeisyqnAGk/bB2ZqqLEs/qkbFeoBMUfjRca5F/6 qHh0YR0BBlwKUG6hn4s7f9nb0js1h/EbbdlMFYR9sLaB2He/Kzades/+73+U4o5EVOXo Nxtv+NrdNJwJ18RIrj42yDuP1he1BF8s2MkC/bW6yK2f1euAbSG+hJBEwwQB2pP53ds7 ssrg== X-Gm-Message-State: ALyK8tLmVc/1Z7gqJwCnJgTfVQrj9O9eh4pF/5D9OY94KhRoMzEOI/DDzBNyVrOJiMgR0BizqgWPjGWrKlR/iQ== X-Received: by 10.194.242.230 with SMTP id wt6mr18462392wjc.143.1465236496038; Mon, 06 Jun 2016 11:08:16 -0700 (PDT) MIME-Version: 1.0 Sender: ismaelj@gmail.com Received: by 10.194.19.71 with HTTP; Mon, 6 Jun 2016 11:07:36 -0700 (PDT) In-Reply-To: References: <4EAE047E-7583-455E-85B1-B880F94C767C@gmail.com> <03513734-670A-4F72-B324-BAC9858EBC34@gmail.com> From: Ismael Juma Date: Mon, 6 Jun 2016 19:07:36 +0100 X-Google-Sender-Auth: F3PcJyn4nqFIScRQ_xDtQ6eyAmM Message-ID: Subject: Re: [DISCUSS] KIP-63: Unify store and downstream caching in streams To: dev@kafka.apache.org Content-Type: multipart/alternative; boundary=089e0141a1fadbc37005349ff412 archived-at: Mon, 06 Jun 2016 18:08:21 -0000 --089e0141a1fadbc37005349ff412 Content-Type: text/plain; charset=UTF-8 On Mon, Jun 6, 2016 at 6:48 PM, Guozhang Wang wrote: > > About using Instrumentation.getObjectSize, yeah we were worried a lot about > its efficiency as well as accuracy when discussing internally, but not a > better solution was proposed. So if people have better ideas, please throw > them here, as it is also the purpose for us to call out such KIP discussion > threads. > Note that this requires a Java agent to be configured. A few links: https://github.com/apache/spark/blob/b0ce0d13127431fa7cd4c11064762eb0b12e3436/core/src/main/scala/org/apache/spark/util/SizeEstimator.scala https://github.com/apache/cassandra/blob/3dcbe90e02440e6ee534f643c7603d50ca08482b/src/java/org/apache/cassandra/utils/ObjectSizes.java https://github.com/jbellis/jamm http://openjdk.java.net/projects/code-tools/jol/ https://github.com/DimitrisAndreou/memory-measurer OK, maybe that's more than what you wanted. :) Ismael --089e0141a1fadbc37005349ff412--