Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 0207F10F54 for ; Fri, 12 Jul 2013 20:48:28 +0000 (UTC) Received: (qmail 62007 invoked by uid 500); 12 Jul 2013 20:48:23 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 61917 invoked by uid 500); 12 Jul 2013 20:48:23 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 61910 invoked by uid 99); 12 Jul 2013 20:48:23 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 12 Jul 2013 20:48:23 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of hadoop.jqi@gmail.com designates 74.125.83.43 as permitted sender) Received: from [74.125.83.43] (HELO mail-ee0-f43.google.com) (74.125.83.43) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 12 Jul 2013 20:48:16 +0000 Received: by mail-ee0-f43.google.com with SMTP id l10so6400097eei.16 for ; Fri, 12 Jul 2013 13:47:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=KPd4w84yjSwVwgEyuf41iBIBjjmmMcBmJMkJpzX5cew=; b=VhAkCFeK5nK3HO6UMXmcOXnj4HRYPq5iWwsNM1BDnMWrd6G9VDnU6oSp3kYoH8mO7R D2GkZqMt7ub1isPcDG5KO7BEX9xYF4hFk4BEk+QnwhcqxuLFtq0aOwNaW50Quqw8nqAa SFT+9/jsEbu2IctnAnpnrB9IiQZxlxmMAHCQ85lrb2eYoxaF+aap7y88If+DhXAm2B/X vuAWh2ssHKgX3pzeEGirNCOIcBPd6BkODYSi3GNbin1Fm2+xIGKgT4300Kq26rQl6k7N PHAEPgSs4lYznkYkfJyJVaBdLQ4az85pAh/eC59/jkm5kXXNO9sSLQEJb18UP+TppDbN RmoQ== MIME-Version: 1.0 X-Received: by 10.15.81.136 with SMTP id x8mr49158061eey.4.1373662075757; Fri, 12 Jul 2013 13:47:55 -0700 (PDT) Received: by 10.223.203.66 with HTTP; Fri, 12 Jul 2013 13:47:55 -0700 (PDT) In-Reply-To: References: Date: Fri, 12 Jul 2013 13:47:55 -0700 Message-ID: Subject: Re: How are 'PHYSICAL_MEMORY_BYTES' and 'VIRTUAL_MEMORY_BYTES' calculated? From: hadoop qi To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=089e016813d011500104e156a023 X-Virus-Checked: Checked by ClamAV on apache.org --089e016813d011500104e156a023 Content-Type: text/plain; charset=ISO-8859-1 Thanks for the response. So they represent the total physical memory (virtual memory) has been allocated to the job (e.g., from heap and stack) during its entire life time? I am still confused how to get the cumulative number from /proc/meminfo. I think from /proc/meminfo we can only get the memory usage of a process in a particular time point (looked like a snapshot of the status of the process). If these numbers are added, the sum would be much more than memory allocated to the program. On Fri, Jul 12, 2013 at 6:47 AM, Shahab Yunus wrote: > I think they are cumulative but per task. > > Physical memory bytes > (PHYSICAL_MEMORY_BYTES) > The physical memory being used by a task in bytes, as reported by > /proc/meminfo. > Virtual memory bytes > (VIRTUAL_MEMORY_BYTES) > The virtual memory being used by a task in bytes, as reported by > /proc/meminfo. > > This is from the Definitive Guide book. Page 260. > > Regards, > Shhab > > > On Thu, Jul 11, 2013 at 12:47 PM, hadoop qi wrote: > >> Hello, >> >> I am wondering how memory counters 'PHYSICAL_MEMORY_BYTES' and >> 'VIRTUAL_MEMORY_BYTES' are calculated? They are peaks of memory usage or >> cumulative usage? >> >> Thanks for help, >> > > --089e016813d011500104e156a023 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Thanks for the response. So they represent the total physi= cal memory (virtual memory) has been allocated to the job (e.g., from heap = and stack) during its entire life time? I am still confused how to get the = cumulative number from /proc/meminfo. I think from /proc/meminfo we can onl= y get the memory usage of a=A0 process in a particular time point (looked l= ike a snapshot of the status of the process). If these numbers are added, t= he sum would be much more than memory allocated to the program.


On Fri,= Jul 12, 2013 at 6:47 AM, Shahab Yunus <shahab.yunus@gmail.com>= ; wrote:
I think they are cumul= ative but per task.

Physical memory bytes
(PHYSICAL_MEMORY_BYTES)
The physical memory being used by a task = in bytes, as reported by /proc/meminfo.
Virtual memory bytes
(VIRTUAL_MEMORY_BYTES)
The vi= rtual memory being used by a task in bytes, as reported by /proc/meminfo.

This is from the Definitive Guide book. Page 260.

Regards,
Shhab


On Thu, Jul 11, 2013 at 12:47 PM, hadoop qi <<= a href=3D"mailto:hadoop.jqi@gmail.com" target=3D"_blank">hadoop.jqi@gmail.c= om> wrote:
Hello,
<= br>
I am wondering how memory counters=A0 'PHYSICAL_MEMORY_BYTES&#= 39;=A0 and 'VIRTUAL_MEMORY_BYTES'=A0 are calculated? They are peaks= of memory usage or cumulative usage?

Thanks for help,


--089e016813d011500104e156a023--