Return-Path: X-Original-To: apmail-giraph-user-archive@www.apache.org Delivered-To: apmail-giraph-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 0E4AF11145 for ; Fri, 18 Apr 2014 14:52:53 +0000 (UTC) Received: (qmail 56765 invoked by uid 500); 18 Apr 2014 14:52:52 -0000 Delivered-To: apmail-giraph-user-archive@giraph.apache.org Received: (qmail 56698 invoked by uid 500); 18 Apr 2014 14:52:52 -0000 Mailing-List: contact user-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@giraph.apache.org Delivered-To: mailing list user@giraph.apache.org Received: (qmail 56689 invoked by uid 99); 18 Apr 2014 14:52:51 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 18 Apr 2014 14:52:51 +0000 X-ASF-Spam-Status: No, hits=2.5 required=5.0 tests=FREEMAIL_REPLY,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of ghufran1malik@gmail.com designates 209.85.128.176 as permitted sender) Received: from [209.85.128.176] (HELO mail-ve0-f176.google.com) (209.85.128.176) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 18 Apr 2014 14:52:47 +0000 Received: by mail-ve0-f176.google.com with SMTP id db11so3046884veb.35 for ; Fri, 18 Apr 2014 07:52:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=qmZJjldi/jsEf8NBQXgkxIyeVzX5Hxd0svjLjA+fERE=; b=MUnOYsYgahWnTcJ7ipsvRnZA5x33mQ0/jSnzOFRdC8qKEIJCfwhYlhhUbTv7SW7d3C /VhKMkM/HTAT6+Zl7Ojmv4GT/vBQblKIPZG9ds9/O/ZllfUQ4AWqQLqk0dVgYzaX8c1X M9KZWYHHlxOQdb14HEwv0bc0Kc9EbRuBFmqsdKUrFl1id6vjiOaVBzjB/bMXvICZ6rCW KSIStbIMmc7EWTOOn6sRzFexcFM0kBFxF0Wx8sld7mkJWDiFe78VIBrcMKWwAsAI79aO rA3A8IEWKKtKfckojtggffIoT6S6S5N3MNYqsf2NIVXY6DO8IOgio354IFlGSaf9VKJe Mh3Q== MIME-Version: 1.0 X-Received: by 10.52.175.166 with SMTP id cb6mr12330508vdc.1.1397832746941; Fri, 18 Apr 2014 07:52:26 -0700 (PDT) Received: by 10.58.99.67 with HTTP; Fri, 18 Apr 2014 07:52:26 -0700 (PDT) In-Reply-To: References: Date: Fri, 18 Apr 2014 15:52:26 +0100 Message-ID: Subject: Re: input superstep of giraph. From: ghufran malik To: user@giraph.apache.org Content-Type: multipart/alternative; boundary=bcaec50fdff356431304f7524c2f X-Virus-Checked: Checked by ClamAV on apache.org --bcaec50fdff356431304f7524c2f Content-Type: text/plain; charset=UTF-8 Hi, Thank you for the explanation :) It was confusing when reading it, some of the timers I can intuitively understand, however I think it would be beneficial if these explanations were added to the API docs, then if anyone else is confused they can look up the meanings there. https://giraph.apache.org/giraph-core/apidocs/org/apache/giraph/counters/GiraphTimers.html Thanks, Ghufran On Fri, Apr 18, 2014 at 3:25 PM, Pavan Kumar A wrote: > I wrote the Initialize counter :) > Please tell me if the name seems confusing > > So, > Initialize = the time spent by job waiting for resources. In a shared pool > the job you launch may not get all the machines needed to start the job. So > for instance you want to run a job with 200 workers, giraph does not start > until all the workers have are allocated & register with the master. > > Setup = once you have all the machines allocated, how much time it takes > before starting to read input > > Shutdown = once you have written your output howmuch time it takes to stop > verify that everything is done & shutdown resources & notify user - for > instance wait for all network connections to close, all threads to join, > etc. > > Total = sum of input time + sum of time in all supersteps > i.e., actual time taken to run by your application after it got all the > resources (does not include time waiting to get resources which is > initialize or shutdown time) > > ------------------------------ > Date: Fri, 18 Apr 2014 13:28:47 +0100 > Subject: Re: input superstep of giraph. > From: ghufran1malik@gmail.com > To: user@giraph.apache.org > > > Hi, > > Could you also explain what the following timers correspond to as well > please: > > Giraph Timers > Initialize (ms)=775 > Setup (ms)=105 > > Shutdown (ms)=12537 > > Total (ms)=27075 > > Thanks, > > Ghufran > > > > On Thu, Apr 17, 2014 at 9:10 PM, Pavan Kumar A wrote: > > Input consists of > > reading the input (vertices and/or edges as provided) into memory on > individual workers > > assigning vertices to partitions and partitions to workers > > moving all partitions (i.e., vertices & their out-edges) to a worker > (which owns the partition) > > doing some bookkeeping of internal data-structures to be used during > computation > > ------------------------------ > Date: Thu, 17 Apr 2014 10:06:03 -0500 > Subject: input superstep of giraph. > From: suijian.zhou@gmail.com > To: user@giraph.apache.org > > > Hi, > From the screen output of a successful giraph program run, what does the > following line mean? > > Input superstep (ms)=22884 > > Does it mean the time used to load the input graph into memory? Thanks. > > Best Regards, > Suijian > > > > --bcaec50fdff356431304f7524c2f Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi,=C2=A0

Thank you for the explanation= :)=C2=A0

It was confusing when reading it, some o= f the timers I can intuitively understand, however I think it would be bene= ficial if these explanations were added to the API docs, then if anyone els= e is confused they can look up the meanings there.=C2=A0

https://giraph.apache.org/giraph-core/apido= cs/org/apache/giraph/counters/GiraphTimers.html=C2=A0

Thanks,=C2=A0

Ghufran


On Fri, Apr 18, 2014 at= 3:25 PM, Pavan Kumar A <pavanka@outlook.com> wrote:
I wrote the Initialize counter :)=C2=A0
Please te= ll me if the name seems confusing

So,
In= itialize =3D the time spent by job waiting for resources. In a shared pool = the job you launch may not get all the machines needed to start the job. So= for instance you want to run a job with 200 workers, giraph does not start= until all the workers have are allocated & register with the master.

Setup =3D once you have all the machines allocated, how= much time it takes before starting to read input

= Shutdown =3D once you have written your output howmuch time it takes to sto= p verify that everything is done & shutdown resources & notify user= - for instance wait for all network connections to close, all threads to j= oin, etc.

Total =3D sum of input time + sum of time in all supers= teps
i.e., actual time taken to run by your application after it = got all the resources (does not include time waiting to get resources which= is initialize or shutdown time)

Date: Fri, 18 Apr 2014 13:28:47 +0100
Subject: Re: input su= perstep of giraph.
From: ghufran1malik@gmail.com
To: user@giraph.apache.org


Hi,=C2=A0

Could you also = explain what the following timers correspond to as well please:=C2=A0
Giraph Timers
= Initialize (ms)=3D775
Setup (ms)=3D105

Shutdown (ms)=3D12537

Total (ms= )=3D27075=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=C2=A0

Thanks,=C2=A0

Ghufran



On Thu, Apr 17, 2014 at 9:10 PM, Pavan Kumar A <pavanka@outlook.com&= gt; wrote:
Input consists of=C2=A0
> reading the input (v= ertices and/or edges as provided) into memory on individual workers
> assigning vertices to partitions and partitions to workers
> moving all partitions (i.e., vertices & their out-edges) to a work= er (which owns the partition)
> doing some bookkeeping of inte= rnal data-structures to be used during computation

Date: Th= u, 17 Apr 2014 10:06:03 -0500
Subject: input superstep of giraph.
From: suijian.zhou@gmail.com
To: user@giraph.apache.org=


Hi,
=C2=A0 From the screen out= put of a successful giraph program run, what does the following line mean?<= br>

Input superstep (ms)=3D22884

=C2=A0Does it mean the time used to load the input graph into memory? Thank= s.

=C2=A0 Best Regards,
=C2=A0 Suijian
<= br>



--bcaec50fdff356431304f7524c2f--