Mailing-List: contact user-help@giraph.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@giraph.apache.org
Received-SPF: pass (athena.apache.org: domain of write2vishal@gmail.com
 designates 209.85.212.176 as permitted sender)
MIME-Version: 1.0
From: Vishal Patel <write2vishal@gmail.com>
Date: Fri, 10 Aug 2012 16:10:33 -0700
Message-ID: 
 <CAPPQaJ7bQDjB1V_n23Vg=hE-=P51bebrHC=HtjrbyQz9jKqL2Q@mail.gmail.com>
Subject: Saving checkpoints?
To: user <user@giraph.apache.org>
Content-Type: multipart/alternative; boundary=f46d04138a59407c0204c6f1755c

--f46d04138a59407c0204c6f1755c
Content-Type: text/plain; charset=ISO-8859-1

Hi,

How do I specify the interval for saving checkpoints? When working with
Amazon's Elastic Mapreduce on a large number of workers (> 80 workers, 40 x
m1.xlarge machines), sometimes there is RPC communication errors and
Zookeeper waits on that worker for a while before timing out and killing
the job all together.

As my graph and number of workers is becoming larger I would like to learn
how to save it since that extra cost might be well worth it-- say every 50
supersteps. Here is the command I use currently, how should I modify it.

hadoop jar giraph-0.2-SNAPSHOT-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner
org.apache.giraph.examples.ConnectedComponentsVertex \
--inputFormat org.apache.giraph.examples.IntIntNullIntTextInputFormat \
--inputPath giraph_in/adj_list.txt \
--outputFormat
org.apache.giraph.examples.VertexWithComponentTextOutputFormat \
--outputPath giraph_out
--combiner org.apache.giraph.examples.MinimumIntCombiner
--workers 95

Also, how do I restart from a specific checkpoint. The help for the
GiraphRunner class did not have instructions on this.

Thank you!

Vishal

--f46d04138a59407c0204c6f1755c
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div>Hi,=A0</div><div><br></div><div>How do I specify the interval for savi=
ng checkpoints? When working with Amazon&#39;s Elastic Mapreduce on a large=
 number of workers (&gt; 80 workers, 40 x m1.xlarge machines), sometimes th=
ere is RPC communication errors and Zookeeper waits on that worker for a wh=
ile before timing out and killing the job all together. =A0</div>

<div><br></div><div>As my graph and number of workers is becoming larger I =
would like to learn how to save it since that extra cost might be well wort=
h it-- say every 50 supersteps. Here is the command I use currently, how sh=
ould I modify it.=A0</div>

<div><br></div><div>hadoop jar giraph-0.2-SNAPSHOT-jar-with-dependencies.ja=
r org.apache.giraph.GiraphRunner org.apache.giraph.examples.ConnectedCompon=
entsVertex \</div><div>--inputFormat org.apache.giraph.examples.IntIntNullI=
ntTextInputFormat \</div>

<div>--inputPath giraph_in/adj_list.txt \</div><div>--outputFormat org.apac=
he.giraph.examples.VertexWithComponentTextOutputFormat \</div><div>--output=
Path giraph_out</div><div>--combiner org.apache.giraph.examples.MinimumIntC=
ombiner</div>

<div>--workers 95 =A0</div><div><br></div><div>Also, how do I restart from =
a specific checkpoint. The help for the GiraphRunner class did not have ins=
tructions on this.=A0</div><div><br></div><div>Thank you!=A0</div><div><br>=
</div>

<div>Vishal=A0</div><div><br></div><div><br></div>

--f46d04138a59407c0204c6f1755c--