Mailing-List: contact giraph-user-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: giraph-user@incubator.apache.org
Received-SPF: pass (nike.apache.org: domain of castagna.lists@googlemail.com
 designates 74.125.82.175 as permitted sender)
Message-ID: <4F881110.9040508@googlemail.com>
Date: Fri, 13 Apr 2012 12:42:08 +0100
From: Paolo Castagna <castagna.lists@googlemail.com>
User-Agent: Thunderbird 2.0.0.24 (X11/20101027)
MIME-Version: 1.0
To: giraph-user@incubator.apache.org
Subject: Re: A simple use case: shortest paths on a FOAF (i.e. Friend of a
 Friend) graph
References: <4F84872E.4050101@googlemail.com> <4F84B1BC.6080807@apache.org>
 <4F852330.7050002@googlemail.com> <4F85277C.403@googlemail.com>
 <4F85C07E.5030201@apache.org> <4F87EFEA.1080705@googlemail.com>
In-Reply-To: <4F87EFEA.1080705@googlemail.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Paolo Castagna wrote:
> Avery Ching wrote:
>> It shouldn't be, your code looks very similar to the unittests (i.e.
>> TestManualCheckpoint.java).  So, you're trying to run your test with the
>> local hadoop (similar to the unittests)?  Or are you using an actual
>> hadoop setup?
> 
> Hi Avery,
> here is a few more details on what I am trying to do, in order to run my Giraph
> jobs on a local Hadoop running (for testing and debugging stuff locally):
> 
>   GiraphJob job = new GiraphJob("shortest paths");
>   Configuration conf = job.getConfiguration();
>   conf.setBoolean(GiraphJob.SPLIT_MASTER_WORKER, false);
>   conf.setBoolean(GiraphJob.LOCAL_TEST_MODE, true);
>   // conf.set(GiraphJob.ZOOKEEPER_JAR,
> "file://target/dependency/zookeeper-3.3.3.jar");
>   job.setWorkerConfiguration(1, 1, 100.0f);
>   job.setVertexClass(SimpleShortestPathsVertex.class);
>   job.setVertexInputFormatClass(SimpleShortestPathsVertexInputFormat.class);
>   job.setVertexOutputFormatClass(SimpleShortestPathsVertexOutputFormat.class);
>   FileInputFormat.addInputPath(job.getInternalJob(), new
> Path("src/main/resources/giraph1.txt"));
>   Path outputPath = new Path("target/giraph1");
>   FileSystem hdfs = FileSystem.get(conf);
>   hdfs.delete(outputPath, true);
>   FileOutputFormat.setOutputPath(job.getInternalJob(), outputPath);
>   job.run(true);
> 
> Am I doing something wrong/stupid here?
> Am I missing something important? (probably! but I do not see what I am missing)

This is a better way:

  Iterable<String> results = InternalVertexRunner.run(
    SimpleShortestPathsVertex.class,
    SimpleShortestPathsVertex.SimpleShortestPathsVertexInputFormat.class,
    SimpleShortestPathsVertex.SimpleShortestPathsVertexOutputFormat.class,
    params, graph);

... which starts a local ZooKeeper properly.

However, I still have a question: when I run it in a unit test everything is
fine. When I run it on a Java main method, it hangs towards the end.

Paolo