giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claudio Martella <claudio.marte...@gmail.com>
Subject Re: Modifying a benchmark to use real input
Date Fri, 24 May 2013 15:33:19 GMT
You can still use the classes in the examples package, which are similar to
those in the benchmark package but are more flexible for your own tests.


On Fri, May 24, 2013 at 3:42 PM, Matt Molek <mpmolek@gmail.com> wrote:

> Oh, never mind, I think I found it by looking trough GiraphRunner.java
>
> GiraphFileInputFormat.addVertexInputPath(conf, new Path("/some/path"));
>
>
> On Thu, May 23, 2013 at 5:22 PM, Matt Molek <mpmolek@gmail.com> wrote:
>
>> Hi,
>>
>> I'm just getting started with Giraph, and struggling a bit to understand
>> what exactly is needed to run a minimal Giraph computation on real data,
>> rather than the PseudoRandomVertexInputFormat.
>>
>> Apologies if this is covered somewhere in the docs or mailing list
>> archives. I looked but couldn't find anything applying to the current
>> version, and I couldn't figure out exactly how things have changed through
>> the versions. Some older code that I tried was clearly incompatible with
>> the current version.
>>
>> Trying to learn by example, I copied the current
>> o.a.g.benchmark.ShortestPathsBenchmark and
>> o.a.g.benchmark.ShortestPathsComputation into my own project, and modified
>> them to run on their own without GiraphBenchmark, and BenchmarkOption. Here
>> is the new ShortestPathsBenchmark I ended up with:
>> http://pastebin.com/h3rH6jTm
>>
>> When using the PseudoRandomVertexInputFormat, and some hard coded options
>> for aggregateVertices and edgesPerVertex, this runs fine from my jar with
>> the command:
>>
>> hadoop jar giraph-testing-jar-with-dependencies.jar
>> modified_benchmarks.ShortestPathsBenchmark --workers 10
>>
>> Now I'd like to use JsonLongDoubleFloatDoubleVertexInputFormat with some
>> real data, but I see no way to specify the input path. If this was plain
>> hadoop, I'd expect to be able to say something like
>> JsonLongDoubleFloatDoubleVertexInputFormat.addInputPath(job, new
>> Path("/some/path"));
>>
>> That's not available though. Could someone point me in the right
>> direction with this?
>>
>> Am I going about this all wrong?
>>
>> Thanks for any help,
>> Matt
>>
>
>


-- 
   Claudio Martella
   claudio.martella@gmail.com

Mime
View raw message