giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ferenc Béres <ferdzs...@gmail.com>
Subject Re: How to output into multiple files through a GiraphJob
Date Sat, 21 Jun 2014 08:42:21 GMT
Hi John,

Yes, I also maintain several states within my vertex values so I use
BasicComputation.

Regarding the output I use this configuration:
GiraphConfiguration conf = new GiraphConfiguration();

 conf.setVertexOutputFormatClass(ColumnOutputFormat.class);
     FileOutputFormat.setOutputPath(giraphJob.getInternalJob(), resultPath);

(ColumnOutputFormat extends TextVertexOutputFormat)

But how could I set multiple output path's for my GiraphJob?  Maybe I'm
wrong but as I've seen it only one outputformat and outputpath can be
configured for the job.

Can you give me a short code example, how should I configure the GiraphJob
or the VertexOutputFormat to achieve multiple output file usage? :)

Thank you,
Ferenc




2014-06-19 14:02 GMT+02:00 John Yost <soozandjohnyost@gmail.com>:

> Hi Ferenc,
>
> I have an Giraph job that outputs from the Computation class as opposed to
> the MasterCompute because I need to maintain alot of state within
> VertexValues as opposed to Aggregators.  This is one way of outputting
> results as multiple files.  I am assuming that you want to scope output
> files per sub-graph groupings of vertices, of course. :)
>
> --John
>
>
> On Thu, Jun 19, 2014 at 4:02 AM, Ferenc Béres <ferdzso05@gmail.com> wrote:
>
>> Hi Everyone,
>>
>> Currently I'm working on an ALS implementation in giraph 1.1.0 and I
>> would like to output the values of the vertices into multiple output files,
>> but I could not figure it out how to do it.
>>
>> I found that in Hadoop it can be done by using *org.apache.hadoop.mapreduce.lib.output.MultipleOutputs<KEYOUT,VALUEOUT>,
>> *but it didn't work with the GiraphJob.
>>
>> Is it possible to output into multiple files by configuring the
>> GiraphJob, or there is an other way?
>>
>> I would appreciate any idea in this matter.
>>
>> Thank you,
>> Ferenc Béres
>>
>
>

Mime
View raw message