giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claudio Martella <claudio.marte...@gmail.com>
Subject Re: Giraph avro input format
Date Mon, 24 Mar 2014 16:11:50 GMT
I think you can try to see how the avro mapreduce inputformat works and use
it on your own in giraph. all the different inputformats we have do that,
so you can copy them. What exactly of the GoraInputFormat do you find
overkill and why?


On Mon, Mar 24, 2014 at 3:22 PM, Martin Neumann <mneumann@spotify.com>wrote:

> I'm trying to put the GiraphJob I wrote into production so I need to fix
> this.
>
> The avro file I need to read is structured in the (bytes) schema. Each
> datum is actually just a number of words separated by whitespace.
> The whole thing is not yet in graph format so that I need some logic in
> the giraph input format to build it.
>
> I tried using the MapReduce avro formats but I could not get it to work.
> I haven't tried Gora yet but its seems a bit of overkill to just add it to
> load and store avro.
>
> Anyone any Ideas?
>
> cheers Martin
>
>
>
> On Tue, Feb 18, 2014 at 8:22 PM, Roman Shaposhnik <roman@shaposhnik.org>wrote:
>
>> It would be interesting to see what can be done with Avro
>> in Giraph natively along the lines of:
>>    http://avro.apache.org/docs/1.7.6/mr.html
>>
>> Thanks,
>> Roman.
>>
>> On Mon, Feb 17, 2014 at 11:49 AM, Claudio Martella
>> <claudio.martella@gmail.com> wrote:
>> > I'm not sure about what I'm going to say, but Gora should read from
>> Avro,
>> > and we do support reading transparently through Gora. you could check
>> that
>> > out.
>> >
>> >
>> > On Mon, Feb 17, 2014 at 1:32 PM, Martin Neumann <mneumann@spotify.com>
>> > wrote:
>> >>
>> >> Hej,
>> >>
>> >> Is there an avro input format for Giraph? I saw some older (july 2013)
>> >> entries on the mailing list and none existed by then. Have things
>> changed
>> >> since then, or do I have to write my own?
>> >>
>> >> When I write my own what's a good base class to start from?
>> >>
>> >> cheers Martin
>> >
>> >
>> >
>> >
>> > --
>> >    Claudio Martella
>> >
>>
>
>


-- 
   Claudio Martella

Mime
View raw message