giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Martin Neumann <mneum...@spotify.com>
Subject Re: Giraph avro input format
Date Mon, 24 Mar 2014 14:22:49 GMT
I'm trying to put the GiraphJob I wrote into production so I need to fix
this.

The avro file I need to read is structured in the (bytes) schema. Each
datum is actually just a number of words separated by whitespace.
The whole thing is not yet in graph format so that I need some logic in the
giraph input format to build it.

I tried using the MapReduce avro formats but I could not get it to work.
I haven't tried Gora yet but its seems a bit of overkill to just add it to
load and store avro.

Anyone any Ideas?

cheers Martin



On Tue, Feb 18, 2014 at 8:22 PM, Roman Shaposhnik <roman@shaposhnik.org>wrote:

> It would be interesting to see what can be done with Avro
> in Giraph natively along the lines of:
>    http://avro.apache.org/docs/1.7.6/mr.html
>
> Thanks,
> Roman.
>
> On Mon, Feb 17, 2014 at 11:49 AM, Claudio Martella
> <claudio.martella@gmail.com> wrote:
> > I'm not sure about what I'm going to say, but Gora should read from Avro,
> > and we do support reading transparently through Gora. you could check
> that
> > out.
> >
> >
> > On Mon, Feb 17, 2014 at 1:32 PM, Martin Neumann <mneumann@spotify.com>
> > wrote:
> >>
> >> Hej,
> >>
> >> Is there an avro input format for Giraph? I saw some older (july 2013)
> >> entries on the mailing list and none existed by then. Have things
> changed
> >> since then, or do I have to write my own?
> >>
> >> When I write my own what's a good base class to start from?
> >>
> >> cheers Martin
> >
> >
> >
> >
> > --
> >    Claudio Martella
> >
>

Mime
View raw message