Mailing-List: contact hama-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hama-dev@incubator.apache.org
Received-SPF: pass (athena.apache.org: domain of
 thomas.jungblut@googlemail.com designates 209.85.212.47 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CADYHM8wYOe8Sykzh6LtaE4ie_rNcEqSirtv_+ORnCxsbEzMHJA@mail.gmail.com>
References: 
 <CADYHM8wdrY28WxhwAcSPmrkZ0cC_z6-4EwkYpAusv29O8P75Uw@mail.gmail.com>
	<CA+jdVcxJHT3Tm4qWypZQ_eiC3S93R-YRqCCFQyY7GVaSXYa5Rw@mail.gmail.com>
	<CADYHM8yyEpJFR+s3PPV7UzCBD3aXFBitv73vi0NRnCKhxmaCcg@mail.gmail.com>
	<CAJ-=ysm2ZYA==phZa4=mz3NdUJ1E7CK2ysFZWFz4BczGH+9t2g@mail.gmail.com>
	<CADYHM8wYOe8Sykzh6LtaE4ie_rNcEqSirtv_+ORnCxsbEzMHJA@mail.gmail.com>
Date: Sun, 25 Mar 2012 19:27:16 +0200
Message-ID: 
 <CAJ-=ysmZo3PJ10ez4W4Smf63fNo4iek8nuP6SvfSUjdS+f7rtg@mail.gmail.com>
Subject: Re: InputFormats for Hama
From: Thomas Jungblut <thomas.jungblut@googlemail.com>
To: hama-dev@incubator.apache.org
Content-Type: multipart/alternative; boundary=20cf307d00f8b6a04d04bc1491b4

--20cf307d00f8b6a04d04bc1491b4
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Thanks for your time.
I have tweeted about the graph db formats, I know some of my followers are
working with them, so they might be interested.

Am 25. M=E4rz 2012 19:25 schrieb Praveen Sripati <praveensripati@gmail.com>=
:

> I have created Umbrella JIRA HAMA-536 for creating the
> InputFormats/OutputFormats with three sub-tasks. For now I have assigned
> the tasks to me, let me know if anyone is interested.
>
> Praveen
>
> On Sun, Mar 25, 2012 at 6:40 PM, Thomas Jungblut <
> thomas.jungblut@googlemail.com> wrote:
>
> > >
> > > I can open a JIRA. I need input on what all InputFormat makes sense a=
nd
> > the
> > > their priority. Some we can port from Hadoop.
> >
> >
> > Yep, you're right. I guess a single JIRA would be enough for the alread=
y
> > implemented formats in Hadoop, for the others we need subclasses.
> > Formats that I really wanted to have would be:
> >
> >   - DBInputFormat[1]
> >   - XMLInputFormat
> >   - NLineInputFormat
> >   - CSVInputFormat (we could use OpenCSV for that in conjunction with
> >   TextInputFormat)
> >   - JSONInputFormat (for OpenGraph stuff)
> >   - The graph DB formats Neo4J and how the others are called
> >
> > Anything I missed for a "full" coverage?
> >
> > Could you please elaborate on this?
> >
> >
> > Sure, DMOZ is some kind of crawled website database. It is used in some
> > pagerank examples to test it, don't know if it was in Mahout. We could
> also
> > use it since we have pagerank as well.
> > CommonCrawl is a new up-coming DMOZ-like database of many crawled sites=
,
> it
> > is hosted on S3 in Amazon Cloud. We run on EC2 via Whirr so this could
> be a
> > cool example as well.
> >
> > [1]
> >
> >
> http://hadoop.apache.org/mapreduce/docs/r0.21.0/api/org/apache/hadoop/map=
reduce/lib/db/DBInputFormat.html
> >
> >
> > Am 25. M=E4rz 2012 14:56 schrieb Praveen Sripati <praveensripati@gmail.=
com
> >:
> >
> > > Thomas et al,
> > >
> > > > Would someone please open JIRAs for that?
> > >
> > > I can open a JIRA. I need input on what all InputFormat makes sense a=
nd
> > the
> > > their priority. Some we can port from Hadoop.
> > >
> > > > Based on XML we can implement a format that parses DMOZ or
> commoncrawl
> > on
> > > Amzon S3.
> > >
> > > Could you please elaborate on this?
> > >
> > > Praveen
> > >
> > >
> > > On Sun, Mar 25, 2012 at 5:14 PM, Chia-Hung Lin <clin4j@googlemail.com
> > > >wrote:
> > >
> > > > As I understand, many iterative applications don't require key valu=
e
> > > > input/ output and additionally need random access (read/ write) to
> > > > particular file. I/O interface e.g. mpi may increase flexibility
> here.
> > > >
> > > > https://issues.apache.org/jira/browse/MAPREDUCE-2911
> > > >
> > > > On 25 March 2012 10:01, Praveen Sripati <praveensripati@gmail.com>
> > > wrote:
> > > > > Hi,
> > > > >
> > > > > For Hama there are limited input formats
> > > > >
> > > > > CombineFileInputFormat, FileInputFormat, NullInputFormat,
> > > > > SequenceFileInputFormat, TextInputFormat
> > > > >
> > > > > Does it make sense to have to have more input formats? I was
> thinking
> > > > > InputFormats for Graph Databases.
> > > > >
> > > > > Any feedback for the different input formats is welcome.
> > > > >
> > > > > I quickly glanced Giraph and Hadoop and they have more InputForma=
ts
> > > which
> > > > > makes it easy to plug them with external systems.
> > > > >
> > > > > Praveen
> > > >
> > >
> >
> >
> >
> > --
> > Thomas Jungblut
> > Berlin <thomas.jungblut@gmail.com>
> >
>


--=20
Thomas Jungblut
Berlin <thomas.jungblut@gmail.com>

--20cf307d00f8b6a04d04bc1491b4--