Return-Path: X-Original-To: apmail-giraph-user-archive@www.apache.org Delivered-To: apmail-giraph-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7789810C54 for ; Fri, 6 Sep 2013 07:43:36 +0000 (UTC) Received: (qmail 42054 invoked by uid 500); 6 Sep 2013 07:43:35 -0000 Delivered-To: apmail-giraph-user-archive@giraph.apache.org Received: (qmail 41810 invoked by uid 500); 6 Sep 2013 07:43:24 -0000 Mailing-List: contact user-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@giraph.apache.org Delivered-To: mailing list user@giraph.apache.org Received: (qmail 41802 invoked by uid 99); 6 Sep 2013 07:43:22 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Sep 2013 07:43:22 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of claudio.martella@gmail.com designates 209.85.128.175 as permitted sender) Received: from [209.85.128.175] (HELO mail-ve0-f175.google.com) (209.85.128.175) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Sep 2013 07:43:16 +0000 Received: by mail-ve0-f175.google.com with SMTP id jy13so1313336veb.20 for ; Fri, 06 Sep 2013 00:42:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=D9ePdTacQFaa/tMZKglEE3Xbtb+SwgiDAB7vAVQcHp8=; b=Fxh7zWaq85UvfJsA2xxbvD8xKYDzehEjEFHbcrDmXrvF4K+KaAoNT3BcP9kKSAPsOm mr4NNY4D9JVzciRIHRrJLxcKK4ghiXKS2woMrjCMpVn7SD0Cvv5Yii9vb7i61WcuOHjW B7glk00tD3GFeJFbMR7wew2Otsup9imeqdpI+ZVBFH4eTDfQrfha0O5asNxfght4S4QD kq/HSS7mqvTzQ6Ux6KkgBQKnb20W1Q/O81r6B61HEmh/Q41tBkoVXmwV/9eSN9RNWd+8 91pu1y2fsqIWDYkRHuaepr613dx7wYgdS57lLldl5a/zxi2jZUYnSAqOFudkwDXNiG9m t1xg== X-Received: by 10.58.44.37 with SMTP id b5mr1079123vem.4.1378453375865; Fri, 06 Sep 2013 00:42:55 -0700 (PDT) MIME-Version: 1.0 Received: by 10.220.141.206 with HTTP; Fri, 6 Sep 2013 00:42:35 -0700 (PDT) In-Reply-To: References: From: Claudio Martella Date: Fri, 6 Sep 2013 09:42:35 +0200 Message-ID: Subject: Re: MySQL Table To: "user@giraph.apache.org" Content-Type: multipart/alternative; boundary=089e013a132acedb5704e5b22f1b X-Virus-Checked: Checked by ClamAV on apache.org --089e013a132acedb5704e5b22f1b Content-Type: text/plain; charset=ISO-8859-1 Hi Bu, no, currently we do not have a DBInputFormat. We have an open issue with a google summer of code student working on a GoraInputFormat, which supports also reading from RDBMs through Gora. However, if/when it will get it, it will not provide a rich semantic as DBInputFormat, e.g. you'll be able to only provide scan-like/range queries, instead of ANY query like DBInputFormat. I think that creating an DB[Vertex|Edge]InputFormat starting from the hadoop DBInputFormat should not be too hard and could prove to be a very useful contribution. If you think about providing an implementation, I can provide guidance. Best, Claudio On Fri, Sep 6, 2013 at 1:45 AM, Bu Xiao wrote: > Hi Girapher, > > I am currently working on algorithm that requires reading the > vertices from MySQL table and not from HDFS. I thought that there has to be > a way of reading data from SQL table since Giraph is built on top of > Hadoop. But I do not seem to figure this part out. Do you have a class > similar to the DBInputFormat in Hadoop? Thank you very much for your help. > > > -- Claudio Martella claudio.martella@gmail.com --089e013a132acedb5704e5b22f1b Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Hi Bu,

no, currently we do not have a D= BInputFormat. We have an open issue with a google summer of code student wo= rking on a GoraInputFormat, which supports also reading from RDBMs through = Gora. However, if/when it will get it, it will not provide a rich semantic = as DBInputFormat, e.g. you'll be able to only provide scan-like/range q= ueries, instead of ANY query like DBInputFormat.

I think that creating an DB[Vertex|Edge]InputFormat sta= rting from the hadoop DBInputFormat should not be too hard and could prove = to be a very useful contribution. If you think about providing an implement= ation, I can provide guidance.

Best,
Claudio


On Fri, Sep 6, 2013 at 1:45 AM, B= u Xiao <buxiao82@gmail.com> wrote:
Hi Girapher,

=
=A0 =A0 =A0 =A0I am currently working on algorithm that requires readi= ng the vertices from MySQL table and not from HDFS. I thought that there ha= s to be a way of reading data from SQL table since Giraph is built on top o= f Hadoop. But I do not seem to figure this part out. Do you have a class si= milar to the DBInputFormat in Hadoop? Thank you very much for your help.





--
=A0 =A0Clau= dio Martella
=A0 =A0claudio.martella@gmail.com=A0 =A0
--089e013a132acedb5704e5b22f1b--