Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of sonalgoyal4@gmail.com
 designates 209.85.216.48 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=EWEyKl4kV45XiIh2Lmb9ufNX+JHkyngfETM9VOdmLIZCXhEoiBlIxcMKM6HJzaOp9O
         Mbxmc6Qb1TEBNqVWKwe4sGqzl8Z+lXqrObD7V16Fn/rnI5QyI9/SGX1sIM1cxDn2kcBE
         kA9TFJQvxwOn1cM7X6gtaVXN0QC8U4tyb6TMo=
MIME-Version: 1.0
In-Reply-To: <AANLkTims32Y-pfq=eq8Ydjsq-uvyodQX2WtPpQu38Z6A@mail.gmail.com>
References: <AANLkTims32Y-pfq=eq8Ydjsq-uvyodQX2WtPpQu38Z6A@mail.gmail.com>
Date: Sun, 9 Jan 2011 08:27:34 +0530
Message-ID: <AANLkTi=S_35p4mVs4GhHXP6DSC6fwAzMXS53qopEBxPO@mail.gmail.com>
Subject: Re: Import data from mysql
From: Sonal Goyal <sonalgoyal4@gmail.com>
To: common-user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=001485ebb03e5bf4570499610391

--001485ebb03e5bf4570499610391
Content-Type: text/plain; charset=ISO-8859-1

Hi Brian,

You can check HIHO at https://github.com/sonalgoyal/hiho which can help you
load data from any JDBC database to the Hadoop file system. If your table
has a date or id field, or any indicator for modified/newly added rows, you
can import only the altered rows every day. Please let me know if you need
help.

Thanks and Regards,
Sonal
<https://github.com/sonalgoyal/hiho>Connect Hadoop with databases,
Salesforce, FTP servers and others <https://github.com/sonalgoyal/hiho>
Nube Technologies <http://www.nubetech.co>

<http://in.linkedin.com/in/sonalgoyal>


On Sun, Jan 9, 2011 at 5:03 AM, Brian McSweeney
<brian.mcsweeney@gmail.com>wrote:

> Hi folks,
>
> I'm a TOTAL newbie on hadoop. I have an existing webapp that has a growing
> number of rows in a mysql database that I have to compare against one
> another once a day from a batch job. This is an exponential problem as
> every
> row must be compared against every other row. I was thinking of
> parallelizing this computation via hadoop. As such, I was thinking that
> perhaps the first thing to look at is how to bring info from a database to
> a
> hadoop job and vise versa. I have seen the following relevant info
>
> https://issues.apache.org/jira/browse/HADOOP-2536
>
> and also
>
> http://architects.dzone.com/articles/tools-moving-sql-database
>
> any advice on what approach to use?
>
> cheers,
> Brian
>

--001485ebb03e5bf4570499610391--