hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Min Zhou <coderp...@gmail.com>
Subject Re: how to dump data from a mysql cluster to hdfs?
Date Thu, 06 Aug 2009 10:31:59 GMT
I guess I havenot expressed clearly. Neither datanodes nor namenodes is
allowed to be directly connected.
Even though, namenode is often in heavy work, it would be burdened dumping
data on them.

On Thu, Aug 6, 2009 at 1:52 PM, Yang Zhou <yangzhou.ebay@gmail.com> wrote:

> Write a Java program which will dump data from mysql cluster and save them
> into HDFS at the same time.
> Run it on namenode. I assume namenode should be able to connect to mysql
> gateway.
> Will it work?
>
> On Thu, Aug 6, 2009 at 12:02 PM, Min Zhou <coderplay@gmail.com> wrote:
>
> > Hi Aaron,
> >
> > We couldnot run mysqldump on the nodes mysqld runs on. The only way is
> > handling a connection to a gateway of the mysql cluster. Our hadoop
> cluster
> > serves us with also gateways, it's not allowed hadoop datanodes directly
> > connect to mysql gateway.
> >
> > Min
> >
> > On Thu, Aug 6, 2009 at 1:27 AM, Aaron Kimball <aaron@cloudera.com>
> wrote:
> >
> > > mysqldump to local files on all 50 nodes, scp them to datanodes, and
> then
> > > bin/hadoop fs -put?
> > > - Aaron
> > >
> > > On Mon, Aug 3, 2009 at 8:15 PM, Min Zhou <coderplay@gmail.com> wrote:
> > >
> > > > hi all,
> > > >
> > > > We need to dump data from a mysql cluster with about 50 nodes to a
> hdfs
> > > > file. Considered about the issues on security , we can't use tools
> like
> > > > sqoop, where all datanodes must hold a connection to mysql. any
> > > > suggestions?
> > > >
> > > >
> > > > Thanks,
> > > > Min
> > > > --
> > > > My research interests are distributed systems, parallel computing and
> > > > bytecode based virtual machine.
> > > >
> > > > My profile:
> > > > http://www.linkedin.com/in/coderplay
> > > > My blog:
> > > > http://coderplay.javaeye.com
> > > >
> > >
> >
> >
> >
> > --
> > My research interests are distributed systems, parallel computing and
> > bytecode based virtual machine.
> >
> > My profile:
> > http://www.linkedin.com/in/coderplay
> > My blog:
> > http://coderplay.javaeye.com
> >
>



-- 
My research interests are distributed systems, parallel computing and
bytecode based virtual machine.

My profile:
http://www.linkedin.com/in/coderplay
My blog:
http://coderplay.javaeye.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message