mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "XiaoboGu" <guxiaobo1...@gmail.com>
Subject RE: Can all the algorithms in Mahout be run locally without a Hadoop cluster.
Date Sat, 25 Jun 2011 02:21:50 GMT
I have found this, will this configuration start the corresponding task trackers too?

http://hadoop-karma.blogspot.com/2010/05/hadoop-cookbook-4-how-to-run-multiple.html


> -----Original Message-----
> From: Ted Dunning [mailto:ted.dunning@gmail.com]
> Sent: Saturday, June 25, 2011 10:12 AM
> To: dev@mahout.apache.org
> Cc: user@mahout.apache.org
> Subject: Re: Can all the algorithms in Mahout be run locally without a Hadoop cluster.
> 
> I have done this with VM's but I would not generally recommend it.  Without
> VM's you will have a pretty ugly configuration issue because Hadoop usually
> assumes it owns the machine.
> 
> Besides, this is a seriously square peg into a round hole kind of problem
> here.  Hadoop (map-reduce) was designed so that you could use several little
> machines instead of one big one.  It just isn't going to work well on a
> single computer.
> 
> On Fri, Jun 24, 2011 at 6:49 PM, XiaoboGu <guxiaobo1982@gmail.com> wrote:
> 
> > Do you have any experience  in running multiple data nodes and task
> > trackers on a single SMP server.
> >
> > > -----Original Message-----
> > > From: Ted Dunning [mailto:ted.dunning@gmail.com]
> > > Sent: Saturday, June 25, 2011 9:26 AM
> > > To: user@mahout.apache.org
> > > Cc: dev@mahout.apache.org
> > > Subject: Re: Can all the algorithms in Mahout be run locally without a
> > Hadoop cluster.
> > >
> > > Pretty big.  SHould scream for local classifier learning.
> > >
> > > Local Hadoop should run pretty fast as well.
> > >
> > > On Fri, Jun 24, 2011 at 5:54 PM, XiaoboGu <guxiaobo1982@gmail.com>
> > wrote:
> > >
> > > > 32Core, 256G RAM
> > > >
> > > > > -----Original Message-----
> > > > > From: Ted Dunning [mailto:ted.dunning@gmail.com]
> > > > > Sent: Saturday, June 25, 2011 1:37 AM
> > > > > To: user@mahout.apache.org
> > > > > Cc: dev@mahout.apache.org
> > > > > Subject: Re: Can all the algorithms in Mahout be run locally without
> > a
> > > > Hadoop cluster.
> > > > >
> > > > > Big iron is fine for some of the classifier stuff, but throughput
per
> > $
> > > > can
> > > > > be higher for other algorithms with a cluster of smaller machines.
> > > > >
> > > > > How big a machine are you talking about?  Even relatively small
> > machines
> > > > are
> > > > > pretty massive any more.  8 core = 16 hyper-thread machines with
48GB
> > > > seem
> > > > > to be not even very impressive any more.
> > > > >
> > > > > On Fri, Jun 24, 2011 at 1:47 AM, XiaoboGu <guxiaobo1982@gmail.com>
> > > > wrote:
> > > > >
> > > > > > We will put a big SMP server to deploy Mahout.
> > > > > >
> > > > > > Regards,
> > > > > >
> > > > > > Xiaobo Gu
> > > > > >
> > > > > >
> > > >
> > > >
> >
> >


Mime
View raw message