mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jake Mannix <jake.man...@gmail.com>
Subject Re: PFPGrowth - not able to pass hadoop any parameters
Date Wed, 27 Jan 2010 04:02:39 GMT
Yeah, this was one of my thoughts with MAHOUT-185 - turn some of our Driver
classes to just fire off a Tool.  It is very convenient to be able to do
this, and
it's becoming more standard as well.

I need to dig up my stuff in decomposer/contrib-hadoop and pull that in and
integrate
it with Drew's patch on that ticket.

  -jake

On Tue, Jan 26, 2010 at 7:19 PM, Robin Anil <robin.anil@gmail.com> wrote:

> Mahout algorithms are not using ToolRunner of Hadoop. I guess many core
> hadoop-ers like that feature. I think we should be supporting that feature
> by 0.3
>
>
> Robin
>
> On Wed, Jan 27, 2010 at 5:59 AM, Sean Owen <srowen@gmail.com> wrote:
>
> > These look like Hadoop params, to the hadoop command? why wouldn't
> > hadoop be parsing those, or, why would the Job command have to shuttle
> > them to Hadoop? I thought these were typically set in the config .xml
> > files anyhow.
> >
> > On Tue, Jan 26, 2010 at 11:43 PM, Aurora Skarra-Gallagher
> > <aurora@yahoo-inc.com> wrote:
> > > Hi,
> > >
> > > I'm using the PFPGrowth code (
> > http://issues.apache.org/jira/browse/MAHOUT-157) from Mahout 0.3 and it
> > works fine on my local box. However, when I try to get it to run on our
> grid
> > cluster, it amazingly does not allow any parameters to be passed to
> Hadoop.
> > When I look at the code
> >
> (mahout/core/src/main/java/org/apache/mahout/fpm/pfpgrowth/PFPGrowth.java),
> > I see that there is no way to pass custom configuration parameters (like
> -
> > Dmapred.job.queue.name=X or -libjars or any other parameter for that
> > matter).
> > >
> > > I am shocked that it would be done this way. To get this to work, I
> need
> > to go change the actual PFPGrowth.java file, add my conf.set("key",
> "val")
> > lines, and recompile. Is there any other way to do this? Why would it be
> > written in such a way that all hadoop parameters are disallowed?
> > >
> > > Thanks,
> > > Aurora
> > >
> >
>
>
>
> --
> ------
> Robin Anil
> Blog: http://techdigger.wordpress.com
> -------
> Try out Swipeball for iPhone
> Video: http://www.youtube.com/watch?v=3hvEbWHciwU
> iTunes: http://itunes.com/apps/swipeball
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message