mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jake Mannix <jake.man...@gmail.com>
Subject Re: PFPGrowth - not able to pass hadoop any parameters
Date Wed, 27 Jan 2010 18:38:31 GMT
I wouldn't say that they're being *hijacked* - the class just doesn't
implement Tool, so it
doesn't let them become hadoop parameters to be passed into the
Configuration.

I agree that we should, because while it's not an API requirement that Tool
/ ToolRunner
be used, it's pretty standard at this point, for exactly the reasons you're
mentioning.

  -jake

On Wed, Jan 27, 2010 at 10:26 AM, Aurora Skarra-Gallagher <
aurora@yahoo-inc.com> wrote:

> These are hadoop parameters, and the OptionBuilder used in the PFPGrowth
> examples hijacks them.
>
> -Aurora
>
>
> On 1/26/10 4:29 PM, "Sean Owen" <srowen@gmail.com> wrote:
>
> These look like Hadoop params, to the hadoop command? why wouldn't
> hadoop be parsing those, or, why would the Job command have to shuttle
> them to Hadoop? I thought these were typically set in the config .xml
> files anyhow.
>
> On Tue, Jan 26, 2010 at 11:43 PM, Aurora Skarra-Gallagher
> <aurora@yahoo-inc.com> wrote:
> > Hi,
> >
> > I'm using the PFPGrowth code (
> http://issues.apache.org/jira/browse/MAHOUT-157) from Mahout 0.3 and it
> works fine on my local box. However, when I try to get it to run on our grid
> cluster, it amazingly does not allow any parameters to be passed to Hadoop.
> When I look at the code
> (mahout/core/src/main/java/org/apache/mahout/fpm/pfpgrowth/PFPGrowth.java),
> I see that there is no way to pass custom configuration parameters (like -
> Dmapred.job.queue.name=X or -libjars or any other parameter for that
> matter).
> >
> > I am shocked that it would be done this way. To get this to work, I need
> to go change the actual PFPGrowth.java file, add my conf.set("key", "val")
> lines, and recompile. Is there any other way to do this? Why would it be
> written in such a way that all hadoop parameters are disallowed?
> >
> > Thanks,
> > Aurora
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message