Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-user@hadoop.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
Message-ID: <d6d7c4410811031237r26bef6edta09e9ada0d4eeab0@mail.gmail.com>
Date: Mon, 3 Nov 2008 14:37:00 -0600
From: "Aaron Kimball" <aaron@cloudera.com>
To: core-user@hadoop.apache.org
Subject: Re: Passing Constants from One Job to the Next
In-Reply-To: <74f4d40b0810301502u5de4f271h18a443ff25e7cdde@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_62871_14766544.1225744620740"
References: <e6d767480810221452ha563be8t6f6c0f75baf1fbb8@mail.gmail.com>
	 <62DC94C5-38EE-4A32-A198-FF74B73FCB58@yahoo-inc.com>
	 <e6d767480810221503q68966875kf14e3ab87f0a9a89@mail.gmail.com>
	 <d6d7c4410810230240q28ad96f4s19e3ef713c484c6c@mail.gmail.com>
	 <74f4d40b0810301502u5de4f271h18a443ff25e7cdde@mail.gmail.com>

------=_Part_62871_14766544.1225744620740
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

The Mapper and Reducer interfaces both provide a method 'void
configure(JobConf conf) throws IOException'; if you extend MapReduceBase,
this will provide a dummy implementation of configure(). You can add your
own implementation; it will be called before the first call to map() or
reduce(). You can read your initialization data at this time.

- Aaron

On Thu, Oct 30, 2008 at 4:02 PM, Erik Holstad <erikholstad@gmail.com> wrote:

> Hi!
> Is there a way of using the value read in the configure() in the Map or
> Reduce phase?
>
> Erik
>
> On Thu, Oct 23, 2008 at 2:40 AM, Aaron Kimball <aaron@cloudera.com> wrote:
>
> > See Configuration.setInt() in the API. (JobConf inherits from
> > Configuration). You can read it back in the configure() method of your
> > mappers/reducers
> > - Aaron
> >
> > On Wed, Oct 22, 2008 at 3:03 PM, Yih Sun Khoo <yskhoo@gmail.com> wrote:
> >
> > > Are you saying that I can pass, say, a single integer constant with
> > either
> > > of these three: JobConf? A HDFS file? DistributedCache?
> > > Or are you asking if I can pass given the context of: JobConf? A HDFS
> > file?
> > > DistributedCache?
> > > I'm thinking of how to pass a single int so from one Jobconf to the
> next
> > >
> > > On Wed, Oct 22, 2008 at 2:57 PM, Arun C Murthy <acm@yahoo-inc.com>
> > wrote:
> > >
> > > >
> > > > On Oct 22, 2008, at 2:52 PM, Yih Sun Khoo wrote:
> > > >
> > > >  I like to hear some good ways of passing constants from one job to
> the
> > > >> next.
> > > >>
> > > >
> > > > Unless I'm missing something: JobConf? A HDFS file? DistributedCache?
> > > >
> > > > Arun
> > > >
> > > >
> > > >
> > > >> These are some ways that I can think of:
> > > >> 1)  The obvious solution is to carry the constant as part of your
> > value
> > > >> from
> > > >> one job to the next, but that would mean every value would hold that
> > > >> constant
> > > >> 2)  Use the reporter as a hack so that you can set the status
> message
> > > and
> > > >> then get the status message back when u need the constant
> > > >>
> > > >> Any other ideas?  (Also please do not include code)
> > > >>
> > > >
> > > >
> > >
> >
>

------=_Part_62871_14766544.1225744620740--