hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amogh Vasekar <am...@yahoo-inc.com>
Subject Re: Multiple Input Paths
Date Mon, 02 Nov 2009 16:45:22 GMT
Set-up for a mapred job consumes a considerable amount of time and resources and so, if possible
a single job is preferred.
You can add multiple paths to your job, and if you need different processing logic depending
upon the input being consumed, you can use parameter map.input.file in your mapper to decide.


On 11/2/09 8:53 PM, "Mark Vigeant" <mark.vigeant@riskmetrics.com> wrote:

Hey, quick question:

I'm writing a program that parses data from 2 different files and puts the data into a table.
Currently I have 2 different map functions and so I submit 2 separate jobs to the job client.
Would it be more efficient to add both paths to the same mapper and only submit one job? Thanks
a lot!

Mark Vigeant
RiskMetrics Group, Inc.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message