hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gera Shegalov <g...@shegalov.com>
Subject Re: Sorting the inputSplits
Date Wed, 29 Jul 2015 23:31:39 GMT
Can you clarify the requirement "processed first"? Maps run in parallel
without any ordering guarantees. If you want to affect the mapping
file->split number, you can implement your own getSplits in the custom
input format and return splits ordered anyway your like.

On Wed, Jul 22, 2015 at 12:06 PM, Nishanth S <chinchu2884@gmail.com> wrote:

> Hey folks,
> Is their a way to sort the input splits  in map reduce.We have a case
> where there are two files file1 and file2 in the input directory.Since we
>  have custominputformat which   has issplittable return false always each
> of  these files would be processed  by  a different mapper.How could I make
> sure that  file1 is processed   before  file2(I want the oldest file to  be
> processed first).Is this possible?.
> Thanks,
> Nishan

View raw message