hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nikoli Dryden <ndry...@ndryden.com>
Subject Streaming and the TotalOrderPartitioner
Date Wed, 22 Jul 2015 23:03:18 GMT
Hi,

Is there a nice way to use the TotalOrderPartitioner through the Hadoop
streaming interface? It can be specified just fine with the -partitioner
option, but I cannot find a way to generate the partition file from
streaming.

Is there a way to generate it from the streaming interface? Or alternately,
is there a way to manually write out the file in the correct format? Or
have I missed something obvious?

I would prefer not to have to use one reducer, as that would be a big
performance limiter in my application.

Thanks,

Nikoli

Mime
View raw message