hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devendra Bhat (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (MAPREDUCE-614) Streaming partitioner should allow command, not just Java class
Date Fri, 17 Nov 2017 04:42:00 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16256463#comment-16256463
] 

Devendra Bhat edited comment on MAPREDUCE-614 at 11/17/17 4:41 AM:
-------------------------------------------------------------------

I don't know most of the thing which you guys were saying but I want to express clearly what
thing I want from this partitioner function.

I have been trying to study mapreduce with python and while using python I have to pass the
python codes as command and also provide no of reducers in command only.

Can you please implement what you suggested about this "it might be easier to define a suitable
partitioner command (when using shell tools it might be easier to output a string instead
of a specific number for example),"  as @Klass suggested above.


was (Author: devendrabhat):
I don't know most of the thing which you guys were saying but I want to express clearly what
thing I want from this partitioner function.

I have been trying to study mapreduce with python and while using python I have to pass the
python codes as command.

Can you please implement what you suggested about this "it might be easier to define a suitable
partitioner command (when using shell tools it might be easier to output a string instead
of a specific number for example),"  as @Klass suggested above.

> Streaming partitioner should allow command, not just Java class
> ---------------------------------------------------------------
>
>                 Key: MAPREDUCE-614
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-614
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: contrib/streaming
>            Reporter: Klaas Bosteels
>
> Since HADOOP-4842 got committed, Streaming allows both commands and Java classes to be
specified as mapper, reducer, and combiner, but the {{-partitioner}} option is still limited
to Java classes only. Allowing commands to be specified as partitioner as well would greatly
improve the flexibility of Streaming programs.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message