hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-12547) Deprecate hadoop-pipes
Date Thu, 05 Nov 2015 18:45:27 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-12547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14992218#comment-14992218
] 

Allen Wittenauer commented on HADOOP-12547:
-------------------------------------------

bq. Because the Java MR API only supports Java (and possibly other JVM languages), whereas
streaming supports Perl, Python, Ruby, C, C++, and any other non-JVM programming language
you can think of.

I'm pretty sure streaming supports JVM languages too since I'd be surprised if Java couldn't
read and write from stdin and stdout... which, by your own argument, means we should drop
the Java client APIs too.  After all, that would reduce the code footprint, limit the testing
needs, etc, etc, too right?

bq. What specifically are you -1 on? Removal, deprecation, or both?

At this point, both.  Perhaps deprecation in trunk if the native task stuff actually works.

bq. Can you explain when you would advise one of your customers to use pipes instead of streaming?

https://www.quora.com/Why-would-anyone-use-Hadoop-Pipes

https://www.quora.com/If-my-current-job-involves-purely-C-C++-coding-what-are-the-best-ways-to-learn-hadoop-and-contribute-to-the-apache-hadoop-project-I-understand-most-of-hadoop-code-is-Java-Are-there-any-C-C++-bindings-for-hadoop-used-in-production-clusters

bq. If you feel that pipes is worth maintaining, can you file JIRAs to reinstate the documentation,
fix the compiler warnings, and fix the security bugs?

Sure, I'll file JIRAs for these.

> Deprecate hadoop-pipes
> ----------------------
>
>                 Key: HADOOP-12547
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12547
>             Project: Hadoop Common
>          Issue Type: Improvement
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>            Priority: Minor
>
> Development appears to have stopped on hadoop-pipes upstream for the last few years,
aside from very basic maintenance.  Hadoop streaming seems to be a better alternative, since
it supports more programming languages and is better implemented.
> There were no responses to a message on the mailing list asking for users of Hadoop pipes...
and in my experience, I have never seen anyone use this.  We should remove it to reduce our
maintenance burden and build times.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message