hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1362) Pipes should be ported to the new mapreduce API
Date Thu, 07 Jan 2010 17:30:29 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12797694#action_12797694
] 

Todd Lipcon commented on MAPREDUCE-1362:
----------------------------------------

bq. bin/hadoop should support both pipes (old api) and pipes2 (new api)

Is it possible to detect using reflection which API to use, rather than making the user specify?

Otherwise sounds reasonable.

> Pipes should be ported to the new mapreduce API
> -----------------------------------------------
>
>                 Key: MAPREDUCE-1362
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1362
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: pipes
>            Reporter: Bassam Tabbara
>             Fix For: 0.20.2
>
>
> Pipes is still currently using the old mapred API. This prevents us from using pipes
with HBase's TableInputFormat, HRegionPartitioner, etc. 
> Here is a rough proposal for how to accomplish this:
> * Add a new package org.apache.hadoop.mapreduce.pipes that uses the new mapred API.
> * the new pipes package will run side by side with the old one. old one should get deprecated
at some point.
> * the wire protocol used between PipesMapper and PipesReducer and C++ programs must not
change.
> * bin/hadoop should support both pipes (old api) and pipes2 (new api)
> Does this sound reasonable?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message