hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-1247) Make Hadoop Abacus work with Hadoop Streaming
Date Mon, 23 Apr 2007 22:24:15 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-1247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Owen O'Malley updated HADOOP-1247:
----------------------------------

    Status: Open  (was: Patch Available)

This is pretty close. 

1. Please do NOT use string literals like:

"org.apache.hadoop.mapred.SequenceFileAsTextInputFormat".equals(cls)

rather, you should do:

SequenceFileAsTextInputFormat.class.getName().equals(cls)

to avoid just the kind of bug that you are fixing.

2. Also, using case insensitive comparison for class names, seems wrong.

3. It bothers me, but not fatally so, that you are causing the abacus class files to be also
compiled into the streaming.jar. This has the potential to be very confusing as the user could
somehow end up with version conflicts. We should seriously consider moving abacas into the
main code base/jar.


> Make Hadoop Abacus work with Hadoop Streaming
> ---------------------------------------------
>
>                 Key: HADOOP-1247
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1247
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: contrib/streaming
>            Reporter: Runping Qi
>         Assigned To: Runping Qi
>         Attachments: patch_1247.txt
>
>
> It will be nice if Hadoop streaming can use Hadoop Abacus.
> This requires to make some changes on StreamJob and StreamMapRed classes so that the
streaming job uses the Abacus reducer/combiner classes, and streaming mapper generates intermediate
data conforming to Hadoop Abacus protocol.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message