hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-954) The new interface's Context objects should be interfaces
Date Fri, 18 Sep 2009 23:42:16 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12757525#action_12757525
] 

Arun C Murthy commented on MAPREDUCE-954:
-----------------------------------------

bq.  I'm not sure that WrappedMapper and WrappedReducer belong in a "lib" package, since the
classes in "lib" are user-facing, and these are framework classes. (People might see them
and wonder how they can use a WrappedMapper in their application, for example.) They would
be better in the task package I think.

Owen & I went back forth on this - finally Owen convinced me it's better to put it lib.

bq. Can we move org.apache.hadoop.mapreduce.task to org.apache.hadoop.mapreduce.server.task
to better emphasise that this is non-user code. This reflects the packaging of HDFS more,
where things that run on the cluster are under a "server" package. We should have another
JIRA to move org.apache.hadoop.mapreduce.task.reduce to org.apache.hadoop.mapreduce.server.task.reduce.

We already have the new shuffle in org.apache.hadoop.mapreduce.task.reduce, thus  org.apache.hadoop.mapreduce.task
seemed logical.

bq. It's a shame JobContextImpl is public and in org.apache.hadoop.mapreduce since users shouldn't
be exposed to it. Can we move it to another package?

Done.

bq. Since Job extends JobContextImpl you don't need the changes that change the *_ATTR constants
(e.g. OUTPUT_FORMAT_CLASS_ATTR) to JobContextImpl.*_ATTR - they can be referred to directly.

Done.

    * What's the compatibility story for previous releases? Would a 0.20 MR program written
to the new ("mapreduce" package) API work with the new interfaces unchanged? What about a
0.20 program using the old MR API - will it continue to work with the old MR API with these
changes?

Yes to both.

> The new interface's Context objects should be interfaces
> --------------------------------------------------------
>
>                 Key: MAPREDUCE-954
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-954
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: client
>            Reporter: Owen O'Malley
>            Assignee: Arun C Murthy
>             Fix For: 0.21.0
>
>         Attachments: MAPREDUCE-954.patch, MAPREDUCE-954.patch, MAPREDUCE-954.patch, MAPREDUCE-954.patch,
MAPREDUCE-954.patch, MAPREDUCE-954.patch
>
>
> When I was doing HADOOP-1230, I was persuaded to make the Context objects as classes.
I think that was a serious mistake. It caused a lot of information leakage into the public
classes.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message