hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (JIRA)" <j...@apache.org>
Subject [jira] Updated: (MAPREDUCE-2141) Add an "extra data" field to Task for use by Mesos
Date Tue, 19 Oct 2010 02:46:30 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Arun C Murthy updated MAPREDUCE-2141:
-------------------------------------

    Status: Open  (was: Patch Available)

Matei, the patch itself is small enough that it probably doesn't matter, but I'd still like
to call out that this is a first for Hadoop Map-Reduce i.e. explicit support for non-Apache
software such as Mesos.

Personally I am happy to commit this, especially given it's innocuous nature - but I'd like
to ensure others in the community are fine with this. Tom? Dhruba? (I see you guys are watching
this.)

----

Also, it might be worth-while to make this a 'context object' rather than a String.

> Add an "extra data" field to Task for use by Mesos
> --------------------------------------------------
>
>                 Key: MAPREDUCE-2141
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2141
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Matei Zaharia
>            Assignee: Matei Zaharia
>            Priority: Minor
>         Attachments: mapreduce-2141-v1.patch
>
>
> In order to support running Hadoop on the Mesos cluster manager (http://mesos.berkeley.edu/),
I'd like to add an extra String field to the Task class to allow extra data (a Mesos task
ID) to be associated with each task. This should have no impact on normal operation other
than making the serialized form of Task a few bytes longer. In the Mesos support patch for
Hadoop, this field is set by a pluggable Hadoop scheduler implementation to allow code on
the TaskTracker side to see which Mesos task corresponds to each Hadoop task. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message