hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-2780) Standardize the value of token service
Date Mon, 08 Aug 2011 19:24:27 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081142#comment-13081142
] 

Daryn Sharp commented on MAPREDUCE-2780:
----------------------------------------

bq.  It forces to set ip/host/port in the service.

Per the title of this jira, that is the goal. :)  Every token producer is required to use
the same service field format to comply with the format needed by the token's selector. Currently
every token producer has a copy-n-paste chunk selector's code to construct the service format.
 If the selector and the producer get out of sync, there's a big problem.

bq. If we later decide to store something else in the service for example, uri with scheme,
we will have to change it again.

Please elaborate?  This appears as a non-sequitur since token producers that choose to use
a URI (someday), for instance, will require a change in either case.

Here is the evolution from the original code of:
{code}
token.setService(new Text(addr.getAddress().getHostAddress() + ":" + addr.getPort()));
selector.selectToken(new Text(addr.getAddress().getHostAddress() + ":" + addr.getPort()),
tokens);
{code}
I think is your suggestion?  It's incrementally better, but continues to require a copy-n-paste
in each token producer, and every token producer continues to have intimate knowledge of the
service format.
{code}
token.setService(SecurityUtil.buildDTAuthority(addr));
selector.selectToken(SecurityUtil.buildDTAuthority(addr), tokens);
{code}
The patch applies another layer of abstraction.  The format is privatized to the token, instead
of publicly diffused over all the tokens in hadoop.
{code}
token.setService(addr);
selector.selectToken(Token.createService(addr), tokens);
// I removed this due to your earlier concerns in the parent jira
// selector.selectToken(addr, tokens);
{code}


Do you believe this a persuasive case for the patch?

> Standardize the value of token service
> --------------------------------------
>
>                 Key: MAPREDUCE-2780
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2780
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>             Fix For: 0.20.205.0
>
>         Attachments: MAPREDUCE-2780-2.patch, MAPREDUCE-2780.patch
>
>
> The token's service field must (currently) be set to "ip:port".  All the producers of
a token are independently building the service string.  This should be done via a common method
to reduce the chance of error, and to facilitate the field value being easily changed in the
(near) future.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message