hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3782) teragen terasort jobs fail when using webhdfs://
Date Wed, 25 Jul 2012 19:38:34 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422539#comment-13422539

Daryn Sharp commented on MAPREDUCE-3782:

Teragen:  Notice it's the DFClient reporting that it got the token.  It shouldn't be getting
a token via RPC at all.  I think it's getting a token for the default fs (hdfs), not the webhdfs
path on the cmdline.  It "works" because they happen to be the same host and the RPC port
isn't blocked.

Terasort: Again, DFSClient shouldn't be getting the token at all.

Task failure: Awhile back, I think Nicholas fixed an issue with 0 not being considered valid
if the file length is zero.  This may no longer be an issue.
> teragen terasort jobs fail when using webhdfs:// 
> -------------------------------------------------
>                 Key: MAPREDUCE-3782
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3782
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.1, 0.24.0
>            Reporter: Arpit Gupta
>            Priority: Critical
> When running a teragen job with a webhdfs:// url the delegation token that is retrieved
is an hdfs delegation token. 
> And the subsequent terasort job on the output fails with java io exception

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message