hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1376) Support for varied user submission in Gridmix
Date Fri, 12 Mar 2010 22:33:27 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12844723#action_12844723

Chris Douglas commented on MAPREDUCE-1376:

The attached patch, even with security disabled, did not work for me using {{RoundRobinUserResolver}}
(user from trace replaced with {{<traceuser>}}, user from userlist replaced with {{<targetuser>}}):
10/03/12 20:20:52 WARN gridmix.JobSubmitter: Failed to submit GRIDMIX00106 as <targetuser>
via <traceuser>
org.apache.hadoop.ipc.RemoteException: User: <traceuser> is not allowed to impersonate
        at org.apache.hadoop.ipc.Client.call(Client.java:873)
        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:222)
        at org.apache.hadoop.mapred.$Proxy1.getProtocolVersion(Unknown Source)
        at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:360)
        at org.apache.hadoop.mapred.JobClient.createRPCProxy(JobClient.java:443)
        at org.apache.hadoop.mapred.JobClient.init(JobClient.java:437)
        at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:422)
        at org.apache.hadoop.mapreduce.Job$1.run(Job.java:477)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:766)
        at org.apache.hadoop.mapreduce.Job.connect(Job.java:475)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:464)
        at org.apache.hadoop.mapred.gridmix.GridmixJob.call(GridmixJob.java:230)
        at org.apache.hadoop.mapred.gridmix.JobSubmitter$SubmitTask.run(JobSubmitter.java:119)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)

Other feedback:
* The following should be removed:
** This method is not used:
+  private void throwException(Throwable exception) throws Throwable{
+    throw exception;
+  }
** GridmixJob contains a few lines with empty statements (see {{doAs}} blocks)
** {{GridmixTestUtils}} is not a useful abstraction. Its functionality should remain in/be
added to {{TestGridmixSubmission}}
* Consider {{Collections.emptyList()}} instead of creating and returning new, empty collections
(e.g. {{EchoUserResolver}})
* {{EchoUserResolver}} only needs to extend {{ShellBasedUnixGroupsMapping}} for the unit test,
right? The group mapping isn't important for the type. A separate group mapping class in test
would be appropriate (presumably one already exists)
* {{RoundRobinUserResolver::parseUserList}} should be protected so subclasses may override
* Since {{UserResolver}} can remain an abstract class (no need to extend any groups mapping),
{{parseUserList}} can remain there.
* The default policy should be {{REPLAY}}, not {{STRESS}}
* {{JobSubmitter}} should not re-resolve the resolved UGI before calling {{buildSplits}}.
* It is not sufficient to fix the failure above, but the job is not submitted in a {{doAs}}

> Support for varied user submission in Gridmix
> ---------------------------------------------
>                 Key: MAPREDUCE-1376
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1376
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: contrib/gridmix
>            Reporter: Chris Douglas
>            Assignee: Chris Douglas
>         Attachments: 1376-2-yhadoop-security.patch, 1376-yhadoop-security.patch, M1376-0.patch,
M1376-1.patch, M1376-2.patch, M1376-3.patch, M1376-4.patch
> Gridmix currently submits all synthetic jobs as the client user. It should be possible
to map users in the trace to a set of users appropriate for the target cluster.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message