hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7278) Add a command that allows sysadmins to manually trigger full block reports from a DN
Date Thu, 23 Oct 2014 21:08:34 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14181969#comment-14181969

Colin Patrick McCabe commented on HDFS-7278:

bq. ATM wrote: Seems like we should restrict this command to require super user privileges.
As it stands I believe any user could connect to the DN to trigger a full BR, which though
not super harmful doesn't seem right, either.

It does check superuser privilieges here:

+  @Override // ClientDatanodeProtocol
+  public void triggerBlockReport(BlockReportOptions options)
+      throws IOException {
+    checkSuperuserPrivilege();

As non-superuser, you get this:
woot@keter:/home/cmccabe/hadoop1> /h/bin/hdfs dfsadmin -triggerBlockReport -full
triggerBlockReport error: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException):
Permission denied.
        at org.apache.hadoop.hdfs.server.datanode.DataNode.checkSuperuserPrivilege(DataNode.java:757)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.triggerBlockReport(DataNode.java:2915)

bq. I think there may be a small race condition in the test case. Since you create a file
and then immediately create a spy object to examine calls between the DN and NN, and then
assert that no calls of blockReceivedAndDeleted were made, I think it's possible that the
DN RPC to send an immediate incremental BR for that file creation might be delayed until after
you've created the spy, which would cause the test to unnecessarily fail. I think more reliable
would be to create the spy object before creating the file, and then assert that exactly one
IBR was sent.

Good catch.  I will switch it over to creating the spy first and asserting that one IBR is

bq. I suspect that the most common use of this command will be to trigger full block reports,
not incremental block reports, given that those are sent rather frequently in a busy cluster
anyway. Perhaps we should change the default behavior of the command to send a full BR, and
change the optional flag to be "-incremental" instead?


bq. Akira wrote: By the way, I suggest to add a command for admin to manually trigger DirectoryScanner
as well in order to avoid inconsistency between the disk and the memory.

Sounds reasonable.  I think we should do that in a separate JIRA, though.

> Add a command that allows sysadmins to manually trigger full block reports from a DN
> ------------------------------------------------------------------------------------
>                 Key: HDFS-7278
>                 URL: https://issues.apache.org/jira/browse/HDFS-7278
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode
>    Affects Versions: 2.6.0
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>         Attachments: HDFS-7278.002.patch
> We should add a command that allows sysadmins to manually trigger full block reports
from a DN.

This message was sent by Atlassian JIRA

View raw message