Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Thu, 23 Oct 2014 21:08:34 +0000 (UTC)
From: "Colin Patrick McCabe (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12749929.1414022453000.326323.1414098514525@Atlassian.JIRA>
In-Reply-To: <JIRA.12749929.1414022453000@Atlassian.JIRA>
References: <JIRA.12749929.1414022453000@Atlassian.JIRA>
 <JIRA.12749929.1414022453843@arcas>
Subject: [jira] [Commented] (HDFS-7278) Add a command that allows sysadmins
 to manually trigger full block reports from a DN
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-7278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14181969#comment-14181969 ] 

Colin Patrick McCabe commented on HDFS-7278:
--------------------------------------------

bq. ATM wrote: Seems like we should restrict this command to require super user privileges. As it stands I believe any user could connect to the DN to trigger a full BR, which though not super harmful doesn't seem right, either.

It does check superuser privilieges here:

{code}
+  @Override // ClientDatanodeProtocol
+  public void triggerBlockReport(BlockReportOptions options)
+      throws IOException {
+    checkSuperuserPrivilege();
{code}

As non-superuser, you get this:
{code}
woot@keter:/home/cmccabe/hadoop1> /h/bin/hdfs dfsadmin -triggerBlockReport -full 127.0.0.1:6102
triggerBlockReport error: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): Permission denied.
        at org.apache.hadoop.hdfs.server.datanode.DataNode.checkSuperuserPrivilege(DataNode.java:757)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.triggerBlockReport(DataNode.java:2915)
{code}

bq. I think there may be a small race condition in the test case. Since you create a file and then immediately create a spy object to examine calls between the DN and NN, and then assert that no calls of blockReceivedAndDeleted were made, I think it's possible that the DN RPC to send an immediate incremental BR for that file creation might be delayed until after you've created the spy, which would cause the test to unnecessarily fail. I think more reliable would be to create the spy object before creating the file, and then assert that exactly one IBR was sent.

Good catch.  I will switch it over to creating the spy first and asserting that one IBR is sent.

bq. I suspect that the most common use of this command will be to trigger full block reports, not incremental block reports, given that those are sent rather frequently in a busy cluster anyway. Perhaps we should change the default behavior of the command to send a full BR, and change the optional flag to be "-incremental" instead?

OK.

bq. Akira wrote: By the way, I suggest to add a command for admin to manually trigger DirectoryScanner as well in order to avoid inconsistency between the disk and the memory.

Sounds reasonable.  I think we should do that in a separate JIRA, though.

> Add a command that allows sysadmins to manually trigger full block reports from a DN
> ------------------------------------------------------------------------------------
>
>                 Key: HDFS-7278
>                 URL: https://issues.apache.org/jira/browse/HDFS-7278
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode
>    Affects Versions: 2.6.0
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>         Attachments: HDFS-7278.002.patch
>
>
> We should add a command that allows sysadmins to manually trigger full block reports from a DN.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)