hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Weiwei Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11668) Ozone: misc improvements for SCM CLI
Date Wed, 19 Apr 2017 12:34:41 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15974550#comment-15974550
] 

Weiwei Yang commented on HDFS-11668:
------------------------------------

Hi [~anu], [~xyao]

Please help to review this JIRA, I am trying to get SCM CLI work nicely on a pseudo cluster,
during my testing, I found the problems I wrote in the description. This patch includes following
fixes

*hdfs*

Integrate SCM CLI into hdfs script, this is the followup work after HDFS-11649. With this
patch, {{bin/hdfs scm}} will start scm daemon, and if there is more arguments given, e.g {{bin/hdfs
scm -container}}, it will parse to SCM CLI classes.

*InitDatanodeState*

Make sure datanode id file (ozone.scm.datanode.id) is properly set in configuration, otherwise
set next datanode state to SHUTDOWN. And it prompts user a meaningful error message "A valid
file path is needed for config setting ozone.scm.datanode.id" to help user understand what
was wrong.

*StateContext*

Added a log message to indicate how datanode state is transited, log looks like

{noformat}
17/04/19 04:57:33 INFO statemachine.StateContext: Task InitDatanodeState executed, state transited
from INIT to RUNNING
{noformat}

this log is only printed when state changes.

*TestDatanodeStateMachine*

Renamed test case {{testDatanodeStateMachineWithInvalidConfiguration}} and added test coverage
for a negative case that {{ozone.scm.datanode.id}} was not set.

I tested this on my single node cluster, with this patch, I am able to create a container
with following command
{noformat}
[wwei@ozone1 hadoop-3.0.0-alpha3-SNAPSHOT]$ ./bin/hdfs scm -container -create -p 20170418c1
17/04/19 05:18:10 INFO cli.OzoneCommandHandler: Create container : 20170418c1
17/04/19 05:18:10 INFO client.ContainerOperationClient: Created container 20170418c1 leader:172.16.165.133:9866
machines:[172.16.165.133:9866]

[wwei@ozone1 hadoop-3.0.0-alpha3-SNAPSHOT]$ ./bin/hdfs scm -container -create -p 20170418c1
17/04/19 05:18:27 INFO cli.OzoneCommandHandler: Create container :20170418c1
Error executing command:org.apache.hadoop.ipc.RemoteException(java.io.IOException): Specified
container already exists. key : 20170418c1
...
{noformat}

first operation succeed and second failed because container already exists.

Please kindly help to review, thanks.


> Ozone: misc improvements for SCM CLI
> ------------------------------------
>
>                 Key: HDFS-11668
>                 URL: https://issues.apache.org/jira/browse/HDFS-11668
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Weiwei Yang
>            Assignee: Weiwei Yang
>              Labels: cli, command-line
>         Attachments: HDFS-11668-HDFS-7240.001.patch
>
>
> Once HDFS-11649 is done, there are some improvements need to be done in order to make
SCM CLI better to use in a pseudo cluster, this includes
> # HDFS-11649 adds java classes for CLIs, we will need to add shell code to expose these
commands
> # Better error messages when missing some key ozone configurations, e.g {{ozone.scm.names}},
{{ozone.scm.datanode.id}} ... etc
> # Better logging. Currently {{DatanodeStateMachine}} prints very limited logs, adds some
more logs to indicate state transition is necessary.
> The ultimate goal of this ticket is to ensure SCM CLI can work nicely on a pseudo cluster.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message