hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron Fabbri (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14094) Rethink S3GuardTool options
Date Fri, 03 Mar 2017 00:20:45 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15893363#comment-15893363
] 

Aaron Fabbri commented on HADOOP-14094:
---------------------------------------

Ok.. I think we should fix the indent levels that checkstyle is flagging (USAGE string literals,
etc).

After that I am +1.

I built the tool and tested it.  Ran diff / import.   Also ran all integration tests in us-west-2
w/ DDB. Only failure I saw was HADOOP-14036.

BTW, I may have found another bug in the CLI: I ran import then diff and saw (1) diff was
non-empty after import, and (2) diff printed everything twice.  I'll open a new JIRA as it
appears to be unrelated to this change.

> Rethink S3GuardTool options
> ---------------------------
>
>                 Key: HADOOP-14094
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14094
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>            Reporter: Sean Mackrory
>            Assignee: Sean Mackrory
>         Attachments: HADOOP-14094-HADOOP-13345.001.patch, HADOOP-14094-HADOOP-13345.002.patch,
HADOOP-14094-HADOOP-13345.003.patch, HADOOP-14094-HADOOP-13345.003.patch, HADOOP-14094-HADOOP-13345.004.patch
>
>
> I think we need to rework the S3GuardTool options. A couple of problems I've observed
in the patches I've done on top of that and seeing other developers trying it out:
> * We should probably wrap the current commands in an S3Guard-specific command, since
'init', 'destroy', etc. don't touch the buckets at all.
> * Convert to whole-word options, as the single-letter options are already getting overloaded.
Some patches I've submitted have added functionality where the obvious flag is already in
use (e.g. -r for region, and read throughput, -m for minutes, and metadatastore uri).  I may
do this early as part of HADOOP-14090.
> * We have some options that must be in the config in some cases, and can be in the command
in other cases. But I've seen someone try to specify the table name in the config and leave
out the -m option, with no luck. Also, since commands hard-code table auto-creation, you might
have configured table auto-creation, try to import to a non-existent table, and it tells you
table auto-creation is off.
> We need a more consistent policy for how things should get configured that addresses
these problems and future-proofs the command a bit more.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message