hudi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pratyaksh Sharma (Jira)" <j...@apache.org>
Subject [jira] [Commented] (HUDI-96) Use Command line options instead of positional arguments when launching spark applications from various CLI commands
Date Fri, 03 Jan 2020 11:13:00 GMT

    [ https://issues.apache.org/jira/browse/HUDI-96?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007412#comment-17007412
] 

Pratyaksh Sharma commented on HUDI-96:
--------------------------------------

[~vbalaji] I have resumed the work for this ticket and have tried to address most of the
comments that you already gave. I have raised a fresh PR for this. Please have a look and
let me know your thoughts. Here is the PR - [https://github.com/apache/incubator-hudi/pull/1174].

> Use Command line options instead of positional arguments when launching spark applications
from various CLI commands
> --------------------------------------------------------------------------------------------------------------------
>
>                 Key: HUDI-96
>                 URL: https://issues.apache.org/jira/browse/HUDI-96
>             Project: Apache Hudi (incubating)
>          Issue Type: Improvement
>          Components: CLI, newbie
>            Reporter: Balaji Varadarajan
>            Assignee: Pratyaksh Sharma
>            Priority: Minor
>              Labels: newbie, pull-request-available
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> Hoodie CLI commands like compaction/rollback/repair/savepoints/parquet-import relies
on launching a spark application to perform their operations (look at SparkMain.java). 
> SparkMain (Look at SparkMain.main()) relies on positional arguments for passing  various
CLI options. Instead we should define proper CLI options in SparkMain and use them (using
Jcommander)  to improve readability and avoid accidental errors at call sites. For e.g :
See com.uber.hoodie.utilities.HoodieCompactor



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message