hudi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinoth Chandar (Jira)" <j...@apache.org>
Subject [jira] [Commented] (HUDI-658) Make ClientUtils spark-free
Date Fri, 06 Mar 2020 18:07:00 GMT

    [ https://issues.apache.org/jira/browse/HUDI-658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17053659#comment-17053659
] 

Vinoth Chandar commented on HUDI-658:
-------------------------------------

May be.. but I have a concern with how we approaching this multi engine refactoring.. we need
a top down design and probably proof of concept few things at the high level first, before
we can go down the list to smaller items like this.. We cannot do this class by class IMO..
lets hash out a high level plan first? 

> Make ClientUtils spark-free
> ---------------------------
>
>                 Key: HUDI-658
>                 URL: https://issues.apache.org/jira/browse/HUDI-658
>             Project: Apache Hudi (incubating)
>          Issue Type: Sub-task
>            Reporter: vinoyang
>            Assignee: vinoyang
>            Priority: Major
>
> {{ClientUtils#createMetaClient}} require {{JavaSparkContext}} only for getting the hadoop
configuration obejct. We can pass the {{Configuration}} object directly so that we can make
{{ClientUtils}} spark-free.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message