hudi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "vinoyang (Jira)" <j...@apache.org>
Subject [jira] [Created] (HUDI-659) Make HoodieCommitArchiveLog spark free
Date Thu, 05 Mar 2020 08:52:00 GMT
vinoyang created HUDI-659:
-----------------------------

             Summary: Make HoodieCommitArchiveLog spark free
                 Key: HUDI-659
                 URL: https://issues.apache.org/jira/browse/HUDI-659
             Project: Apache Hudi (incubating)
          Issue Type: Sub-task
            Reporter: vinoyang
            Assignee: vinoyang


Currently, {{HoodieCommitArchiveLog}} depends on {{JavaSparkContext}} in its two methods:
{{archiveIfRequired}} and {{getInstantsToArchive}}. These two methods pass {{JavaSparkContext}}
to get {{HoodieTable}} object. After diving into the call chain, I found we can replace {{JavaSparkContext}}
with {{Configuration}} and other some cleanup. After that, we can make {{HoodieCommitArchiveLog}}
spark free.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message