hudi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yixue Zhu (Jira)" <j...@apache.org>
Subject [jira] [Created] (HUDI-603) HoodieDeltaStreamer should periodically fetch table schema update
Date Thu, 06 Feb 2020 18:26:00 GMT
Yixue Zhu created HUDI-603:
------------------------------

             Summary: HoodieDeltaStreamer should periodically fetch table schema update
                 Key: HUDI-603
                 URL: https://issues.apache.org/jira/browse/HUDI-603
             Project: Apache Hudi (incubating)
          Issue Type: Bug
          Components: DeltaStreamer
            Reporter: Yixue Zhu


HoodieDeltaStreamer create SchemaProvider instance and delegate to DeltaSync for periodical
sync. However, default implementation of SchemaProvider does not refresh schema, which can
change due to schema evolution. DeltaSync snapshot the schema when it creates writeClient,
using the SchemaProvider instance or pick up from source, and the schema for writeClient is
not refreshed during the loop of Sync.

I think this needs to be addressed to support schema evolution fully.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message