spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Blue <>
Subject [DISCUSS] Catalog APIs and multi-catalog support
Date Thu, 29 Mar 2018 19:34:04 GMT
Hi everyone,

As a follow-up to the SPIP to clean up SparkSQL logical plans
I've written up a proposal for catalog APIs that are required for Spark to
implement reliable high-level operations like CTAS. This includes an API
that is an extension to DataSourceV2 that provides table operations, and
proposes a public API to create and alter tables.

The proposal is here: Spark Catalog APIs

Comments and feedback are welcome! Feel free to comment on the doc or reply
to this thread.


Ryan Blue
Software Engineer

View raw message