ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Manoj G T (Jira)" <j...@apache.org>
Subject [jira] [Commented] (IGNITE-9228) Spark SQL Table Schema Specification
Date Thu, 05 Sep 2019 05:43:00 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-9228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16923059#comment-16923059

Manoj G T commented on IGNITE-9228:

[~NIzhikov] [~stuartmacd] I have gone through the implementation of supporting schema name Ignite
Spark SQL and it's only having support to schema name to read data, not for write. Any specific
reason for the same? Is it in the future roadmap?

> Spark SQL Table Schema Specification
> ------------------------------------
>                 Key: IGNITE-9228
>                 URL: https://issues.apache.org/jira/browse/IGNITE-9228
>             Project: Ignite
>          Issue Type: Improvement
>          Components: spark
>    Affects Versions: 2.6
>            Reporter: Stuart Macdonald
>            Assignee: Stuart Macdonald
>            Priority: Major
>             Fix For: 2.8
> The Ignite Spark SQL interface currently takes just “table name” as a
> parameter which it uses to supply a Spark dataset with data from the
> underlying Ignite SQL table with that name.
> To do this it loops through each cache and finds the first one with the
> given table name [1]. This causes issues if there are multiple tables
> registered in different schema with the same table name as you can only
> access one of those from Spark. We could either:
> 1. Pass an extra parameter through the Ignite Spark data source which
> optionally specifies the schema name.
> 2. Support namespacing in the existing table name parameter, ie
> “schemaName.tableName”
> [1 ]https://github.com/apache/ignite/blob/ca973ad99c6112160a305df05be9458e29f88307/modules/spark/src/main/scala/org/apache/ignite/spark/impl/package.scala#L119

This message was sent by Atlassian Jira

View raw message