carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sounakr <>
Subject [GitHub] incubator-carbondata pull request #723: [CARBONDATA-845] Insert Select Into ...
Date Mon, 03 Apr 2017 09:42:39 GMT
GitHub user sounakr opened a pull request:

    [CARBONDATA-845] Insert Select Into Same Table

    Problem : Insert Select Implementation in current Spark-2.1 prohibits to read and write
into same table. 
    Analysis : This restriction is imposed by spark in case InsertableRelation is extended.
Current insert select implementation in spark-2.1 CarbonDatasourceHadoopRelation extends InsertableRelation
and override the insert method. Spark rules in case of InsertableRelation checks if source
and target tables are same, then it fails and prompts  "Cannot insert overwrite into table
that is also being read from."
    Fix:: Spark 1.5 implementation of insert select in carbon allows reading and writing into
same table, so as Hive and Parquet. Therfore as part of current fix reimplemented insert-select
logic of Spark-2.1 similar to Spark-1.5. 
     a) CarbonDatasourceHadoopRelation doesn't extends InsertableRelation.
     b) New CarbonPreInsertionCasts Rule is written for spark-2.1 (For Spark-1.5 it was already
     d) This new Rule is registered CarbonSessionState, 
     c) During Insert Select processing the Logical Plan wrapped by InsertIntoCarbonTable
and later execute LoadTableByInsert to process insert select.

You can merge this pull request into a Git repository by running:

    $ git pull insert_select

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #723
commit beb26625dc8698574f9be4701a0b80504097f049
Author: sounakr <>
Date:   2017-04-03T07:18:20Z

    Insert Select Into Same Table


If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

View raw message