carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jackylk <>
Subject [GitHub] carbondata issue #1321: [CARBONDATA-1438] Unify the sort column and sort sco...
Date Wed, 06 Sep 2017 02:42:12 GMT
Github user jackylk commented on the issue:
    I suggest we consider these two requirements separately:
    1. Streaming ingestion to carbon table
    2. Want to override the sort scope after table is created.
    For 1, I think we should probably have another table property in the create table to indicate
this table is streaming ingestable. And we should have separate API instead of  SQL to do
the streaming ingest. Anyway, I think it is better to design after carbon 1.2 in streaming
ingest branch.
    For 2, I feel we should not allow this, at least for carbon 1.2, because this will open
up many complexity. Not just loading, but also compaction a lot of validation and configuration
to make it work for different level. We should allow this only if all combination scenario
has been tested. 
    In one word, we should remove the SORT_SCOPE option in LOAD DATA statement for carbondata
1.2. If there are requirement for it in future, we always can add it later.


View raw message