carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Akash R Nilugal (Jira)" <j...@apache.org>
Subject [jira] [Resolved] (CARBONDATA-4037) Improve the table status and segment file writing
Date Thu, 22 Apr 2021 09:57:00 GMT

     [ https://issues.apache.org/jira/browse/CARBONDATA-4037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Akash R Nilugal resolved CARBONDATA-4037.
-----------------------------------------
    Fix Version/s: 2.2.0
       Resolution: Fixed

> Improve the table status and segment file writing
> -------------------------------------------------
>
>                 Key: CARBONDATA-4037
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-4037
>             Project: CarbonData
>          Issue Type: Improvement
>            Reporter: SHREELEKHYA GAMPA
>            Priority: Minor
>             Fix For: 2.2.0
>
>         Attachments: Improve table status and segment file writing_1.docx
>
>          Time Spent: 27.5h
>  Remaining Estimate: 0h
>
> Currently, we update table status and segment files multiple times for a single iud/merge/compact
operation and delete the index files immediately after merge. When concurrent queries are
run, there may be situations like user query is trying to access the segment index files and
they are not present, which is availability issue.
>  * To solve above issue, we can make mergeindex files generation mandatory and fail load/compaction
if mergeindex fails. Then if merge index is success, update table status file and can delete
index files immediately. However, in legacy stores when alter segment merge is called, after
merge index success, do not delete index files immediately as it may cause issues for parallel
queries.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message