carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Manish Gupta (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CARBONDATA-1486) Fixed issue of table status updation on insert overwrite failure and exception thrown while deletion of stale folders
Date Tue, 19 Sep 2017 07:01:00 GMT

     [ https://issues.apache.org/jira/browse/CARBONDATA-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Manish Gupta updated CARBONDATA-1486:
-------------------------------------
    Description: 
Problems to be fixed:

1. HDFS file system throwing exception while deleting a non-existent folder. This happens
because on multiple run of insert overwrite operation, same folder is getting added to the
stale folder list even though it has been deleted during first time insert overwrite operation
run and when the same folder is tried to be deleted again HDFS file system throws an IO exception.

2. Insert or Load operation should throw exception if:
       a. If insert overwrite is in progress and any other load or insert operation is triggered
       b. If load or insert into operation is in progress and insert overwrite operation is
triggered

3. Table status not getting updated after failure of insert/load job. In case of insert overwrite
job status in table status file is "Overwrite in Progress". In this case no other load for
same table can run in parallel and if this status is not changes on success of failure jobs,
the system will still assume that insert overwrite is in progress and fail all next insert/load
jobs

4. Removed Unused Code

  was:
Problems to be fixed:

1. HDFS file system throwing exception while deleting a non-existent folder. This happens
because on multiple run of insert overwrite operation, same folder is getting added to the
stale folder list even though it has been deleted during first time insert overwrite operation
run and when the same folder is tried to be deleted again HDFS file system throws an IO exception.

2. Insert or Load operation should throw exception if:
       a. If insert overwrite is in progress and any other load or insert operation is triggered
       b. If load or insert into operation is in progress and insert overwrite operation is
triggered

3. Table status not getting updated after failure of insert/load job. In case of insert overwrite
job status in table status file is "Overwrite in Progress". In this case no other load for
same table can run in parallel and if this status is not changes on success of failure jobs,
the system will still assume that insert overwrite is in progress and fail all next insert/load
jobs

4. Fortify issue for the category "Identical code for different branches"


> Fixed issue of table status updation on insert overwrite failure and exception thrown
while deletion of stale folders
> ---------------------------------------------------------------------------------------------------------------------
>
>                 Key: CARBONDATA-1486
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-1486
>             Project: CarbonData
>          Issue Type: Bug
>            Reporter: Manish Gupta
>            Assignee: Manish Gupta
>            Priority: Minor
>             Fix For: 1.2.0
>
>
> Problems to be fixed:
> 1. HDFS file system throwing exception while deleting a non-existent folder. This happens
because on multiple run of insert overwrite operation, same folder is getting added to the
stale folder list even though it has been deleted during first time insert overwrite operation
run and when the same folder is tried to be deleted again HDFS file system throws an IO exception.
> 2. Insert or Load operation should throw exception if:
>        a. If insert overwrite is in progress and any other load or insert operation is
triggered
>        b. If load or insert into operation is in progress and insert overwrite operation
is triggered
> 3. Table status not getting updated after failure of insert/load job. In case of insert
overwrite job status in table status file is "Overwrite in Progress". In this case no other
load for same table can run in parallel and if this status is not changes on success of failure
jobs, the system will still assume that insert overwrite is in progress and fail all next
insert/load jobs
> 4. Removed Unused Code



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message