carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chetan Bhat (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CARBONDATA-1747) (Carbon1.3.0- DB creation external path) - Owner name of compacted segment and segment after update is not correct
Date Thu, 16 Nov 2017 15:45:00 GMT

     [ https://issues.apache.org/jira/browse/CARBONDATA-1747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Chetan Bhat updated CARBONDATA-1747:
------------------------------------
    Description: 
Steps :
In spark Beeline user executes the following queries
drop database if exists test_db1 cascade;
create database test_db1 location 'hdfs://hacluster/user/test1';
use test_db1;
create table if not exists ORDERS(O_ORDERDATE string,O_ORDERPRIORITY string,O_ORDERSTATUS
string,O_ORDERKEY string,O_CUSTKEY string,O_TOTALPRICE double,O_CLERK string,O_SHIPPRIORITY
int,O_COMMENT string) STORED BY 'org.apache.carbondata.format'TBLPROPERTIES ('table_blocksize'='128');
load data inpath "hdfs://hacluster/chetan/orders.tbl.1" into table ORDERS options('DELIMITER'='|','FILEHEADER'='O_ORDERKEY,O_CUSTKEY,O_ORDERSTATUS,O_TOTALPRICE,O_ORDERDATE,O_ORDERPRIORITY,O_CLERK,O_SHIPPRIORITY,O_COMMENT','batch_sort_size_inmb'='32');
load data inpath "hdfs://hacluster/chetan/orders.tbl.1" into table ORDERS options('DELIMITER'='|','FILEHEADER'='O_ORDERKEY,O_CUSTKEY,O_ORDERSTATUS,O_TOTALPRICE,O_ORDERDATE,O_ORDERPRIORITY,O_CLERK,O_SHIPPRIORITY,O_COMMENT','batch_sort_size_inmb'='32');
load data inpath "hdfs://hacluster/chetan/orders.tbl.1" into table ORDERS options('DELIMITER'='|','FILEHEADER'='O_ORDERKEY,O_CUSTKEY,O_ORDERSTATUS,O_TOTALPRICE,O_ORDERDATE,O_ORDERPRIORITY,O_CLERK,O_SHIPPRIORITY,O_COMMENT','batch_sort_size_inmb'='32');
load data inpath "hdfs://hacluster/chetan/orders.tbl.1" into table ORDERS options('DELIMITER'='|','FILEHEADER'='O_ORDERKEY,O_CUSTKEY,O_ORDERSTATUS,O_TOTALPRICE,O_ORDERDATE,O_ORDERPRIORITY,O_CLERK,O_SHIPPRIORITY,O_COMMENT','batch_sort_size_inmb'='32');
alter table ORDERS compact 'major';
update orders set (O_ORDERKEY)=(1) where O_CUSTKEY=6259021;

After compaction and update user checks the Owner name of compacted segment and segment name
after update in HDFS UI.

Issue : In HDFS UI before compaction and update the owner name of the existing segment folders
was "anonymous". After compaction and update the owner name of the compacted segment folder
and segment which is impacted by update is displayed as "root".

Expected : After compaction and update the owner name of the compacted segment folder and
segment which is impacted by update should be "anonymous".

  was:
Steps :
User executes the following queries
drop database if exists test_db1 cascade;
create database test_db1 location 'hdfs://hacluster/user/test1';
use test_db1;
create table if not exists ORDERS(O_ORDERDATE string,O_ORDERPRIORITY string,O_ORDERSTATUS
string,O_ORDERKEY string,O_CUSTKEY string,O_TOTALPRICE double,O_CLERK string,O_SHIPPRIORITY
int,O_COMMENT string) STORED BY 'org.apache.carbondata.format'TBLPROPERTIES ('table_blocksize'='128');
load data inpath "hdfs://hacluster/chetan/orders.tbl.1" into table ORDERS options('DELIMITER'='|','FILEHEADER'='O_ORDERKEY,O_CUSTKEY,O_ORDERSTATUS,O_TOTALPRICE,O_ORDERDATE,O_ORDERPRIORITY,O_CLERK,O_SHIPPRIORITY,O_COMMENT','batch_sort_size_inmb'='32');
load data inpath "hdfs://hacluster/chetan/orders.tbl.1" into table ORDERS options('DELIMITER'='|','FILEHEADER'='O_ORDERKEY,O_CUSTKEY,O_ORDERSTATUS,O_TOTALPRICE,O_ORDERDATE,O_ORDERPRIORITY,O_CLERK,O_SHIPPRIORITY,O_COMMENT','batch_sort_size_inmb'='32');
load data inpath "hdfs://hacluster/chetan/orders.tbl.1" into table ORDERS options('DELIMITER'='|','FILEHEADER'='O_ORDERKEY,O_CUSTKEY,O_ORDERSTATUS,O_TOTALPRICE,O_ORDERDATE,O_ORDERPRIORITY,O_CLERK,O_SHIPPRIORITY,O_COMMENT','batch_sort_size_inmb'='32');
load data inpath "hdfs://hacluster/chetan/orders.tbl.1" into table ORDERS options('DELIMITER'='|','FILEHEADER'='O_ORDERKEY,O_CUSTKEY,O_ORDERSTATUS,O_TOTALPRICE,O_ORDERDATE,O_ORDERPRIORITY,O_CLERK,O_SHIPPRIORITY,O_COMMENT','batch_sort_size_inmb'='32');
alter table ORDERS compact 'major';
update orders set (O_ORDERKEY)=(1) where O_CUSTKEY=6259021;

After compaction and update user checks the Owner name of compacted segment and segment name
after update in HDFS UI.

Issue : In HDFS UI before compaction and update the owner name of the existing segment folders
was "anonymous". After compaction and update the owner name of the compacted segment folder
and segment which is impacted by update is displayed as "root".

Expected : After compaction and update the owner name of the compacted segment folder and
segment which is impacted by update should be "anonymous".


> (Carbon1.3.0- DB creation external path) - Owner name of compacted segment and segment
after update is not correct
> ------------------------------------------------------------------------------------------------------------------
>
>                 Key: CARBONDATA-1747
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-1747
>             Project: CarbonData
>          Issue Type: Bug
>          Components: other
>    Affects Versions: 1.3.0
>         Environment: 3 node ant cluster
>            Reporter: Chetan Bhat
>              Labels: security
>
> Steps :
> In spark Beeline user executes the following queries
> drop database if exists test_db1 cascade;
> create database test_db1 location 'hdfs://hacluster/user/test1';
> use test_db1;
> create table if not exists ORDERS(O_ORDERDATE string,O_ORDERPRIORITY string,O_ORDERSTATUS
string,O_ORDERKEY string,O_CUSTKEY string,O_TOTALPRICE double,O_CLERK string,O_SHIPPRIORITY
int,O_COMMENT string) STORED BY 'org.apache.carbondata.format'TBLPROPERTIES ('table_blocksize'='128');
> load data inpath "hdfs://hacluster/chetan/orders.tbl.1" into table ORDERS options('DELIMITER'='|','FILEHEADER'='O_ORDERKEY,O_CUSTKEY,O_ORDERSTATUS,O_TOTALPRICE,O_ORDERDATE,O_ORDERPRIORITY,O_CLERK,O_SHIPPRIORITY,O_COMMENT','batch_sort_size_inmb'='32');
> load data inpath "hdfs://hacluster/chetan/orders.tbl.1" into table ORDERS options('DELIMITER'='|','FILEHEADER'='O_ORDERKEY,O_CUSTKEY,O_ORDERSTATUS,O_TOTALPRICE,O_ORDERDATE,O_ORDERPRIORITY,O_CLERK,O_SHIPPRIORITY,O_COMMENT','batch_sort_size_inmb'='32');
> load data inpath "hdfs://hacluster/chetan/orders.tbl.1" into table ORDERS options('DELIMITER'='|','FILEHEADER'='O_ORDERKEY,O_CUSTKEY,O_ORDERSTATUS,O_TOTALPRICE,O_ORDERDATE,O_ORDERPRIORITY,O_CLERK,O_SHIPPRIORITY,O_COMMENT','batch_sort_size_inmb'='32');
> load data inpath "hdfs://hacluster/chetan/orders.tbl.1" into table ORDERS options('DELIMITER'='|','FILEHEADER'='O_ORDERKEY,O_CUSTKEY,O_ORDERSTATUS,O_TOTALPRICE,O_ORDERDATE,O_ORDERPRIORITY,O_CLERK,O_SHIPPRIORITY,O_COMMENT','batch_sort_size_inmb'='32');
> alter table ORDERS compact 'major';
> update orders set (O_ORDERKEY)=(1) where O_CUSTKEY=6259021;
> After compaction and update user checks the Owner name of compacted segment and segment
name after update in HDFS UI.
> Issue : In HDFS UI before compaction and update the owner name of the existing segment
folders was "anonymous". After compaction and update the owner name of the compacted segment
folder and segment which is impacted by update is displayed as "root".
> Expected : After compaction and update the owner name of the compacted segment folder
and segment which is impacted by update should be "anonymous".



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message