carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Geetika Gupta (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CARBONDATA-1964) SET command does not set the parameters correctly
Date Wed, 17 Jan 2018 08:12:03 GMT

     [ https://issues.apache.org/jira/browse/CARBONDATA-1964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Geetika Gupta updated CARBONDATA-1964:
--------------------------------------
    Description: 
I created the following table:
 CREATE TABLE uniqdata(CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION string, DOB timestamp,
DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10),
DECIMAL_COLUMN2 decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES('DICTIONARY_INCLUDE'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');

then I set the parameter carbon.options.bad.records.action using:
 *set carbon.options.bad.records.action=fail;
 *
 Load command:
 LOAD DATA INPATH 'hdfs://localhost:54311/Files/2000_UniqData.csv' into table uniqdata OPTIONS('DELIMITER'=',',
'QUOTECHAR'='"','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1','timestampformat'='dd/mm/yyyy');
 +----------+-+
|Result|

+----------+-+
 +----------+-+
 No rows selected (1.43 seconds)

The load executed successfully. However, no data is loaded into the table due to mismatch
of timestamp format.

Then I again set the parameter carbon.options.bad.records.action using:
 *set carbon.options.bad.records.action=FAIL;
 *
 This time the same load command gave me the following exception:

Error: java.lang.Exception: Data load failed due to bad record: The value with column name
dob and column data type TIMESTAMP is not a valid TIMESTAMP type.Please enable bad record
logger to know the detail reason. (state=,code=0)

The first case should behave in the same as manner as the second case. So the SET command
does not set the parameter values correctly and it does not even throw an exception when the
value is not set correctly.

  was:
I created the following table:
CREATE TABLE uniqdata(CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION string, DOB timestamp,
DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10),
DECIMAL_COLUMN2 decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES('DICTIONARY_INCLUDE'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');

then I set the parameter carbon.options.bad.records.action using:
*set carbon.options.bad.records.action=fail;
*
Load command:
LOAD DATA INPATH 'hdfs://localhost:54311/Files/2000_UniqData.csv' into table uniqdata OPTIONS('DELIMITER'=',',
'QUOTECHAR'='"','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1','timestampformat'='dd/mm/yyyy');
+---------+--+
| Result  |
+---------+--+
+---------+--+
No rows selected (1.43 seconds)

The load executed successfully. However, no data is loaded into the table due to mismatch
of timestamp format.

Then I again set the parameter carbon.options.bad.records.action using:
*set carbon.options.bad.records.action=FAIL;
*
This time the same load command gave me the following exception:

Error: java.lang.Exception: Data load failed due to bad record: The value with column name
dob and column data type TIMESTAMP is not a valid TIMESTAMP type.Please enable bad record
logger to know the detail reason. (state=,code=0)

The first case should behave in the same as manner as the second case. So the SET command
does not set the parameter values correctly and it does not even throw an exception when the
value is not set correctly.




> SET command does not set the parameters correctly
> -------------------------------------------------
>
>                 Key: CARBONDATA-1964
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-1964
>             Project: CarbonData
>          Issue Type: Bug
>          Components: data-query
>    Affects Versions: 1.3.0
>         Environment: spark2.1
>            Reporter: Geetika Gupta
>            Priority: Major
>             Fix For: 1.3.0
>
>         Attachments: 2000_UniqData.csv
>
>
> I created the following table:
>  CREATE TABLE uniqdata(CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION string, DOB timestamp,
DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10),
DECIMAL_COLUMN2 decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES('DICTIONARY_INCLUDE'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> then I set the parameter carbon.options.bad.records.action using:
>  *set carbon.options.bad.records.action=fail;
>  *
>  Load command:
>  LOAD DATA INPATH 'hdfs://localhost:54311/Files/2000_UniqData.csv' into table uniqdata
OPTIONS('DELIMITER'=',', 'QUOTECHAR'='"','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1','timestampformat'='dd/mm/yyyy');
>  +----------+-+
> |Result|
> +----------+-+
>  +----------+-+
>  No rows selected (1.43 seconds)
> The load executed successfully. However, no data is loaded into the table due to mismatch
of timestamp format.
> Then I again set the parameter carbon.options.bad.records.action using:
>  *set carbon.options.bad.records.action=FAIL;
>  *
>  This time the same load command gave me the following exception:
> Error: java.lang.Exception: Data load failed due to bad record: The value with column
name dob and column data type TIMESTAMP is not a valid TIMESTAMP type.Please enable bad record
logger to know the detail reason. (state=,code=0)
> The first case should behave in the same as manner as the second case. So the SET command
does not set the parameter values correctly and it does not even throw an exception when the
value is not set correctly.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message