carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vandana Yadav (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CARBONDATA-1664) Abnormal behavior of timestamp data type in carbondata
Date Thu, 02 Nov 2017 08:27:00 GMT

     [ https://issues.apache.org/jira/browse/CARBONDATA-1664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Vandana Yadav updated CARBONDATA-1664:
--------------------------------------
    Description: 
Abnormal behavior of timestamp data type in carbondata

Steps to Reproduce:

1) Create Table:
CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION string, DOB timestamp,
DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10),
DECIMAL_COLUMN2 decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES ("TABLE_BLOCKSIZE"= "256 MB")

2)Load Data:
LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into table uniqdata
OPTIONS('DELIMITER'=',' , 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1')

3) Execute Query:
a) select DOB from UNIQDATA where DOB ='1970-01-01 10:00:03.0' or DOB = '1970-01-04 01:00:03.0';

output:
+------------------------+--+
|          DOB           |
+------------------------+--+
| 1970-01-01 10:00:03.0  |
| 1970-01-04 01:00:03.0  |
+------------------------+--+

b) select DOB from UNIQDATA where DOB in ('1970-01-01 10:00:03.0','1970-01-04 01:00:03.0');

output:

+------+--+
| DOB  |
+------+--+
+------+--+

c)select DOB from UNIQDATA where DOB in (cast('1970-01-01 10:00:03.0' as timestamp),cast('1970-01-04
01:00:03.0' as timestamp));

output:

+------------------------+--+
|          DOB           |
+------------------------+--+
| 1970-01-01 10:00:03.0  |
| 1970-01-04 01:00:03.0  |
+------------------------+--+

Abnormality of timestamp datatype:
In the select query (a) it fetch the records containing DOB  1970-01-01 10:00:03.0 and 1970-01-04
01:00:03.0 but for query (b) while using IN operator it shows no data and again in the same
query when we cast it to timestamp as in query (c) it displays result.

There should be a strict type checking for timestamp values.



  was:
Abnormal behavior of timestamp data type in carbondata

Steps to Reproduce:

1) Create Table:
CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION string, DOB timestamp,
DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10),
DECIMAL_COLUMN2 decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES ("TABLE_BLOCKSIZE"= "256 MB")

2)Load Data:
LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into table uniqdata
OPTIONS('DELIMITER'=',' , 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1')

3) Execute Query:
a) select DOB from UNIQDATA where DOB ='1970-01-01 10:00:03.0' or DOB = '1970-01-04 01:00:03.0';
output:
+------------------------+--+
|          DOB           |
+------------------------+--+
| 1970-01-01 10:00:03.0  |
| 1970-01-04 01:00:03.0  |
+------------------------+--+

b) select DOB from UNIQDATA where DOB in ('1970-01-01 10:00:03.0','1970-01-04 01:00:03.0');
+------+--+
| DOB  |
+------+--+
+------+--+

c)select DOB from UNIQDATA where DOB in (cast('1970-01-01 10:00:03.0' as timestamp),cast('1970-01-04
01:00:03.0' as timestamp));
+------------------------+--+
|          DOB           |
+------------------------+--+
| 1970-01-01 10:00:03.0  |
| 1970-01-04 01:00:03.0  |
+------------------------+--+

Abnormality of timestamp datatype:
In the select query (a) it fetch the records containing DOB  1970-01-01 10:00:03.0 and 1970-01-04
01:00:03.0 but for query (b) while using IN operator it shows no data and again in the same
query when we cast it to timestamp as in query (c) it displays result.

There should be a strict type checking for timestamp values.




> Abnormal behavior of timestamp data type in carbondata
> ------------------------------------------------------
>
>                 Key: CARBONDATA-1664
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-1664
>             Project: CarbonData
>          Issue Type: Bug
>          Components: data-query
>    Affects Versions: 1.3.0
>         Environment: spark 2.1
>            Reporter: Vandana Yadav
>         Attachments: 2000_UniqData.csv
>
>
> Abnormal behavior of timestamp data type in carbondata
> Steps to Reproduce:
> 1) Create Table:
> CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION string, DOB timestamp,
DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10),
DECIMAL_COLUMN2 decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES ("TABLE_BLOCKSIZE"= "256 MB")
> 2)Load Data:
> LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into table uniqdata
OPTIONS('DELIMITER'=',' , 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1')
> 3) Execute Query:
> a) select DOB from UNIQDATA where DOB ='1970-01-01 10:00:03.0' or DOB = '1970-01-04 01:00:03.0';
> output:
> +------------------------+--+
> |          DOB           |
> +------------------------+--+
> | 1970-01-01 10:00:03.0  |
> | 1970-01-04 01:00:03.0  |
> +------------------------+--+
> b) select DOB from UNIQDATA where DOB in ('1970-01-01 10:00:03.0','1970-01-04 01:00:03.0');
> output:
> +------+--+
> | DOB  |
> +------+--+
> +------+--+
> c)select DOB from UNIQDATA where DOB in (cast('1970-01-01 10:00:03.0' as timestamp),cast('1970-01-04
01:00:03.0' as timestamp));
> output:
> +------------------------+--+
> |          DOB           |
> +------------------------+--+
> | 1970-01-01 10:00:03.0  |
> | 1970-01-04 01:00:03.0  |
> +------------------------+--+
> Abnormality of timestamp datatype:
> In the select query (a) it fetch the records containing DOB  1970-01-01 10:00:03.0 and
1970-01-04 01:00:03.0 but for query (b) while using IN operator it shows no data and again
in the same query when we cast it to timestamp as in query (c) it displays result.
> There should be a strict type checking for timestamp values.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message